[GRASSLIST:8437] Re: "Large" file problem. Can do clipping with arcview but not GRASS...

[Craig Aumann]

Thanks!  I certainly appreciate all the work you do Radim and I posted
this more to find out if I was doing something stupid and if there were
work arounds.  Thanks again!



On Tue, 2005-09-27 at 01:24, Radim Blazek wrote:
> I added Vect_set_release_support because the map is
> reopened in update mode. Should be better but probably still
> needs a lot of memory.
> Everything can be optimized but I don't have time to do everything.
> 
> Radim
> 
> On 9/26/05, Craig Aumann <caumann at ualberta.ca> wrote:
> > This is related to my earlier post 8400.  To get around the problem of
> > importing the files, I simply used v.external.  However, when I went to
> > do some overlays all of the overlays crashed as well:
> >
> > On one file:
> >
> > Breaking lines ...
> > Intersections: 832406 (line 888364)File size limit exceeded
> >
> >
> > And on another file:
> >
> > Cleaning boundaries at nodes ...
> > Modifications:  8082
> > -----------------------------------------------------------------------------
> > Breaking lines ...
> > Intersections:     1 (line 2955912)Unexpected error.
> > Aborted
> > GRASS 6.1.cvs (wavelets):~/DATA/ILM/scalingup/analysis/readin_setup >
> >
> >
> > What really upsets me is that I was able to do the overlays (clippings
> > actually) using ArcView's geoprocessing tool on these large files - each
> > one taking approximately 10-15 minutes on a Windows XP machine with only
> > 512 MB of memory!
> >
> > The Linux machine has Grass60, has 1.5 Gig of memory and 2 gig of swap
> > space.  I don't know the reasons here, but I'm having to eat crow for
> > not being able to handle these files with GRASS...
> >
> > Cheers!
> > Craig
> >
> >
> > On Mon, 2005-09-26 at 01:16, Radim Blazek wrote:
> > > It can be that it consumes a lot of memory. Probably you have
> > > to add swap.
> > >
> > > Radim
> > >
> > > On 9/23/05, Craig Aumann <caumann at ualberta.ca> wrote:
> > > > I was reading a large shape file:
> > > >
> > > > -rw-rw-r--    1 caumann  caumann  111771178 Apr 17 10:44 neforest.dbf
> > > > -rw-rw-r--    1 caumann  caumann   2355756 Apr  7  2003 neforest.sbn
> > > > -rw-rw-r--    1 caumann  caumann    119812 Apr  7  2003 neforest.sbx
> > > > -rw-rw-r--    1 caumann  caumann  746124196 Dec 10  2002 neforest.shp
> > > > -rw-rw-r--    1 caumann  caumann   1973924 Dec 10  2002 neforest.shx
> > > >
> > > > and got the error:
> > > >
> > > > ERROR: G_realloc: out of memory
> > > >
> > > > Can v.in.ogr not handle this size of file?
> > > >
> > > > Call and output are below
> > > >
> > > > Cheers!
> > > > Craig
> > > > -------------------------------------------------------
> > > >
> > > > v.in.ogr -o dsn="$SHPDIR" \
> > > >         output="$SHPFILE" layer="$SHPFILE" snap=1
> > > > Over-riding projection check.
> > > > Proceeding with import...
> > > > Layer: neforest
> > > > WARNING: 'user' in database definition is not supported, use db.login
> > > > -----------------------------------------------------
> > > > Building topology ...
> > > > 259136 primitives registered
> > > > Building areas:  100%
> > > > 204742 areas built
> > > > 194638 isles built
> > > > Attaching islands:  100%
> > > > Attaching centroids:  100%
> > > > Topology was built.
> > > > Number of nodes     :   226019
> > > > Number of primitives:   259136
> > > > Number of points    :   0
> > > > Number of lines     :   0
> > > > Number of boundaries:   259136
> > > > Number of centroids :   0
> > > > Number of areas     :   204742
> > > > Number of isles     :   194638
> > > > Number of incorrect boundaries   :   63856
> > > > Number of areas without centroid :   204742
> > > > -----------------------------------------------------
> > > > WARNING: Cleaning polygons, result is not guaranteed!
> > > > Building topology ...
> > > > Topology was built.
> > > > Number of nodes     :   226019
> > > > Number of primitives:   259136
> > > > Number of points    :   0
> > > > Number of lines     :   0
> > > > Number of boundaries:   259136
> > > > Number of centroids :   0
> > > > Number of areas     :   -
> > > > Number of isles     :   -
> > > > -----------------------------------------------------
> > > > Snap boundaries (threshold = 1.000e+00):
> > > > Registering points ... 20377235ERROR: G_realloc: out of memory
> > > >
> > > >
> > >
> >
> >
>