[GRASSLIST:8432] Re: "Large" file problem. Can do clipping with arcview but not GRASS...

[Radim Blazek]

I added Vect_set_release_support because the map is
reopened in update mode. Should be better but probably still
needs a lot of memory.
Everything can be optimized but I don't have time to do everything.

Radim

On 9/26/05, Craig Aumann <caumann at ualberta.ca> wrote:
> This is related to my earlier post 8400.  To get around the problem of
> importing the files, I simply used v.external.  However, when I went to
> do some overlays all of the overlays crashed as well:
>
> On one file:
>
> Breaking lines ...
> Intersections: 832406 (line 888364)File size limit exceeded
>
>
> And on another file:
>
> Cleaning boundaries at nodes ...
> Modifications:  8082
> -----------------------------------------------------------------------------
> Breaking lines ...
> Intersections:     1 (line 2955912)Unexpected error.
> Aborted
> GRASS 6.1.cvs (wavelets):~/DATA/ILM/scalingup/analysis/readin_setup >
>
>
> What really upsets me is that I was able to do the overlays (clippings
> actually) using ArcView's geoprocessing tool on these large files - each
> one taking approximately 10-15 minutes on a Windows XP machine with only
> 512 MB of memory!
>
> The Linux machine has Grass60, has 1.5 Gig of memory and 2 gig of swap
> space.  I don't know the reasons here, but I'm having to eat crow for
> not being able to handle these files with GRASS...
>
> Cheers!
> Craig
>
>
> On Mon, 2005-09-26 at 01:16, Radim Blazek wrote:
> > It can be that it consumes a lot of memory. Probably you have
> > to add swap.
> >
> > Radim
> >
> > On 9/23/05, Craig Aumann <caumann at ualberta.ca> wrote:
> > > I was reading a large shape file:
> > >
> > > -rw-rw-r--    1 caumann  caumann  111771178 Apr 17 10:44 neforest.dbf
> > > -rw-rw-r--    1 caumann  caumann   2355756 Apr  7  2003 neforest.sbn
> > > -rw-rw-r--    1 caumann  caumann    119812 Apr  7  2003 neforest.sbx
> > > -rw-rw-r--    1 caumann  caumann  746124196 Dec 10  2002 neforest.shp
> > > -rw-rw-r--    1 caumann  caumann   1973924 Dec 10  2002 neforest.shx
> > >
> > > and got the error:
> > >
> > > ERROR: G_realloc: out of memory
> > >
> > > Can v.in.ogr not handle this size of file?
> > >
> > > Call and output are below
> > >
> > > Cheers!
> > > Craig
> > > -------------------------------------------------------
> > >
> > > v.in.ogr -o dsn="$SHPDIR" \
> > >         output="$SHPFILE" layer="$SHPFILE" snap=1
> > > Over-riding projection check.
> > > Proceeding with import...
> > > Layer: neforest
> > > WARNING: 'user' in database definition is not supported, use db.login
> > > -----------------------------------------------------
> > > Building topology ...
> > > 259136 primitives registered
> > > Building areas:  100%
> > > 204742 areas built
> > > 194638 isles built
> > > Attaching islands:  100%
> > > Attaching centroids:  100%
> > > Topology was built.
> > > Number of nodes     :   226019
> > > Number of primitives:   259136
> > > Number of points    :   0
> > > Number of lines     :   0
> > > Number of boundaries:   259136
> > > Number of centroids :   0
> > > Number of areas     :   204742
> > > Number of isles     :   194638
> > > Number of incorrect boundaries   :   63856
> > > Number of areas without centroid :   204742
> > > -----------------------------------------------------
> > > WARNING: Cleaning polygons, result is not guaranteed!
> > > Building topology ...
> > > Topology was built.
> > > Number of nodes     :   226019
> > > Number of primitives:   259136
> > > Number of points    :   0
> > > Number of lines     :   0
> > > Number of boundaries:   259136
> > > Number of centroids :   0
> > > Number of areas     :   -
> > > Number of isles     :   -
> > > -----------------------------------------------------
> > > Snap boundaries (threshold = 1.000e+00):
> > > Registering points ... 20377235ERROR: G_realloc: out of memory
> > >
> > >
> >
>
>