[GRASSLIST:8416] "Large" file problem. Can do clipping with arcview but not GRASS...

[Craig Aumann]

This is related to my earlier post 8400.  To get around the problem of
importing the files, I simply used v.external.  However, when I went to
do some overlays all of the overlays crashed as well:

On one file: 

Breaking lines ...
Intersections: 832406 (line 888364)File size limit exceeded


And on another file:

Cleaning boundaries at nodes ...
Modifications:  8082
-----------------------------------------------------------------------------
Breaking lines ...
Intersections:     1 (line 2955912)Unexpected error.
Aborted
GRASS 6.1.cvs (wavelets):~/DATA/ILM/scalingup/analysis/readin_setup >


What really upsets me is that I was able to do the overlays (clippings
actually) using ArcView's geoprocessing tool on these large files - each
one taking approximately 10-15 minutes on a Windows XP machine with only
512 MB of memory!

The Linux machine has Grass60, has 1.5 Gig of memory and 2 gig of swap
space.  I don't know the reasons here, but I'm having to eat crow for
not being able to handle these files with GRASS... 

Cheers!
Craig


On Mon, 2005-09-26 at 01:16, Radim Blazek wrote:
> It can be that it consumes a lot of memory. Probably you have
> to add swap.
> 
> Radim
> 
> On 9/23/05, Craig Aumann <caumann at ualberta.ca> wrote:
> > I was reading a large shape file:
> >
> > -rw-rw-r--    1 caumann  caumann  111771178 Apr 17 10:44 neforest.dbf
> > -rw-rw-r--    1 caumann  caumann   2355756 Apr  7  2003 neforest.sbn
> > -rw-rw-r--    1 caumann  caumann    119812 Apr  7  2003 neforest.sbx
> > -rw-rw-r--    1 caumann  caumann  746124196 Dec 10  2002 neforest.shp
> > -rw-rw-r--    1 caumann  caumann   1973924 Dec 10  2002 neforest.shx
> >
> > and got the error:
> >
> > ERROR: G_realloc: out of memory
> >
> > Can v.in.ogr not handle this size of file?
> >
> > Call and output are below
> >
> > Cheers!
> > Craig
> > -------------------------------------------------------
> >
> > v.in.ogr -o dsn="$SHPDIR" \
> >         output="$SHPFILE" layer="$SHPFILE" snap=1
> > Over-riding projection check.
> > Proceeding with import...
> > Layer: neforest
> > WARNING: 'user' in database definition is not supported, use db.login
> > -----------------------------------------------------
> > Building topology ...
> > 259136 primitives registered
> > Building areas:  100%
> > 204742 areas built
> > 194638 isles built
> > Attaching islands:  100%
> > Attaching centroids:  100%
> > Topology was built.
> > Number of nodes     :   226019
> > Number of primitives:   259136
> > Number of points    :   0
> > Number of lines     :   0
> > Number of boundaries:   259136
> > Number of centroids :   0
> > Number of areas     :   204742
> > Number of isles     :   194638
> > Number of incorrect boundaries   :   63856
> > Number of areas without centroid :   204742
> > -----------------------------------------------------
> > WARNING: Cleaning polygons, result is not guaranteed!
> > Building topology ...
> > Topology was built.
> > Number of nodes     :   226019
> > Number of primitives:   259136
> > Number of points    :   0
> > Number of lines     :   0
> > Number of boundaries:   259136
> > Number of centroids :   0
> > Number of areas     :   -
> > Number of isles     :   -
> > -----------------------------------------------------
> > Snap boundaries (threshold = 1.000e+00):
> > Registering points ... 20377235ERROR: G_realloc: out of memory
> >
> >
>