[GRASS-dev] [GRASS GIS] #494: break in importing and cleaning very large vector datasets

[GRASS GIS]

#494: break in importing and cleaning very large vector datasets
---------------------------------------+------------------------------------
 Reporter:  gisboa                     |       Owner:  grass-dev at lists.osgeo.org
     Type:  defect                     |      Status:  new                      
 Priority:  major                      |   Milestone:  6.4.0                    
Component:  default                    |     Version:  6.4.0 RCs                
 Keywords:  vector import clean build  |    Platform:  All                      
      Cpu:  All                        |  
---------------------------------------+------------------------------------
 I try to use Grass with large vector files, with sizes up to 0,5GB, this
 usually results in errors during import, or if not during import, then at
 least while cleaning them. Both on Mac (Leopard intel) and Linux (openSuse
 11 (32 and 64bit). It doesn't seem to be a problem with the data, as I can
 process all data when splitting the import, and handle the various slices
 separately.

 Files are too large to attach...

 Wouter

 Mac OSX
 ======
 GRASS 6.4.0RC3 (nl-rdn):/data/grassdb/nl-rdn > v.clean --o
 input=top_top10vec_gebouwen output=top_top10vec_geb2 tool=bpol
 --------------------------------------------------
 Tool: Threshold
 Break polygons: 0.000000e+00
 --------------------------------------------------
 Copying vector lines...
 Rebuilding parts of topology...
 Building topology for vector map <top_top10vec_geb2>...
 Registering primitives...
 6041168 primitives registered
 25000849 vertices registered
 Number of nodes: 6037532
 Number of primitives: 6041168
 Number of points: 0
 Number of lines: 0
 Number of boundaries: 3021055
 Number of centroids: 3020113
 Number of areas: -
 Number of isles: -
 --------------------------------------------------
 Tool: Break polygons
 v.clean(22861) malloc: *** mmap(size=275042304) failed (error code=12)
 *** error: can't allocate region
 *** set a breakpoint in malloc_error_break to debug
 ERROR: G_realloc: unable to allocate 275040036 bytes at
        break_polygons.c:188

 Linux 64bit
 =======
 Rebuilding parts of topology...
 Building topology for vector map <top_top10vec_geb2>...
 Registering primitives...
 6041168 primitives registered
 25000849 vertices registered
 Number of nodes: 6037532
 Number of primitives: 6041168
 Number of points: 0
 Number of lines: 0
 Number of boundaries: 3021055
 Number of centroids: 3020113
 Number of areas: -
 Number of isles: -
 --------------------------------------------------
 Tool: Break polygons
 ERROR: G_realloc: unable to allocate 34400040 bytes at
 break_polygons.c:188

-- 
Ticket URL: <http://trac.osgeo.org/grass/ticket/494>
GRASS GIS <http://grass.osgeo.org>