[GRASS-dev] [GRASS GIS] #2045: r.to.vect: use less memory

[GRASS GIS]

#2045: r.to.vect: use less memory
--------------------------+-------------------------
  Reporter:  mlennert     |      Owner:  grass-dev@…
      Type:  enhancement  |     Status:  new
  Priority:  normal       |  Milestone:  7.0.5
 Component:  Raster       |    Version:  svn-trunk
Resolution:               |   Keywords:  r.to.vect
       CPU:  Unspecified  |   Platform:  Unspecified
--------------------------+-------------------------

Comment (by mlennert):

 Replying to [comment:15 mlennert]:
 > Replying to [comment:14 mmetz]:
 > > Replying to [comment:13 mlennert]:
 > > >
 > > > Without (i.e. grass70) my machine became unusable for a while and
 then the module execution stopped after a bit more than 5 minutes because
 of too little memory. With the fix (grass73) it went all the way to 100%
 of writing areas (17 minutes on my machine), but then I got an error
 "Category index is not up to date".
 > >
 > > The -v and -b flags are mutually exclusive. The -v flag is disabled
 for certain input/flag combinations, but this test was missing. Added in
 trunk r68720.
 >
 > Ah, ok. Why is topology necessary for category values ? Is it because
 areas are only defined via topological connection between centroids and
 boundaries ?
 >
 > This is a pity for us: if you want to use, for example, i.segment
 results both as raster and vector, you need to keep the link between the
 two via the segment ids. As attribute table handling '''really''' slows
 things down, -vt is the  preferred way to create a vector map with
 polygons that have the same category as the segment ids in the raster
 maps...
 >
 > > >
 > > > However, my colleague's problem was not in the vectorization stage,
 but in the topology building stage. I'm currently testing with -vt and
 VECTOR_LOW_MEM=1. It's been running for about 1 h. I'll see tomorrow what
 the result is...
 > >
 > > The topology building stage might need a lot of memory. If a lot of
 memory is still in use because of a memory leak in the extraction stage,
 the topology building stage is more likely to fail because of an out-of-
 memory error. There is no guarantee that topology building will succeed,
 but the chances for success are higher now that the (hopefully last)
 memory leak in the extraction stage has been fixed. If topology building
 still fails with an out-of-memory error, try again with the environment
 variable GRASS_VECTOR_LOWMEM set.
 >
 > I did use GRASS_VECTOR_LOWMEM=1, but the process was stopped. No
 explicit memory error, but AFAIK  such stopping of a process happens in
 case of memory errors:
 >
 >
 > {{{
 > Enregistrement des primitives ...
 > 63282475 primitives registered
 > 397657832 vertices registered
 > Construction des surfaces ...
 > Processus arrêté
 >
 > real    106m12.306s
 > user    28m2.664s
 > sys     68m42.260s
 > }}}
 >
 > Next try will be to use v.build on the vector created by r.to.vect -b...



 {{{
 > export GRASS_VECTOR_LOWMEM=1
 > time v.build seg_full
 Construction de la topologie pour la carte vectorielle
 <seg_full at rtovecttest>...
 Enregistrement des primitives ...
 63282475 primitives registered
 397657832 vertices registered
 Construction des surfaces ...
 Processus arrêté

 real    84m57.309s
 user    19m8.872s
 sys     54m37.868s
 }}}

 :-(

 I, therefore, do not have any means to vectorize this file directly in one
 piece on my computer. Memory usage goes up steadily through v.build.

 I'll try r.to.vect.tiled.

 I'll also try to run v.build through valgrind to see if anything shows up.

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2045#comment:16>
GRASS GIS <https://grass.osgeo.org>