[GRASS-user] i.segment.stats memory usage error

[Moritz Lennert]

On 27/10/16 12:51, James Duffy wrote:
>
>
> On 27 October 2016 at 11:45, Moritz Lennert
> <mlennert at club.worldonline.be <mailto:mlennert at club.worldonline.be>> wrote:
>
>
>
>     Le 27 octobre 2016 12:35:14 GMT+02:00, James Duffy
>     <james.philip.duffy at gmail.com <mailto:james.philip.duffy at gmail.com>>
>     a écrit :
>     >On 27 October 2016 at 11:08, Moritz Lennert
>     ><mlennert at club.worldonline.be <mailto:mlennert at club.worldonline.be>>
>     >wrote:
>     >
>     >>
>     >>
>     >> Le 27 octobre 2016 11:22:02 GMT+02:00, James Duffy <
>     >> james.philip.duffy at gmail.com
>     <mailto:james.philip.duffy at gmail.com>> a écrit :
>     >> >Hello,
>     >> >
>     >> >I'm trying to use i.segment.stats with GRASS 7.0.4 in the osgeolive
>     >> >32bit
>     >> >operating system.
>     >> >
>     >> >Prior to using this tool I have successfully created my segmented
>     >map
>     >> >using
>     >> >i.segment.
>     >> >
>     >> >When I try to execute the following command:
>     >> >
>     >> >i.segment.stats --overwrite --verbose map=gp_seg_optimum at gp1 \
>     >> >rasters=gp_ortho.1 at gp1,gp_ortho.2 at gp1,gp_ortho.3 at gp1,gp_ortho.4 at gp1
>     >\
>     >> >raster_statistics=min,max,mean,stddev,variance,sum \
>     >> >csvfile=/home/jpd205/Wales_GRASS/GarronPill/gp_seg_stats \
>     >> >separator=comma
>     >> >
>     >> >I get this error:
>     >> >
>     >> >Calculating geometry statistics
>     >> >ERROR: G_malloc: unable to allocate 4273800320 bytes of memory at
>     >> >       /tmp/tmpgzUtnA/r.object.geometry/main.c:129
>     >>
>     >> This is coming from r.object.geometry.
>     >>
>     >> What are your region settings (g.region -p) ? How many segments do
>     >you
>     >> have ?
>     >>
>     >
>     >GRASS 7.0.4 (GarronPill):~ > g.region -p
>     >projection: 99 (OSGB 1936 / British National Grid)
>     >zone:       0
>     >datum:      osgb36
>     >ellipsoid:  airy
>     >north:      208007.00931776
>     >south:      207952.59780698
>     >west:       200993.90853302
>     >east:       201097.28911076
>     >nsres:      0.00430914
>     >ewres:      0.00430914
>     >rows:       12627
>     >cols:       23991
>     >cells:      302934357
>
>     Are you sure 4mm is correct for the resolution ?
>
>
> Yes. It's high resolution imagery from a drone.
>
>
>
>     What does r.info <http://r.info> on
>
>
>     >
>     >
>     >>
>     >> Try running I.segment.stats without requesting form statistics.
>     >
>     >
>     >Running:
>     >
>     >i.segment.stats --overwrite --verbose map=gp_seg_optimum at gp1 \
>     >csvfile=/home/jpd205/Wales_GRASS/GarronPill/gp_seg_stats \
>     >separator=comma
>
>     Default is to do form statistics, so the above line also does. I
>     have to admit that I don't know what happens when you give an empty
>     parameter such as
>
>     area_measures= or area_measures=""
>
>     What does r.info <http://r.info> gp_seg_optimum give you ?
>
>
> +----------------------------------------------------------------------------+
>  | Map:      gp_seg_optimum                 Date: Thu Oct 27 06:45:57
> 2016    |
>  | Mapset:   gp1                            Login of Creator:
> jpd205          |
>  | Location:
> GarronPill                                                       |
>  | DataBase:
> /home/jpd205/Wales_GRASS                                         |
>  | Title:     ( gp_seg_optimum
> )                                              |
>  | Timestamp:
> none                                                            |
>  |----------------------------------------------------------------------------|
>  |
> |
>  |   Type of Map:  raster               Number of Categories:
> 0               |
>  |   Data Type:
> CELL                                                       |
>  |   Rows:
> 12627                                                      |
>  |   Columns:
> 23991                                                      |
>  |   Total Cells:
> 302934357                                                  |
>  |        Projection: OSGB 1936 / British National
> Grid                       |
>  |            N: 208007.00931776    S: 207952.59780698   Res:
> 0.00430914      |
>  |            E: 201097.28911076    W: 200993.90853302   Res:
> 0.00430914      |
>  |   Range of data:    min = 35  max =
> 133556294                              |
>  |

Ok, I think I see at least part of the problem: in GRASS 7.0 i.segment 
numbers segments non sequentially, i.e. whenever a new segment is 
created out of the merger of two existing segments, a new id is 
assigned. In GRASS trunk (the development version) i.segment is 
rewritten to, at the end, number all segments sequentially from 1 to the 
number of segments.

r.object.geometry allocates memory according to the new system. As it 
says in the code:

     /* REMARK: The following is only true if object ids are 
continuously numbered */
     n_objects = max - min + 1;
     obj_geos = G_malloc(n_objects * sizeof(struct obj_geo));

So, one solution would be to use the development version of GRASS. 
Another would be to run r.clump on the result of the segmentation in 
order to renumber the segments before running it through i.segment.stats.

A third option, potentially a bit faster than the second, would be to 
write a little script which gets all category values from the segment 
map (using r.category), and reassigns sequential category values to 
those using r.reclass. I.e. something like this:

r.category segment_map | awk 'BEGIN{cat=1} {printf"%d=%d\n", $1, cat; 
cat++}' | r.reclass segment_map out=sequential_segment_map rules=-

Moritz