[GRASS-user] i.segment.stats memory usage error
Moritz Lennert
mlennert at club.worldonline.be
Thu Oct 27 08:33:58 PDT 2016
On 27/10/16 16:15, Moritz Lennert wrote:
> On 27/10/16 15:38, James Duffy wrote:
>>
>>
>> On 27 October 2016 at 13:53, Moritz Lennert
>> <mlennert at club.worldonline.be <mailto:mlennert at club.worldonline.be>>
>> wrote:
>>
>> On 27/10/16 12:51, James Duffy wrote:
>>
>>
>>
>> On 27 October 2016 at 11:45, Moritz Lennert
>> <mlennert at club.worldonline.be
>> <mailto:mlennert at club.worldonline.be>
>> <mailto:mlennert at club.worldonline.be
>> <mailto:mlennert at club.worldonline.be>>> wrote:
>>
>>
>>
>> Le 27 octobre 2016 12:35:14 GMT+02:00, James Duffy
>> <james.philip.duffy at gmail.com
>> <mailto:james.philip.duffy at gmail.com>
>> <mailto:james.philip.duffy at gmail.com
>> <mailto:james.philip.duffy at gmail.com>>>
>> a écrit :
>> >On 27 October 2016 at 11:08, Moritz Lennert
>> ><mlennert at club.worldonline.be
>> <mailto:mlennert at club.worldonline.be>
>> <mailto:mlennert at club.worldonline.be
>> <mailto:mlennert at club.worldonline.be>>>
>> >wrote:
>> >
>> >>
>> >>
>> >> Le 27 octobre 2016 11:22:02 GMT+02:00, James Duffy <
>> >> james.philip.duffy at gmail.com
>> <mailto:james.philip.duffy at gmail.com>
>> <mailto:james.philip.duffy at gmail.com
>> <mailto:james.philip.duffy at gmail.com>>> a écrit :
>>
>> >> >Hello,
>> >> >
>> >> >I'm trying to use i.segment.stats with GRASS 7.0.4 in
>> the osgeolive
>> >> >32bit
>> >> >operating system.
>> >> >
>> >> >Prior to using this tool I have successfully created my
>> segmented
>> >map
>> >> >using
>> >> >i.segment.
>> >> >
>> >> >When I try to execute the following command:
>> >> >
>> >> >i.segment.stats --overwrite --verbose
>> map=gp_seg_optimum at gp1 \
>> >>
>>
>> >rasters=gp_ortho.1 at gp1,gp_ortho.2 at gp1,gp_ortho.3 at gp1,gp_ortho.4 at gp1
>> >\
>> >> >raster_statistics=min,max,mean,stddev,variance,sum \
>> >>
>> >csvfile=/home/jpd205/Wales_GRASS/GarronPill/gp_seg_stats \
>> >> >separator=comma
>> >> >
>> >> >I get this error:
>> >> >
>> >> >Calculating geometry statistics
>> >> >ERROR: G_malloc: unable to allocate 4273800320 bytes of
>> memory at
>> >> > /tmp/tmpgzUtnA/r.object.geometry/main.c:129
>> >>
>> >> This is coming from r.object.geometry.
>> >>
>> >> What are your region settings (g.region -p) ? How many
>> segments do
>> >you
>> >> have ?
>> >>
>> >
>> >GRASS 7.0.4 (GarronPill):~ > g.region -p
>> >projection: 99 (OSGB 1936 / British National Grid)
>> >zone: 0
>> >datum: osgb36
>> >ellipsoid: airy
>> >north: 208007.00931776
>> >south: 207952.59780698
>> >west: 200993.90853302
>> >east: 201097.28911076
>> >nsres: 0.00430914
>> >ewres: 0.00430914
>> >rows: 12627
>> >cols: 23991
>> >cells: 302934357
>>
>> Are you sure 4mm is correct for the resolution ?
>>
>>
>> Yes. It's high resolution imagery from a drone.
>>
>>
>>
>> What does r.info <http://r.info> <http://r.info> on
>>
>>
>> >
>> >
>> >>
>> >> Try running I.segment.stats without requesting form
>> statistics.
>> >
>> >
>> >Running:
>> >
>> >i.segment.stats --overwrite --verbose
>> map=gp_seg_optimum at gp1 \
>> >csvfile=/home/jpd205/Wales_GRASS/GarronPill/gp_seg_stats \
>> >separator=comma
>>
>> Default is to do form statistics, so the above line also
>> does. I
>> have to admit that I don't know what happens when you give
>> an empty
>> parameter such as
>>
>> area_measures= or area_measures=""
>>
>> What does r.info <http://r.info> <http://r.info>
>> gp_seg_optimum give you ?
>>
>>
>>
>>
>> +----------------------------------------------------------------------------+
>>
>> | Map: gp_seg_optimum Date: Thu Oct 27
>> 06:45:57
>> 2016 |
>> | Mapset: gp1 Login of Creator:
>> jpd205 |
>> | Location:
>>
>> GarronPill |
>> | DataBase:
>>
>> /home/jpd205/Wales_GRASS |
>> | Title: ( gp_seg_optimum
>> ) |
>> | Timestamp:
>> none |
>>
>> |----------------------------------------------------------------------------|
>>
>> |
>> |
>> | Type of Map: raster Number of Categories:
>> 0 |
>> | Data Type:
>> CELL |
>> | Rows:
>> 12627 |
>> | Columns:
>> 23991 |
>> | Total Cells:
>> 302934357 |
>> | Projection: OSGB 1936 / British National
>> Grid |
>> | N: 208007.00931776 S: 207952.59780698 Res:
>> 0.00430914 |
>> | E: 201097.28911076 W: 200993.90853302 Res:
>> 0.00430914 |
>> | Range of data: min = 35 max =
>> 133556294 |
>> |
>>
>>
>> Ok, I think I see at least part of the problem: in GRASS 7.0
>> i.segment numbers segments non sequentially, i.e. whenever a new
>> segment is created out of the merger of two existing segments, a new
>> id is assigned. In GRASS trunk (the development version) i.segment
>> is rewritten to, at the end, number all segments sequentially from 1
>> to the number of segments.
>>
>> r.object.geometry allocates memory according to the new system. As
>> it says in the code:
>>
>> /* REMARK: The following is only true if object ids are
>> continuously numbered */
>> n_objects = max - min + 1;
>> obj_geos = G_malloc(n_objects * sizeof(struct obj_geo));
>>
>> So, one solution would be to use the development version of GRASS.
>>
>>
>>
>> Another would be to run r.clump on the result of the segmentation in
>> order to renumber the segments before running it through
>> i.segment.stats.
>>
>>
>> This option seemed the most straightforward given my current setup. I
>> ran r.clump with diagnoal enabled, and the output looked good. It only
>> took 2mins to run.
>>
>> I then proceeded with the original i.segment.stats command and got the
>> following:
>>
>> Calculating geometry statistics
>> Calculating statistics for raster gp_ortho.1 at gp1
>> ERROR: G_realloc: unable to allocate 572000 bytes of memory at
>> raster/r.univar/r.univar_main.c:324
>>
>> It looks like the tool got further, but is still getting stuck on
>> something...
>
> Yes, now there a memory issue in the part where it uses r.univar to
> calculate statistics per segment on one of the spectral bands.
Could you run
r.univar -et gp_ortho.1 zones=ClumpedSegmentMap output=testoutput.csv
to if r.univar by itself handles this (which would mean I could try to
change the data handling in i.segment.stats to make it more memory
efficient), or not ?
Thanks !
Moritz
More information about the grass-user
mailing list