[GRASS-user] i.segment.stats memory usage error
James Duffy
james.philip.duffy at gmail.com
Thu Oct 27 08:45:47 PDT 2016
On 27 October 2016 at 16:33, Moritz Lennert <mlennert at club.worldonline.be>
wrote:
> On 27/10/16 16:15, Moritz Lennert wrote:
>
>> On 27/10/16 15:38, James Duffy wrote:
>>
>>>
>>>
>>> On 27 October 2016 at 13:53, Moritz Lennert
>>> <mlennert at club.worldonline.be <mailto:mlennert at club.worldonline.be>>
>>> wrote:
>>>
>>> On 27/10/16 12:51, James Duffy wrote:
>>>
>>>
>>>
>>> On 27 October 2016 at 11:45, Moritz Lennert
>>> <mlennert at club.worldonline.be
>>> <mailto:mlennert at club.worldonline.be>
>>> <mailto:mlennert at club.worldonline.be
>>> <mailto:mlennert at club.worldonline.be>>> wrote:
>>>
>>>
>>>
>>> Le 27 octobre 2016 12:35:14 GMT+02:00, James Duffy
>>> <james.philip.duffy at gmail.com
>>> <mailto:james.philip.duffy at gmail.com>
>>> <mailto:james.philip.duffy at gmail.com
>>> <mailto:james.philip.duffy at gmail.com>>>
>>> a écrit :
>>> >On 27 October 2016 at 11:08, Moritz Lennert
>>> ><mlennert at club.worldonline.be
>>> <mailto:mlennert at club.worldonline.be>
>>> <mailto:mlennert at club.worldonline.be
>>> <mailto:mlennert at club.worldonline.be>>>
>>> >wrote:
>>> >
>>> >>
>>> >>
>>> >> Le 27 octobre 2016 11:22:02 GMT+02:00, James Duffy <
>>> >> james.philip.duffy at gmail.com
>>> <mailto:james.philip.duffy at gmail.com>
>>> <mailto:james.philip.duffy at gmail.com
>>> <mailto:james.philip.duffy at gmail.com>>> a écrit :
>>>
>>> >> >Hello,
>>> >> >
>>> >> >I'm trying to use i.segment.stats with GRASS 7.0.4 in
>>> the osgeolive
>>> >> >32bit
>>> >> >operating system.
>>> >> >
>>> >> >Prior to using this tool I have successfully created my
>>> segmented
>>> >map
>>> >> >using
>>> >> >i.segment.
>>> >> >
>>> >> >When I try to execute the following command:
>>> >> >
>>> >> >i.segment.stats --overwrite --verbose
>>> map=gp_seg_optimum at gp1 \
>>> >>
>>>
>>> >rasters=gp_ortho.1 at gp1,gp_ortho.2 at gp1,gp_ortho.3 at gp1,gp_ortho.4 at gp1
>>> >\
>>> >> >raster_statistics=min,max,mean,stddev,variance,sum \
>>> >>
>>> >csvfile=/home/jpd205/Wales_GRASS/GarronPill/gp_seg_stats \
>>> >> >separator=comma
>>> >> >
>>> >> >I get this error:
>>> >> >
>>> >> >Calculating geometry statistics
>>> >> >ERROR: G_malloc: unable to allocate 4273800320 bytes of
>>> memory at
>>> >> > /tmp/tmpgzUtnA/r.object.geometry/main.c:129
>>> >>
>>> >> This is coming from r.object.geometry.
>>> >>
>>> >> What are your region settings (g.region -p) ? How many
>>> segments do
>>> >you
>>> >> have ?
>>> >>
>>> >
>>> >GRASS 7.0.4 (GarronPill):~ > g.region -p
>>> >projection: 99 (OSGB 1936 / British National Grid)
>>> >zone: 0
>>> >datum: osgb36
>>> >ellipsoid: airy
>>> >north: 208007.00931776
>>> >south: 207952.59780698
>>> >west: 200993.90853302
>>> >east: 201097.28911076
>>> >nsres: 0.00430914
>>> >ewres: 0.00430914
>>> >rows: 12627
>>> >cols: 23991
>>> >cells: 302934357
>>>
>>> Are you sure 4mm is correct for the resolution ?
>>>
>>>
>>> Yes. It's high resolution imagery from a drone.
>>>
>>>
>>>
>>> What does r.info <http://r.info> <http://r.info> on
>>>
>>>
>>> >
>>> >
>>> >>
>>> >> Try running I.segment.stats without requesting form
>>> statistics.
>>> >
>>> >
>>> >Running:
>>> >
>>> >i.segment.stats --overwrite --verbose
>>> map=gp_seg_optimum at gp1 \
>>> >csvfile=/home/jpd205/Wales_GRASS/GarronPill/gp_seg_stats \
>>> >separator=comma
>>>
>>> Default is to do form statistics, so the above line also
>>> does. I
>>> have to admit that I don't know what happens when you give
>>> an empty
>>> parameter such as
>>>
>>> area_measures= or area_measures=""
>>>
>>> What does r.info <http://r.info> <http://r.info>
>>> gp_seg_optimum give you ?
>>>
>>>
>>>
>>>
>>> +-----------------------------------------------------------
>>> -----------------+
>>>
>>> | Map: gp_seg_optimum Date: Thu Oct 27
>>> 06:45:57
>>> 2016 |
>>> | Mapset: gp1 Login of Creator:
>>> jpd205 |
>>> | Location:
>>>
>>> GarronPill |
>>> | DataBase:
>>>
>>> /home/jpd205/Wales_GRASS |
>>> | Title: ( gp_seg_optimum
>>> ) |
>>> | Timestamp:
>>> none |
>>>
>>> |-----------------------------------------------------------
>>> -----------------|
>>>
>>> |
>>> |
>>> | Type of Map: raster Number of Categories:
>>> 0 |
>>> | Data Type:
>>> CELL |
>>> | Rows:
>>> 12627 |
>>> | Columns:
>>> 23991 |
>>> | Total Cells:
>>> 302934357 |
>>> | Projection: OSGB 1936 / British National
>>> Grid |
>>> | N: 208007.00931776 S: 207952.59780698 Res:
>>> 0.00430914 |
>>> | E: 201097.28911076 W: 200993.90853302 Res:
>>> 0.00430914 |
>>> | Range of data: min = 35 max =
>>> 133556294 |
>>> |
>>>
>>>
>>> Ok, I think I see at least part of the problem: in GRASS 7.0
>>> i.segment numbers segments non sequentially, i.e. whenever a new
>>> segment is created out of the merger of two existing segments, a new
>>> id is assigned. In GRASS trunk (the development version) i.segment
>>> is rewritten to, at the end, number all segments sequentially from 1
>>> to the number of segments.
>>>
>>> r.object.geometry allocates memory according to the new system. As
>>> it says in the code:
>>>
>>> /* REMARK: The following is only true if object ids are
>>> continuously numbered */
>>> n_objects = max - min + 1;
>>> obj_geos = G_malloc(n_objects * sizeof(struct obj_geo));
>>>
>>> So, one solution would be to use the development version of GRASS.
>>>
>>>
>>>
>>> Another would be to run r.clump on the result of the segmentation in
>>> order to renumber the segments before running it through
>>> i.segment.stats.
>>>
>>>
>>> This option seemed the most straightforward given my current setup. I
>>> ran r.clump with diagnoal enabled, and the output looked good. It only
>>> took 2mins to run.
>>>
>>> I then proceeded with the original i.segment.stats command and got the
>>> following:
>>>
>>> Calculating geometry statistics
>>> Calculating statistics for raster gp_ortho.1 at gp1
>>> ERROR: G_realloc: unable to allocate 572000 bytes of memory at
>>> raster/r.univar/r.univar_main.c:324
>>>
>>> It looks like the tool got further, but is still getting stuck on
>>> something...
>>>
>>
>> Yes, now there a memory issue in the part where it uses r.univar to
>> calculate statistics per segment on one of the spectral bands.
>>
>
> Could you run
>
> r.univar -et gp_ortho.1 zones=ClumpedSegmentMap output=testoutput.csv
>
> to if r.univar by itself handles this (which would mean I could try to
> change the data handling in i.segment.stats to make it more memory
> efficient), or not ?
>
> Thanks !
>
> Moritz
>
r.info on the clumped map gives:
+----------------------------------------------------------------------------+
| Map: gp_seg_optimum_clump Date: Thu Oct 27 14:28:30
2016 |
| Mapset: gp1 Login of Creator:
jpd205 |
| Location:
GarronPill |
| DataBase:
/home/jpd205/Wales_GRASS |
| Title: gp_seg_optimum_clump ( gp_seg_optimum_clump
) |
| Timestamp:
none |
|----------------------------------------------------------------------------|
|
|
| Type of Map: raster Number of Categories:
0 |
| Data Type:
CELL |
| Rows:
12627 |
| Columns:
23991 |
| Total Cells:
302934357 |
| Projection: OSGB 1936 / British National
Grid |
| N: 208007.00931776 S: 207952.59780698 Res:
0.00430914 |
| E: 201097.28911076 W: 200993.90853302 Res:
0.00430914 |
| Range of data: min = 1 max =
1264957 |
|
|
| Data
Source: |
| gp_seg_optimum at gp1
|
|
|
|
|
| Data
Description: |
| generated by
r.clump |
|
|
|
Comments: |
| r.clump --overwrite --verbose -d input="gp_seg_optimum at gp1"
output="\ |
| gp_seg_optimum_clump"
title="gp_seg_optimum_clump" |
|
|
+----------------------------------------------------------------------------+
Running:
r.univar -et gp_ortho.1 zones=gp_seg_optimum_clump output=testoutput2.csv
Returns:
Current region rows: 12627, cols: 23991
ERROR: G_realloc: unable to allocate 224000 bytes of memory at
raster/r.univar/r.univar_main.c:324
James
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20161027/0f634987/attachment-0001.html>
More information about the grass-user
mailing list