[GRASS-user] i.segment.stats memory usage error

[Markus Metz]

On Wed, Nov 2, 2016 at 5:15 PM, Markus Metz
<markus.metz.giswork at gmail.com> wrote:
> On Wed, Nov 2, 2016 at 3:47 PM, Moritz Lennert
> <mlennert at club.worldonline.be> wrote:
>> On 02/11/16 11:05, James Duffy wrote:
>>>
>>> I can't register with osgeo to submit a bug as no-one has replied with a
>>> 'mantra' for me to do so... what a convoluted bug reporting system!
>>>
>>> And in your opinion Moritz, does it look like my workflow will not be
>>> possible unless I find a 64bit machine with a decent amount of RAM?
>>
>>
>> I'm not sure about the necessity of 64bit, but more RAM probably.
>>
>> IIUC, all the stats of all the zones are put into memory during the run of
>> the module. Maybe a file-based store of the information might be possible
>> which would avoid this memory usage, but I'm not sure.
>>
>> @MarkusM: could you have a look at this ? See
>> https://lists.osgeo.org/pipermail/grass-user/2016-October/075493.html for
>> the beginning of the thread. In short: James seems to be hitting memory
>> limits of r.univar and I'm wondering out loud whether there is something
>> that could be done about this...
>
> As Anna wrote, the -e flag increases memory consumption of r.univar
> considerably. A file based approach for extended stats of r.univar is
> difficult because the number of values per segment is initially
> unknown to r.univar, thus the amount of disk space needed for extended
> stats for each segment is unknown.
>
> I would rather use r.univar for standard stats and r.stats.quantile
> for extended stats.

There were some bugs in r.stats.quantile, fixed in r69770-2 in all G7 branches.

About the limit to MAX_CATS categories in the base map (ticket #3198),
there is not really a reason for it. I guess it is meant to protect
against out-of-memory errors.

Markus M

> The options for raster_statistics of
> i.segment.stats would then reduce to "min, max, range, mean,
> mean_of_abs, stddev, variance, coeff_var, sum, sum_abs", and a new
> option, e.g. raster_percentiles could be added, accepting a list of
> percentiles which can be directly fed by i.segment.stats to
> r.stats.quantile percentiles=<list of percentiles>.
>
> Markus M