[GRASS-user] i.segment.stats memory usage error

[Moritz Lennert]

On 02/11/16 17:15, Markus Metz wrote:
> On Wed, Nov 2, 2016 at 3:47 PM, Moritz Lennert
> <mlennert at club.worldonline.be> wrote:
>> On 02/11/16 11:05, James Duffy wrote:
>>>
>>> I can't register with osgeo to submit a bug as no-one has replied with a
>>> 'mantra' for me to do so... what a convoluted bug reporting system!
>>>
>>> And in your opinion Moritz, does it look like my workflow will not be
>>> possible unless I find a 64bit machine with a decent amount of RAM?
>>
>>
>> I'm not sure about the necessity of 64bit, but more RAM probably.
>>
>> IIUC, all the stats of all the zones are put into memory during the run of
>> the module. Maybe a file-based store of the information might be possible
>> which would avoid this memory usage, but I'm not sure.
>>
>> @MarkusM: could you have a look at this ? See
>> https://lists.osgeo.org/pipermail/grass-user/2016-October/075493.html for
>> the beginning of the thread. In short: James seems to be hitting memory
>> limits of r.univar and I'm wondering out loud whether there is something
>> that could be done about this...
>
> As Anna wrote, the -e flag increases memory consumption of r.univar
> considerably. A file based approach for extended stats of r.univar is
> difficult because the number of values per segment is initially
> unknown to r.univar, thus the amount of disk space needed for extended
> stats for each segment is unknown.
>
> I would rather use r.univar for standard stats and r.stats.quantile
> for extended stats. The options for raster_statistics of
> i.segment.stats would then reduce to "min, max, range, mean,
> mean_of_abs, stddev, variance, coeff_var, sum, sum_abs", and a new
> option, e.g. raster_percentiles could be added, accepting a list of
> percentiles which can be directly fed by i.segment.stats to
> r.stats.quantile percentiles=<list of percentiles>.

Ok, I'll look into this. The only disadvantage is that r.stats.quantile 
creates a new raster map for each percentile, which then have to be 
treated to extract the statistics, instead of allowing to print the 
results directly to a file. But I guess the overhead of that is worth 
the fact that it reduces memory usage.

Moritz