[GRASS-user] i.segment.stats memory usage error

[Moritz Lennert]

On 03/11/16 10:58, Moritz Lennert wrote:
> On 02/11/16 17:15, Markus Metz wrote:
>> On Wed, Nov 2, 2016 at 3:47 PM, Moritz Lennert
>> <mlennert at club.worldonline.be> wrote:
>>> On 02/11/16 11:05, James Duffy wrote:
>>>>
>>>> I can't register with osgeo to submit a bug as no-one has replied with a
>>>> 'mantra' for me to do so... what a convoluted bug reporting system!
>>>>
>>>> And in your opinion Moritz, does it look like my workflow will not be
>>>> possible unless I find a 64bit machine with a decent amount of RAM?
>>>
>>>
>>> I'm not sure about the necessity of 64bit, but more RAM probably.
>>>
>>> IIUC, all the stats of all the zones are put into memory during the run of
>>> the module. Maybe a file-based store of the information might be possible
>>> which would avoid this memory usage, but I'm not sure.
>>>
>>> @MarkusM: could you have a look at this ? See
>>> https://lists.osgeo.org/pipermail/grass-user/2016-October/075493.html for
>>> the beginning of the thread. In short: James seems to be hitting memory
>>> limits of r.univar and I'm wondering out loud whether there is something
>>> that could be done about this...
>>
>> As Anna wrote, the -e flag increases memory consumption of r.univar
>> considerably. A file based approach for extended stats of r.univar is
>> difficult because the number of values per segment is initially
>> unknown to r.univar, thus the amount of disk space needed for extended
>> stats for each segment is unknown.
>>
>> I would rather use r.univar for standard stats and r.stats.quantile
>> for extended stats. The options for raster_statistics of
>> i.segment.stats would then reduce to "min, max, range, mean,
>> mean_of_abs, stddev, variance, coeff_var, sum, sum_abs", and a new
>> option, e.g. raster_percentiles could be added, accepting a list of
>> percentiles which can be directly fed by i.segment.stats to
>> r.stats.quantile percentiles=<list of percentiles>.
>
> Ok, I'll look into this. The only disadvantage is that r.stats.quantile
> creates a new raster map for each percentile, which then have to be
> treated to extract the statistics, instead of allowing to print the
> results directly to a file. But I guess the overhead of that is worth
> the fact that it reduces memory usage.

Forget that, I just saw the -p option. Need new glasses, I guess ;-)

Moritz