[GRASS-user] i.segment.stats memory usage error

[James Duffy]

Moritz, have you seen the reply on the bug rep?
https://trac.osgeo.org/grass/ticket/3197

Do you think the alternative r.univar2 could be used in your function?

On 3 November 2016 at 10:01, Moritz Lennert <mlennert at club.worldonline.be>
wrote:

> On 03/11/16 10:58, Moritz Lennert wrote:
>
>> On 02/11/16 17:15, Markus Metz wrote:
>>
>>> On Wed, Nov 2, 2016 at 3:47 PM, Moritz Lennert
>>> <mlennert at club.worldonline.be> wrote:
>>>
>>>> On 02/11/16 11:05, James Duffy wrote:
>>>>
>>>>>
>>>>> I can't register with osgeo to submit a bug as no-one has replied with
>>>>> a
>>>>> 'mantra' for me to do so... what a convoluted bug reporting system!
>>>>>
>>>>> And in your opinion Moritz, does it look like my workflow will not be
>>>>> possible unless I find a 64bit machine with a decent amount of RAM?
>>>>>
>>>>
>>>>
>>>> I'm not sure about the necessity of 64bit, but more RAM probably.
>>>>
>>>> IIUC, all the stats of all the zones are put into memory during the run
>>>> of
>>>> the module. Maybe a file-based store of the information might be
>>>> possible
>>>> which would avoid this memory usage, but I'm not sure.
>>>>
>>>> @MarkusM: could you have a look at this ? See
>>>> https://lists.osgeo.org/pipermail/grass-user/2016-October/075493.html
>>>> for
>>>> the beginning of the thread. In short: James seems to be hitting memory
>>>> limits of r.univar and I'm wondering out loud whether there is something
>>>> that could be done about this...
>>>>
>>>
>>> As Anna wrote, the -e flag increases memory consumption of r.univar
>>> considerably. A file based approach for extended stats of r.univar is
>>> difficult because the number of values per segment is initially
>>> unknown to r.univar, thus the amount of disk space needed for extended
>>> stats for each segment is unknown.
>>>
>>> I would rather use r.univar for standard stats and r.stats.quantile
>>> for extended stats. The options for raster_statistics of
>>> i.segment.stats would then reduce to "min, max, range, mean,
>>> mean_of_abs, stddev, variance, coeff_var, sum, sum_abs", and a new
>>> option, e.g. raster_percentiles could be added, accepting a list of
>>> percentiles which can be directly fed by i.segment.stats to
>>> r.stats.quantile percentiles=<list of percentiles>.
>>>
>>
>> Ok, I'll look into this. The only disadvantage is that r.stats.quantile
>> creates a new raster map for each percentile, which then have to be
>> treated to extract the statistics, instead of allowing to print the
>> results directly to a file. But I guess the overhead of that is worth
>> the fact that it reduces memory usage.
>>
>
> Forget that, I just saw the -p option. Need new glasses, I guess ;-)
>
> Moritz
>



-- 
*James Duffy*
PhD Researcher
Environment and Sustainability Institute
Penryn Campus
University of Exeter
Penryn
Cornwall
TR10 9FE
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20161104/4ed25304/attachment.html>