[GRASS-dev] i.colors.enhance: G_calloc() error in r.quantile with large maps

[Markus Metz]

On Sun, Apr 30, 2017 at 12:51 PM, Markus Neteler <neteler at osgeo.org> wrote:
>
> On Sun, Apr 30, 2017 at 11:47 AM, Markus Neteler <neteler at osgeo.org>
wrote:
> > ....
> > .On Sat, Apr 29, 2017 at 11:16 PM, Markus Neteler <neteler at osgeo.org>
wrote:
> >> On Fri, Apr 28, 2017 at 5:05 PM, Markus Metz
> > ...
> >> rows:       88265
> >> cols:       169411
> >> cells:      14953061915
> > ...
> >> I have locally backported it, works too (I'd suggest to backport).
> >
> ..
> > ops. It seems that the memory footprint becomes big at some point in
> > r.quantile. Could there be a leak somewhere? Or is it due to the 14,95
> > gigapixels?
> >
> > I'll add another swap file and try again.
>
> This helped!
>
> Here the memory situation during the "Sorting bins" step which appears
> to be most RAM intensive:
>
> top - 12:22:16 up 55 days, 18 min,  5 users,  load average: 3.55, 3.97,
4.09
> Tasks: 678 total,   2 running, 662 sleeping,   0 stopped,  14 zombie
> %Cpu(s): 14.1 us,  1.8 sy,  0.0 ni, 54.9 id, 28.2 wa,  0.5 hi,  0.5 si,
 0.0 st
> KiB Mem : 32772892 total,   279812 free, 32281888 used,   211192
buff/cache
> KiB Swap: 39845880 total, 16045316 free, 23800564 used.    64196 avail Mem
>
>   PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+
COMMAND
> 27472 mundial+  20   0 16.382g 0.013t    956 R  99.7 41.8  16:57.87
r.quantile
> 27474 mundial+  20   0 17.026g 8.091g    156 R  99.7 25.9  15:43.12
r.quantile
> 27473 mundial+  20   0 16.322g 9.260g    152 D   5.0 29.6  15:40.62
> r.quantile  <<--- !
>

16+GB of RAM still seems too much to me, unless the histograms of the cell
values are highly skewed. What is the output of r.stats -c for B04_255,
B11_255, B8A_255?

Markus M
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20170430/b9ade37c/attachment.html>