Thank you!<br> r.series is working fine. I am now trying to solve the problem of the limit for the number of files within my linux distribution. <br>It is rather fast, much faster then using R. However my rasters are rather small (15000 cells), then I do not know how it would behave with a much larger file.<br>
<br>Thank you again<br><br>Laura<br><br><div class="gmail_quote">2009/9/17 Glynn Clements <span dir="ltr"><<a href="mailto:glynn@gclements.plus.com">glynn@gclements.plus.com</a>></span><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div class="im"><br>
Laura Poggio wrote:<br>
<br>
> I would need to calculate the resulting percentile map from a large number<br>
> of maps (>10000), obtaining the values for 5 and 95 percentile for each<br>
> pixel. Is there any function in GRASS?<br>
<br>
</div>Recent versions of r.series can calculate arbitrary quantiles.<br>
<br>
You may have to make some configuration changes in order to allow a<br>
single process to have >10000 open file descriptors (for Linux, check<br>
/etc/security/limits.conf).<br>
<br>
Memory shouldn't be an issue, as r.series only needs to hold one row<br>
from each map at a time.<br>
<br>
This won't be particularly fast, as r.series method=quantile sorts all<br>
of the values then picks out the requested quantile. And if you<br>
calculate multiple quantiles, it will sort the values once for each<br>
quantile.<br>
<br>
CC to grass-dev:<br>
<br>
The duplicate sorting can be gotten around in a several ways:<br>
<br>
1. Have the sorting routine first check for an already sorted list.<br>
<br>
2. Add a global variable to lib/stats to indicate that the data is<br>
already sorted.<br>
<br>
3. Add yet another parameter to the stats functions to indicate that<br>
the data is sorted.<br>
<br>
4. Use the closure parameter to indicate that the data is sorted (for<br>
c_quant(), this would mean that it would need to point to a structure<br>
containing both the quantile and the sorted flag).<br>
<br>
#1 is inefficient, #2 is a bit of a hack, #3 adds another parameter<br>
which most functions won't use (only diversity, median, mode,<br>
quantiles need sorted data), #4 is awkward to use.<br>
<br>
The inefficient quantile algorithm can be replaced (or augmented) with<br>
the more efficient (and more complex) one used by r.quantile and<br>
r.statistics3. But is it worth the effort and complexity?<br>
<br>
Essentially, it only matters if the number of maps is large enough<br>
that the qsort() dominates. For a few maps, the setup overhead of the<br>
more complex algorithm will make it slower. For an intermediate<br>
number, the speed gain may be small compared to other overheads. It's<br>
worth it for calculating quantiles over millions of values (although<br>
the primary motivation was to avoid having to store all values in<br>
memory), but I don't know if it's worth it for 10,000 values.<br>
<font color="#888888"><br>
--<br>
Glynn Clements <<a href="mailto:glynn@gclements.plus.com">glynn@gclements.plus.com</a>><br>
</font></blockquote></div><br>