[GRASS-dev] r.series with file option - too many files open

[Veronica Andreo]

Perfect! I'll sync them, and also r.series.accumulate :)

cheers,
Vero

2018-01-15 22:00 GMT+01:00 Markus Metz <markus.metz.giswork at gmail.com>:

>
>
> On Mon, Jan 15, 2018 at 8:41 PM, Veronica Andreo <veroandreo at gmail.com>
> wrote:
> >
> > Hi all,
> >
> > 2018-01-15 20:11 GMT+01:00 Markus Neteler <neteler at osgeo.org>:
> >>
> >> On Mon, Jan 15, 2018 at 5:27 PM, Paulo van Breugel
> >> <p.vanbreugel at gmail.com> wrote:
> >> > On 1/15/18 4:39 PM, Markus Metz wrote:
> >> ...
> >> >> the manual is wrong:
> >> >> Use the file option to analyze large amount of raster maps without
> hitting
> >> >> open files limit and the size limit of command line arguments.
> >> >>
> >> >> must be
> >> >> Use the -z flag to analyze large amount of raster maps without
> hitting open
> >> >> files limit and the file option to avoid hitting the size limit of
> command
> >> >> line arguments.
> >> >
> >> > Ah, yes, I totally forgot about that flag. Thanks (also Anna and
> Veronica).
> >> > Perhaps somebody with write permission can correct his in the manual
> page?
> >>
> >> Done in r72081: please verify. I think it could still be better.
> >> https://trac.osgeo.org/grass/changeset/72081
> >
> >
> > AFAIU, the file option does not necessarily make the process slower (it
> just replaces a list of comma separated maps in a command by a file), it is
> the -z flag that does that, because it will open and close files once and
> again, instead of keeping them open (Am I right, @markusM?)
> >
> > Moreover, I checked r.hants and r.series.lwr, which are based on
> r.series and have the same file option and -z flag. They all have different
> wording. Maybe we could agree here on the 3 of them. From my understanding,
> they are not entirely correct. Here the text snippets:
> >
> > r.hants:
> > "Use the file option to analyze large amount of raster maps without
> hitting open files limit and the size limit of command line arguments. The
> computation is slower than with the input option method. For every single
> row in the output map(s) all input maps are opened and closed. The amount
> of RAM will rise linearly with the number of specified input maps. The
> input and file options are mutually exclusive. The option input is a comma
> separated list of raster map names and the option file is a text file with
> a new line separated list of raster map names. Note that the order of maps
> in one option or the other is very important."
> >
> > r.series.lwr:
> > "Use the -z flag to analyze large amounts of raster maps without hitting
> the open files limit and the size limit of command line arguments. This
> will however increase the processing time. For every single row in the
> output map(s) all input maps are opened and closed. The amount of RAM used
> will rise linearly with the number of specified input maps.The input and
> file options are mutually exclusive. Input is a text file with a new line
> separated list of raster map names"
> >
> > I can change them accordingly once we understand correctly the effect of
> both file option and -z flag.
>
> The manual of r.series is IMHO now correct. The file option avoids hitting
> the size limit of command line arguments, and the -z flag avoids hitting
> the limit of the number of open files.
>
> In all three manuals, the sections starting with "The maximum number of
> raster maps" (in r.series "Number of raster maps to be processed") should
> be identical. Ideally, the manual of r.hants or r.series.lwr would be used
> as template, adding the latest change of r.series with regard to the file
> option and the -z flag, then sync all manuals.
>
> > Which other modules share this -z flag?
>
> r.series.accumulate
>
> t.rast.series and t.rast.aggregate do not have the -z flag, but should
> probably have it because they call r.series.
>
> Markus M
>
> >
> > best,
> > Vero
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20180115/648972a1/attachment.html>