[GRASS-dev] any special reason why choice of raster compression method is done via environmental variable ?

[Markus Metz]

On Thu, Jan 16, 2020 at 10:56 AM Moritz Lennert <
mlennert at club.worldonline.be> wrote:
>
> Coming back to this to continue the discussion:
>
> On 15/12/19 21:41, Markus Metz wrote:
> >
> >
> > On Tue, Dec 10, 2019 at 6:47 AM Moritz Lennert
> > <mlennert at club.worldonline.be <mailto:mlennert at club.worldonline.be>>
wrote:
> >  >
> >  > On 9/12/19 22:50, Markus Metz wrote:
> >  > >   Hi Moritz,
> >  > >
> >  > > On Wed, Dec 4, 2019 at 10:28 AM Moritz Lennert
> >  > > <mlennert at club.worldonline.be <mailto:mlennert at club.worldonline.be>
> > <mailto:mlennert at club.worldonline.be
> > <mailto:mlennert at club.worldonline.be>>> wrote:
> >  > >  >
> >  > >  > Hi Markus,
> >  > >  >
> >  > >  > In recent days, I've been confronted several times with the
issue of
> >  > >  > people trying to share data among themselves, but using
different
> >  > >  > versions of GRASS, and so raster data compressed in a more
recent
> >  > >  > version of GRASS was not usable in an older version of GRASS.
> >  > >  >
> >  > >  > Now, I agree that generally the solution is to tell people to
> > use the
> >  > >  > latest and greatest, but this is not always possible / it is not
> >  > >  > necessarily highest on the list of priorities of people to see
> > how they
> >  > >  > can install the latest version of GRASS within their particular
> >  > > environment.
> >  > >  >
> >  > >  > Obviously, those with the latest version of GRASS can simple
> > recompress
> >  > >  > using ZLIB. However, compression method is defined as an
environment
> >  > >  > variable. This is somewhat daunting to many MS Windows users out
> > there.
> >  > >  > Is there any specific reason that lead to the choice of not
using a
> >  > >  > parameter to allow the choice of compression method (possibly to
> >  > >  > override a default that is still defined by an environment
> > variable) ?
> >  > >
> >  > > Such a parameter would need to be added to every module creating
raster
> >  > > output.
> >  >
> >  > My request is more linked to use cases where one would like to share
> >  > data (e.g. with r.pack) with other GRASS GIS users who do not
> >  > necessarily have access to the same compression method, not
necessarily
> >  > to changing the default compression method. I was just wondering
whether
> >  > it might be easily possible to just implement r.compress method= as a
> >  > quick way to recompress a specific map with a chosen method,
overriding
> >  > the default method. Currently, to do that, you have to change the
> >  > default method by changing the env variable, run r.compress, then
change
> >  > the variable back to the value one wishes generally to use as
default.
> >
> > recompressing an existing raster map seems like a waste of time. Why not
> > compress it in the first place with the appropriate method?
>
> Because what you consider the appropriate method at the time creation of
> the data (and still the appropriate method for your actual work) might
> not be the appropriate method for sharing with someone using a different
> system.

True, for sharing the highest (smallest size) compression method is usually
preferred, while for actual work the fastest compression method is usually
preferred.
> >
> > About the use cases:
> > People want to share a mapset: in this case all GRASS raster maps should
> > be compressed with the most compatible compression method (ZLIB).
> > Currently, this can be done with an env var. Alternatively, a new
> > mechanism via g.gisenv could be established.
>
> This would already make life easier for Windows users who (for the
> majority) are not used to env variables (and this is becoming more and
> more true of GNU/Linux users as well...).
>
> > Another option could be to set the default compression method per
> > mapset, to be stored in in the mapset's VAR file, to be set with e.g.
> > r.compress method=.
>
> That would probably be a nice solution. One could then create an
> "Export" mapset which can then be easily shared.

You could do the same by creating some "Export" mapset, then setting the
compression method and creating raster maps to be exported in this "Export"
mapset.

Implementation of a compression method set with g.gisenv is relatively easy.
Implementation of a compression method defined in a mapset's VAR file is a
bit more work and requires a mechanism to set the compression method in a
mapset's VAR file, e.g. r.compress method=.

But, if e.g. the aim is to reduce file size, the best compression method
depends on the datatype and size of the individual raster map to be
compressed. There is no "one size fits all" method.
>
> >  > Obviously, you can always just export as tiff and share that, but
that
> >  > just feels less elegant. Anyway, this is probably somewhat of a
luxury
> >  > problem :-)
> >
> > This only a luxury problem if you don't need to care about processing
> > speed and disk usage. The different compression methods by now available
> > in GRASS allow for optimization for speed (LZ4) or for optimization for
> > disk usage (BZIP2 for larger CELL maps). An easy (easier) way to set the
> > compression method seems reasonable.
>
> I didn't mean that compression is a luxury problem, but rather that
> since the option exists of just exporting the map as a tiff, readable by
> most systems (and you can chose a sharable tiff compression method at
> export time), all the above is probably a bit less of a necessity, and
> I'm just making noise ;-)

A sharable tiff compression method is not trivial, it depends on the
software and its capabilities used by the receiver.But this is a bit
off-topic.

Markus M
>
> Moritz
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20200116/61469aec/attachment.html>