[GRASS-dev] [GRASS GIS] #2750: LZ4 when writing raster rows; better than double I/O bound r.mapcalc speed

[Paulo van Breugel]

On 28-12-15 17:44, GRASS GIS wrote:
> #2750: LZ4 when writing raster rows; better than double I/O bound r.mapcalc speed
> --------------------------+---------------------------
>    Reporter:  sprice       |      Owner:  grass-dev@…
>        Type:  enhancement  |     Status:  new
>    Priority:  normal       |  Milestone:  7.1.0
>   Component:  Raster       |    Version:  svn-trunk
> Resolution:               |   Keywords:  ZLIB LZ4 ZSTD
>         CPU:  OSX/Intel    |   Platform:  MacOSX
> --------------------------+---------------------------

I was assuming this is for all platforms, but see above MacOSX being 
ticked as platform?
>
> Comment (by neteler):
>
>   '''To summarize the current state'''
>
>   We currently have the following compression methods in trunk (according to
>   lib/gis/compress.c; set by `export GRASS_COMPRESSOR=XXX`):
>
>    * `NONE` (uncompressed)
>    * `RLE`  (generic Run-Length Encoding of single bytes)
>    * `ZLIB` (DEFLATE, good speed and compression)
>      * with zlib compression levels (`export GRASS_ZLIB_LEVEL=X`): -1..9 (-1
>   is default which is level 6)
>      * Notes from #comment:23: ZLIB level = 0 tells ZLIB to copy the data as
>   is from source to destination. With CELL maps, the rasterlib will then
>   still trim high zero bytes with trim_bytes() which can already reduce the
>   data size considerably, but ZLIB will not compress the data.
>    * `LZ4`  (fastest, low compression)
>    * `BZIP2` (slowest, high compression)
>
>   NULL file compression: At time it must be explicitly turned on (IMHO it
>   should become the default) with
>    * `export GRASS_COMPRESS_NULLS=1`
If I want to test this, is there an overview page how to use this? E.g., 
do I need to run the above every time I start GRASS? And will it then 
compress every NULL file when opening, or is there a way to batch 
compress all NULL files in a mapset for example. And if compressed, can 
I only open the raster layers when compression is (explicitly) enabled?

>
>   Backward NULL file compression compatibility could be implemented in
>   relbranch70 only by ZLIB's DEFLATE (see #comment:36). Hence ZLIB may
>   qualify for the default NULL compression algorithm.
>
> --
> Ticket URL: <https://trac.osgeo.org/grass/ticket/2750#comment:40>
> GRASS GIS <https://grass.osgeo.org>
>
> _______________________________________________
> grass-dev mailing list
> grass-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/grass-dev