[GRASS-dev] [GRASS GIS] #2750: LZ4 when writing raster rows; better than double I/O bound r.mapcalc speed

[GRASS GIS]

#2750: LZ4 when writing raster rows; better than double I/O bound r.mapcalc speed
--------------------------+---------------------------
  Reporter:  sprice       |      Owner:  grass-dev@…
      Type:  enhancement  |     Status:  new
  Priority:  normal       |  Milestone:  7.1.0
 Component:  Raster       |    Version:  svn-trunk
Resolution:               |   Keywords:  ZLIB LZ4 ZSTD
       CPU:  OSX/Intel    |   Platform:  MacOSX
--------------------------+---------------------------

Comment (by mmetz):

 Replying to [comment:45 neteler]:
 > Thanks. It would be good to now also pave the way to compress the NULL
 files by default (chose default method to be able to have read-support in
 relbr70).

 Why have this also in relbr70? Backporting new features from trunk to a
 release branch is a bad idea and against GRASS policy, I thought. You
 could instead release 7.1 if you want these features in a release branch.

 ZLIB is not really the best method for null bits compression. With large
 chunks of NULL values and/or large chunks of non-NULL values, the new RLE
 method seems to be as fast as no null file compression and provides the
 best compression. For special cases, LZ4 should compress better than RLE
 at the same speed.

 Some tests with a CELL map (MODIS LST Europe) with 18711 rows and 22195
 columns using r.compress (processing time and size of the null2 file):

 {{{
 no null compression
 14.8 sec, 51.9 MB
 RLE
 14.8 sec, 1.2 MB
 ZLIB
 16.8 sec, 1.4 MB
 LZ4
 14.8 sec, 1.4 MB
 BZIP2
 18.1 sec, 1.7 MB
 }}}

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2750#comment:46>
GRASS GIS <https://grass.osgeo.org>