[GRASS-dev] Raster data compression confusion: identical CELL file size

[Markus Neteler]

On Tue, Sep 6, 2016 at 3:23 PM, Markus Metz <markus.metz.giswork at gmail.com>
wrote:
> On Tue, Sep 6, 2016 at 1:44 PM, Markus Neteler <neteler at osgeo.org> wrote:
>> Hi,
>>
>> since my local drive was filled up again :) I checked raster how data are
>> currently compressed in GRASS GIS 7.2.svn.
>> According to
>>
>> https://grass.osgeo.org/grass72/manuals/r.compress.html#
used-compression-algorithms
>>
...
> The manual says, as you cited:
> "Raster maps are by default ZLIB compressed."
> What exactly is unclear about this? Should it say "All raster maps ..." ?

Well, it is perhaps ok but the software appeared to behave differently.

...
> The default ZLIB compression level was invalid, causing ZLIB to not
> compress at all. Fixed in r69387,8.

Ah! So, that changes it dramatically :-) Thanks for the quick fix:

-rw-r--r-- 1 mneteler mneteler 2539235691 Jun 14 17:19 s2_20151225_B03_10m
         <<-- before
-rw-r--r-- 1 mneteler mneteler 1463080868 Sep  6 16:46
s2_20151225_B03_10m_NEWCOPY  <<-- now, generated with r.mapcalc = operator

Great.
And comparing to the previously full file-based gzip test:

-rw-r--r-- 1 mneteler mneteler 1456248453 Jun 14 17:19
s2_20151225_B03_10m.gz

> 1463080868 / 1456248453
[1] 1.004692

... which is now almost the same compression rate.

Some more values:

- before today's bugfix:
du -hs PERMANENT/
30G    PERMANENT/

- after the bugfix (copies created with r.mapcalc, original raster maps
removed):
du -hs PERMANENT/
25G    PERMANENT/

- using export GRASS_COMPRESS_NULLS=1 and running r.null -z on all raster
maps
  which generates cell_misc/nullcmpr and removes the old uncompressed
cell_misc/null:
du -hs PERMANENT/
21G    PERMANENT/

Now a notable amount of (SSD) disk space is saved - 21GB usage instead of
30GB!
Goal of trac #2349 <https://trac.osgeo.org/grass/ticket/2349> achieved.

Thanks again,
markusN

PS: it would be great to know through a user message is r.null -z is
actually compressing or uncompressing...

-- 
Markus Neteler
http://www.mundialis.de - free data with free software
http://grass.osgeo.org
http://courses.neteler.org/blog
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20160906/f1b552ca/attachment.html>