[GRASS-dev] Raster data compression confusion: identical CELL file size

[Markus Neteler]

Hi,

since my local drive was filled up again :) I checked raster how data are
currently compressed in GRASS GIS 7.2.svn.
According to

https://grass.osgeo.org/grass72/manuals/r.compress.html#used-compression-algorithms

all maps are DEFLATE compressed by default:

"Raster maps are by default ZLIB compressed.
...

Floating point (FCELL, DCELL) raster maps never use RLE compression; they
are either compressed with ZLIB, LZ4, BZIP2 or are uncompressed.
"

Ehm, now how *are* FCELL, DCELL compressed by default? Not quite clear to
me! This document needs improvements.

Reality check with Sentinel-2 data (3 different bands, same regional
extent):

GRASS 7.2.svn (utm37n):~ > r.info -r s2_20151225_B02_10m
min=0
max=22937
GRASS 7.2.svn (utm37n):~ > r.info -r s2_20151225_B04_10m
min=0
max=18849
GRASS 7.2.svn (utm37n):~ > r.info -r s2_20151225_B8A_20m
min=0
max=17210

GRASS 7.2.svn (utm37n):~ >:~ > r.compress s2_20151225_B02_10m -p
<s2_20151225_B02_10m> is compressed (method 2: ZLIB). Data type:
CELL

GRASS 7.2.svn (utm37n):~ >:~ > r.compress s2_20151225_B04_10m -p
<s2_20151225_B04_10m> is compressed (method 2: ZLIB). Data type:
CELL

GRASS 7.2.svn (utm37n):~ >:~ > r.compress s2_20151225_B8A_20m -p
<s2_20151225_B8A_20m> is compressed (method 2: ZLIB). Data type:
CELL

So far so nice. Now the suprising part, while the channels are not
identical (obviously, since covering different spectral parts), the map
sizes are identical!

GRASS 7.2.svn (utm37n):~ > ls -la
...
-rw-r--r--  1 neteler neteler 2539235691 Jun 14 17:14 s2_20151225_B02_10m
-rw-r--r--  1 neteler neteler 2539235691 Jun 14 17:19 s2_20151225_B03_10m
-rw-r--r--  1 neteler neteler 2539235691 Jun 14 17:25 s2_20151225_B04_10m
-rw-r--r--  1 neteler neteler  634878630 Jun 14 20:36 s2_20151225_B05_20m
-rw-r--r--  1 neteler neteler  634878630 Jun 14 20:37 s2_20151225_B06_20m
-rw-r--r--  1 neteler neteler  634878630 Jun 14 20:39 s2_20151225_B07_20m
-rw-r--r--  1 neteler neteler  634878630 Jun 14 20:40 s2_20151225_B11_20m
-rw-r--r--  1 neteler neteler  634878630 Jun 14 20:42 s2_20151225_B12_20m
-rw-r--r--  1 neteler neteler  634878630 Jun 14 20:43 s2_20151225_B8A_20m

I would expect different sizes, compression can hardly lead to identical
file sizes.

Next test: gzip the file

GRASS 7.2.svn (utm37n):~/grassdata/utm37n/PERMANENT/cell > ls -la
s2_20151225_B03_10m
-rw-r--r-- 1 mneteler mneteler 2539235691 Jun 14 17:19 s2_20151225_B03_10m

GRASS 7.2.svn (utm37n):~/grassdata/utm37n/PERMANENT/cell > gzip
s2_20151225_B03_10m

GRASS 7.2.svn (utm37n):~/grassdata/utm37n/PERMANENT/cell > ls -la
s2_20151225_B03_10m.gz
-rw-r--r-- 1 mneteler mneteler 1456248453 Jun 14 17:19
s2_20151225_B03_10m.gz

R
> 1456248453/2539235691
[1] 0.5734987

Quite smaller! So I am not at all convinced that these CELL files are
currently ZLIB compressed.

>From this ticket I would expect something else:
https://trac.osgeo.org/grass/ticket/2349

Ah, and no specific environment variables are set:

GRASS 7.2.svn (utm37n):~ > echo $GRASS_<tab>
$GRASS_ADDON_BASE   $GRASS_GNUPLOT    $GRASS_HTML_BROWSER  $GRASS_PAGER
$GRASS_PROJSHARE    $GRASS_PYTHON     $GRASS_VERSION


A bug?

Markus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20160906/6c2f119f/attachment.html>