[GRASS-dev] GRASS GIS raster files: LZW compression?

[Nikos Alexandris]

* Markus Neteler <neteler at osgeo.org> [2017-12-05 10:08:00 +0100]:

>On Mon, Dec 4, 2017 at 7:32 PM, Markus Metz
><markus.metz.giswork at gmail.com> wrote:
>> ZSTD compression has been added to trunk with r71889-92.
>
>Great!
>
>> ZSTD compression can be added with configure --with-zstd=yes or simpler
>> configure --with-zstd
>>
>> In order to get some wider testing, ZSTD is now the default compression
>> method in trunk if ZSTD is available.
>>
>> ZSTD is an improvement over ZLIB: generally it compresses faster and higher
>> than ZLIB. Decompression is consistently much faster than with ZLIB. For
>> more information on ZSTD (Zstandard), see
>> http://facebook.github.io/zstd/
>> https://github.com/facebook/zstd
>>
>> Please test!
>
>Here an initial test with a WorldClim DECLL map:
>
[..]

Another quick test with some MERIT DEM tiles:

→ find . -name n00w060 |sort |xargs du -h
512     ./cats/n00w060
4.5K    ./cellhd/n00w060
1.3M    ./cell_misc/n00w060
512     ./cell/n00w060
279M    ./fcell/n00w060
4.5K    ./hist/n00w060

→ find . -name n00w060_zstd |sort |xargs du -h
512     ./cats/n00w060_zstd
4.5K    ./cellhd/n00w060_zstd
187K    ./cell_misc/n00w060_zstd
512     ./cell/n00w060_zstd
278M    ./fcell/n00w060_zstd
4.5K    ./hist/n00w060_zstd

→ r.compress -p n00w060
<n00w060> is compressed (method 2: ZLIB). Data type: FCELL
<n00w060> has an uncompressed NULL file

time -p r.univar -g n00w060
n=81725535
null_cells=242274465
cells=324000000
min=-116.987380981445
max=2043.755859375
range=2160.74324035645
mean=175.473662699091
mean_of_abs=175.474989313581
stddev=141.00352182084
variance=19881.9931658801
coeff_var=80.3559461015174
sum=14340678962.4927
real 17.31
user 16.94
sys 0.25

→ r.compress -p n00w060_zstd
<n00w060_zstd> is compressed (method 5: ZSTD). Data type: FCELL
<n00w060_zstd> has a compressed NULL file

→ time -p r.univar -g n00w060_zstd
n=81725535
null_cells=242274465
cells=324000000
min=-116.987380981445
max=2043.755859375
range=2160.74324035645
mean=175.473662699091
mean_of_abs=175.474989313581
stddev=141.00352182084
variance=19881.9931658801
coeff_var=80.3559461015174
sum=14340678962.4927
real 13.21
user 12.98
sys 0.16

Thank you, Nikos
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 228 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20171205/439c94c4/attachment.sig>