[GRASS-user] Problem with zstd compressed rasters

[Markus Metz]

On Sat, Mar 30, 2019 at 4:23 PM Micha Silver <tsvibar at gmail.com> wrote:
>
>
> On 29/03/2019 16:04, Markus Metz wrote:
>
>
>
> On Fri, Mar 29, 2019 at 11:20 AM Micha Silver <tsvibar at gmail.com> wrote:
> >
> > Hello all:
> >
> > I have some GRASS data that I share between different computers. The
newer machine got updated recently to version 7.6.1, and rasters are by
default compressed with zstd. The older machine (Linux Mint 18.3) was
running 7.4 and (I think) did not support zstd. So those rasters created on
the newer machine will no longer open on the older one. I updated to GRASS
7.6.1 (a word of thanks to the ubuntugis-unstable packagers!) and added the
libzstd libraries, but still rasters created on the newer machine will not
open on the older one. The difference between them is the version of
libzstd. Is there a minimum required version of zstd??
>
> Not really, if anything, version 1.x.
> >
> >
> > On the old machine:
> >
> >
> > micha at RMS ~ $ g.version
> > GRASS 7.6.1 (2019)
> > micha at RMS ~ $ zstd --version
> > *** zstd command line interface 64-bits v1.3.1, by Yann Collet ***
> > micha at RMS ~ $ r.univar slope
> > WARNING: ZSTD compression error -2: Unknown frame descriptor
> > ERROR: Error uncompressing raster data for row 2913 of <slope>
> > micha at RMS ~ $ ldd /usr/lib/grass76/bin/r.univar | grep zstd
> >     libzstd.so.0 => /usr/lib/x86_64-linux-gnu/libzstd.so.0
(0x00007fadf663c000)
> >
> >
> > The new machine has zstd version 1.3.8, and everything works fine. But
not on the old machine. Is my problem the difference in zstd versions?
>
> I had something similar before, the error was caused by a corrupted file.
If the old machine uses a copy of the mapset of the new machine, check e.g.
with md5sum if the corresponding cell or fcell files are identical. It
could also be a disk IO problem.
>
> I restored the LOCATION to the old machine, and checked that the md5sums
of fcell/* on both computers were identical.
>
> Still the same problem.

I think I found something: ZSTD can be compiled with or without legacy
support. If compiled without legacy support, you can get this error
"Unknown frame descriptor". That means it could be possible that ZSTD 1.3.1
on the old machine has been built without ZSTD legacy support and ZSTD
1.3.8 on the new machine has been compiled with ZSTD legacy support. In
this case I would consider it a bug in the packaging of ZSTD for the old
machine. If this is correct, there is nothing GRASS can do, apart from
using another compression method.

This is a wild guess but the only explanation I can come up with.

Markus M
>
> <shrugging>Maybe it's time to upgrade</shrugging>
>
>
>
> HTH,
>
> Markus M
>
> --
> Micha Silver
> Ben Gurion Univ.
> Sde Boker, Remote Sensing Lab
> cell: +972-523-665918
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20190331/23425905/attachment.html>