[gdal-dev] GeoTIFF and LZWDecode (LZWEncode?) errors

[Even Rouault]

Le jeudi 11 juillet 2013 15:56:56, Casper Børgesen (CABO) a écrit :
> Hi Even.
> 
> I cannot reproduce the error, it just occurs when I'm batch processing and
> not necessarily at the same place/file. That's why I asked for experience.
> However, when I get the error again, I can provide the corrupted file and
> then its possible to see that ex. Gdal_translate doesn't set ERRORLEVEL 1
> or similar even though the process cannot continue.

Normally it should return an error, but if the error happens during the 
closing of the dataset (writes might be deferred), then it would go unnoticed. 
That's potentially what happens in your case, but difficult to know without 
closer investigation.

> 
> Kind regards, Casper
> 
> 
> -----Original Message-----
> From: Even Rouault [mailto:even.rouault at mines-paris.org]
> Sent: 11. juli 2013 15:40
> To: Casper Børgesen (CABO)
> Cc: gdal-dev at lists.osgeo.org
> Subject: Re: [gdal-dev] GeoTIFF and LZWDecode (LZWEncode?) errors
> 
> Casper,
> 
> I'm not sure to have understood if you have this problem while reading or
> generating compressed files. Could you provide a way (that is to say
> provide both the input data and the command line used) of reproducing this
> ?
> 
> Best regards,
> 
> Even
> 
> > Hi
> > 
> > When batch processing GeoTIFF files, I tend to get this problem a bit
> > too
> > often:
> > 
> > Warning 1: LZWDecode:LZWDecode: Strip 11 not terminated with EOI code
> > ERROR 1: LZWDecode:Not enough data at scanline 11 (short 30 bytes)
> > ERROR 1: TIFFReadEncodedStrip() failed.
> > 
> > band 1: IReadBlock failed at X offset 0, Y offset 11 ERROR 1:
> > GetBlockRef failed at X block offset 0, Y block offset 11
> > 
> > The strip, scanline and offsets vary. It normally happens when I
> > process from clients (computers) with the files located on servers (on
> > local network), and I can reprocess a file again without getting the
> > error, or possible an error with a different strip, scanline or
> > offset. This special case is caused by gdaldem.exe and the error is
> > reported by gdal_translate.exe using GDAL 1.9, but I have experienced
> > this error using GDAL 1.9.2 and I think I saw it using GDAL 1.10 also.
> > 
> > I have two issues here:
> > 
> > 1.       Why the problem is happening to begin with?
> > 
> > 2.       Why does the Warning and/or Error not trigger ERRORLEVEL 1?
> > 
> > The first bullet I think can be very difficult to debug, so I seek
> > experiences here. The second bullet is really frustrating, because I
> > have to look through all my clients looking for output similar to the
> > message above - and then reprocess all crashed files again.
> > 
> > One solution I have seen somewhere is to not compressing the files,
> > but this results in quite an increase in disk space usage - way too high
> > for me.
> > 
> > 
> > Kind regards, Casper

-- 
Geospatial professional services
http://even.rouault.free.fr/services.html