[gdal-dev] Problem when creating overviews for large input files with gdaladdo
armin.burger at gmx.net
Mon Dec 19 13:02:43 PST 2016
I ran into a problem when trying to create overviews for large
LZW-compressed GeoTIFF images using gdaladdo. The TIFF's are internally
tiled, the file type is Byte. GDAL versions tested are 2.1.1 and 1.11,
running on 64bit Linux.
The first try was using an input file of size 1038336 x 565248 pixels.
Setting CPL_DEBUG=ON the processing goes up to the message
...10...20...30...40...50...60...70..GDAL: Potential thrashing on band 1
and then it remains at this "70.." % forever, with no further log
output. This happened for all files of similar size, on multiple
machines, on both GDAL versions. Always hangs at this "70.." state, the
process remains alive but nothing is written any more to the output
file. I used the command like
gdaladdo -ro --config COMPRESS_OVERVIEW LZW INPUT_FILE 2 4 8 16 32 64
Trying to specify BIGTIFF or not for the overview did not make any
Using a smaller file or converting the large input file to 50% size and
running the gdaladdo on this smaller file finished successfully, but
also here I got some warnings of a similar kind, and the process got
stuck for a while after the ".70..GDAL: Potential thrashing..." warning,
but then continued to the end. The full log is attached.
An additional observation I made was that the overview file stopping at
70% was already bigger than the input file.
I was thinking to create single lower resolution files and merging them
to an overview file, but there's no built-in functionality for this, so
did not follow it up further.
Could there be a sort of limitation of maximum pixels x*y, or with the
LZW compression? Any ideas how this could be solved?
Thanks in advance for any hint
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 5367 bytes
Desc: not available
More information about the gdal-dev