[gdal-dev] Question on TIFF compression alternatives
even.rouault at spatialys.com
Fri Oct 14 04:16:03 PDT 2016
Le vendredi 14 octobre 2016 13:01:50, Neumann, Andreas a écrit :
> Hi Even,
> Thank you for your detailed reply.
> I should have added that my concern is mainly about random access to
> parts of the images - for QGIS server and QGIS Desktop.
> I have internal pyramids. I am not so sure if it is tiled or not.
Internal pyramids created by GDAL are tiled with tiles of 128x128 pixel (can
be changed with the GDAL_TIFF_OVR_BLOCKSIZE configuration option:
http://gdal.org/frmt_gtiff.html#overviews . But the default is generally fine)
> gdalinfo reveal whether the TIFF was created with "TILED=YES"?
If you see something "Band 1 Block=256x256" in the gdalinfo output, the full
resolution image is tiled. If the block width (the first figure after Block=) is
the image width, then it is not. You definitely want to use a tiled image for
> So from your analysis it looks like lzma is the slowest, but overall,
> the differences aren't that huge. I don't know how random access to
> parts of the image changes the game a bit, compared to your analysis
> where you examine the decompression of the whole image - right?
I used whole image decompression otherwise the timings would have been hard to
measure accurately. You can expect the same ratios of performance to apply to
partial image decompression.
My timing were done on "hot" runs. Depending on use cases, I/O might be the
> On 2016-10-14 12:09, Even Rouault wrote:
> > Le vendredi 14 octobre 2016 09:45:38, Neumann, Andreas a écrit :
> >> Hi,
> >> I have a question on TIFF compression alternatives.
> >> My data of concern is 8bit gray value. Lots of white pixel (>95%), the
> >> rest is black pixels (content) and some gray pixels because of
> >> antialiasing.
> >> I would like to compress to save some storage, but I need a good
> >> performance. What is the recommended compression that is fast on
> >> decompression?
> >> I tried DEFLATE, but it is rather slow to decompress. Would LZW or
> >> PACKBITS be preferred in this situation? How about LZMA? I would like to
> >> avoid non-compressed data, because I use SSD storage for performance
> >> reasons.
> > My findings on a 37614 x 18816 8 bit image (a scan of a text page) that
> > should be similar to what you describe.
> > Decompression time is the result of the second run of "time gdalinfo -
> > checksum" (so you can consider that I/O is cached by the OS and you
> > measure essentially CPU time. storage is spinning disk)
> > No tiling:
> > Size Compression Decompression time
> > 707895750 out_nocompression.tif 9.3 s
> > 11239972 out_deflate.tif 10.8 s
> > 17749306 out_lzw.tif 10.5 s
> > 12337710 out_lzma.tif 12.6 s
> > 36201320 out_packbits.tif 9.9 s
> > Tiling:
> > Size Compression Decompression time
> > 8813833 out_deflate_tiled.tif 11.1 s
> > 14614121 out_lzw_tiled.tif 11.0 s
> > 8479238 out_lzma_tiled.tif 13.0 s
> > 37018470 out_packbits_tiled.tif 10.2 s
> > Note: in the tiling case, the gdalinfo -checksum benchmark might not be
> > ideal as it proceeds line by line (and not block by block), thus there's
> > a lot of copying between cached blocks and output buffer that is done.
> > So I've done time gdal_translate XXX.tif /vsimem/out.tif -q :
> > deflate tiled: 2.5 s
> > lzw tiled: 2.3 s
> > packbits tiled: 1.5 s
> > lzma tiled: 4.5 s
> > Times for non tiled files are essentially the same.
> > Note: if you use tiling and you have whole tiles that are at white value,
> > if you use GDAL trunk, set the SPARSE_OK=YES creation option and set the
> > nodata value to 255, then those tiles will be entirely omitted from the
> > compressed file. This might save a few extra bytes, and also some
> > decompression time.
> > gdal_translate out_nocompression.tif out_packbits_tiled.tif -co tiled=yes
> > -co compress=packbits
> > gdal_translate out_nocompression.tif out_packbits_tiled_sparse.tif -co
> > tiled=yes -a_nodata 255 -co compress=packbits -co sparse_ok=yes
> > 37018470 out_packbits_tiled.tif
> > 30409074 out_packbits_tiled_sparse.tif
> > time gdal_translate out_packbits_tiled_sparse.tif /vsimem/out.tif -q: 1.3
> > s
> > Even
Spatialys - Geospatial professional services
More information about the gdal-dev