Compression in TIF files: disk space vs. performance

[Ed McNierney]

Andreas -

If you are using scanned black and white cadastral images, I suggest you
do a test with LZW-compressed TIFFs.  You may get excellent results.

The LZW compression scheme requires that the entire data set be
decompressed starting from the beginning in order to decompress any
piece of the data.  This can cause a performance problem since you may
have to decompress the whole image just to get the lower-right corner.

However, you can specify that your image use a small number of rows per
strip, rather than treat the entire image as one strip.  This will make
each strip a separate LZW data set, so you will only need to decompress
the strips you need rather than the whole image.  This will reduce the
compression ratio a little bit (making your output image a little
larger) but can make it quite a bit faster.  You can use geotifcp -c LZW
-r 1, for example, to create an LZW-compressed copy of your image with
one row per strip; you'll probably find the best results with a small
number for -r, but 1 is probably too small.

	- Ed

Ed McNierney
President and Chief Mapmaker
TopoZone.com / Maps a la carte, Inc.
73 Princeton Street, Suite 305
North Chelmsford, MA  01863
Phone: +1 (978) 251-4242
Fax: +1 (978) 251-1396
ed at topozone.com 

-----Original Message-----
From: Andreas Neumann [mailto:neumann at karto.baug.ethz.ch] 
Sent: Sunday, May 14, 2006 3:48 PM
To: Ed McNierney
Subject: Re: [UMN_MAPSERVER-USERS] Compression in TIF files: disk space
vs. performance


>Compression will almost always cost you performance.  If you have the 
>unusual situation of having lots of RAM, a fast processor, and a VERY 
>slow disk system, then it might not cost you performance, but I've 
>never seen that happen.
>
>  
>

that's what I thought. Thanks for confirming.

>However, if you're out of space, you're out of space!  My first 
>reminder is that disk is inexpensive - and my first suggestion is to 
>buy more disk space.
>  
>

true - we'll probably end up buying more disk

>Lossless TIFF compression (Packbits, LZW, etc.) will not cost you much 
>in performance at all.  It is also unlikely to save you much disk
space.
>  
>

Since our images are originally 1bit b+w it makes a huge difference if
we use a lossless compression or not. LZW and packpits compresses very
well if you have huge areas of the same color, doesn't it? At least I
see a huge difference between filesizes of uncompressed and lzw
compressed files.

My only concern is that the lzw compressed files will take considerable
longer than the uncompressed ones. Is that true?

Well, maybe we just have to test ...

>You should experiment with your own data, but I have found that it's 
>unusual to save more than 10% if you're doing lossless compression on 
>aerial or satellite photos.  If your rasters are scanned images of line

>art (scanned maps) then you are likely to do much better, but that's 
>not likely to be the case because it's rare to have a large amount of 
>such data.
>
yes, in my case we have "line art" images - they are b+w cadastral map
sheets. We don't deal with aerial images.

Thanks for the answer, anyway.

Andreas

--
----------------------------------------------
Andreas Neumann
Institute of Cartography
ETH Zurich
Wolfgang-Paulistrasse 15
CH-8093  Zurich, Switzerland

Phone: ++41-44-633 3031, Fax: ++41-44-633 1153
e-mail: neumann at karto.baug.ethz.ch
www: http://www.carto.net/neumann/
SVG.Open: http://www.svgopen.org/
Carto.net: http://www.carto.net/