Compression in TIF files: disk space vs. performance

[Ed McNierney]

Andreas -

Compression will almost always cost you performance.  If you have the
unusual situation of having lots of RAM, a fast processor, and a VERY
slow disk system, then it might not cost you performance, but I've never
seen that happen.

However, if you're out of space, you're out of space!  My first reminder
is that disk is inexpensive - and my first suggestion is to buy more
disk space.

Lossless TIFF compression (Packbits, LZW, etc.) will not cost you much
in performance at all.  It is also unlikely to save you much disk space.
You should experiment with your own data, but I have found that it's
unusual to save more than 10% if you're doing lossless compression on
aerial or satellite photos.  If your rasters are scanned images of line
art (scanned maps) then you are likely to do much better, but that's not
likely to be the case because it's rare to have a large amount of such
data.

Lossy compression with MrSID or ECW tools will definitely cost you
performance.  However, if you have a light server load that might not be
a problem, particularly if you have a lot of RAM and very few
simultaneous users.  But you can save a LOT of disk space that way.

	- Ed

Ed McNierney
President and Chief Mapmaker
TopoZone.com / Maps a la carte, Inc.
73 Princeton Street, Suite 305
North Chelmsford, MA  01863
Phone: +1 (978) 251-4242
Fax: +1 (978) 251-1396
ed at topozone.com 

-----Original Message-----
From: UMN MapServer Users List [mailto:MAPSERVER-USERS at LISTS.UMN.EDU] On
Behalf Of Andreas Neumann
Sent: Sunday, May 14, 2006 1:10 PM
To: MAPSERVER-USERS at LISTS.UMN.EDU
Subject: [UMN_MAPSERVER-USERS] Compression in TIF files: disk space vs.
performance

Hello list,

I have a question on compression in tiff files.

I read that for best mapserver performance it is recommended to store
tiff files on the server in uncompressed format. We currently have a
disk space problem and need to serve a large number of raster images. Is
there any compression technique that allows to save diskspace while
maintaining a reasonable or good performance? Or is compression not
recommended at all?

Another question: we need to serve 1bit (b+w) rasterimages that are
delivered as gray/rgb jpegs later because we want to use resampling. 
What is the best color depth to store the original images. I noticed
that when keeping them in 1bit color depth one cannot create pyramids
using gdaladdo. Is it better to store them in 1 bit with no pyramids or
in 8bit gray value with pyramids?

Thank you for any help on my questions,

All the best,
Andreas

--
----------------------------------------------
Andreas Neumann
Institute of Cartography
ETH Zurich
Wolfgang-Paulistrasse 15
CH-8093  Zurich, Switzerland

Phone: ++41-44-633 3031, Fax: ++41-44-633 1153
e-mail: neumann at karto.baug.ethz.ch
www: http://www.carto.net/neumann/
SVG.Open: http://www.svgopen.org/
Carto.net: http://www.carto.net/