[Mapserver-users] raster image server: optimal performance

[Ed McNierney]

Hubert -

This is a complex subject.  The most important thing is to have a good idea of how your images will be used.  You want to always be thinking about how a typical map image request translates into disk reading activity.  In the ideal world, each map image request would translate into opening exactly one source image file and reading consecutive bytes from that file to produce exactly the image you need and no more.  The real world doesn't work that way, but you should always be thinking about how to come as close to that goal as you can.  Here are some thoughts.

1. JPEG is usually a bad idea, relative to GeoTIFF.  That's because the JPEG format requires the entire image to be decoded in order to return a small portion of it.  GeoTIFF allows you to only read a specific portion of the file.  However, if your test images are set up such that you need to read most of each file anyway, you may see little difference.

2. Smaller image tiles with a TILEINDEX are usually a good idea, but don't overdo it.  Disk seek time (going from one file to another) is typically much slower than reading from a single file.  So don't make very small images such that each map image request requires many image files to be opened and read.  Images that are perhaps 2 to 5 times the size of the map image in each dimension might be a good target - that means you'll have many cases in which only one source image needs to be read.  Making the files larger (if they're uncompressed GeoTIFF) isn't really a problem either.  If you make the image files too large, you will end up doing multiple disk seeks inside the file, which aren't as bad as multiple-file seeks but which should be avoided if possible.

3. For aerial photography, TIFFs typically don't compress very much, so uncompressed is fine.  This also eliminates some issues you might have with various compression schemes.

4. For some people it's not obvious that zooming OUT on the image (showing a large area on the map) is much more work than zooming IN on it.  If you expect your users to be viewing zoomed-out images often, look into GDAL's support for image pyramids.  This allows you to create pre-resampled, lower-resolution versions of each image that are automatically used for zoomed-out display.  The disadvantage is that they require more disk space, but you can get significant improvements in performance for only a 15% - 20% increase in storage space.

5. Disk space is cheap - use it.  400GB of good IDE disk storage costs around USD$650 right now, or just about 70,000 times cheaper than the first hard disk I ever purchased <g>.

6. Make sure your testing models the way your maps will really be used.  The zoom level is the most important factor - if your users are mostly going to view the photos at full-scale, test that way.  If they're mostly zoomed out, be sure to test that way, because there are different factors involved.

Good luck!

	- Ed

Ed McNierney
President and Chief Mapmaker
TopoZone.com / Maps a la carte, Inc.
73 Princeton Street, Suite 305
North Chelmsford, MA  01863
ed at topozone.com
(978) 251-4242 

-----Original Message-----
From: Hubert Fröhlich [mailto:hubert.froehlich at bvv.bayern.de]
Sent: Thursday, January 23, 2003 10:45 AM
To: mapserver-users at lists.gis.umn.edu
Subject: [Mapserver-users] raster image server: optimal performance


Hello list,

I want to use MapServer 3.6.3 as a raster server for rectified aerial photos
, working on a Linux box (SuSE 8.0), I have also gdal.

For testing, I worked with a dataset of jpg's (plus worldfiles) or
geotiffs.

I have a test dataset of aerial photos (jpg plus worldfile, alernatively
geotiff) of an area 40 x 25 km in 1km-square tiles with resolution 40
cm, i.e. the tiles are 2500x2500 pixels, about 800 tiles.

To get things working at all, I proceeded the usual way
for raster images setting up an index
shapefile (e.g. with gdaltindex). So far, everything OK ...


Now I want to speed up things, as my original dataset covers 200x350 km,
all in 40cm resolution. (IN jpeg 80 GByte, in GeoTiff about 400 GByte!),
   in 1km-squares that would mean 70000 tiles. The files are stored in a
simple file system.

a) I tried out jpeg's vs (uncompressed) geotiffs. With the test dataset,
jpegs (also diskspace-saving!) seemed to perform slightly better. Can
anybody share the experience or give me hints about the best performing
data format?

b) What effect on MapServer response time does the tile size have? Any
recommendations on the list for optimal tile size (bigger means less
tiles) for best performance? Any dependencies on the data format?

c) At the moment, I use b/w photos. What changes do I face in
performance if I switch to color?


Thank you

Hubert
-- 
-------------------------------------------------------------------------------
Dr.-Ing. Hubert Fröhlich			
Bezirksfinanzdirektion München 			
Alexandrastr. 3, D-80538 München, GERMANY
Tel. :+49 (0)89 / 2190 - 2980
Fax  :+49 (0)89 / 2190 - 2459
hubert.froehlich at bvv.bayern.de




_______________________________________________
Mapserver-users mailing list
Mapserver-users at lists.gis.umn.edu
http://lists.gis.umn.edu/mailman/listinfo/mapserver-users