[mapserver-users] Ed's Rules for the Best Raster Performance

[Jim Klassen]

Just out of curiosity, has anyone tested the performance of Jpegs vs. GeoTiffs?

I would expect at some point the additional disk access time required for GeoTiffs (of the same pixel count) as Jpegs would outweigh the additional processor time required to decompress the Jpegs. (Also the number of Jpegs that can fit in disk cache is greater than for similar GeoTiffs.)

For reference we use 1000px by 1000px Jpeg tiles (with world files). We store multiple resolutions of the dataset, each in its own directory. We start at the native dataset resolution, and half that for each step, stopping when there are less than 10 tiles produced at that particular resolution. (I.e for one of our county wide datasets 6in/px, 1ft/px, 2ft/px, ... 32ft/px). A tileindex is then created for each resolution (using gdaltindex followed by shptree) and a layer is created in the mapfile for each tileindex and appropriate min/maxscales are set. The outputformat in the mapfile is set to jpeg.

Our typical tile size is 200KB. There are about 20k tiles in the 6in/px dataset, 80k tiles in the 3in/px dataset (actually 4in data, but stored in 3in so it fits with the rest of the datasets well). I have tested and this large number of files in a directory doesn't seem to effect performance on our system.

Average access time for a 500x500px request to mapserver is 300ms measured at the client using perl/LWP and about 220ms with shp2img.

Machine is mapserver 5.2.0/x86-64/2.8GHz Xeon/Linux 2.6.16/ext3 filesystem.

Jim Klassen
City of Saint Paul

>>> "Fawcett, David" <David.Fawcett at state.mn.us> 09/15/08 1:10 PM >>>
Better yet, 

Add your comments to:

http://mapserver.gis.umn.edu/docs/howto/optimizeraster

and 

http://mapserver.gis.umn.edu/docs/howto/optimizevector

I had always thought that all we needed to do to make these pages great
was to grok the list for all of Ed's posts...

David.

-----Original Message-----
From: mapserver-users-bounces at lists.osgeo.org
[mailto:mapserver-users-bounces at lists.osgeo.org] On Behalf Of Brent
Fraser
Sent: Monday, September 15, 2008 12:55 PM
To: mapserver-users at lists.osgeo.org
Subject: [mapserver-users] Ed's Rules for the Best Raster Performance


In honor of Ed's imminent retirement from the Mapserver Support Group,
I've put together "Ed's List for the Best Raster Performance":


#1. Pyramid the data 
        - use MAXSCALE and MINSCALE in the LAYER object.

#2. Tile the data (and merge your upper levels of the pyramid for fewer
files).
        - see the TILEINDEX object

#3. Don't compress your data
        - avoid jpg, ecw, and mrsid formats.

#4. Don't re-project your data on-the-fly.

#5. Get the fastest disks you can afford.


(Ed, feel free to edit...)

Brent Fraser
_______________________________________________
mapserver-users mailing list
mapserver-users at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
_______________________________________________
mapserver-users mailing list
mapserver-users at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users