TIFF performance: overviews and internal tiles: Surprise!

[Ed McNierney]

Gregor -

Could you go into a bit more detail about your 4-disk RAID-5 system?  What kind of controller?  What capacity, interface, and rotation speed are the disks?  Since even a RAID-5 array is the slowest component in a MapServer application, small differences in disk design can make a big difference.  Thanks.

     - Ed


Ed McNierney
Chief Mapmaker
Demand Media / TopoZone.com
73 Princeton Street, Suite 305
North Chelmsford, MA  01863
Phone: 978-251-4242, Fax: 978-251-1396
ed at topozone.com




-----Original Message-----
From: UMN MapServer Users List [mailto:MAPSERVER-USERS at LISTS.UMN.EDU] On Behalf Of Gregor Mosheh
Sent: Saturday, September 01, 2007 7:00 PM
To: MAPSERVER-USERS at LISTS.UMN.EDU
Subject: [UMN_MAPSERVER-USERS] TIFF performance: overviews and internal tiles: Surprise!

This test compares the effect of overviews and internal tiling on the 
performance of a TIFF raster data source.

*** SETUP

The data is USGS DOQQs (black-n-white) of San Francisco, California. 
Several DOQQs were downloaded and merged into a single GeoTIFF using 
gdal_merge.py.

I then created three copies of the TIFF, with overviews, with internal 
tiling, and with both:
    cp original.tif overviews.tif
    gdaladdo overviews.tif 2 4 8 16 32
    gdal_translate -co "TILED=YES" original.tif tiled.tif
    gdal_translate -co "TILED=YES" original.tif tiledandoverviews.tif
    gdaladdo tiledandoverviews.tif 2 4 8 16 32

This set of overviews for this particular TIFF gives the following 
overview resolutions:
    5984x21140, 2992x10570, 1496x5285, 748x2643, 374x1322

All images use the same projection: EPSG 26910, aka UTM zone 10N with 
NAD83 datum.
The spatial extent observed is:
    543577.000 4150151.000 555545.000 4192431.000

A mapfile is created specifying three layers, named BARE, OVERVIEW, 
TILE, BOTH, each one reading from the corresponding raster. No 
reprojection is being done in the mapfile.


*** STORAGE SPACE

original.tif            483 MB
overviews.tif           646 MB
tiled.tif               487 MB
tiledandoverviews.tif   651 MB

As was expected, adding overviews increases the file size by some 33% 
while internal tiling adds only 4-5 MB to the file size.



*** RUNTIME SPEED

# full view of the region
shp2img -m mapfile.map -l BARE     -o bare-fullview.png
Time: 0.484s
shp2img -m mapfile.map -l OVERVIEW -o overview-fullview.png
Time: 0.460s
shp2img -m mapfile.map -l TILE     -o tile-fullview.png
Time: 1.187s
shp2img -m mapfile.map -l BOTH     -o both-fullview.png
Time: 0.457s

# 1 square kilometer pulled from the map
shp2img -m mapfile.map -l BARE     -e 549561 4150150 550561 4151150 -o 
bare-1km.png
Time: 1.072s
shp2img -m mapfile.map -l OVERVIEW -e 549561 4150150 550561 4151150 -o 
overview-1km.png
Time: 1.066s
shp2img -m mapfile.map -l TILE     -e 549561 4150150 550561 4151150 -o 
tile-1km.png
Time: 1.034s
shp2img -m mapfile.map -l BOTH     -e 549561 4150150 550561 4151150 -o 
both-1km.png
Time: 1.042s

# a 3km square pulled from the map
shp2img -m mapfile.map -l BARE     -e 547561 4149150 550561 4152150 -o 
bare-1km.png
Time: 0.839s
shp2img -m mapfile.map -l OVERVIEW -e 547561 4149150 550561 4152150 -o 
overview-1km.png
Time: 0.822s
shp2img -m mapfile.map -l TILE     -e 547561 4149150 550561 4152150 -o 
tile-1km.png
Time: 0.840s
shp2img -m mapfile.map -l BOTH     -e 547561 4149150 550561 4152150 -o 
both-1km.png
Time: 0.825s


At the full view, tiling actually hurt performance; presumably this was 
due to it seeking tiles and eventually not saving any time/seeks anyway.

At the close-up views, I was quite surprised to see that tiles and 
overviews did indeed have an effect, but that the effect was only in the 
dozens of microseconds.



I then repeated this experiment by fetching several more DOQQs and 
merging them into a 1 GB TIFF, then repeating the same generation and 
testing steps as above.

* The tiling and overview size increase was basically the same: 4 MB 
fixed growth for tiles, and 33% growth for overviews.

* The full views, all times remained in the same ratio, but increased by 
50%. Not bad, considering that there was a 100% increase in file size.

* For both the 1km and 2km extraction, the times were basically the same 
as for the 500 MB test above. The increased file size made a difference 
of 0.016s in the most dramatic case, which was that of the "bare" 
GeoTIFF. Other increases were 7 and 10ms.


I then repeated again with a 2 GB GeoTIFF (just keep adding counties til 
I hit the limit, right?).

Full view:
   BARE       0.806s
   OVERVIEW   0.759s
   TILE       2.238s
   BOTH       0.747s
2km square:
   BARE       0.867s
   OVERVIEW   0.832s
   TILE       0.840s
   BOTH       0.828s

Again the same results!


The conclusions are quite surprising:

* The presence of overviews makes little difference in the performance 
otherwise, about 1/20 of a second in the most dramatic case.

* Internal tiling seems likewise ineffectual, a difference of several 
microseconds.

* However, internal tiling will hurt performance substantially in a case 
where the request ends up grabbing the entire image anyway.


Is anybody else able to replicate these findings, or to achieve 
different results in a controlled experiment? Our hardware is rather 
beefy (8 CPU cores and a 4-disk RAID-5) so perhaps that's confounding my 
experiment? Perhaps these performance enhancers are mostly useful on a 
slower single-disk system?