TIFF Performance tests: Found it

[Gregor Mosheh]

Per Frank's and Umberto's suggestion, I repeated my experiment on much 
less beefy hardware: a dual-core laptop with one hard disk, running in 
VMWare. I'll put the conclusion first, for folks like myself who are 
impatient. :)


*** Conclusion:

Both tiling and overviews are useful, but only on machines with resource 
shortages, such as slow disks or a lack of spare RAM for caching.

Hardly surprising is the conclusion that throwing more RAM into the 
machine makes a bigger difference than any of these data-tweaking 
approaches.

Although it's intuitive that faster hardware makes for better results,
I was surprised to find a level of "fast enough hardware" that makes 
these performance enhancements useless.


--- And now the more boring experimental design stuff. :)

This test compares the effect of overviews and internal tiling on the 
performance of a TIFF raster data source,
on a low-end system where disk performance is substandard and filesystem 
caching is minimal due to low RAM.


*** PLATFORM

The hardware is a mid-budget laptop: Core Duo 1.7GHz, single SATA hard 
disk, 1 GB RAM.

The software is HostGIS Linux 4.1 running under VMWare Server. The VM 
was given 512 MB of memory, and the disk file (virtual IDE) was created 
entirely before booting the machine.

(Hey, if the goal is a system that's slow enough to need performance 
enhancement, VMWare is as good as it gets <grin>)


*** DATA SETUP

The data is 48 USGS DOQQs (black-n-white), downloaded and merged into a 
single GeoTIFF using gdal_merge.py. I then created three copies of the 
TIFF: with overviews, with internal tiling, and with both.

All images use the same projection: UTM zone 10N with NAD83 datum (EPSG 
26910).
The spatial extent observed is:
    543502 4150151 577642 4206427

The commands used were as follows:
    gdal_merge.py -o original.tif o37122e2ne.tif o37122e2nw.tif 
o37122e2se.tif o37122e2sw.tif o37122e3se.tif o37122e3sw.tif \
               o37122e4ne.tif o37122e4nw.tif o37122e4se.tif 
o37122e4sw.tif o37122f2ne.tif o37122f2nw.tif o37122f2se.tif o37122f3ne.tif \
               o37122f3nw.tif o37122f3sw.tif o37122f4ne.tif 
o37122f4nw.tif o37122f4se.tif o37122f4sw.tif o37122g2ne.tif o37122g2nw.tif \
               o37122g2se.tif o37122g2sw.tif o37122g3ne.tif 
o37122g3nw.tif o37122g3se.tif o37122g3sw.tif o37122g4ne.tif o37122g4nw.tif \
               o37122g4se.tif o37122g4sw.tif o37122h2ne.tif 
o37122h2nw.tif o37122h2se.tif o37122h2sw.tif o37122h3ne.tif o37122h3nw.tif \
               o37122h3se.tif o37122h3sw.tif o37122h4ne.tif 
o37122h4nw.tif o37122h4se.tif o37122h4sw.tif o37122e3ne.tif o37122e3nw.tif
    gdal_translate -co "TILED=YES" original.tif tiled.tif
    cp original.tif overviews.tif
    gdaladdo overviews.tif 2 4 8 16 32
    cp tiled.tif tilesandoverviews.tif
    gdaladdo tilesandoverviews.tif 2 4 8 16 32

The resulting "original" image is 34140 x 56276 pixels, 1.79 GB in size.
The ones with overviews have them are 2.04 GB and have these overviews:
    17070x28163, 8535x14082, 4268x7041, 2134x3521, 1067x1761


*** TESTING

A mapfile is created specifying three layers, named BARE, OVERVIEW, 
TILE, BOTH, each one reading from the corresponding raster. No 
reprojection is being done in the mapfile.

The mapfile specifies an output image size of 1000 x 1000 pixels.

The virtual machine was rebooted after each test, to ensure that 
filesystem caching was not occurring, and each test was repeated several 
times in random order to ensure that no unknown time-sensitive variable 
was affecting the first or last in each series.

# full view of the region
shp2img -m mapfile.map -l BARE     -o output.png
Time: 40.196s 27.812s 28.473s
shp2img -m mapfile.map -l OVERVIEW -o output.png
Time: 1.787s 2.947s 2.905s
shp2img -m mapfile.map -l TILE     -o output.png
Time: 78.947s 77.175s 75.600s
shp2img -m mapfile.map -l BOTH     -o output.png
Time: 2.572s 3.647s 1.828s

Ah-ha! Now that's what we're expecting!
Tiling hurts performance in a case where you end up showing the whole 
thing anyway.
Overviews have dramatic (20 to 1) effect on rendering the full view.

# 1 square kilometer pulled from the map
shp2img -m mapfile.map -l BARE     -e 549561 4150150 550561 4151150 -o 
output.png
Time: 4.514s 5.903s 5.458s
shp2img -m mapfile.map -l OVERVIEW -e 549561 4150150 550561 4151150 -o 
output.png
Time: 8.225s 2.980s 3.556s
shp2img -m mapfile.map -l TILE     -e 549561 4150150 550561 4151150 -o 
output.png
Time: 2.529s 1.852s 1.868s
shp2img -m mapfile.map -l BOTH     -e 549561 4150150 550561 4151150 -o 
output.png
Time: 2.533s 1.956s 1.970s

Again, not too surprising, though the margin is a lot less dramatic than 
the full-view tests. Use of tiles
gave a 2:1 improvement over not using tiles, and use of overviews gave a 
lesser but still useful boost.