[Gdal-dev] comparing overviews (pyramids) - arcgis, gdal, ossim

[Matt Wilkie]

Hi All,

I've just run an experiment on the various methods available to me of 
adding overviews, pyramids in arcgis parlance, to rasters and thought 
others might be interested in the results.

Source data: 14 LZW compressed tiff images, a mix of 8bit grey and RGB.

source                                      1.710 gb
arcgis LZ7 compressed (.rrd)                2.069 gb
ossim img2rr compress deflate (.ovr)        2.348 gb
gdaladdo with COMPRESS_OVERVIEW LZW (.tif)  2.623 gb
gdaladdo with USE_RRD YES (.aux)            3.169 gb

(totals are source + overviews together)


So, the best absolute storage savings are achieved wimith arcgis.

The best open source option in terms of storage space is ossim, however 
only ossim tools will use the overviews. Ossim is also the fastest of 
the bunch (subjective, I didn't think to actually measure the process 
until halfway through).

The best open source option for sharing amoung multiple applications is 
gdaladdo with embedded overviews in the tiff, but be careful of bug 
#1758 http://trac.osgeo.org/gdal/ticket/1758

Gdal tools (e.g. openev) will use arcgis .rrd even if it can't make 
them. So if you have a mixed shop this is a good choice.

It would be nice to see more collaboration between ossim and gdal. The 
external .ovr file used by ossim is geotiff so I imagine this would not 
be that hard(?).

Ossim is generating 10-12 overviews per file while arcgis does 6-7 and 
gdal 4 (I used gdaladdo ... 2 4 8 16 as per the example on the manual 
page). Since the ossim .ovr is a deflate compressed geotiff I am 
mystified as to why it's images are so much smaller than gdal's.  On a 
single image, there is less than a 1mb difference in 366mb using 
gdaladdo deflate vs lzw compression.

source lzw tif    218 mb
arcgis: 7 overs   255 mb
ossim:  12 overs  272 mb
gdal tif: 4 overs 367 mb

For more strangeness yet, gdaladdo doesn't like deflate compressed tiff:

  gdal_translate -co compress=deflate image.tif deflated.tif

  gdaladdo deflated.tif 2 4 8 16
   0...10...20...30.ERROR 1: ZIPDecode:ZLib error: (null)
   ERROR 1: TIFFReadEncodedTile() failed.
   ERROR 1: IReadBlock failed at X offset 0, Y offset 1
   ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 1
   ERROR 1: ZIPDecode:Decoding error at scanline 128d, unknown
    compression method
   ERROR 1: TIFFReadEncodedTile() failed.

And same error using '--config compress_overview deflate'

cheers,

-- 
matt wilkie
--------------------------------------------
Geographic Information,
Information Management and Technology,
Yukon Department of Environment
10 Burns Road * Whitehorse, Yukon * Y1A 4Y9
867-667-8133 Tel * 867-393-7003 Fax
http://environmentyukon.gov.yk.ca/geomatics/
--------------------------------------------