[gdal-dev] Slow access to compressed HDF4 files

[Joaquim Luis]

Hi,
I reopened ticket http://trac.osgeo.org/gdal/ticket/2208 which reports a 
very low efficiency issue reading (some ?) hdf files

An example file can be get from
ftp://data.nodc.noaa.gov/pub/data.nodc/pathfinder/Version5.0/Monthly/1991/199101.s04m1pfv50-sst-16b.hdf

The timings below (in seconds) respect my MEX for doing gdal read inside 
Matlab. That program is called gdalread.
Hardware is a 2.5 GHz core 2 duo

Let tt.hdf = 199101.s04m1pfv50-sst-16b.hdf

tic,Z = gdalread('c:\tt.hdf');toc
elapsed_time =  21.5

However, using the internal Matlab solution one get:

tic,sst=hdfread('c:\tt.hdf', 'sst', 'index', {[1 1],[1 1], [4096 8192]});toc
elapsed_time =  0.8120

Just to show that the problem is not in "gdalread" I converted the hdf 
file to getiff (lixo.tiff), which took about 25 sec

tic,Z = gdalread('c:\lixo.tiff');toc
elapsed_time =   2.0930

The slow performance is noticeable also when fetching only the file's 
metadata

tic,Z = gdalread('c:\tt.hdf','-M');toc
elapsed_time = 1.0310

% Matlab's way
tic,Z=hdfinfo('c:\tt.hdf');toc
elapsed_time =  0.0310

% On the geotiff file
tic,Z = gdalread('c:\lixo.tiff','-M');toc
elapsed_time =  0.0310

Joaquim Luis