[Gdal-dev] Re: DTED Level 2

[Frank Warmerdam]

Martin Daly wrote:
> Frank,
> 
> One of our users is having performance problems with DTED Level 2 via
> GDAL.

... Martin goes on to mention that OpenEV can load DTED Level 2 files
fairly quickly, but in freestanding use of GDAL read performance is
terrible for DTED Level 2.


Martin,

OpenEV does not (normally) use the dted.dll.  It is part of OGDI but is
essentially just cruft. In fact OpenEV

The problem you are encountering is that your GDAL raster cache is smaller
than the size of the DTED level 2 file.  The DTED file is read by column
and is cached in columns.  When a scanline is read by an application each
of the columns needs to be read in but is kept in the cache.  However, if the
cache isn't large enough to hold all the blocks (the entire DTED file) then
by the time the last column is being read, the first column will be expired
from the cache. That means that every scanline read means re-reading all the
profiles.  Essentially this is a cache thrashing situation.

OpenEV has a different default GDAL cache size which is why it is fairly fast.

So, my suggestion is that you review the GDAL cache size you set from your
application.  The default in GDAL is 10MB but I think a DTED level 2 file is
roughly 25MB.  You can verify that this caching is the issue by calling
GDALSetCacheMax() to set the max to 30MB or so.

One step that you might want to take would be to check the block size of any
file you open, and ensure that the GDAL cache size is forced at least large
enough to hold enough blocks to satisfy one complete scanline request.

The danger of setting the GDAL cache too high is that it will tend to soak up
alot of memory when accessing large files.  But the danger of having the
cache too small is terrible cache thrashing in some cases.  The worse case
is profile oriented files like DTED.  But there can also be problems with
large files that have big tiles or that are all in one tile.

I have taken the liberty of cc:ing my response to gdal-dev since I think the
answer is of broad interest.  I hope you don't mind.

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent