[Gdal-dev] RE: DTED Level 2

[Martin Daly]

> Martin Daly wrote:
> > Frank,
> > 
> > One of our users is having performance problems with DTED 
> Level 2 via 
> > GDAL.
> 
> ... Martin goes on to mention that OpenEV can load DTED Level 
> 2 files fairly quickly, but in freestanding use of GDAL read 
> performance is terrible for DTED Level 2.
> 
> 
> Martin,
> 
> OpenEV does not (normally) use the dted.dll.  It is part of 
> OGDI but is essentially just cruft. In fact OpenEV
> 
> The problem you are encountering is that your GDAL raster 
> cache is smaller than the size of the DTED level 2 file.  The 
> DTED file is read by column and is cached in columns.  When a 
> scanline is read by an application each of the columns needs 
> to be read in but is kept in the cache.  However, if the 
> cache isn't large enough to hold all the blocks (the entire 
> DTED file) then by the time the last column is being read, 
> the first column will be expired from the cache. That means 
> that every scanline read means re-reading all the profiles.  
> Essentially this is a cache thrashing situation.
> 
> OpenEV has a different default GDAL cache size which is why 
> it is fairly fast.
> 
> So, my suggestion is that you review the GDAL cache size you 
> set from your application.  The default in GDAL is 10MB but I 
> think a DTED level 2 file is roughly 25MB.  You can verify 
> that this caching is the issue by calling
> GDALSetCacheMax() to set the max to 30MB or so.
> 
> One step that you might want to take would be to check the 
> block size of any file you open, and ensure that the GDAL 
> cache size is forced at least large enough to hold enough 
> blocks to satisfy one complete scanline request.
> 
> The danger of setting the GDAL cache too high is that it will 
> tend to soak up alot of memory when accessing large files.  
> But the danger of having the cache too small is terrible 
> cache thrashing in some cases.  The worse case is profile 
> oriented files like DTED.  But there can also be problems 
> with large files that have big tiles or that are all in one tile.

That worked a treat.

> I have taken the liberty of cc:ing my response to gdal-dev 
> since I think the answer is of broad interest.  I hope you don't mind.

You didn't, but I don't, so I did.

Thanks,
Martin