[Gdal-dev] MrSID Driver Optimization

[Frank Warmerdam]

Folks,

Many folks have run into problems with the performance of the MrSID 
driver in GDAL.  This essentially because the MrSID SDK 4.x based
version had no optimization for subwindowed access and it used
a blocksize of imagewidth * 64.  That meant to extra a 200x200 
window you would end up reading several blocks the width of the
whole image.   Not a big issue for small images, but for 50K x 50K
images alot of extra data was being processed.

I have made two major changes:
 1) I now use a block size of 1024x128.   So cached reads will
     end up having to do several reads for a full image width request,
     but small region reads that are cached will be much more reasonable.

 2) I have overridden the RasterIO() call to do direct IO (skipping the
     block cache) for many cases with the notable exception of 
     single scanline requests. 

Things to be wary of: 
 o Doing lots of small RasterIO() that are treated directly wil have
    high overhead.   You can forced cached IO by setting the 
    GDAL_FORCE_CACHING option to YES. 

 o For very large images that are read one scanline at a time it is
    critical that you have the GDAL cache size set large enough to 
    whole one whole row of blocks (times the number of bands).  
    Otherwise you wll have a cache thrashing situation where the
    blocks are re-read for each scanline requested! 

In other notes, I have split out the MrSID SDK 3.x support into a
file named mrsiddataset3.cpp.  I haven't tried building it, and there is
no automatic build support for it.  If you are using the old 3.x 
SDK you could likely rename mrsiddataset3.cpp to mrsiddataset.cpp
and fiddle a bit till it works.  I didn't want to get rid of the MrSID 3 
SDK code but I am not intending to actively support it anymore.  It
was too confusing have all the code for both cases in one giant
file. 

Let me or the list know if you run into problems with the new MrSID
driver implementation. 

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent