[Gdal-dev] [RFC] RasterIO speedup patch

[Frank Warmerdam]

Tim Beckmann wrote:
> Frank and the rest of the list,
> 
> We've been working to incorporate GDAL into our software.  One drawback 
> we've found so far is that the performance of gdal when it needs to do 
> data type conversions is extremely slow.  Here is a patch that I've 
> generated that seems to work to improve the performance.  It works by 
> changing the GDALRasterBand::IRasterIO to attempt to convert more than a 
> single pixel for every call to GDALCopyWords.  For the testing I've done, 
> it seems to work well.  I'm not sure if I've covered all the possible 
> combinations though.  There are still cases where it falls back to calling 
> GDALCopyWords once for each pixel.
> 
> Some preliminary timings show that the runtime to convert a 100 MB Float32 
> image to Byte data took 49 seconds with the original code, but only 27 
> seconds with the patch below.  This is using our unreleased LAS driver on 
> an SGI Origin 2000 with 250 MHz processors.
> 
> Any comments?

Tim,

Based on a quick inspection the code looks OK.  Could you submit this
patch into bugzilla as an enhancement request?  I will have Andrey apply
and test it.

> Oh yeah, once for the record - I really dislike the naming convention used 
> for variables in the routine.  It makes it much harder to understand the 
> code.  >  Also, some of the variable names could use some comments to explain
> what they do (I'd add them, but I'm not 100% sure).  Example: the 
> nSrcPixelOffset parameter to GDALCopyWords.  If I understand it correctly, 
> it isn't an offset, it is the value to stride through the data with.

Well, I think of it as the offset from the beginning of one pixel to the
beginning of the next.  I would agree of course that more internal
documentation would be helpful.

> I also have another set of changes that cut the time nearly in half 
> again...  But one step at a time.

I look forward to receiving them.

There is clearly lots of room for improvement in GDAL, at least in
cases where up/down sampling is done and/or when data type conversion is
done.   I have tried to ensure that the 1:1 IO logic (especially on
block boundaries) is efficient.

I would also add that I think I need to write some regression tests that
exercise lots of the cases so that when we tune it we can have greater
confidence that we haven't broken anything.

Best regards,

-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent