[gdal-dev] Help with idiomatic GDAL solution for raster algebra benchmark

[Alex HighViz]

On 19/06/2018 18:41, Mateusz Loskot wrote:
> On 19 June 2018 at 11:50, Alex HighViz <alexhighviz at hotmail.com> wrote:
>> On 19/06/2018 10:28, Mateusz Loskot wrote:
>>> On 19 June 2018 at 11:22, Alex HighViz <alexhighviz at hotmail.com> wrote:
>>>> Regrettably I overstated the performance in my previous post, due to a bug that is now solved. With the solved bug, Pronto is about 50% slower than GDAL directly. I believe this is still good considering the benefits, but not as glamorous as I first thought. I have also applied it on a set of 100000 x 1000 rasters and the results are consistent.
>>>>
>>>>
>>>>
>>>> I was wondering if the slow performance may be due to the standard iterators
>>>> debugging enabled.
>>>>
>>>>
>>>> Best regards,

I have looked at the benchmark a bit further, and considered that it 
might have to do with the way in which the different solutions access 
the data.

The Pronto Raster code uses band->GetLockedBlockRef(...), whereas my the 
GDAL-only reference uses band->ReadBlock(...).

This means that Pronto uses the cache and the GDAL-only reference 
doesn't.  My next thought was to use band->RasterIO(...) as the 
reference because that is more appropriate, considering that it also 
uses the cache. And finally, I used band->GetLockedBlockRef() in a GDAL 
only solution.

I ended up with the following timings:

GDAL-Only using ReadBlock(...)                         :   7.7 seconds
GDAL-Only using RasterIO(...)                            : 30.1 seconds
GDAL-Only using GetLockedBlockRef(...)         :   8.9 seconds
Pronto Raster using GetLockedBlockRef(...)     : 10.6 seconds

I suppose the relatively poor performance of RasterIO is known 
(https://lists.osgeo.org/pipermail/gdal-dev/2008-March/016357.html and 
https://trac.osgeo.org/gdal/ticket/2266), although I don't understand 
the reason for it.