[gdal-dev] Help with idiomatic GDAL solution for raster algebra benchmark

[Alex HighViz]

Thanks for the suggestions. These extra benchmarks should be feasible in the next week. Especially if I manage to get started properly with Google benchmark. I need to look into sse2/ave2. My feeling is that will be easier and more effective for the reference case than for Pronto, because pronto is geared to express all operations at the pixel level.
________________________________
From: Even Rouault <even.rouault at spatialys.com>
Sent: 14 June 2018 21:22:36
To: gdal-dev at lists.osgeo.org
Cc: Alex HighViz; Hagen-Zanker AH Dr (Civil & Env. Eng.)
Subject: Re: [gdal-dev] Help with idiomatic GDAL solution for raster algebra benchmark

Hi,

Interesting work.

A few thoughts:

- the benchmark_3_rasters_reference() could receive a few comments for non-
advanced users to mention that it is correct only if all the rasters use Byte
datatype, and all have the same block size.

- It would be interesting to see a variant of this reference benchmark that
does no computation at all, that is to say remove the for (int iY = 0; iY <
nYValid; iY++) loop . That would show how much CPU computation time is really
involved

- 1000x1000 is somewhat small. Perhaps benchmark on larger rasters

- with some tricks, it should probably be possible to have a compiler generate
SSE2/AVX2 instructions for the computation loop of the reference benchmark,
since it looks like an excellent candidate for vectorization. Or that could be
done at hand with a few _mm_ intrinsincs. Do you think that Pronto could use
that ?

- it would be interested if you could also benchmark a VRT with Python numpy-
based pixel function, and enable Numba. See
http://www.gdal.org/gdal_vrttut.html#gdal_vrttut_derived_python (or just pure
numpy based computation + Numba)

- the benchmark only involve integer computations. What if you do linear
combination of bands with floating-point coefficients for example ?

Even

> Dear all,
>
> I few times I have posted to the list trying to promote the idea of
> providing iterators over pixels in a raster band , and more generally to
> make raster data accessible using (future) standard conforming ranges. It
> would make implementing algorithms on raster data a lot more intuitive.
> These ideas are implemented in Pronto Raster which is an OSGeo Community
> C++ library. Feedback in this list has been that such a solution would
> incur costly computational overheads.
>
> I now have some preliminary benchmark results, where I compare Pronto Raster
> solutions for a simple raster overlay operation (OUT = 3 * A + B * C) to a
> direct and idiomatic GDAL implementation. The results seem to indicate that
> overheads can be negligible, depending on which Pronto Raster functions are
> used. I would very much appreciate it if some more experienced GDAL C++
> users could look at my "idiomatic GDAL implementation" to see if it really
> is what it claims to be and I am not overstating the results. I can use
> your help.
>
> I'd also be interested to hear any opinion about these results and the costs
> / benefits associated with providing pixel ranges for raster bands.
>
> For details on the benchmark see here:
> https://github.com/ahhz/raster/blob/master/docs/_posts/2018-06-14-Prelimina
> ry-benchmark-results-are-promising.md or here:
> http://ahhz.github.io/raster/Preliminary-benchmark-results-are-promising/
>
> Many thanks,  Alex


--
Spatialys - Geospatial professional services
http://www.spatialys.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20180614/75d56778/attachment-0001.html>