[gdal-dev] VRT derived band pixel functions written in Python
kassies at gmail.com
Tue Sep 13 02:07:39 PDT 2016
I overlooked the fact that it still moves through Python, is that the 'only'
hurdle preventing parallel IO? Since gdalwarp for example has the -multi
flag, it seems as if GDAL is capable of it, or is that a
Numba has several options which might eliminate using Python during
execution. There are c-callbacks:
It probably works by using the Numpy C-API (which i have zero experience
with). I don't know if its possible that other programs, like GDAL, can also
use those compiled functions without moving to Python first.
There is also ahead-of-time compilation (AOT):
AOT has the benefit that users only need Numpy as a dependency, and don't
need Numba/llvm. There some drawbacks as well, like no longer being able to
compile optimizations for the hardware its running on.
Even Rouault-2 wrote
> Le mardi 13 septembre 2016 09:02:09, Rutger a écrit :
> I've only scratched up the surface of Numba (didn't know it 2 days ago). I
> guess this might work (might only be interested if the
> VRTDerivedRasterBand::IRasterIO() is called with a big enough region.
> depends on the pixel access pattern of upper layers). The VRT drivers just
> calls a Python function that takes numpy arrays and a few extra args.
>> And would that give parallel
>> processing for both IO and calculations?
> Only calculations. I/O is done before going to Python and after returning
> Actually... if you specify zero source in the VRT, then it is up to you to
> the read operations the way you like in Python, so you could possibly
> parallelize them there.
> Spatialys - Geospatial professional services
> gdal-dev mailing list
> gdal-dev at .osgeo
View this message in context: http://osgeo-org.1560.x6.nabble.com/gdal-dev-VRT-derived-band-pixel-functions-written-in-Python-tp5285323p5285482.html
Sent from the GDAL - Dev mailing list archive at Nabble.com.
More information about the gdal-dev