[gdal-dev] OGR drivers written in Python

[Sean Gillies]

Hi Even,

On Wed, Apr 26, 2017 at 8:32 PM, Even Rouault <even.rouault at spatialys.com>
wrote:

> Hi,
>
>
>
> I've experimented lately with an infrastructure to write drivers in Python
> :
>
> https://github.com/rouault/gdal2/tree/pythondrivers
>
>
>
> This is a further step to the GDAL 2.2 new capability of writing pixel
>
> functions in Python in VRT files.
>
>
>
> For now, only read-only vector capabilities are supported.
>
>
>
> The Python drivers don't depend on the SWIG Python bindings, so that can
> potentially be used
>
> like native drivers, in all GDAL valid use cases: integrated in native
> code (GDAL command line
>
> utilities, QGIS, etc..), GDAL SWIG Python bindings, fiona, etc...
>
>
>
> Example: given the dummy driver in
>
> https://github.com/rouault/gdal2/blob/pythondrivers/gdal/
> pythondrivers/ogr_MYPLUGIN.py
>
>
>
> $ GDAL_PYTHON_DRIVER_PATH=pythondrivers ogrinfo -al -ro MYPLUGIN:
>
>
>
> INFO: Open of `MYPLUGIN:'
>
> using driver `MYPLUGIN' successful.
>
> Metadata:
>
> foo=bar
>
>
>
> Layer name: my layer
>
> Geometry: Point
>
> Feature Count: 5
>
> Extent: (2.100000, 49.000000) - (3.000000, 50.000000)
>
> Layer SRS WKT:
>
> GEOGCS["WGS 84",
>
> DATUM["WGS_1984",
>
> SPHEROID["WGS 84",6378137,298.257223563,
>
> AUTHORITY["EPSG","7030"]],
>
> AUTHORITY["EPSG","6326"]],
>
> PRIMEM["Greenwich",0,
>
> AUTHORITY["EPSG","8901"]],
>
> UNIT["degree",0.0174532925199433,
>
> AUTHORITY["EPSG","9122"]],
>
> AUTHORITY["EPSG","4326"]]
>
> FID Column = my_fid
>
> Geometry Column = geomField
>
> boolField: Integer(Boolean) (0.0)
>
> intField: Integer (0.0)
>
> int64Field: Integer64 (0.0)
>
> realField: Real (0.0)
>
> strField: String (0.0)
>
> strNullField: String (0.0)
>
> strUnsetField: String (0.0)
>
> binaryField: Binary (0.0)
>
> datetimeField: DateTime (0.0)
>
> OGRFeature(my layer):1
>
> boolField (Integer(Boolean)) = 1
>
> intField (Integer) = 2
>
> int64Field (Integer64) = 1234567890123
>
> realField (Real) = 1.23
>
> strField (String) = foo
>
> strNullField (String) = (null)
>
> binaryField (Binary) = 010002
>
> datetimeField (DateTime) = 2017/04/26 12:34:56.789+00
>
> POINT (2 49)
>
>
>
> [...]
>
>
>
> OGRFeature(my layer):5
>
> boolField (Integer(Boolean)) = 1
>
> intField (Integer) = 6
>
> int64Field (Integer64) = 1234567890123
>
> realField (Real) = 1.23
>
> strField (String) = foo
>
> strNullField (String) = (null)
>
> binaryField (Binary) = 010002
>
> datetimeField (DateTime) = 2017/04/26 12:34:56.789+00
>
> POINT (2 49)
>
>
>
> The PYTHONSO env variable can be used to force a given Python shared
> library
>
> to be used, e.g PYTHONSO=libpython3.5m.so . I've tested with Python 2.6,
> 2.7,
>
> 3.2 and 3.5
>
>
>
> Currently there's some copy&paste from the VRT Python derived band code
> that I'd
>
> like to factor (the logic to dynamically load the Python interpreter and
> load
>
> needed symbols)
>
>
>
> The mechanism could probably be extended to support write operations
>
> and the raster side too. I'm not sure how far I'll go with that, but I'll
> probably
>
> write a RFC at some point with what exists.
>
>
>
> Feedback welcome.
>

The sketch of the Python interface for drivers looks fine to me. Some of
the function signatures could be more Pythonic, but these are not going to
be called from Python code (correct?), and that's probably only a aesthetic
issue.

I still don't understand what the problems are that are best solved by
embedding Python in GDAL. To be frank: I'm dreading the day that someone
delivers to me a VRT file with a Python script embedded in it. These
scripts are unlikely to be well tested (what tools would one use?) and very
difficult to debug when they fail.

What good is a VRT that only opens with Python 2? Or only with Python 3? Or
only if Rasterio or ArcPy is installed? How does a VRT with embedded Python
specify what its requirements are? Specifically, how would a VRT specify
that it requires Numpy or scikit-learn, or OpenCV, and how would a user
install the Python or other library other requirements of a VRT? All of the
above are solved (for some value of "solved") for the Python *extension*
use case, but are unsolved when it comes to embedding Python in GDAL.

Respectfully,

-- 
Sean Gillies
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20170427/26dc2fad/attachment-0001.html>