<div dir="ltr">Hi Even,<div><br></div><div>On Wed, Apr 26, 2017 at 8:32 PM, Even Rouault <span dir="ltr"><<a href="mailto:even.rouault@spatialys.com" target="_blank">even.rouault@spatialys.com</a>></span> wrote:<br></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><u></u>
<div style="font-family:monospace;font-size:9pt;font-weight:400;font-style:normal">
<p style="margin:0px;text-indent:0px">Hi,</p>
<p style="margin:0px;text-indent:0px"> </p>
<p style="margin:0px;text-indent:0px">I've experimented lately with an infrastructure to write drivers in Python :</p>
<p style="margin:0px;text-indent:0px"><a href="https://github.com/rouault/gdal2/tree/pythondrivers" target="_blank">https://github.com/rouault/<wbr>gdal2/tree/pythondrivers</a></p>
<p style="margin:0px;text-indent:0px"> </p>
<p style="margin:0px;text-indent:0px">This is a further step to the GDAL 2.2 new capability of writing pixel</p>
<p style="margin:0px;text-indent:0px">functions in Python in VRT files.</p>
<p style="margin:0px;text-indent:0px"> </p>
<p style="margin:0px;text-indent:0px">For now, only read-only vector capabilities are supported.</p>
<p style="margin:0px;text-indent:0px"> </p>
<p style="margin:0px;text-indent:0px">The Python drivers don't depend on the SWIG Python bindings, so that can potentially be used</p>
<p style="margin:0px;text-indent:0px">like native drivers, in all GDAL valid use cases: integrated in native code (GDAL command line</p>
<p style="margin:0px;text-indent:0px">utilities, QGIS, etc..), GDAL SWIG Python bindings, fiona, etc...</p>
<p style="margin:0px;text-indent:0px"> </p>
<p style="margin:0px;text-indent:0px">Example: given the dummy driver in</p>
<p style="margin:0px;text-indent:0px"><a href="https://github.com/rouault/gdal2/blob/pythondrivers/gdal/pythondrivers/ogr_MYPLUGIN.py" target="_blank">https://github.com/rouault/<wbr>gdal2/blob/pythondrivers/gdal/<wbr>pythondrivers/ogr_MYPLUGIN.py</a></p>
<p style="margin:0px;text-indent:0px"> </p>
<p style="margin:0px;text-indent:0px">$ GDAL_PYTHON_DRIVER_PATH=<wbr>pythondrivers ogrinfo -al -ro MYPLUGIN:</p>
<p style="margin:0px;text-indent:0px"> </p>
<p style="margin:0px;text-indent:0px">INFO: Open of `MYPLUGIN:'</p>
<p style="margin:0px;text-indent:0px">      using driver `MYPLUGIN' successful.</p>
<p style="margin:0px;text-indent:0px">Metadata:</p>
<p style="margin:0px;text-indent:0px">  foo=bar</p>
<p style="margin:0px;text-indent:0px"> </p>
<p style="margin:0px;text-indent:0px">Layer name: my layer</p>
<p style="margin:0px;text-indent:0px">Geometry: Point</p>
<p style="margin:0px;text-indent:0px">Feature Count: 5</p>
<p style="margin:0px;text-indent:0px">Extent: (2.100000, 49.000000) - (3.000000, 50.000000)</p>
<p style="margin:0px;text-indent:0px">Layer SRS WKT:</p>
<p style="margin:0px;text-indent:0px">GEOGCS["WGS 84",</p>
<p style="margin:0px;text-indent:0px">    DATUM["WGS_1984",</p>
<p style="margin:0px;text-indent:0px">        SPHEROID["WGS 84",6378137,298.257223563,</p>
<p style="margin:0px;text-indent:0px">            AUTHORITY["EPSG","7030"]],</p>
<p style="margin:0px;text-indent:0px">        AUTHORITY["EPSG","6326"]],</p>
<p style="margin:0px;text-indent:0px">    PRIMEM["Greenwich",0,</p>
<p style="margin:0px;text-indent:0px">        AUTHORITY["EPSG","8901"]],</p>
<p style="margin:0px;text-indent:0px">    UNIT["degree",0.<wbr>0174532925199433,</p>
<p style="margin:0px;text-indent:0px">        AUTHORITY["EPSG","9122"]],</p>
<p style="margin:0px;text-indent:0px">    AUTHORITY["EPSG","4326"]]</p>
<p style="margin:0px;text-indent:0px">FID Column = my_fid</p>
<p style="margin:0px;text-indent:0px">Geometry Column = geomField</p>
<p style="margin:0px;text-indent:0px">boolField: Integer(Boolean) (0.0)</p>
<p style="margin:0px;text-indent:0px">intField: Integer (0.0)</p>
<p style="margin:0px;text-indent:0px">int64Field: Integer64 (0.0)</p>
<p style="margin:0px;text-indent:0px">realField: Real (0.0)</p>
<p style="margin:0px;text-indent:0px">strField: String (0.0)</p>
<p style="margin:0px;text-indent:0px">strNullField: String (0.0)</p>
<p style="margin:0px;text-indent:0px">strUnsetField: String (0.0)</p>
<p style="margin:0px;text-indent:0px">binaryField: Binary (0.0)</p>
<p style="margin:0px;text-indent:0px">datetimeField: DateTime (0.0)</p>
<p style="margin:0px;text-indent:0px">OGRFeature(my layer):1</p>
<p style="margin:0px;text-indent:0px">  boolField (Integer(Boolean)) = 1</p>
<p style="margin:0px;text-indent:0px">  intField (Integer) = 2</p>
<p style="margin:0px;text-indent:0px">  int64Field (Integer64) = 1234567890123</p>
<p style="margin:0px;text-indent:0px">  realField (Real) = 1.23</p>
<p style="margin:0px;text-indent:0px">  strField (String) = foo</p>
<p style="margin:0px;text-indent:0px">  strNullField (String) = (null)</p>
<p style="margin:0px;text-indent:0px">  binaryField (Binary) = 010002</p>
<p style="margin:0px;text-indent:0px">  datetimeField (DateTime) = 2017/04/26 12:34:56.789+00</p>
<p style="margin:0px;text-indent:0px">  POINT (2 49)</p>
<p style="margin:0px;text-indent:0px"> </p>
<p style="margin:0px;text-indent:0px">[...]</p>
<p style="margin:0px;text-indent:0px"> </p>
<p style="margin:0px;text-indent:0px">OGRFeature(my layer):5</p>
<p style="margin:0px;text-indent:0px">  boolField (Integer(Boolean)) = 1</p>
<p style="margin:0px;text-indent:0px">  intField (Integer) = 6</p>
<p style="margin:0px;text-indent:0px">  int64Field (Integer64) = 1234567890123</p>
<p style="margin:0px;text-indent:0px">  realField (Real) = 1.23</p>
<p style="margin:0px;text-indent:0px">  strField (String) = foo</p>
<p style="margin:0px;text-indent:0px">  strNullField (String) = (null)</p>
<p style="margin:0px;text-indent:0px">  binaryField (Binary) = 010002</p>
<p style="margin:0px;text-indent:0px">  datetimeField (DateTime) = 2017/04/26 12:34:56.789+00</p>
<p style="margin:0px;text-indent:0px">  POINT (2 49)</p>
<p style="margin:0px;text-indent:0px"> </p>
<p style="margin:0px;text-indent:0px">The PYTHONSO env variable can be used to force a given Python shared library</p>
<p style="margin:0px;text-indent:0px">to be used, e.g PYTHONSO=<a href="http://libpython3.5m.so" target="_blank">libpython3.5m.so</a> . I've tested with Python 2.6, 2.7,</p>
<p style="margin:0px;text-indent:0px">3.2 and 3.5</p>
<p style="margin:0px;text-indent:0px"> </p>
<p style="margin:0px;text-indent:0px">Currently there's some copy&paste from the VRT Python derived band code that I'd</p>
<p style="margin:0px;text-indent:0px">like to factor (the logic to dynamically load the Python interpreter and load</p>
<p style="margin:0px;text-indent:0px">needed symbols)</p>
<p style="margin:0px;text-indent:0px"> </p>
<p style="margin:0px;text-indent:0px">The mechanism could probably be extended to support write operations</p>
<p style="margin:0px;text-indent:0px">and the raster side too. I'm not sure how far I'll go with that, but I'll probably</p>
<p style="margin:0px;text-indent:0px">write a RFC at some point with what exists.</p>
<p style="margin:0px;text-indent:0px"> </p>
<p style="margin:0px;text-indent:0px">Feedback welcome.</p></div></blockquote><div><br></div><div>The sketch of the Python interface for drivers looks fine to me. Some of the function signatures could be more Pythonic, but these are not going to be called from Python code (correct?), and that's probably only a aesthetic issue.</div><div><br></div><div>I still don't understand what the problems are that are best solved by embedding Python in GDAL. To be frank: I'm dreading the day that someone delivers to me a VRT file with a Python script embedded in it. These scripts are unlikely to be well tested (what tools would one use?) and very difficult to debug when they fail.</div><div><br></div><div>What good is a VRT that only opens with Python 2? Or only with Python 3? Or only if Rasterio or ArcPy is installed? How does a VRT with embedded Python specify what its requirements are? Specifically, how would a VRT specify that it requires Numpy or scikit-learn, or OpenCV, and how would a user install the Python or other library other requirements of a VRT? All of the above are solved (for some value of "solved") for the Python *extension* use case, but are unsolved when it comes to embedding Python in GDAL.</div><div><br></div><div>Respectfully,</div><div><br></div></div>-- <br><div class="gmail_signature"><div dir="ltr">Sean Gillies</div></div>
</div></div>