[gdal-dev] OGR drivers written in Python

Thu Apr 27 14:00:38 PDT 2017

Hi Even and Sean

Allowing OGR drivers written in Python looks IMHO very promising!

I actually can't follow Seans concerns (except for one, see below): Is
it the fear that Python programmers step in who are less skilled?
There is quite some experience with QGIS plugins and yes, there are
many disfunct and outdated plugins there. But this allowed innovative
QGIS extensions which eventually have been moved to the QGIS core or
converted to a QGS C++ extension.

One important concern remains in fact: Keeping dependencies under
control! But now it's the right time to introduce some kinde of
requirements file mechanism, isn't it?

There ar quite some use cases which come to my mind; here's the most
important one: There is no real data provider support in QGIS for
Python extensions. OGR drivers as Python extensions could be a
solution. As a side effect, these providers are not only available in
QGIS GUI but also at the ogr CLI (I hope).

Just my 2 cents
:Stefan

2017-04-27 14:38 GMT+02:00 Even Rouault <even.rouault at spatialys.com>:
> Sean,
>
>
>
>>
>
>> The sketch of the Python interface for drivers looks fine to me. Some of
>
>> the function signatures could be more Pythonic, but these are not going to
>
>> be called from Python code (correct?),
>
>
>
> Yes, this is driver code that is mostly dedicated to be called from GDAL
> core itself. That could potentially be also called externally for unit
> testing, who knows, but with some caveats, like the gdal_base module
> containing the base classes being a built-in module inside GDAL. But it can
> be easily mocked up as being essentially a no-op. Currently the only thing
> it has is the OGRLayer.feature_count() method that can be called from the
> specialized feature_count() method to ultimately call the C++ base method
> OGRLayer::GetFeatureCount() that does the slow part of iterating through all
> features and taking them into account if they match the spatial and/or
> attribute filters.
>
>
>
>> and that's probably only a aesthetic
>
>> issue.
>
>
>
> Your suggestions are welcome to improve that. Note that I did some effort to
> apply PEP 8 naming conventions (well I tried at least :-))
>
>
>
>>
>
>> I still don't understand what the problems are that are best solved by
>
>> embedding Python in GDAL. To be frank: I'm dreading the day that someone
>
>> delivers to me a VRT file with a Python script embedded in it. These
>
>> scripts are unlikely to be well tested (what tools would one use?) and
>> very
>
>> difficult to debug when they fail.
>
>>
>
>> What good is a VRT that only opens with Python 2? Or only with Python 3?
>> Or
>
>> only if Rasterio or ArcPy is installed? How does a VRT with embedded
>> Python
>
>> specify what its requirements are? Specifically, how would a VRT specify
>
>> that it requires Numpy or scikit-learn, or OpenCV, and how would a user
>
>> install the Python or other library other requirements of a VRT? All of
>> the
>
>> above are solved (for some value of "solved") for the Python *extension*
>
>> use case, but are unsolved when it comes to embedding Python in GDAL.
>
>
>
> This is a good point, especially regarding VRT. I see those Python
> capabilities are mostly a way of quickly prototyping stuff for your internal
> needs. And I know at least one user using VRT Python pixel functions for
> such prototyping work (to display the result of the simulation of some
> physical processus in QGIS)
>
>
>
> Regarding drivers themselves, this could be the same use case. You have that
> custom in-house format for which you don't really want to write a dedicated
> converter to a more standard format, because you have too much of that data
> and can't afford the conversion stage. Or because you just need to code the
> reading part and are happy that ogr2ogr handles the conversion to
> shapefile/GPKG/GeoJSON/whatever for you.
>
> Someone could also write a driver in Python for less esoteric formats,
> because that's the language (s)he masters and/or there's an underlying
> Python lib that does the hard job of decyphering the format and you just
> need to expose it through OGR formalism. In that case, you could potentially
> publish the plugin as a "proper" Python module with adequate requirement
> specification through your favorite Python distribution & packaging system.
> Like "pip install ogr-myoddformatdriver" or "apt-get install
> python-ogr-myoddformatdriver". In the pip use case, that would probably
> require addition in GDAL core of some logic to know where to find those
> modules since I don't think you can constraint a file to be installed in the
> same directory as files from other packages (can you?). In the apt-get
> install case, that would mostly require to express package dependencies in
> the .deb formalism and installing one of several files in
> /usr/lib/gdalplugins/2.3
>
>
>
> Or perhaps there isn't any valid use case after all. I just tried it for fun
> and because it could be done :-)
>
>
>
> Even
>
>
>
> --
>
> Spatialys - Geospatial professional services
>
> http://www.spatialys.com
>
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev