[gdal-dev] VRT derived band pixel functions written in Python
James Ramm
jamessramm at gmail.com
Thu Sep 15 08:46:37 PDT 2016
I agree...could be slightly inelegant to figure out what version of python
is used though...A system call to `where python` on windows would return
the interpreter on the path, and you could replace the '.exe' with '.dll'
(this is assuming python dll is always alongside the exe, but i think this
is the case on all windows distributions), or Evens' suggest by calling
python.
I cant think of an (easy) way to do it without a system call though.
On 15 September 2016 at 14:56, Sean Gillies <sean at mapbox.com> wrote:
> Hi Even, James:
>
> I suspect users will be find it very surprising if `which python` and the
> python interpreter used by VRT are not the same. Imagine starting python in
> an environment created by virtualenv or conda that contains extension
> modules like scikit-image, scikit-learn, &c. You import osgeo.gdal, open
> your super duper VRT with `gdal.Open()`, and get an error because your
> environment's extension modules aren't available in the VRT script. This
> could be tough to debug for a beginning to intermediate level Python
> programmer.
>
>
> On Thu, Sep 15, 2016 at 10:29 AM, James Ramm <jamessramm at gmail.com> wrote:
>
>> excellent, working setting the PYTHONSO variable. Trying to match the
>> version of python on the path would be smart, but it is probably 'good
>> enough' to give PYTHONSO some prominence in the documentation - some kind
>> of warning on how python is discovered perhaps?
>>
>> On 14 September 2016 at 20:20, James Ramm <jamessramm at gmail.com> wrote:
>>
>>> Ah, that makes sense. I'll have to try the config option in the morning,
>>> but it sounds like that could be it
>>>
>>> On 14 Sep 2016 8:06 p.m., "Even Rouault" <even.rouault at spatialys.com>
>>> wrote:
>>>
>>>> Le mercredi 14 septembre 2016 20:50:04, James Ramm a écrit :
>>>> > Yes, it is loading a different python....The path printed by sys.path
>>>> is
>>>> > different to if I open the command window and type:
>>>> >
>>>> > python
>>>> >
>>>> > >>> import sys
>>>> > >>> print(sys.path)
>>>> >
>>>> > Gdal and the python bindings were compiled with vs2015 and python
>>>> 3.5, and
>>>> > I can correctly import python in the 3.5 interpreter, yet somehow a
>>>> > different python DLL (2.7) is being loaded at runtime.
>>>>
>>>> Might be related to the default try order in
>>>>
>>>> const char* const apszPythonSO[] = { "python27.dll",
>>>> "python26.dll",
>>>> "python34.dll",
>>>> "python35.dll",
>>>> "python36.dll",
>>>> "python33.dll",
>>>> "python32.dll" };
>>>>
>>>> First found, first served.
>>>>
>>>> Hum maybe we should try to match the version that issuing python on the
>>>> command line would start. We could potentially look at the PATH to see
>>>> if
>>>> there's something like "bla:\pythonXX" and try the related .dll... Or
>>>> more
>>>> costly, but more reliable, try issuing a 'python -c "import sys;
>>>> print(str(sys.version_info[0]) + str(sys.version_info[1]))"' command
>>>>
>>>> That's one of the downside of requiring no dependency at build time.
>>>>
>>>> Anyway you can override the default guess by setting the PYTHONSO
>>>> config option
>>>> to point to the desired python dll.
>>>>
>>>> By the way I've committed the doc in
>>>> https://trac.osgeo.org/gdal/changeset/35441 . Should be reflected
>>>> online in a
>>>> few hours.
>>>>
>>>>
>> On 14 September 2016 at 20:20, James Ramm <jamessramm at gmail.com> wrote:
>>
>>> Ah, that makes sense. I'll have to try the config option in the morning,
>>> but it sounds like that could be it
>>>
>>> On 14 Sep 2016 8:06 p.m., "Even Rouault" <even.rouault at spatialys.com>
>>> wrote:
>>>
>>>> Le mercredi 14 septembre 2016 20:50:04, James Ramm a écrit :
>>>> > Yes, it is loading a different python....The path printed by sys.path
>>>> is
>>>> > different to if I open the command window and type:
>>>> >
>>>> > python
>>>> >
>>>> > >>> import sys
>>>> > >>> print(sys.path)
>>>> >
>>>> > Gdal and the python bindings were compiled with vs2015 and python
>>>> 3.5, and
>>>> > I can correctly import python in the 3.5 interpreter, yet somehow a
>>>> > different python DLL (2.7) is being loaded at runtime.
>>>>
>>>> Might be related to the default try order in
>>>>
>>>> const char* const apszPythonSO[] = { "python27.dll",
>>>> "python26.dll",
>>>> "python34.dll",
>>>> "python35.dll",
>>>> "python36.dll",
>>>> "python33.dll",
>>>> "python32.dll" };
>>>>
>>>> First found, first served.
>>>>
>>>> Hum maybe we should try to match the version that issuing python on the
>>>> command line would start. We could potentially look at the PATH to see
>>>> if
>>>> there's something like "bla:\pythonXX" and try the related .dll... Or
>>>> more
>>>> costly, but more reliable, try issuing a 'python -c "import sys;
>>>> print(str(sys.version_info[0]) + str(sys.version_info[1]))"' command
>>>>
>>>> That's one of the downside of requiring no dependency at build time.
>>>>
>>>> Anyway you can override the default guess by setting the PYTHONSO
>>>> config option
>>>> to point to the desired python dll.
>>>>
>>>> By the way I've committed the doc in
>>>> https://trac.osgeo.org/gdal/changeset/35441 . Should be reflected
>>>> online in a
>>>> few hours.
>>>>
>>>>
>>>> > I am on a
>>>> > 'inherited' PC right now, so final thing to do is to ensure that the
>>>> > gdal_translate I am running is the one I compiled and there isn't
>>>> another
>>>> > version lurking somewhere....
>>>> >
>>>> >
>>>> > On 14 September 2016 at 16:50, Even Rouault <
>>>> even.rouault at spatialys.com>
>>>> >
>>>> > wrote:
>>>> > > Le mercredi 14 septembre 2016 17:24:53, jramm a écrit :
>>>> > > > Trying to run this using a function relying on scipy.ndimage...
>>>> > > >
>>>> > > > When running gdal_translate on the VRT, I get ImportError: No
>>>> module
>>>> > >
>>>> > > named
>>>> > >
>>>> > > > scipy.ndimage
>>>> > > > This comes after successfully import numpy. scipy.ndimage will
>>>> happily
>>>> > > > import within the python interpreter.
>>>> > >
>>>> > > Works for me for both inline or offline functions.
>>>> > >
>>>> > > Are you sure GDAL loads the same python lib as the python version
>>>> used in
>>>> > > the
>>>> > > python interpreter ? (check the debug traces with CPL_DEBUG=ON)
>>>> > >
>>>> > > You can also add at the top of your script
>>>> > >
>>>> > > import sys
>>>> > > print(sys.path)
>>>> > >
>>>> > > and check if the output points to a location where your scipy
>>>> package can
>>>> > > be
>>>> > > found.
>>>> > >
>>>> > > > Any tips on how to track this down/debug?
>>>> > >
>>>> > > > The entire VRT file is as follows:
>>>> > > I guess this is not the entire VRT since it refers to an inline
>>>> > > definition of
>>>> > > the script but <PixelFunctionCode> has empty content.
>>>> > >
>>>> > > > <VRTDataset RasterXSize="111090" RasterYSize="259376">
>>>> > > >
>>>> > > > <SRS>PROJCS["OSGB 1936 / British National Grid",GEOGCS["OSGB
>>>> > > >
>>>> > > > 1936",DATUM["OSGB_1936",SPHEROID["Airy
>>>> > > > 1830",6377563.396,299.3249646,AUTHORITY["EPSG","7001"]],TOWG
>>>> > >
>>>> > > S84[446.448,-12
>>>> > >
>>>> > > > 5.157,542.06,0.15,0.247,0.842,-20.489],AUTHORITY["EPSG","627
>>>> > >
>>>> > > 7"]],PRIMEM["Gr
>>>> > >
>>>> > > > eenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532
>>>> > >
>>>> > > 925199433,AUTHO
>>>> > >
>>>> > > > RITY["EPSG","9122"]],AUTHORITY["EPSG","4277"]],PROJECTION["
>>>> > >
>>>> > > Transverse_Merca
>>>> > >
>>>> > > > tor"],PARAMETER["latitude_of_origin",49],PARAMETER["central_
>>>> > >
>>>> > > meridian",-2],P
>>>> > >
>>>> > > > ARAMETER["scale_factor",0.9996012717],PARAMETER["false_easti
>>>> > >
>>>> > > ng",400000],PAR
>>>> > >
>>>> > > > AMETER["false_northing",-100000],UNIT["metre",1,AUTHORITY["
>>>> > >
>>>> > > EPSG","9001"]],A
>>>> > >
>>>> > > > XIS["Easting",EAST],AXIS["Northing",NORTH],AUTHORITY["EPSG",
>>>> > >
>>>> > > "27700"]]</SRS>
>>>> > >
>>>> > > > <GeoTransform>100180.0,5.0,0.0,1215730.0,0.0,-5.0</GeoTransform>
>>>> > > > <VRTRasterBand dataType="Float32" band="1"
>>>> > > > subClass="VRTDerivedRasterBand">
>>>> > > >
>>>> > > > <SimpleSource>
>>>> > > >
>>>> > > > <SourceFilename
>>>> > > >
>>>> > > > relativeToVrt="0">F:\tif_data\large_sparse.tif</SourceFilename>
>>>> > > >
>>>> > > > <SourceProperties BlockXSize="256" BlockYSize="256"
>>>> > > >
>>>> > > > RasterXSize="111090" RasterYSize="259376"/>
>>>> > > >
>>>> > > > <OpenOptions>
>>>> > > >
>>>> > > > <OOI key="NUM_THREADS">4</OOI>
>>>> > > > <OOI key="SPARSE_OK">TRUE</OOI>
>>>> > > >
>>>> > > > </OpenOptions>
>>>> > > >
>>>> > > > </SimpleSource>
>>>> > > > <PixelFunctionType>extract_blobs</PixelFunctionType>
>>>> > > > <PixelFunctionLanguage>Python</PixelFunctionLanguage>
>>>> > > > <PixelFunctionCode>
>>>> > > >
>>>> > > > </PixelFunctionCode>
>>>> > > >
>>>> > > > <BufferRadius>5</BufferRadius>
>>>> > > > <PixelFunctionArguments nodata="0" count_threshold="5"/>
>>>> > > >
>>>> > > > </VRTRasterBand>
>>>> > > >
>>>> > > > </VRTDataset>
>>>> > > >
>>>> > > >
>>>> > > >
>>>> > > >
>>>> > > > --
>>>> > > > View this message in context:
>>>> > > > http://osgeo-org.1560.x6.nabble.com/gdal-dev-VRT-derived-
>>>> > >
>>>> > > band-pixel-functi
>>>> > >
>>>> > > > ons-written-in-Python-tp5285323p5285882.html Sent from the GDAL
>>>> - Dev
>>>> > > > mailing list archive at Nabble.com.
>>>> > > > _______________________________________________
>>>> > > > gdal-dev mailing list
>>>> > > > gdal-dev at lists.osgeo.org
>>>> > > > http://lists.osgeo.org/mailman/listinfo/gdal-dev
>>>> > >
>>>> > > --
>>>> > > Spatialys - Geospatial professional services
>>>> > > http://www.spatialys.com
>>>>
>>>> --
>>>> Spatialys - Geospatial professional services
>>>> http://www.spatialys.com
>>>>
>>>
>>
>> _______________________________________________
>> gdal-dev mailing list
>> gdal-dev at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>>
>
>
>
> --
> Sean Gillies
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20160915/b7f9bf2a/attachment-0001.html>
More information about the gdal-dev
mailing list