[gdal-dev] VRT derived band pixel functions written in Python

[James Ramm]

Incredible!!!!
Really cool initiative.

I had a play around and wanted a way to be able to write the pixel function
in a 'stand alone' python module and generate the VRT from it. This would
allow independent testing and easier maintenance of the python code. It is
fairly easy using lxml to build up the VRT dynamically:

https://github.com/JamesRamm/gdal_pixel_functions

In the example I more-or-less recreate the hillshade example Even posted,
although there is a lot of functionality missing from the VRT-builder
still, so some nodes are not present.
It would be quite easy therefore, to expand this to a sort of GDAL
'algorithms library' (I think it can wholesale replace what I had tried
here: https://github.com/JamesRamm/GeoAlg).

One thing I would note (if I understand correctly) is that you have
specified any "import" statement as unsafe except numpy/math and numba? It
might be better to either allow imports or not allow them at all (unless
the config option is set) as I don't see why those libraries would be any
more or any less 'safe' than another library...e.g. another standard
library module.

I would agree with Sean and remove 'IF_SAFE' and leave it entirely to the
users discretion. Many users who are working with their own local data will
likely have no problems about just setting it to 'YES' and I imagine users
for whom it would be a probably might already have their own rules on what
constitutes 'safe'?

On a sidenote - does this new functionality overlap considerably with the
gdal map algebra project?

On 12 September 2016 at 19:25, Even Rouault <even.rouault at spatialys.com>
wrote:

> > I found http://nedbatchelder.com/blog/201206/eval_really_is_
> dangerous.html
> > to be a good intro to the risks of eval'ing untrusted Python code.
> > Mentioned in there is a notable attempt to make a secure subset of Python
> > called "pysandbox", but its developer has since declared it "broken by
> > design": https://lwn.net/Articles/574215/. I'm not knowledgeable enough
> > about sandboxing (OS or otherwise) to say if that's right.
>
> Those links are fascinating. I wouldn't have imagined that there was valid
> Python code that could crash the Python interpreter, almost by design ! (so
> the osgeo.gdal gotchas are not so bad after all :-) )
> OK, so I've followed your suggestion:
> * IF_SAFE mode is removed (actually between #if 0 / #endif if someone
> wants to
> pursue the sandboxing effort),
> * and the default of GDAL_VRT_ENABLE_PYTHON is now NO.
>
> >
> > I see that in GDAL 2.0+ we can set options in the VRT XML itself. Is it
> > possible to set GDAL_VRT_ENABLE_PYTHON=YES in a VRT and thus override the
> > reader's own trust policies?
>
> No, that's not possible. GDAL_VRT_ENABLE_PYTHON is a config option only,
> so can
> only be set as a env variable or through CPLSetConfigOption().
>
> We could imagine to provide it as an option of the VRT indeed, but then
> there
> would be interesting situations like the following. Imagine something
> called
> "my.tif" with the following content :
>
> <VRTDataset rasterXSize="20" rasterYSize="20">
>   <VRTRasterBand dataType="Byte" band="1">
>     <ColorInterp>Gray</ColorInterp>
>     <SimpleSource>
>       <SourceFilename relativeToVRT="1"><![CDATA[
> <VRTDataset .... evil Python code here ... .... >]]></SourceFilename>
>       <OpenOptions>
>         <OOI key="ENABLE_PYTHON">YES</OOI>
>       </OpenOptions>
>     </SimpleSource>
>   </VRTRasterBand>
> </VRTDataset>
>
> The internalized VRT would then be allowed with ENABLE_PYTHON=YES.
>
> So it's probably a good idea to not provide an open option for now. One
> could
> perhaps imagine to introduce one, provided we make it illegal as an item in
> <OpenOptions>, so it can only be provided by "top-level" GDALOpenEx()
> calls.
>
> > My ignorance of how GDAL separates "open"
> > options from "config" options might be on display in this question.
>
> They are just 2 different mechanisms of providing options. There's no
> automatic
> bridge between both concepts. Sometimes some option may exist in both
> worlds
> (because historically there was only global config option, but in some use
> cases, some options mostly make sense per dataset) or in just one of them.
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20160912/d240ad13/attachment.html>