[postgis-users] PL/Python

[Rémi Cura]

Hey,
postgres list PostgreSQL General <pgsql-general at postgresql.org>,
or postgres dev list PostgreSQL Hackers <pgsql-hackers at postgresql.org>.

I would more likely say dev list.

I do not know how you use your in memory layer, but you have to keep in
mind that postgres wipes memory of pl functions.
In python, one way to bypass that (for the same session) is to use the
global dictionary (it is basically a python memory that is persistent).
http://www.postgresql.org/docs/9.4/static/plpython-sharing.html
I tried that and it works (I created a cache with it, easy).

Another fairly simple problem you could have is that you would try to put
in memory a raster that is too big.
For my setting, I end up doing python out of the database rather than with
plpython when the consumed RAM is over 500 MB. It is a rule of thumb.

I'm afraid plpython stuff does not rise lots of interest in the postgres
community,
a shame!

Good luck,

Cheers,
Rémi-C


2016-05-23 22:51 GMT+02:00 David Haynes <haynesd2 at gmail.com>:

> Hello Remi,
>
> Do you know who, what list I should report this to? In my work I found is
> that plpython doesn't work well with the memory drivers for gdal or ogr.
> Specifically when I would run the my code it might create the raster
> dataset in memory and the for no apparent reason it would terminate the
> database connection. I thought this was specific to gdal memory until I
> encountered some downstream issues with vector datasets.
>
> The error was resolved choosing actual directories and writing the data
> there. I suppose that there would have to a specific plpython memory driver.
>
>
>
> On Sat, May 21, 2016 at 5:04 AM, Rémi Cura <remi.cura at gmail.com> wrote:
>
>> Hey,
>> debuggin plpython is a bit of a pain.
>> Maybe you are aware that you can use
>> plpy.notice to print values during execution.
>>
>> This blog
>> <http://geoexamples.blogspot.fr/2012/01/creating-files-in-ogr-and-gdal-with.html>
>> seems to indicate that what you do should work:
>>
>> driver = gdal.GetDriverByName( 'MEM' )
>> ds = driver.Create( '', 255, 255, 1, gdal.GDT_Int32)
>>
>>
>> Maybe you can check that executing that in plain python while logged as postgres works?
>>
>> Cheers,
>> Rémi-C
>>
>>
>>
>>
>>
>> 2016-05-18 21:26 GMT+02:00 David Haynes <haynesd2 at gmail.com>:
>>
>>> Yes because the second function in my doesn't error.
>>>
>>> On Wed, May 18, 2016 at 12:06 PM, Martijn Meijers <
>>> b.m.meijers at tudelft.nl> wrote:
>>>
>>>> I think the gdal.getDriver call returns None. Hence the attribute error
>>>> on the line afterwards. Are you sure that 19 is what you should pass as
>>>> input?
>>>>
>>>> Martijn
>>>>
>>>> Verzonden vanaf mobiel.
>>>>
>>>>
>>>> -------- Oorspronkelijk bericht --------
>>>> Van: David Haynes
>>>> Datum:18-05-2016 17:35 (GMT+01:00)
>>>> Aan: postgis-users at lists.osgeo.org
>>>> Onderwerp: [postgis-users] PL/Python
>>>>
>>>> Hello,
>>>>
>>>> I have a question regarding importing the gdal library using pl/python.
>>>> Specifically, I want to create an in memory raster using the gdal library.
>>>> This is a snippet of code that I have written,
>>>>
>>>> CREATE OR REPLACE FUNCTION terrapop_area_level_to_raster2(
>>>> sample_geog_level_id bigint, raster_variable_id bigint, raster_band bigint)
>>>> RETURNS text AS
>>>> $BODY$
>>>> from osgeo import gdal
>>>> import numpy as np
>>>> from osgeo import ogr, osr
>>>>
>>>> memDriver = gdal.GetDriver(19)
>>>> #memDriver = gdal.GetDriverByName('MEM')
>>>> memRast = memDriver.Create('', 10, 10, 1, gdal.GDT_Int32)
>>>>
>>>> This is the error I receive. A python NoneType error. Which seems to
>>>> that the gdal class has not been imported. I have verified that gdal is
>>>> available on the system.
>>>>
>>>> ERROR: AttributeError: 'NoneType' object has no attribute 'Create'
>>>> SQL state: XX000
>>>>
>>>> However, this function on the same server and database returns to me
>>>> all the gdal drivers. Any idea how I can diagnose this problem?
>>>>
>>>> CREATE OR REPLACE FUNCTION gdal_drivers()
>>>>   RETURNS SETOF text AS
>>>> $BODY$
>>>>
>>>> from osgeo import gdal
>>>> driver = gdal.GetDriverByName('MEM')
>>>> rast = driver.Create('', 2, 4, 1, gdal.GDT_Int32)
>>>>
>>>> names = []
>>>> driverall = gdal.GetDriverCount()
>>>> for i in range(gdal.GetDriverCount()):
>>>> driver = gdal.GetDriver(i)
>>>> drivername = driver.ShortName
>>>> names.append([i,drivername])
>>>>
>>>> return names
>>>>
>>>> $BODY$
>>>>   LANGUAGE plpythonu VOLATILE
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> postgis-users mailing list
>>>> postgis-users at lists.osgeo.org
>>>> http://lists.osgeo.org/mailman/listinfo/postgis-users
>>>>
>>>
>>>
>>> _______________________________________________
>>> postgis-users mailing list
>>> postgis-users at lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/postgis-users
>>>
>>
>>
>> _______________________________________________
>> postgis-users mailing list
>> postgis-users at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/postgis-users
>>
>
>
> _______________________________________________
> postgis-users mailing list
> postgis-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/postgis-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/postgis-users/attachments/20160524/2ba8601b/attachment.html>