[GRASS-dev] Working with TGIS without starting GRASS explicitly

[Laurent C.]

Hello all,

My code is working fine in Linux, but I run into a (related?) problem
running in Windows.
Running from the Anaconda shell:

  File "e:\itzi\gis.py", line 24, in <module>
    import grass.temporal as tgis
  File "C:\OSGEO4~1\apps\grass\grass-7.0.5\etc\python\grass\temporal\__init__.py",
line 1, in <module>
    from core import *
  File "C:\OSGEO4~1\apps\grass\grass-7.0.5\etc\python\grass\temporal\core.py",
line 38, in <module>
    from c_libraries_interface import *
  File "C:\OSGEO4~1\apps\grass\grass-7.0.5\etc\python\grass\temporal\c_libraries_interface.py",
line 19, in <module>
    import grass.lib.gis as libgis
  File "C:\OSGEO4~1\apps\grass\grass-7.0.5\etc\python\grass\lib\gis.py",
line 23, in <module>
    _libs["grass_gis.7.0.5"] = load_library("grass_gis.7.0.5")
  File "C:\OSGEO4~1\apps\grass\grass-7.0.5\etc\python\grass\lib\ctypes_loader.py",
line 55, in load_library
    return self.load(path)
  File "C:\OSGEO4~1\apps\grass\grass-7.0.5\etc\python\grass\lib\ctypes_loader.py",
line 221, in load
    return _WindowsLibrary(path)
  File "C:\OSGEO4~1\apps\grass\grass-7.0.5\etc\python\grass\lib\ctypes_loader.py",
line 207, in __init__
    self.cdll = ctypes.cdll.LoadLibrary(path)
  File "C:\Users\lcourty\Anaconda2\lib\ctypes\__init__.py", line 440,
in LoadLibrary
    return self._dlltype(name)
  File "C:\Users\lcourty\Anaconda2\lib\ctypes\__init__.py", line 362,
in __init__
    self._handle = _dlopen(self._name, mode)
WindowsError: [Error 126] The specified module could not be found


This is using the OSGeo4W version. If using the WinGRASS binary, the
code fail at the same place but without any traceback.

>From the GRASS shell, it fail earlier:

Traceback (most recent call last):
  File "C:\Users\lcourty\Anaconda2\Scripts\itzi-script.py", line 5, in <module>
    from pkg_resources import load_entry_point
ImportError: No module named pkg_resources


I have the feeling that it could be related to the fact that Anaconda
and GRASS use their own version of Python. But I'm not sure and I
don't know how to fix that.
Any clue?

Regards,
Laurent


2016-10-05 8:20 GMT-05:00 Laurent C. <lrntct at gmail.com>:
> Hi Sören,
>
> Thanks for your answer. I did something similar and it works well under
> Linux.
> Making those hacks works with Windows seems to be a bit more complex,
> though.
>
> Regards,
> Laurent
>
> 2016-09-24 4:35 GMT-05:00 Sören Gebbert <soerengebbert at googlemail.com>:
>> Hi Laurent,
>> i have attached a Python script that shows the handling of tgis.init() in
>> a
>> subprocess in case location and mapsets change using grass72.
>> The following output was generated with this script, it seems to work on
>> my
>> system:
>>
>> soeren at knecht:~$ python test_tgis.py
>> New envrionment: {u'MAPSET': u'PERMANENT', u'GISDBASE':
>> u'/home/soeren/grassdata', u'LOCATION_NAME': u'nc_spm_08_grass7'}
>> TGIS init:
>> database: /home/soeren/grassdata
>> location: nc_spm_08_grass7
>> mapset: PERMANENT
>> New envrionment: {u'MAPSET': u'user1', u'GISDBASE':
>> u'/home/soeren/grassdata', u'LOCATION_NAME': u'nc_spm_08_grass7'}
>> TGIS init:
>> database: /home/soeren/grassdata
>> location: nc_spm_08_grass7
>> mapset: user1
>> New envrionment: {u'MAPSET': u'PERMANENT', u'GISDBASE':
>> u'/home/soeren/grassdata', u'LOCATION_NAME': u'nc_spm_08'}
>> TGIS init:
>> database: /home/soeren/grassdata
>> location: nc_spm_08
>> mapset: PERMANENT
>> New envrionment: {u'MAPSET': u'user1', u'GISDBASE':
>> u'/home/soeren/grassdata', u'LOCATION_NAME': u'nc_spm_08'}
>> TGIS init:
>> database: /home/soeren/grassdata
>> location: nc_spm_08
>> mapset: user1
>>
>>
>> Best regards
>> Soeren
>>
>>
>> 2016-09-24 5:32 GMT+02:00 Laurent C. <lrntct at gmail.com>:
>>>
>>> Hi Sören,
>>>
>>> I followed you suggestion and now each simulation run in its own process
>>> using the multiprocessing library. It works well, thanks.
>>> However, there was indeed something strange going on:
>>> The new mapset, location and accessible mapsets were set properly, as
>>> seen
>>> by the output of g.mapset and g.mapsets
>>> However, the output of ciface.available_mapsets() and therefore
>>> core.get_available_temporal_mapsets() were not updated after the first
>>> simulation, so tgis.init() were failing, not seeing the new current
>>> mapset
>>> in its own list of available mapsets.
>>>
>>> Below was the output of the software:
>>>
>>> g.mapset -p
>>> swashes_1d_1000m
>>> g.mapsets -p
>>> Accessible mapsets:
>>> swashes_1d_1000m PERMANENT
>>> ciface.available_mapsets()
>>> ['swashes_1d_1000m', 'PERMANENT']
>>> get_available_temporal_mapsets()
>>> {'swashes_1d_1000m': ('sqlite',
>>> '/home/laurent/grassdata/flood_test/swashes_1d_1000m/tgis/sqlite.db')}
>>> Starting simulation for configuration file macdo1000.ini...
>>> [...]
>>> g.mapset -p
>>> hull
>>> g.mapsets -p
>>> Accessible mapsets:
>>> hull PERMANENT hull_rain
>>> ciface.available_mapsets()
>>> ['swashes_1d_1000m', 'PERMANENT']
>>> get_available_temporal_mapsets()
>>> {'swashes_1d_1000m': ('sqlite',
>>> '/home/laurent/grassdata/flood_test/swashes_1d_1000m/tgis/sqlite.db')}
>>> ERROR: Unable to execute sql statement. There is no temporal database
>>> connection defined for mapset <hull>
>>>
>>> It seems that tgis.init() is not doing what it is supposed to, according
>>> to the documentation:
>>>
>>> "Re-run this function in case the following GRASS variables change while
>>> the process runs:
>>> - MAPSET
>>> - LOCATION_NAME
>>> - GISDBASE
>>> - TGIS_DISABLE_MAPSET_CHECK
>>> - TGIS_DISABLE_TIMESTAMP_WRITE
>>> "
>>>
>>> Thanks again.
>>>
>>> Regards,
>>> Laurent
>>>
>>> 2016-09-22 13:52 GMT-05:00 Sören Gebbert <soerengebbert at googlemail.com>:
>>> > Hi Laurent,
>>> > the temporal framework was neither tested nor designed to work in a
>>> > script
>>> > that changes the location while running. I would suggest to run the
>>> > temporal
>>> > framework related code in a subprocess that is created each time the
>>> > location or mapset changes. Unfortunately i have no other suggestion.
>>> >
>>> > However, have you tested your code with grass 7.2 or 7.3? Many
>>> > improvements
>>> > have been implement in the recent version that would be beneficial to
>>> > use.
>>> >
>>> > Best regards
>>> > Soeren
>>> >
>>> > 2016-09-22 2:47 GMT+02:00 Laurent C. <lrntct at gmail.com>:
>>> >>
>>> >> Hello all,
>>> >>
>>> >> I've ran into another related issue.
>>> >> One of the goal of running the software outside of GRASS shell is to
>>> >> batch process simulations in various Locations/mapsets.
>>> >>
>>> >> I set the GRASS session for each case. The simulation works well for
>>> >> the first case.
>>> >> However when starting the second case, tgis.init() fails with the
>>> >> following error:
>>> >>
>>> >> ERROR: Unable to execute sql statement. There is no temporal database
>>> >> connection defined for mapset <hull>
>>> >>
>>> >> The mapset and location are properly set.
>>> >> If this case is run first, it works well and it's the second one that
>>> >> fail.
>>> >> Running t.connect -c between two cases does not solve the problem.
>>> >> Actually, t.connect -p shows the correct connection parameters, but
>>> >> tgis.init() still fails.
>>> >>
>>> >> Regards,
>>> >> Laurent
>>> >>
>>> >>
>>> >> 2016-09-21 18:27 GMT-05:00 Laurent C. <lrntct at gmail.com>:
>>> >> > Hi Sören,
>>> >> >
>>> >> > Setting LD_LIBRARY_PATH is working, thanks. But it cannot be set at
>>> >> > run-time, which is not very user-friendly.
>>> >> > I guess this issue is related to the ticket 2424 [1].
>>> >> >
>>> >> > I managed to get it work at run-time by restarting the program with
>>> >> > sys.execv() after setting the path [2], but I find it a bit ugly and
>>> >> > quite verbose to be multi-platform.
>>> >> >
>>> >> > It would be great if an easier option was possible.
>>> >> >
>>> >> > Regards,
>>> >> > Laurent
>>> >> >
>>> >> > [1] https://trac.osgeo.org/grass/ticket/2424
>>> >> > [2]
>>> >> >
>>> >> >
>>> >> > http://stackoverflow.com/questions/6543847/setting-ld-library-path-from-inside-python
>>> >> >
>>> >> >
>>> >> > 2016-09-21 15:40 GMT-05:00 Sören Gebbert
>>> >> > <soerengebbert at googlemail.com>:
>>> >> >> Hi,
>>> >> >> i think you have to put the GRASS libraries into your
>>> >> >> LD_LIBRARY_PATH
>>> >> >> so
>>> >> >> that the python wrapper can access them.
>>> >> >>
>>> >> >> Best regards
>>> >> >> Soeren
>>> >> >>
>>> >> >> 2016-09-21 20:37 GMT+02:00 Laurent C. <lrntct at gmail.com>:
>>> >> >>>
>>> >> >>> Hello all,
>>> >> >>>
>>> >> >>> I'm trying to adapt a python module in order for it to be run
>>> >> >>> outside
>>> >> >>> of
>>> >> >>> GRASS.
>>> >> >>> I followed the instructions from the wiki and
>>> >> >>> lib/python/script/setup.py
>>> >> >>>
>>> >> >>> gsetup.init() run without error and I can list maps like in the
>>> >> >>> example.
>>> >> >>>
>>> >> >>> However, when I try to import the temporal module, I receive this
>>> >> >>> error:
>>> >> >>>
>>> >> >>> import grass.temporal as tgis
>>> >> >>> File "/usr/lib/grass70/etc/python/grass/temporal/__init__.py",
>>> >> >>> line
>>> >> >>> 1, in <module>
>>> >> >>> from core import *
>>> >> >>> File "/usr/lib/grass70/etc/python/grass/temporal/core.py", line
>>> >> >>> 38,
>>> >> >>> in <module>
>>> >> >>> from c_libraries_interface import *
>>> >> >>> File
>>> >> >>>
>>> >> >>>
>>> >> >>> "/usr/lib/grass70/etc/python/grass/temporal/c_libraries_interface.py",
>>> >> >>> line 19, in <module>
>>> >> >>> import grass.lib.gis as libgis
>>> >> >>> File "/usr/lib/grass70/etc/python/grass/lib/gis.py", line 23, in
>>> >> >>> <module>
>>> >> >>> _libs["grass_gis.7.0.4"] = load_library("grass_gis.7.0.4")
>>> >> >>> File "/usr/lib/grass70/etc/python/grass/lib/ctypes_loader.py",
>>> >> >>> line
>>> >> >>> 55, in load_library
>>> >> >>> return self.load(path)
>>> >> >>> File "/usr/lib/grass70/etc/python/grass/lib/ctypes_loader.py",
>>> >> >>> line
>>> >> >>> 71, in load
>>> >> >>> raise ImportError,e
>>> >> >>> ImportError: libgrass_datetime.7.0.4.so: cannot open shared object
>>> >> >>> file: No such file or directory
>>> >> >>>
>>> >> >>>
>>> >> >>> Did I forgot to set-up something or this is a bug?
>>> >> >>>
>>> >> >>> Cheers,
>>> >> >>> Laurent
>>> >> >>> _______________________________________________
>>> >> >>> grass-dev mailing list
>>> >> >>> grass-dev at lists.osgeo.org
>>> >> >>> http://lists.osgeo.org/mailman/listinfo/grass-dev
>>> >> >>
>>> >> >>
>>> >
>>> >
>>
>>