[GRASS-dev] [GRASS GIS] #2134: Create a general exit-safe interface to C libraries

[GRASS GIS]

#2134: Create a general exit-safe interface to C libraries
--------------------------------------------------+-------------------------
 Reporter:  wenzeslaus                            |       Owner:  grass-dev@…              
     Type:  enhancement                           |      Status:  new                      
 Priority:  normal                                |   Milestone:  7.0.0                    
Component:  Python ctypes                         |     Version:  svn-trunk                
 Keywords:  G_fatal_error, exit, multiprocessing  |    Platform:  All                      
      Cpu:  Unspecified                           |  
--------------------------------------------------+-------------------------

Comment(by huhabla):

 Replying to [comment:2 wenzeslaus]:
 > Thanks Soeren, this is what I hoped for and I'm looking forward how this
 will continue. I just quickly looked at this topic for now. So, just some
 small notes now.
 >
 > I needed to do this changes to get it working:
 > {{{
 > #!diff
 > @@ -19,12 +19,13 @@
 >  from multiprocessing import Process, Lock, Pipe
 >  import logging
 >  from ctypes import *
 > -from core import *
 > +from grass.script.core import *
 >  import grass.lib.gis as libgis
 >  import grass.lib.raster as libraster
 >  import grass.lib.vector as libvector
 >  import grass.lib.date as libdate
 >  import grass.lib.raster3d as libraster3d
 > +from datetime import datetime
 > }}}


 Thanks for the fix, the new version should work now in any directory.

 > I though I understand how it works with exceptions and doctest but
 apparently not, even I tried verbose mode:
 > {{{
 > python -m doctest -v c_library_interface.py
 > }}}

 Please try:

 {{{
 python c_library_interface.py -v
 }}}

 > Anyway test passes. Would it be possible to write a test also for the
 case of waiting, killing and restarting the server? I mean the case when I
 run something with `call_no_wait(function=aaa)`, it runs for a long time,
 meanwhile I call `call(function=bbb)` but then `bbb` fails and kills the
 server. Does this make sense?

 Yes it does, but i was not able to fix the deadlock that appeared when
 mixing "waiting calls" with "no waiting calls" and a fatal error killed
 the subprocess. So i removed the no wait calls from the server interface.

 > About the interface. I'm not sure about how to report errors. The
 functions might need to accept keyword arguments in some way. How to deal
 with objects, e.g. how to use pyGRASS through this?

 In the latest implementation the subprocess creates an exception that is
 send through the pipe to the server and is raised there.
 If an object is picklable then it can be send between processes.

 >
 > I was not thinking about how to deal with wrapping all C functions. But
 maybe manual wrapping it is inevitable if we want really Python(ic)
 interface. We already have pyGRASS for this reason.
 >
 > The last not is about naming. Although it is a well established practice
 in GRASS that things are named with two or more different names, I would
 suggest to use only one name for newly created things. We are at the
 beginning, we can always rename but it would be safer to use one name at
 time. `*C*Interface` is a nice name (although it it might by also
 `*Python*Interface`) but it is more for the particular implementation
 (interface for temporal framework or to messages) than for the general
 thing (server). `RPCServer` is very general, not sure about that. What
 about `CFunctionCaller`? I also heard suggestion `NoGFatal_Huray!!!` from
 someone here but that's not a valid Python identifier, although, with
 unicode identifiers, there might be a way. No strong opinion here.

 The latest RPC server incarnation does not care what kind of functions it
 should call, hence it is very generic and takes care about killed
 subprocesses. You can define any arbitrary function with arguments that
 are picklable and pass it to the RPC server to call it in the subprocess.

 But maybe it would be better to use sharedctypes memory in case of
 digitizing rather than a pipe?

-- 
Ticket URL: <http://trac.osgeo.org/grass/ticket/2134#comment:11>
GRASS GIS <http://grass.osgeo.org>