[Qgis-developer] [GRASS-dev] GRASS & QGIS: the future

Tue Apr 22 07:49:02 PDT 2014

Hi Radim,

2014-04-22 16:03 GMT+02:00 Radim Blazek <radim.blazek at gmail.com>:
> On Fri, Apr 18, 2014 at 2:19 PM, Sören Gebbert
> <soerengebbert at googlemail.com> wrote:
>> Hi Radim,
>> IMHO we GRASS developers are  too stubborn to change the design of
>> GRASS GIS to work as library with persistent applications. This would
>> require rewriting plenty of core functionality and modification of all
>> C-modules. But there is a solution for persistent applications, called
>> Remote Procedure Call (RPC), more below.
>
> It sounds interesting but it seems to be a huge work an far future. I

Its far less work then modifying GRASS libraries to run with
persistent applications. We need
such an approach anyway, since vector editing is a GUI task -> GUI is
a persistent
application -> we need an exit safe interface to the GRASS libraries
to implement a reliable
vector editing GUI in GRASS itself.

> am also concerned about complexity, e.g. starting / keeping running
> servers on various platforms. I can imagine endless bug reports about
> "server not running". I am looking for something which will not

Since only QGIS is spawning the "server" process, it is always able to
check if the
process is running and can restart it when needed. We already make use
of this concept in GRASS
and it works nicely[1]. Spawning of processes and keeping track of
them is an easy task with Qt,
and it is completely OS independent. The Qt Bitcoin client for example
makes use of RPC
and runs on Windows, Mac OS and various Linux systems. So i don't see
a problem here.

> require permanent assistance. It is not KISS enough for me.

What would be a better KISS solution?

IMO the RPC approach will make the implementation of the QGIS-GRASS
data provider and the
plugin simpler. There will be no mixing of C++ and C-code anymore. No
setjmp/longjmp/exception
workaround that is really complex and will not prevent QGIS from
terminating or segfaulting eventually in case of a fatal error.
The QGIS GRASS data provider/plugin will be a clean C++ implementation
using the thrift library.
It will be better maintainable since developers do not need to know
all the different
GRASS libraries (gis, raster, vector, database) but only the RPC
interface that make
use of a subset of the GRASS library functions, bundling them in an
object oriented API.
QGIS do not need to link against GRASS libraries anymore, future GRASS
versions will be supported out of the box.

This is much more KISS then the current approach.

Best regards
Soeren

[1] http://grass.osgeo.org/programming7/namespacepython_1_1temporal_1_1c__libraries__interface.html
>
> Radim
>
>> 2014-04-18 11:31 GMT+02:00 Radim Blazek <radim.blazek at gmail.com>:
>>> I have upgraded  the vector provider to GRASS 7, layers may be added
>>> by drag from browser. The raster and the plugin are disabled. Be
>>> careful about multiple versions on the same system
>>> (LD_LIBRARY_PATH..., check with ldd if does not work).
>>>
>>> Unfortunately GRASS 7 moved ahead towards its aim "to make life harder
>>> for anyone trying to use the GRASS libraries" [1]. Basically more and
>>> more low level functions are calling exit() instead of returning error
>>> code if something failed. As I am not willing to implement GRASS
>>> module call for each simple function, we have to think again about
>>> hacks we can use:
>>>
>>> 1) add a requirement that GRASS 7 used with QGIS must be compiled with
>>> -fexceptions
>>>
>>> 2) add a requirement that a patch (it is a single line comment in
>>> fact) must be applied to GRASS 7 to make it usable with QGIS
>>>
>>> 3) use setjmp()/longjmp()
>>>
>>> 4) let QGIS crash whenever GRASS lib function fails
>>
>> I fear that none of these suggestions are good working solutions,
>> since in case a fatal error or a segfault occurs nothing (no
>> exception, no setjmp) will prevent QGIS from crashing. So i would like
>> to suggest the following:
>>
>> 5.) Using a RPC interface for map metadata, vector and raster map
>> access. Hence one or several GRASS "server" processes provide an RPC
>> interface to access GRASS library functions that are needed in a
>> persistent application. Most important is the vector editing, vector
>> reading/writing, database access, raster reading and map metadata
>> support. Everything else can be done using modules. Hence the RPC
>> interface will support only a limited subset of the GRASS library
>> functions. This RPC interface should only be used in the GRASS
>> provider classes, the GRASS plugin itself can be written in C++ or
>> Python.
>>
>> This approach will decouple GRASS from QGIS, since all communication
>> is done via Inter Process Communication (IPC) using pipes or sockets.
>> There is no need anymore to catch a fatal error or to link GRASS
>> libraries directly to QGIS. The GRASS plugin can be implemented GRASS
>> version independently and so the GRASS data provider, since they will
>> not use GRASS functions directly only the RPC interface. The RPC
>> implementation on the side of GRASS will provide a consistent
>> interface that will not change in case the underlying GRASS API
>> changes.
>>
>> I strongly suggest to use an existing RPC framework to implement fast
>> binary data exchange and the IPC. My favorite is apache thrift[1],
>> since it supports plenty of programming languages (C/C++, Python,
>> Java, JavaScript, ...) and provides operating system independent
>> client and server functionality. It supports exceptions on client side
>> that can be emitted in case a GRASS server process died because of a
>> fatal error, SIGINT or segfault.
>>
>> I am absolutely willing to implement the RPC server side in GRASS
>> using thrift and C++, providing plenty of Python unit tests that show
>> howto use it on the client side. We just need to decide what kind of
>> GRASS library functionality is needed as RPC interface and what can be
>> done in QGIS with C++/Python directly (gisrc creation, GRASS
>> environmental variable settings, ...) or using GRASS modules
>> (g.region, g.gisenv, ...).
>>
>> [1] https://thrift.apache.org/
>>
>>
>> Best regards
>> Soeren
>>
>>>
>>> Radim
>>>
>>> [1]https://trac.osgeo.org/grass/ticket/869#comment:1
>>>
>>> On Thu, Mar 27, 2014 at 11:18 AM, Paolo Cavallini <cavallini at faunalia.it> wrote:
>>>> Hi all.
>>>> I learned during dinner that GRASS7 RC1 is due very soon. This opens the
>>>> issue of its functioning in QGIS. IMHO:
>>>>
>>>> * the qgis-grass-plugin might stop working (this has to be tested)
>>>> * some of the module options will be different
>>>> * new modules will not be available in QGIS.
>>>>
>>>> I think we can deal with this in several ways:
>>>>
>>>> * dropping the plugin, and concentrate the work on Processing
>>>> * upgrading both the plugin and Processing.
>>>>
>>>> In the first case, we have two major issues:
>>>>
>>>> * upgrading Processing GRASS modules
>>>> * changing the current Processing behaviour, avoiding the import-export
>>>> phase when piping consecutive GRASS commands; this makes GRASS modules
>>>> slower than the equivalent commands of other backends.
>>>>
>>>> While the first issue can be solved easily by a couple of people in a
>>>> few days, the second one is more tricky, and requires hard coding skills.
>>>> In addition, we'll no longer be able to edit GRASS vectors directly.
>>>>
>>>> In the second case, we'll have more work, and a not-so-nice duplication.
>>>>
>>>> I would like to have an open discussion on this, avoiding things to just
>>>> happen, with the possible negative consequences.
>>>>
>>>> All the best.
>>>> --
>>>> Paolo Cavallini - www.faunalia.eu
>>>> QGIS & PostGIS courses: http://www.faunalia.eu/training.html
>>>> _______________________________________________
>>>> grass-dev mailing list
>>>> grass-dev at lists.osgeo.org
>>>> http://lists.osgeo.org/mailman/listinfo/grass-dev
>>> _______________________________________________
>>> Qgis-developer mailing list
>>> Qgis-developer at lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/qgis-developer