[GRASS-dev] How GRASS should behave when another session is already active?

[Vaclav Petras]

On Tue, Jun 2, 2015 at 7:50 AM, Moritz Lennert <mlennert at club.worldonline.be>
wrote:
>
> On 02/06/15 03:05, Vaclav Petras wrote:
>>
>> Hi all,
>>
>> I need opinions on how to resolve the question whether GRASS session can
>> be started from another GRASS session.
>>
>> Here we want to use GISBASE from environment instead of build constant
>> if available:
>>
>> https://trac.osgeo.org/grass/changeset/45576
>>
>> However, here we cause error when GISBASE already exists:
>>
>> http://trac.osgeo.org/grass/changeset/59210
>>
>> But the check was removed in:
>>
>> http://trac.osgeo.org/grass/changeset/59220
>>
>> because of:
>>
>> http://trac.osgeo.org/grass/ticket/2213
>>
>> The point of r59210 was to make impossible to start another GRASS
>> session within an existing session.
>
>
> Even though I see this happen regularly amongst my students,

Interesting. I actually haven't seen it in this way. I'm concerned about
scripting, other applications and advanced users.

> I'm not sure that I consider this such a problem to justify significantly
changing the existing logic.

... and apparently it is not trivial (r59210 and r59220).

> Have you come across any serious issues related to this ?

I don't know about any problems and even the commit r59210 which did the
check and failed with present GISBASE doesn't disclose any motivation.
Maybe Martin remembers... Markus is saying that it is asking for trouble.
Perhaps, but on the other hand, a new session overrides a lot of variables
and surely allows to at least use different data (but MANPATH and
PYTHONPATH should be OK too). But also, there is a lot of variables which
are not overridden (e.g. GRASS_HTML_BROWSER which is OK, but also discussed
GISBASE which is a catastrophe if the version/installation is different).

> The whole notion of a "session" is a bit misleading as all a "session" is
(IIUC) is a specific state of environment variables.

You are right that whole notion of a "session" is a mess. GRASS session has
two parts.

First, runtime environment for modules which is a set of variables such as
PATH, LD_LIBRARY_PATH or PYTHONPATH.

Second, data linked using GISRC file and variable. In a GRASS session,
data, specifically Mapset, is locked to prevent other GRASS sessions from
modifying it. There is a "gisrc" file somewhere which contains information
about current GRASS Database directory, Location and Mapset (basically full
path to mapset) and optionally other variables. Path to this file is stored
in GISRC variable and it does not change during the GRASS session, although
subprocess may change GISRC variable to do some special things (e.g.
Mapset-based parallelization).

GRASS session by itself can change which data (Mapset) are linked/connected
but it cannot (should not) change the GISRC variable ("gisrc" file path).
Runtime environment cannot be changed by the session itself which is both
technical and design limitation.

If one just sets up the required variables, there is not locking of the
Mapset nor cleaning of temporary directory in a Mapset (other managed
temporary file is for example "gisrc" file). GRASS session can be
understood as a process which starts and ends with all these steps besides
providing the right environment.

Then perhaps it seems that we can allow having runtime environment in the
system all the time and the only thing where we may (or may not) need to
limit nesting is a fully working environment which means that GISRC
environment variable is present. But the latter test also does have to be
present unless we want to really ban interactive session in another session.

This would leave use just with the issue of getting GISBASE in case of
OSGeo4W.

> So if someone wants to switch GRASS version during a working session,
this should remain possible by just changing these environment variables,
i.e. after having launched trunk, switch to the grass70 release branch:
>
> export
GISBASE=/data/home/mlennert/SRC/GRASS/grass70_release/dist.x86_64-unknown-linux-gnu/
> export
LD_LIBRARY_PATH=/data/home/mlennert/SRC/GRASS/grass70_release/dist.x86_64-unknown-linux-gnu/lib/
> export
PATH=/data/home/mlennert/SRC/GRASS/grass70_release/dist.x86_64-unknown-linux-gnu/bin/:/data/home/mlennert/SRC/GRASS/grass70_release/dist.x86_64-unknown-linux-gnu/scripts/:/home/mlennert/.grass7/addons/bin:/home/mlennert/.grass7/addons/scripts:/home/mlennert/.grass7/addons/:/home/mlennert/BIN:/home/mlennert/BIN:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/opt/libreoffice/program:/home/mlennert/SRC/FWTools-2.0.6/bin_safe
>
> or similar (there might be some other variables that need to be changed).
>
> Why should this be bad ? But I see your argument that (in this case) the
grass70 starting (better name would be "env_setup") script should actually
force setting the GISBASE (and the others) to the path where grass70 is
installed, instead of taking the existing setting from the environment,
thus allowing easy switching of versions.

Exactly, but OSGeo4W requires setting of GISBASE from outside because
GISBASE from compilation is wrong. I'm not sure if this is possible on some
other platforms or if there is some other way around than setting a
variable.

I still can see only two possible solutions. Use GRASS_GISBASE (or whatever
name) variable when present in cases that the distribution cannot ensure
the right path in grass.py. GRASS_GISBASE would be used only in a
distribution custom script which subsequently invokes grass.py. I hope that
GRASS_GISBASE wouldn't be used in scripts because the need for it would be
already satisfied in whatever is behind grass7 command (which should be
used in the scripts instead or with the actual GISBASE).

The second solution is to use existing GISBASE variable as a fallback when
the path from compile time does not exist. I'm not sure how reliable this
can be tested. Is it possible that the directly will exist but is unusable
(and user indeed wants to use the existing GISBASE) or there are some
strange permission issues and it would say it does not exist just because
we cannot write there or something? Pseudo-code for both solutions is in my
previous email to Markus.

>> However, r45576 is using exactly
>> same check (GISBASE in env) to see if GISBASE is forced from outside.
>> The two places for it are here:
>>
>>
https://trac.osgeo.org/grass/browser/grass/trunk/lib/init/grass.py?annotate=blame&rev=65348#L48
>>
https://trac.osgeo.org/grass/browser/grass/trunk/lib/init/grass.py?annotate=blame&rev=65348#L1633
>>
>> I think starting non-interactive session (batch/exec mode) should be
>> possible. Perhaps interactive session (--text & --gui) should have the
>> check.
>
>
> Non interactive sessions should always be possible since you can set all
the env variables in the script and IIUC these will only be valid during
the execution of the script.

By non-interactive session I mean an actual run of grass7 command* but
without jumping to the command line. This can be now achieved by --exec or
BATCH_MODE. Perhaps interactive is not a right word since you can actually
start GUI using the --exec flag, so what you need and then close it. The
difference is in the presence of a prompt and execution of a process
specified when starting the session.

Manual setting of the environment with or without the help of
grass.script.setup.init function should be always possible. However, I
think it is a pain and I would like to see it replaced by usage of "grass71
--exec" interface as it is easier, safer and more reliable (e.g. locking,
ctypes works). However, I also see that changing the environment directly
in the script has it's advantages, most importantly you can use
grass.script interface without creating another file, so I'm also trying to
simplify, or at least not complicate, this procedure.

* I really wonder how should I call that. I would like to see all
distributions to use just grass, so one don't have to think if it is 70 or
71 and write specific number only if necessary (like with Python). And it
is a command, program or an app?

>> If the check is present (even just for the interactive sessions), the
>> OSGeo4W version (and probably standalone winGRASS too) is going to break
>> (needs to be changed somehow).
>
>
> IIRC, this is just due to the check being in grass.py, but wingrass
setting the env variable in the bat startup script.

I think that the bat script just calls the grass.py file:

https://trac.osgeo.org/grass/browser/grass/trunk/mswindows/osgeo4w/grass.bat.tmpl

If we don't check for existing session using GISBASE or we use one of the
solutions above for getting the right GISBASE. there should be no problem.

>> Moreover, this would disallow setting
>> some GISBASE to system for scripting purposes ("without starting
>> explictly") and then starting a standard session.
>>
>> The issue with the current state is that you cannot run different
>> version of GRASS from a current session.
>
>
> AFAIU, you can. You just have to set the env variables correctly.

If we would use GISBASE to check for an existing session, you wouldn't be
able to put GISBASE to your system environment and then start GRASS using
grass7 command. If check applied also for --exec, you would have to unset
GISBASE in any script which would use --exec and which should be portable.
This would complicate the manual setup which is not acceptable.

>> More serious issue is that you
>> cannot start another version if you have GISBASE in your system
>> environment for scripting purposes (as mentioned above).
>
>
> If you don't want to have this problem, just don't set a system-wide /
user-session wide GISBASE. You can always set it explicitely in your
scripts.

True, but there are people who want to use GRASS modules directly in the
command line and I also don't want to have complicated scripts with a lot
of hardcoded paths (preferably no hardcoded paths). The alternative for
scripts is to have grass7 command/program on path and then get GISBASE from
it. I like that but I think that it is still quite long which may be the
reason why I haven't seen it used much. The other reason might be that
grass is not on path on MS Windows by default, nor is the possibility
advertised in doc/wiki such as here:

http://grasswiki.osgeo.org/wiki/Working_with_GRASS_without_starting_it_explicitly

>> The current
>> behavior may also differ depending on a platform.
>>
>> I'm not really sure what were the original intentions but I would prefer
>> to switch the logic in r45576 and use GISBASE only if the path from
>> compile time does not exist.
>
>
> This would mean as I said above that calling the "starting" script of a
specific version would always by default set the relevant variables to that
version. I can agree to that.
>
>
>> This allows to override current session
>> (GISBASE set in some way) but allows to supply the GISBASE when there is
>> something wrong with compile time path.
>
>
> You mean that we would have some fallback mechanism allowing to overwrite
the default from the command line (i.e. something like this: grass70
--use-env-variables) ?

Yes, we even already have it but because of usage of GISBASE variable there
is great potential for problems as discussed above. I now suggest that
either invalid GISBASE from compile time or presence of another variable
(e.g. GRASS_GISBASE) would use it. However, I'm not sure what are the
issues with either of these solutions. I wonder if there is some other
solution (like no GISBASE path from compilation, better
distribution/install process, ...). The solution with a parameter
(--use-env-variables or --gisbase "/the/path") also crossed my mind and
there already was a similar interface for Location and Mapset (to
--use-env-variables) but it seems to me that environmental variable is more
appropriate interface in this case considering the use cases (distribution,
system-driven invocation) as opposed to user/programmer interface which
should be as clean as possible (and --gisbase "/the/path" is not nice, even
just in --help doc).

If environmental variable has a special name (such as GRASS_GIS which does
not overlap with in-session name), there is no need for --use-env-variables
but this kind of parameter could be helpful when you would like to use
GRASS default for GRASS_HTML_BROWSER although you have something set in the
system but I don't think that this useful (and if you need it then I think
is OK to do things like unset the variable).

The fallback is need for OSGeo4W and perhaps other distributions as the
actual (runtime) path may not be known during compilation.

> I think this discussion is relevant, but please do not go through any
premature changes before we haven't thought this through.

This is my intention exactly. Thanks for the comments so far, it was very
helpful for me in sorting out things and thinking about solutions. There is
a lot of moving peaces. I also believe that I cleaned up grass.py
significantly, so I invite anybody who is interested to examine actual code
and commend on it.

Vaclav
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20150602/63bb4d51/attachment-0001.html>