[GRASS-user] Announcing new command line interface of `grass` program in trunk

[Rainer M Krug]

Vaclav Petras <wenzeslaus at gmail.com> writes:

> On Thu, Jun 4, 2015 at 3:25 PM, Rainer M Krug <Rainer at krugs.de> wrote:
>>
>> > [1, 2] The code now allows to introduce such interface. The challenge
>> > is where to store the information about active Location and Mapset.
>> > Currently, .gisrc (or .grassrc or .grassgisrc) file stored in the
>> > current working directory seems as the best option to me. Another
>> > option is to use environment variables [3, 4] but I don't consider it
>> > a nice solution.
>>
>> I agree - unless it is in a scrip with shebang #!grass71exec or
>> similar. Then it would be easy to use environmental variables.
>
> Let me clarify here, GRASS modules read Mapset from a "gisrc" file. The
> path to this file is stored in `GISRC` environment variable. The standard
> interface now creates this file (its content) based on what was provided in
> the command line.

OK - got this one.

> In the subcommand interface, only the first command knows the Mapset
> and it must pass it to commands called later. The way how to do this
> easily is to create a file with a given name in the current directory
> (e.g., .grassrc). When grass command is called with subcommand (other
> than setmapset), it looks for that file, if it is there, GISRC is set
> to this file, if it is not there, it fails.

OK. So a command without the mapset specified shoud fail - but it
doesn't, if a .grass7/ exists with an rc file in it which holds the
mapset to be used.

I think that interaction between grass session usage of mapset and grass
--exec should be separated and --exec should have it's own way of
storing the mapset to be used to avoid interfere with an interactive
grass70 session.

By doing this, one could also, if I am not mistaken, use

,----
| grass71 --exec ...
`----

for a different mapset inside of a running interactive grass71 session.

>
> The environment variables I was mentioning are different thing. They
> wouldn't be (cannot) be used inside the GRASS session (so no usage for them
> in shebang mode). If the grass program would respect them, you can get poor
> man's subcommand-like interface without implementing it:
>
> export GRASS_FULL_MAPSET=/path/to/mapset
> grass [--mapset-from-environment] --exec r.univar map=elevation
>
> I don't think this is how the API should look like and the subcommand
> interface seems more natural (or just modern?) and more extensible (but do
> we need it?).

No - this does not look optimal - agreed.
I think one should aim for a syntax which does not limit the further
developments - and the sub command syntax really is widely used
successfully.

I see many possibilities for future developments like e.g. defining
particular default values for a session.

>
>> What about a user specified name? This would make switching mapsets in a
>> script much more symbolic, as one does not has to specify the path of
>> the mapset and just an easy to remember name? This would make it possible
> to
>> setup a whole bunch of mapsets associated with names in a text file and
>> then just load the .grassEXEC file?
>
> Mapset is the data. Should the data be specified in a "config file" or some
> general file and then passed to the program? I don't think so. I think one
> should pass the Mapset as a parameter.

I was somehow never to happy about using parameter in scripts, as this
somehow makes it less reproducible. When one uses a file as input /
config for the session, it can be put under VC, the same for the grass
script, and I can easily go back in time. With parameters one needs an
additional layer of a script which calls the R script with the parameter
and put that one under VC. So I think that the file approach is more
reproducible and, in the longer run, more extensible. But it should be
possible to specify the data (including mapset) in the script as well by
using the

,----
| grass71 set mapset /PATH/TO/THE/MAPSET
`----

For the file one could use a syntax like

,----
| grass71 setFromFile /PATH/TO/THE/FILE/WHATEVER.conf
`----

If a file with a default name exists in the working directory, that one
is loaded when doing

,----
| grass71 start
`----
 or to start with a specific file
,----
| grass71 start /PATH/TO/THE/FILE/WHATEVER.conf
`----
or to just specify a mapset
,----
| grass71 start /PATH/TO/MAPSET
`----

could be used to start the session.

obviously
,----
| grass71 start $1
`----

as well where $1 could be the config file or the mapset.

Initially, the file would contain only the mapset, but further on all
settings for which "set" can be used (default output format for maps
comes to mind, interpreter possibly as well?).

>
> To your suggestion of symbolic names. It might be easier to switch but it
> would complicate the API and implementation a lot. How the file would be
> managed? I don't think it is worth the trouble. Also I don't want to
> implement interfaces which don't have precedence. An existing example of
> something like that could change my opinion about a nickname file but I'm
> unsure how this would fit with the rest.

Was just an idea - not critical.

>
>> >
>> > [1] https://trac.osgeo.org/grass/ticket/2579#Additionalideas
>> > [2] https://trac.osgeo.org/grass/ticket/2579#comment:10
>> > [3] https://trac.osgeo.org/grass/ticket/2679
>> > [4] https://trac.osgeo.org/grass/ticket/2681
>> >
>> >> ,----
>> >> | grass71 --setmapset .../grassdata/test1/PERMANENT/
>> >> | grass71 --exec r.external input=basins.tiff output=basins
>> >> | grass71 --exec r.external input=elevation.tiff output=elevation
>> >> | grass71 --unsetmapset
>> >> `----
>> >
>> > Rather than this, I prefer subcommand interface as known from git,
> docker
>> > or apt-get:
>> >
>> > ,----
>> > | grass71 setmapset .../grassdata/test1/PERMANENT/
>> > | grass71 run r.external input=basins.tiff output=basins
>> > | grass71 run r.external input=elevation.tiff output=elevation
>> > | grass71 unsetmapset
>> > `----
>>
>> Looks much nicer - I agree.
>>
>> >
>> > Which subcommands and how to name them is of course a separate question.
>> > I'm quite comfortable with `run` but not so much with `setmapset` and
>> > `unsetmapset`.
>>
>> They were just descriptive names. And if you want to stick with the
>> subcommand options, something like
>>
>> ,----
>> | grass71 set mapset .../grassdata/test1/PERMANENT/
>> | grass71 run r.external input=basins.tiff output=basins
>> | grass71 run r.external input=elevation.tiff output=elevation
>> | grass71 unset mapset
>> `----
>
> The decision if to nest subcommands or not is one of the main disadvantages
> of this kind of interface. I'm quite unsure about the syntax you
> proposed.

I don't see the "set mapset" as a nested subcommand, rather the
parameter (mapset) which will set to the value
(.../grassdata/test1/PERMANENT/).

> Perhaps something simpler like:
>
> ,----
> | grass71 start .../grassdata/test1/PERMANENT/
> | grass71 run r.external input=basins.tiff output=basins
> | grass71 run r.external input=elevation.tiff output=elevation
> | grass71 stop
> `----
>
> would be nicer and simpler.

I like your "start" and "stop" suggestion - see my suggestion above which fits nicely
into your suggestion.

Thinking about it, probably not use "start" and "stop" as it somehow
implies to me that something is changing / running inbetween the two -
which it is not. Maybe "init" and "cleanup"?

>
>> Then one could also use
>>
>> ,----
>> | grass71 set mapset ./input
>> `----
>>
>> where input would be a file which contains something like
>>
>> ,----
>> | mapset = .../grassdata/test1/PERMANENT/
>> `----
>>
>> This would make it possible to easily run the same script for different
>> mapsets.
>
> I don't see any advantage in that. You should pass the the Mapset to your
> script as a parameter instead of hardcoding path to file in the script and
> then changing the file. If you script has a lot of parameters then some
> kind of config file is appropriate but this should be solved by the script
> itself.
>
> In script (naive bash):
>
> ,----
> | grass71 start $1
> `----
>
> Using script:
>
> ,----
> | ./the_script.sh .../grassdata/test1/PERMANENT/
> `----
>
> I think this is an appropriate syntax.

See my reasoning above of why I think that a file based approach is more
reproducible and extensible.

>
>> Another advantage of the set ... syhtax would be the flexibility to set
>> many more things without having to add additional syntax
>
> I would replace set by start because you anyway need the stop counter part
> (to delete the "gisrc" file) but you got a point that it has a precedence:
>
> ,----
> | git config user.name "John Smith"
> `----
>
> However, I have no idea what we would be setting, so I'm staying with
> start-stop pair for now. set-unset could be an addition.

Yup - that would be the easiest and fit nicely into the suggested
structure above.

>
>> >> and finally introduce a mapper around this (similar to Rscript ore
>> >> littler) so that one could say
>> >>
>> >> ,----
>> >> | grass71exec setmapset .../grassdata/test1/PERMANENT/
>> >> | grass71exec r.external input=basins.tiff output=basins
>> >> | grass71exec r.external input=elevation.tiff output=elevation
>> >> | grass71exec unsetmapset
>> >> `----
>> >
>> > I don't see much advantage in this comparing to grass71 --exec for the
>> > usages other than shebang which requires special approach anyway (see
>> > below).
>>
>> Exactly - the shebang would be the target here.
>
> But with shebang we have just one command to do, so no `setmapset` is
> possible (see below).
>
>> >> and could use grass71exec as a shebang - that would be the ultimate
>> >> convenience.
>> >
>> > This would make sense. GRASS looks like an interpreter by requiring
> things
>> > to run inside, so it should behave as an interpreter.
>> >
>> > Shebang may require to have a special wrapper to set some parameters
> (since
>> > shebang allows just one short parameter and no parameter when
> /usr/bin/env
>> > is used) but the challenge here is much bigger: Who will set the Mapset?
>>
>> The first command has to set the mapset, with something like
>>
>> set mapset ./input
>>
>> or
>>
>> ,----
>> | set mapset .../grassdata/test1/PERMANENT/
>> `----
>
> Perhaps this is where we diverge. But maybe I'm wrong because I'm for a
> strange compromise. I don't think GRASS should actual interpret the file
> itself. It should be possible to use grass in shebang but then GRASS should
> execute the file using whatever interpreter is appropriate (Bash, Python,
> R, ...). Once in GRASS session, no special
>
> g.mapset mapset=PERMANENT -c --q
>
> The only issue is there we need to introduce inconsistency, so far all the
> scripts executed in GRASS were *guaranteed to have* Mapset already set,
> script executed in this way would be *required to set* the Mapset at the
> very beginning.
>
>> and I don't know if it would be feasible (and if it can be done with the
>> same command), if in a shebang script, the cleaning and initialisation
>> only needs to be done when "set mapset" is called.
>>
>> >
>> > As I recently discussed on grass-dev, GRASS session is runtime+data [1].
>> > Shebang version could just set up runtime while the script would be
>> > responsible for the data (setting Mapset).
>>
>> This is how I would see it.
>>
>> > The implementation/design challenge is then who will create and delete
>> > the "gisrc" file which keeps info about the Mapset. GRASS would have
>> > to know that it is not supposed to require Mapset to be set when
>> > starting nor read the last used one from the user settings (I'm not
>> > sure what are the options to achieve this except for special shebang
>> > parameter/mode).
>>
>> I don't think a special shebang parameter / mode would be a problem from
>> the user side, if the only difference is the cleaning up.
>>
>> >
>> > Another challenge is how to execute the script in GRASS session because
> you
>> > need another shebang to tell you how to actually execute the script. So
> my
>> > idea so far is to have:
>> >
>> > #!grass71exec
>> > #!python
>> >
>> > where first line would by used by system to call grass program, then
> GRASS
>> > would copy the file, delete the first line and actually execute this
> copy.
>> > GRASS would have to know that it is supposed to do that (special shebang
>> > parameter/mode, two shebang lines, or special syntax for the second
>> > shebang).
>>
>> Sorry - I don't follow you here. I have used GRASS only from R or bash
>> terminal and I have never used python.
>
> I had to try it to understand. First shebang (#!/usr/bin/grass71exec) would
> be used by system to call GRASS. But GRASS has to execute the file somehow.
> GRASS is using system for that. System knows what to use thanks to the
> shebang, but shebang is set to GRASS now and we don't want that. So GRASS
> has to change the shebang to something else (which could be stored in file
> as another shebang, e.g. #!/usr/bin/python).
>
> Another way would be to have several grass71s for shebang, one for each
> language we want to support. grass71python would use Python associated with
> GRASS. But we would have to explicitly support each interpreter.

I think I get it what you are aiming at. But wouldn't then a command be
a more versatile option? something like in a

,----
| #!grass71exec
| set interpreter /usr/bin/python <- sets the interpreter to python
| start                           <- starts / initialises the session
| DO SOME STUFF in python         <- these are python commands
| stop                            <- ends / finalizes pyhton session
| set interpreter /usr/bin/bash   <- sets bash interpreter 
| start                           <- starts / initialises the session 
| DO SOME STUFF in bash           <- these are bash commands as in GRASS
| stop                            <- ends / finalizes bash session 
`----

or with single commands in bash:

,----
| grass71 set interpreter /usr/bin/python
| grass71 start
| grass71 DO SOME STUFF in python
| grass71 stop
| grass71 set interpreter /usr/bin/bash
| grass71 start
| grass71 DO SOME STUFF in bash
| grass71 stop
`----

shouldn't this work as well, and be more flexible?

And in this case, one could use config files nicely to setup the
different languages (and possibly mapsets as well).

Just one thought: at the moment, I think that only *one* mapset can be
set for one user at a time using --exec - correct? would it be possible to make it
such that multiple independent so that in different shells (bash,
python, ...) different mapsets can be used?


>
>> But I must say this sounds very interesting and I think the sub command
>> would be an important next step. As soon as it is implemented, I will
>> see that I adapt the rgrass7 interface so that I can see how it goes
>> From that side.
>
> If you want to try it, don't wait for subcommand interface. I cannot
> implement it now. You can get a lot from the --exec interface too.

OK - I will look at it. I'll keep you via grass-stats posted about my
progress there.

> The difference is that you have to pass Mapset as a parameter every
> time,

Not really, as if none is specified, the one stored for the normal
startup of grass70 is used - which is not that nice.

> with subcommand interface, you will have to be in one directory for
> things to work (or alternatively set environment variables if
> implemented).

Thanks a lot again for this, I really like it,

Rainer

>
> Vaclav
> _______________________________________________
> grass-user mailing list
> grass-user at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/grass-user

-- 
Rainer M. Krug
email: Rainer<at>krugs<dot>de
PGP: 0x0F52F982
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 494 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20150605/d47ffea7/attachment-0001.pgp>