[GRASS-user] Announcing new command line interface of `grass` program in trunk

Thu Jun 4 08:40:02 PDT 2015

Thank you all for the comments. Here are some more from me.

On Thu, Jun 4, 2015 at 3:53 AM, Rainer M Krug <Rainer at krugs.de> wrote:
>
> Vaclav Petras <wenzeslaus at gmail.com> writes:
>
> > Hi all,
> >
> > I'm happy to announce new command line interface of the `grass` program
[1,
> > 2] available in trunk.
>
> This sounds very exciting - and will make life much easier. Thanks a
> lot.
>
> It might be to early to ask, but are there any penalties is speed when
> using this construct compared with the "traditional" way?

It is not too early. The time you have to currently pay is approximately
0.05 s. However, you pay for having the full session with all checks,
cleanups and Mapset locking. Although the execution time is much longer
than when just setting the variables manually (when you don't execute any
cleanups, for example), the time is relatively short when running some
computation.

Unfortunately, the price has to be paid every time you call a module. Then
the price can grow especially if you call something many times (e.g. in a
loop). This can be dealt with by not executing individual modules directly
and creating a script (or module) for GRASS instead. Although this is less
convenient then direct execution of modules, it brings advantages of
writing Python script with GRASS Python API, script which can become true
GRASS module and finally, and finally it separates GRASS-related
functionality from your script (separation is good for maintenance and
reuse). In any case, the goal is to keep the time as low as possible to
support as many use cases as possible.

And here is my simple benchmark:

time grass71 ~/grassdata/nc_spm_08_grass7/PERMANENT/ --exec echo "from
GRASS"

real    0m0.069s
user    0m0.048s
sys     0m0.008s

>> >
> > Certainly you already know that you can start GRASS GIS session in the
> > specified Location and Mapset in the following way:
> >
> > grass71 .../grassdata/location/mapset/
> >
> > The new interface adds `--exec` flag for specifying a command which
will be
> > executed inside that GRASS session:
> >
> > grass71 .../grassdata/location/mapset/ --exec r.univar map=elevation
>
> Am I right in assuming, that the --exec effectively splits the command
> into the "setup part" (mapset) before and the "command part" after and
> whatever follows will be executed?

Exactly, it splits the command line: standard `grass` program parameters
are before --exec, command to execute is after.

> What happens when one specifies a shell command - I assume it will it
also be
> executed?

Right, anything is executed, e.g.
env
RScript...
python -c "import grass.script as gscript; gscript.run_command('g.region',
flags='g')"
r.external --ui
g.gui -f

Just note that the shell syntax is interpreted in the current shell (e.g.
$VARIABLE), `grass` program gets just plain parameters to execute.

> I guess that if this is working, the next step would be to introduce the
> single command version so that one does not have to specify the mapset
> each time. E.g:

Yes! [1, 2] The code now allows to introduce such interface. The challenge
is where to store the information about active Location and Mapset.
Currently, .gisrc (or .grassrc or .grassgisrc) file stored in the current
working directory seems as the best option to me. Another option is to use
environment variables [3, 4] but I don't consider it a nice solution.

[1] https://trac.osgeo.org/grass/ticket/2579#Additionalideas
[2] https://trac.osgeo.org/grass/ticket/2579#comment:10
[3] https://trac.osgeo.org/grass/ticket/2679
[4] https://trac.osgeo.org/grass/ticket/2681

> ,----
> | grass71 --setmapset .../grassdata/test1/PERMANENT/
> | grass71 --exec r.external input=basins.tiff output=basins
> | grass71 --exec r.external input=elevation.tiff output=elevation
> | grass71 --unsetmapset
> `----

Rather than this, I prefer subcommand interface as known from git, docker
or apt-get:

,----
| grass71 setmapset .../grassdata/test1/PERMANENT/
| grass71 run r.external input=basins.tiff output=basins
| grass71 run r.external input=elevation.tiff output=elevation
| grass71 unsetmapset
`----

Which subcommands and how to name them is of course a separate question.
I'm quite comfortable with `run` but not so much with `setmapset` and
`unsetmapset`.

> and finally introduce a mapper around this (similar to Rscript ore
> littler) so that one could say
>
> ,----
> | grass71exec setmapset .../grassdata/test1/PERMANENT/
> | grass71exec r.external input=basins.tiff output=basins
> | grass71exec r.external input=elevation.tiff output=elevation
> | grass71exec unsetmapset
> `----

I don't see much advantage in this comparing to grass71 --exec for the
usages other than shebang which requires special approach anyway (see
below).

> and could use grass71exec as a shebang - that would be the ultimate
> convenience.

This would make sense. GRASS looks like an interpreter by requiring things
to run inside, so it should behave as an interpreter.

Shebang may require to have a special wrapper to set some parameters (since
shebang allows just one short parameter and no parameter when /usr/bin/env
is used) but the challenge here is much bigger: Who will set the Mapset?

As I recently discussed on grass-dev, GRASS session is runtime+data [1].
Shebang version could just set up runtime while the script would be
responsible for the data (setting Mapset). The implementation/design
challenge is then who will create and delete the "gisrc" file which keeps
info about the Mapset. GRASS would have to know that it is not supposed to
require Mapset to be set when starting nor read the last used one from the
user settings (I'm not sure what are the options to achieve this except for
special shebang parameter/mode).

Another challenge is how to execute the script in GRASS session because you
need another shebang to tell you how to actually execute the script. So my
idea so far is to have:

#!grass71exec
#!python

where first line would by used by system to call grass program, then GRASS
would copy the file, delete the first line and actually execute this copy.
GRASS would have to know that it is supposed to do that (special shebang
parameter/mode, two shebang lines, or special syntax for the second
shebang).

[1] http://lists.osgeo.org/pipermail/grass-dev/2015-June/075203.html

> Thanks a lot,
>
> Rainer
>
> >
> > This starts GRASS session, executes whatever command is specified after
> > `--exec` flag and then ends showing the (text) output of the module as
if
> > the module would be executed manually in the GRASS session. And example
> > showing this in combination with r.external to get the data into the
Mapset
> > is in the documentation [3].
> >
> > This interface is meant to remove the need for lengthy manual setup of
> > environmental variables followed by execution of GRASS modules in the
> > mock-up GRASS session [4]. The idea is to use correct GRASS session
> > maintained by `grass` program which means that any GRASS-related code
must
> > be passed to the `grass` program as opposed to direct execution in the
> > former case.
> >
> > The interface can run any module or custom script with or without
> > parameters. GUI tools including g.gui [5] can be called as well which
opens
> > new possibilities for application developers.
> >
> > The interface is the first implementation and currently is fully
> > operational only on Linux and similar systems but it benefits from a
lot of
> > stable code which was already in place. Try it out and feel free to
comment
> > here, on grass-dev or in the related ticket [6]. There is definitively
many
> > potential improvements and it would be great to know what the community
> > demands.
> >
> > Vaclav
> >
> > [1] https://trac.osgeo.org/grass/changeset/65252
> > [2] https://trac.osgeo.org/grass/changeset/65294
> > [3]
> >
http://grass.osgeo.org/grass71/manuals/grass7.html#exec-interface-example
> > [4]
> >
http://grasswiki.osgeo.org/wiki/Working_with_GRASS_without_starting_it_explicitly
> > [5] https://trac.osgeo.org/grass/changeset/65306
> > [6] https://trac.osgeo.org/grass/ticket/2579
> > _______________________________________________
> > grass-user mailing list
> > grass-user at lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/grass-user
>
> --
> Rainer M. Krug
> email: Rainer<at>krugs<dot>de
> PGP: 0x0F52F982
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20150604/37af48aa/attachment.html>