[GRASS-user] Announcing new command line interface of `grass` program in trunk

Thu Jun 4 12:25:16 PDT 2015

Vaclav Petras <wenzeslaus at gmail.com> writes:

> Thank you all for the comments. Here are some more from me.
>
> On Thu, Jun 4, 2015 at 3:53 AM, Rainer M Krug <Rainer at krugs.de> wrote:
>>
>> Vaclav Petras <wenzeslaus at gmail.com> writes:
>>
>> > Hi all,
>> >
>> > I'm happy to announce new command line interface of the `grass` program
> [1,
>> > 2] available in trunk.
>>
>> This sounds very exciting - and will make life much easier. Thanks a
>> lot.
>>
>> It might be to early to ask, but are there any penalties is speed when
>> using this construct compared with the "traditional" way?
>
> It is not too early. The time you have to currently pay is approximately
> 0.05 s. However, you pay for having the full session with all checks,
> cleanups and Mapset locking. Although the execution time is much longer
> than when just setting the variables manually (when you don't execute any
> cleanups, for example), the time is relatively short when running some
> computation.

0.05s is not to bad ion most cases, as the computation of the command
time will usually be much longer.

>
> Unfortunately, the price has to be paid every time you call a module. Then
> the price can grow especially if you call something many times (e.g. in a
> loop). This can be dealt with by not executing individual modules directly
> and creating a script (or module) for GRASS instead. Although this is less
> convenient then direct execution of modules, it brings advantages of
> writing Python script with GRASS Python API, script which can become true
> GRASS module and finally, and finally it separates GRASS-related
> functionality from your script (separation is good for maintenance and
> reuse). In any case, the goal is to keep the time as low as possible to
> support as many use cases as possible.

That leads to the shebang point below - I will come back to it later.

>
> And here is my simple benchmark:
>
> time grass71 ~/grassdata/nc_spm_08_grass7/PERMANENT/ --exec echo "from
> GRASS"
>
> real    0m0.069s
> user    0m0.048s
> sys     0m0.008s
>

Thanks.

>>> >
>> > Certainly you already know that you can start GRASS GIS session in the
>> > specified Location and Mapset in the following way:
>> >
>> > grass71 .../grassdata/location/mapset/
>> >
>> > The new interface adds `--exec` flag for specifying a command which
> will be
>> > executed inside that GRASS session:
>> >
>> > grass71 .../grassdata/location/mapset/ --exec r.univar map=elevation
>>
>> Am I right in assuming, that the --exec effectively splits the command
>> into the "setup part" (mapset) before and the "command part" after and
>> whatever follows will be executed?
>
> Exactly, it splits the command line: standard `grass` program parameters
> are before --exec, command to execute is after.

Perfect.

>
>> What happens when one specifies a shell command - I assume it will it
> also be
>> executed?
>
> Right, anything is executed, e.g.
> env
> RScript...
> python -c "import grass.script as gscript; gscript.run_command('g.region',
> flags='g')"
> r.external --ui
> g.gui -f
>
> Just note that the shell syntax is interpreted in the current shell (e.g.
> $VARIABLE), `grass` program gets just plain parameters to execute.

Good to know and might be useful (I just don't know for what, at the
moment - but I am sure there is something...).

>
>> I guess that if this is working, the next step would be to introduce the
>> single command version so that one does not have to specify the mapset
>> each time. E.g:
>
> Yes!

Great.

> [1, 2] The code now allows to introduce such interface. The challenge
> is where to store the information about active Location and Mapset.
> Currently, .gisrc (or .grassrc or .grassgisrc) file stored in the
> current working directory seems as the best option to me. Another
> option is to use environment variables [3, 4] but I don't consider it
> a nice solution.

I agree - unless it is in a scrip with shebang #!grass71exec or
similar. Then it would be easy to use environmental variables.

What about a user specified name? This would make switching mapsets in a
script much more symbolic, as one does not has to specify the path of
the mapset and just an easy to remember name? This would make it possible to
setup a whole bunch of mapsets associated with names in a text file and
then just load the .grassEXEC file?

>
> [1] https://trac.osgeo.org/grass/ticket/2579#Additionalideas
> [2] https://trac.osgeo.org/grass/ticket/2579#comment:10
> [3] https://trac.osgeo.org/grass/ticket/2679
> [4] https://trac.osgeo.org/grass/ticket/2681
>
>> ,----
>> | grass71 --setmapset .../grassdata/test1/PERMANENT/
>> | grass71 --exec r.external input=basins.tiff output=basins
>> | grass71 --exec r.external input=elevation.tiff output=elevation
>> | grass71 --unsetmapset
>> `----
>
> Rather than this, I prefer subcommand interface as known from git, docker
> or apt-get:
>
> ,----
> | grass71 setmapset .../grassdata/test1/PERMANENT/
> | grass71 run r.external input=basins.tiff output=basins
> | grass71 run r.external input=elevation.tiff output=elevation
> | grass71 unsetmapset
> `----

Looks much nicer - I agree.

>
> Which subcommands and how to name them is of course a separate question.
> I'm quite comfortable with `run` but not so much with `setmapset` and
> `unsetmapset`.

They were just descriptive names. And if you want to stick with the
subcommand options, something like

,----
| grass71 set mapset .../grassdata/test1/PERMANENT/
| grass71 run r.external input=basins.tiff output=basins
| grass71 run r.external input=elevation.tiff output=elevation
| grass71 unset mapset
`----

Then one could also use

,----
| grass71 set mapset ./input
`----

where input would be a file which contains something like

,----
| mapset = .../grassdata/test1/PERMANENT/
`----

This would make it possible to easily run the same script for different
mapsets.

Another advantage of the set ... syhtax would be the flexibility to set
many more things without having to add additional syntax

>
>> and finally introduce a mapper around this (similar to Rscript ore
>> littler) so that one could say
>>
>> ,----
>> | grass71exec setmapset .../grassdata/test1/PERMANENT/
>> | grass71exec r.external input=basins.tiff output=basins
>> | grass71exec r.external input=elevation.tiff output=elevation
>> | grass71exec unsetmapset
>> `----
>
> I don't see much advantage in this comparing to grass71 --exec for the
> usages other than shebang which requires special approach anyway (see
> below).

Exactly - the shebang would be the target here.

>
>> and could use grass71exec as a shebang - that would be the ultimate
>> convenience.
>
> This would make sense. GRASS looks like an interpreter by requiring things
> to run inside, so it should behave as an interpreter.
>
> Shebang may require to have a special wrapper to set some parameters (since
> shebang allows just one short parameter and no parameter when /usr/bin/env
> is used) but the challenge here is much bigger: Who will set the Mapset?

The first command has to set the mapset, with something like

set mapset ./input

or

,----
| set mapset .../grassdata/test1/PERMANENT/
`----

and I don't know if it would be feasible (and if it can be done with the
same command), if in a shebang script, the cleaning and initialisation
only needs to be done when "set mapset" is called.

>
> As I recently discussed on grass-dev, GRASS session is runtime+data [1].
> Shebang version could just set up runtime while the script would be
> responsible for the data (setting Mapset).

This is how I would see it.

> The implementation/design challenge is then who will create and delete
> the "gisrc" file which keeps info about the Mapset. GRASS would have
> to know that it is not supposed to require Mapset to be set when
> starting nor read the last used one from the user settings (I'm not
> sure what are the options to achieve this except for special shebang
> parameter/mode).

I don't think a special shebang parameter / mode would be a problem from
the user side, if the only difference is the cleaning up.

>
> Another challenge is how to execute the script in GRASS session because you
> need another shebang to tell you how to actually execute the script. So my
> idea so far is to have:
>
> #!grass71exec
> #!python
>
> where first line would by used by system to call grass program, then GRASS
> would copy the file, delete the first line and actually execute this copy.
> GRASS would have to know that it is supposed to do that (special shebang
> parameter/mode, two shebang lines, or special syntax for the second
> shebang).

Sorry - I don't follow you here. I have used GRASS only from R or bash
terminal and I have never used python.

But I must say this sounds very interesting and I think the sub command
would be an important next step. As soon as it is implemented, I will
see that I adapt the rgrass7 interface so that I can see how it goes
From that side.

Cheers,

Rainer

>
> [1] http://lists.osgeo.org/pipermail/grass-dev/2015-June/075203.html
>
>> Thanks a lot,
>>
>> Rainer
>>
>> >
>> > This starts GRASS session, executes whatever command is specified after
>> > `--exec` flag and then ends showing the (text) output of the module as
> if
>> > the module would be executed manually in the GRASS session. And example
>> > showing this in combination with r.external to get the data into the
> Mapset
>> > is in the documentation [3].
>> >
>> > This interface is meant to remove the need for lengthy manual setup of
>> > environmental variables followed by execution of GRASS modules in the
>> > mock-up GRASS session [4]. The idea is to use correct GRASS session
>> > maintained by `grass` program which means that any GRASS-related code
> must
>> > be passed to the `grass` program as opposed to direct execution in the
>> > former case.
>> >
>> > The interface can run any module or custom script with or without
>> > parameters. GUI tools including g.gui [5] can be called as well which
> opens
>> > new possibilities for application developers.
>> >
>> > The interface is the first implementation and currently is fully
>> > operational only on Linux and similar systems but it benefits from a
> lot of
>> > stable code which was already in place. Try it out and feel free to
> comment
>> > here, on grass-dev or in the related ticket [6]. There is definitively
> many
>> > potential improvements and it would be great to know what the community
>> > demands.
>> >
>> > Vaclav
>> >
>> > [1] https://trac.osgeo.org/grass/changeset/65252
>> > [2] https://trac.osgeo.org/grass/changeset/65294
>> > [3]
>> >
> http://grass.osgeo.org/grass71/manuals/grass7.html#exec-interface-example
>> > [4]
>> >
> http://grasswiki.osgeo.org/wiki/Working_with_GRASS_without_starting_it_explicitly
>> > [5] https://trac.osgeo.org/grass/changeset/65306
>> > [6] https://trac.osgeo.org/grass/ticket/2579
>> > _______________________________________________
>> > grass-user mailing list
>> > grass-user at lists.osgeo.org
>> > http://lists.osgeo.org/mailman/listinfo/grass-user
>>
>> --
>> Rainer M. Krug
>> email: Rainer<at>krugs<dot>de
>> PGP: 0x0F52F982
> _______________________________________________
> grass-user mailing list
> grass-user at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/grass-user

-- 
Rainer M. Krug
email: Rainer<at>krugs<dot>de
PGP: 0x0F52F982
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 494 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20150604/e87c2745/attachment.pgp>