[GRASS-dev] GRASS extensions/addons mismatch

[Michael Barton]

Sorry that I come late to this discussion. But g.extension has never worked right for me. It is a credit to you and Martin that it is working well enough now that I finally understand what it is that doesn't work.

First, I've long had GRASS_ADDON_PATH set in my GRASS environmental variables. I *think* it used to work. But clearly from your explanation that is not the case now. This has been one point of confusion for me. Another is that William has designed the Mac startup script to set a shell environmental variable GRASS_ADDON_PATH. This overwrites anything setting I have in a .profile or other settings file. Finally, I think there are some extensions with faulty makefiles (e.g., r.stream in the trunk repository) that further confuse the issue and make testing difficulty. 

Given all that, here are some observations now that I think I understand what is going on with this. 

1) I have been testing with the bash script in GRASS 6.4svn. At least I'm pretty sure that is what I've been doing. It certainly looks like that is what I get if I press "go to command dialog" from the g.extension wrapper gui (is that what you are referring to as g.extension.py??). I go to the command dialog because the wrapper does not let me have any control over where extensions are installed (IMHO, this needs to be remedied). This matches the appearance of the gui I get when I enter g.extension from the command line, and has the same arguments as the command version of g.extension.

2) The default location for extension installation is set by a default value for the g.extension prefix= argument. This is "$GRASS_ADDON_PATH". Apparently this is a shell environmental variable rather than the GRASS environmental variable of the same name that I have set. There is currently nothing I can do to change the value of this variable due to the circumstance explained above. IMHO, that is a problem. More on that below.

3) I still don't follow all the details of the thread between you and Martin about g.extension and how it installs extensions. But it IS clear that there are two different but related functions that need to be taken care of: 1) telling g.extension where to install addons and 2) setting an internal-to-GRASS path so that if you type the name of any executable file (bin or script) in a user-specified addons directory GRASS will know to execute that file. More on that below.

So for my thoughts on #2. I see it as a problematic holdover from earlier days that GRASS needs any shell variables to perform its functions. This is a real pain on Windows and nearly as difficult to set on a Mac. Neither of these start GRASS from a terminal any more, so even exporting the variable before starting up GRASS is no longer an option. These have to be set in .profile or .bashrc for the Mac and something equivalent for Windows. And most users don't know how to alter these and probably should not be messing with them. 

William has solved this by hard coding in a value of GRASS_ADDON_PATH in the Mac startup. While it is the best we can do at the moment, it is not a very satisfying solution. I never want my extensions installed to the desktop. But since that is first in the GRASS_ADDONS_PATH list, that is where they are ALWAYS installed unless I specify another directory EVERY TIME I start g.extension. On the other hand, someone else might like having extensions installed to the desktop. So hard coding a different path order could be an equal pain for someone else.

The user should be able to set this path easily from within GRASS and have it stick. This cannot be done if we must depend on a shell environmental variable set before GRASS starts. But is EASY to do if we use the GRASS environmental variable option. Any GRASS environmental variable can be set using g.gisenv and via a nice wrapper for g.gisenv in the gui. Moreover, all these are saved between sessions. 

There should be no problem in using a GRASS variable for GRASS_ADDON_PATH instead of a shell variable. I don't know what it takes for GRASS to read a GRASS version of GRASS_ADDON_PATH to recognize a path for execution, but fixing this in g.extension is easy. One way would be to have a flag for 'default installation path' rather than inserting "$GRASS_ADDON_PATH" as a default value in the prefix= argument. 

A script would just run...

addonpath=`g.gisenv get=GRASS_ADDON_PATH`

...to store the default path in the variable addonpath

Then 'addonpath' would be used if the 'default path' flag is set or the 'prefix' argument is empty. A similar procedure could be followed with the wxPython wrapper script.

Please, let's not depend on a shell environmental variable for setting this important GRASS parameter. This is probably a large part of why g.extension has never seemed to work for me in the past. Even now, it makes installation of all the really cool GRASS addons much more cumbersome that it needs to be.

Now for some thoughts on #3. I think I understand the issues surrounding the way that GRASS_ADDON_PATH has evolved from just setting a path to also specifying the installation directory for addons. In practicality, if you define an installation directory, you will want to add this into the executables path. So the dual purpose makes sense even if it can also lead to confusion. AFAICT, the only reasons that a single environmental variable cannot be used for both purposes are twofold: 1) a path can be a list of multiple directories while installations (currently) end up in one location only, and 2) an installation path might simply be a prefix for hierarchy of subdirectories (like GISBASE) while an executables path must refer specifically to directories that contain executables. 

The first issue is being "solved" by making the first directory of the path list the default installation directory. That is a royal pain if it cannot be changed. But it would be much less of a problem if we could easily alter the path and the order of directories in the path (e.g., suggestion for #2 above). Additionally, with a little more sophisticated coding, a script could read the list of paths and install addons in all specified directories (though I'm not sure if that's what a user would want).

The second issue may be part of the core issue of the discussion between you and Martin (though I may not understand it correctly). Here, I don't see why an addons installation directory needs to include a complicated hierarchy of sub directories (e.g., replicating the directory structure of the GRASS source tree). The idea is to install binaries and scripts in the end, regardless of whether the binaries come pre-compiled or get compiled locally, right? Do any binaries or scripts NEED any other directories to function? AFAICT from the addons I've installed so far, only other directory that might be useful is the doc directory that is getting installed now. However, none of the addons that I've installed so far have a "manual" tab on their GUI. So any associated docs are not showing up any way. So if there is simply a single installation directory where binaries and scripts get installed, it can be the same as the executable directory in the path list. 

So, while we could create a second GRASS_ADDON_BASE environmental variable, it may not be necessary if the two issues raised above can be solved.

Well enough on all that. Thanks for the explanations. At least now I understand what the problems are can work around them. I do think that we could work it so such work-arounds aren't needed though.

Michael
____________________
C. Michael Barton
Director, Center for Social Dynamics & Complexity 
Professor of Anthropology, School of Human Evolution & Social Change
Arizona State University

voice: 	480-965-6262 (SHESC), 480-727-9746 (CSDC)
fax:          480-965-7671 (SHESC),  480-727-0709 (CSDC)
www: http://www.public.asu.edu/~cmbarton, http://csdc.asu.edu











On Dec 20, 2011, at 6:34 PM, Hamish wrote:

> fwiw it would be possible to have wxGUI preferences save a custom addon
> dir which it could save into ~/.grass/wx & add to the environment when
> you click Apply and also upon startup. And you can add it to ~/.grass.bashrc
> and have the shell script version of g.extension pick it up and use it.
> 
> ... but that still doesn't get it into $PATH to run it.
> 
> 
> alternatively, in init.sh we could move
>  cat "$USERHOME/.grass.bashrc" >> "$bashrc"
> 
> until after
>   echo 'export GRASS_SHELL_PID=$$' >> "$bashrc"
> 
> but then you'd have to add
>  PATH="$GRASS_ADDON_PATH:$PATH"
> 
> to your .grass.bashrc file..
> 
> It is possible to rearrange init.sh to add some conditional magic into
> $MAPSET/.bashrc so a value in ~/.grass.bashrc would be added to the
> shell's $PATH, but it is invasive enough that I would not like to touch
> it before 6.4.2. Also you'd have to do the same for csh and MinGW/other
> shells.
> 
> 
> Hamish