[GRASS-dev] Python handling in winGRASS7 [was: Re: GRASS GIS 7 tech-preview release preparations]

[Moritz Lennert]

On Mon, July 29, 2013 16:13, Moritz Lennert wrote:
> On 26/07/13 20:18, Glynn Clements wrote:
>> Moritz Lennert wrote:
>>> Would it be a possible option to implement only a) and document the
fact
>>> that the user has to take care of creating a working Python
>>> installation
>>> herself if they want b) ?
>> In order to be able to "execute" scripts via the command prompt, batch
files, system(), subprocess.Popen() or similar, the extension must be
associated with an interpreter in the registry. This cannot be
overridden via environment variables.
>> Note that Python 3 includes a launcher utility for Windows:
>> 	http://docs.python.org/3/using/windows.html#launcher
>> The idea is that the .py extension can be associated with the
>> launcher, which can be configured to use a specific version of Python
in a specific context.
>> I don't know whether it's possible to make use of this without
>> installing the rest of Python 3, though.
> I just tested the following on a WinXP machine without Python installed
(except for the Python in the GRASS installation):
> - Go to https://bitbucket.org/vinay.sajip/pylauncher
> - Download launchwin.msi
> (https://bitbucket.org/vinay.sajip/pylauncher/downloads/launchwin.msi) -
Install by double-clicking or with msiexec /i launcher.msi
> Result:
> - I can call python modules from the command line (v.db.update,
> v.db.renamecolumn)
> - Calling a script from a script works (v.db.renamecolumn)
> This is without any py.ini, so IIUC just relying on the /usr/bin/env
python shebangs in the scripts !
> When I installed a 2.7.5 Python from python.org, everything still works
the same way, but launching python at the prompt gives me 2.7.5 (i.e. the
python.org version, not the GRASS installed version which is 2.7.4), so I
suppose that the system-wide installed python is launched, even though its
path is at the very end of %PATH%, after grass\extrabin. When I installed
a 3.3 version from python.org, running python from the GRASS command line
gives me 2.7.4, but running GRASS GUI gives me error
messages as does trying to launch v.db.update or other scripts at the
command line. The error message contains a reference to python33.dll.
Running py -2 gives an error that version 2 is not found. So it seems that
the python launcher does not take into account the PATH,
> PYTHONHOME, and PYTHONPATH settings which all point correctly to the
GRASS version of Python, but rather still calls the installed 3.3
version (even it the path to that version is not in PATH). I guess the
launcher takes its information from the registry.

Continuing my tests. As the details are long, I start with a summary:

****************************
- If no Python or Python 2.7 is installed system-wide, all issues known to
me with Python on Windows are solved by just installing the Python
launcher.

- If Python 3 is installed system-wide, and no Python 2, then the Python
launcher + adding an entry pointing to the GRASS Python installation +
python.exe, pythonw.exe and python27.dll copied to the Python27 directory
in the GRASS installation also solves all the issues known to me.

- Defining a specific grasspython program in the py.ini seems to work if a
default Python 2 or no Python installation exists in the registry, but not
if Python 3 is the default / only system-wide Python installation.

- It thus seems to me that the safest path is the one checking during
installation whether a Python 2.7 installation is know in the registry,
and if that is not the case, to create a registry entry that points to the
GRASS installation and then install the Python launcher. Ideally, any
other existing Python installation should be made the default in the
system-wide py.ini, but that implies that we can create a specific py.ini
for GRASS which I haven't been able to do, yet (see details below).
*****************************

Now in detail:

- Python launcher still installed as explained above.

- In order to be able to call the GRASS embedded version via the python
launcher, I added it to the registry using the script published at [1].

- However, this was not enough as the python installation in GRASS is
split over the extrabin, extralib and Python2.7 directory. Even when I
added extrabin to the PYTHONPATH defined in the registry, the launcher did
find a valid python version. When I copied python.exe and pythonw.exe from
extrabin to the Python27 directory, I got an error message about a missing
python27.dll. When I copied that from extralib/ to Python27/, I can launch
Python from the windows command line by typing py.

- After the above manipulations, I can now launch python scripts and
scripts called from scripts without using extensions and without conflict
with Python 3, even if that is the default version when launching "python"
from the windows command prompt. The only things that does not work is
launching 'py -3' from the GRASS command line as that call Python 3 but
with the GRASS Python environment variables set, leading to
incompatibilities. But I think this is specific enough as a case to put
that into known issues.

So, if during installation, we test whether there is a python 2.7 in the
registry (or whatever version the GRASS installer uses at a given moment)
and if yes, we do not touch the registry, but if not, we edit the registry
to point to the version embedded in GRASS, then things seem to work.

In theory, we should be able to add another test to see if another python
is installed and make that python the default for the launcher in the
default py.ini and configure the needed version for GRASS in the
Application Data directory of GRASS. But I tried setting python=3 in the
system-wide py.ini and python=2.7 in C:\Documents and
Settings\Lamine\Application Data\GRASS7\py.ini, but I came upon the same
conflict with python3. Is \Application Data\GRASS7 "the directory returned
by calling the Windows function SHGetFolderPath with CSIDL_LOCAL_APPDATA"
(as defined in the python docs on the launcher) ?

Finally, I tried to add a grasspython command in the py.ini and erasing
the python2.7 registry entry pointing to the GRASS python installation
(reminder: python 3 is still installed). As expected commands with a
shebang of '/usr/bin/env python' fail because python 3 is called by
default. However, I changed the shebang to '/usr/bin/env grasspython' in
v.db.addcolumn and v.db.renamecolumn. The former worked like a charm, the
second (which calls v.db.addcolumn) did not, as (I imagine) the call to
v.db.addcolumn was again handled by the default python installation, i.e.
Python 3. When I uninstall Python 3 (i.e. no more pointers to Python in
the registry), the grasspython solution works (as does the solution with
only the standard installation of the launcher). (NB: When you uninstall
Python 3 it automatically uninstall the launcher, even if you installed it
as a standalone...).


[1] http://effbot.org/zone/python-register.htm


Moritz