[GRASS-dev] [GRASS GIS] #2132: Create a system for building and running GRASS modules written in Python

[GRASS GIS]

#2132: Create a system for building and running GRASS modules written in Python
-------------------------------------------------------+--------------------
 Reporter:  wenzeslaus                                 |       Owner:  grass-dev@…              
     Type:  defect                                     |      Status:  new                      
 Priority:  normal                                     |   Milestone:  7.0.0                    
Component:  Python                                     |     Version:  svn-trunk                
 Keywords:  makefiles, addons, path, python, packages  |    Platform:  All                      
      Cpu:  All                                        |  
-------------------------------------------------------+--------------------
 The r.modis Makefiles changed [https://trac.osgeo.org/grass/log/grass-
 addons/grass7/raster/r.modis/libmodis/Makefile?rev=54412 several times] in
 last two years and still there are problems in running it (#2097 and it
 does not work for me now).

 The main r.modis Makefile is:
 {{{
 MODULE_TOPDIR =../..

 PGM = r.modis

 SUBDIRS = \
         r.modis.download \
         r.modis.import \
                 libmodis

 include $(MODULE_TOPDIR)/include/Make/Dir.make

 default: parsubdirs htmldir

 install: installsubdirs
 }}}

 The critical r.modis Python package Makefile says:

 {{{
 MODULE_TOPDIR = ../../..

 include $(MODULE_TOPDIR)/include/Make/Other.make
 include $(MODULE_TOPDIR)/include/Make/Python.make

 MODULES = downmodis rmodislib convertmodis parsemodis

 ETCDIR = $(ETC)/r.modis

 PYFILES := $(patsubst %,$(ETCDIR)/%.py,$(MODULES))
 PYCFILES := $(patsubst %,$(ETCDIR)/%.pyc,$(MODULES))

 default: $(PYFILES) $(PYCFILES)

 $(ETCDIR):
         $(MKDIR) $@

 $(ETCDIR)/%: % | $(ETCDIR)
         $(INSTALL_DATA) $< $@

 install:
         cp -r $(ETCDIR) $(INST_DIR)
 }}}

 I was trying to understand and make it work but I don't get any results
 because I don't understand and I don't know where we actually want to
 place the Python scripts and Python packages in distribution and in addons
 if the modules are from addons.

 Can somebody design the Makefile and/or Makefile rules?

 Than we need runtime support for loading the Python packages.

 One possibility is to put all packages needed by different modules into
 one directory. Then it is enough just to import the package in the module.

 For example for core (non-addons) functionality:
 {{{
 .../grass/
 .../rmodis/
 }}}

 The other is to have one directory for each module (or module group) and
 put there their package or packages. Typically there is only one, so:

 {{{
 .../rmodis/rmodis/
 }}}

 or maybe
 {{{
 .../rmodis/rmodis/
 .../rmodis/3rdpartylib/
 }}}

 Probably in both cases we need some function which will determine where
 the package is and will add the path (`sys.path`) correctly. This function
 can be in `grass.utils`, so it will be not necessary to repeat the same
 code in all modules and then change it when there is an error in it. It
 could be something like:

 {{{
 #!python
 def add_pythonlib_to_path(name):
     libpath = None
     # this is specified by Makefile
     # maybe this needs to be different for core modules
     # and for addons
     if os.path.isdir(os.path.join(os.getenv('GISBASE'), 'etc', name)):
         libpath = os.path.join(os.getenv('GISBASE'), 'etc', name)
     elif os.getenv('GRASS_ADDON_BASE') and \
             os.path.isdir(os.path.join(os.getenv('GRASS_ADDON_BASE'),
 'etc',
                                        name)):
         libpath = os.path.join(os.getenv('GRASS_ADDON_BASE'), 'etc', name)
     # this is the directory name
     # when we run the script from source
     elif os.path.join(os.path.dirname(__file__), '..', name):
         libpath = os.path.join(os.path.dirname(__file__), '..')
     # the file with script is somewhere but package is `../package`
     # maybe this should be removed because it is strange case
     # maybe replace with current directory but that's strange too
     elif os.path.isdir(os.path.join('..', name)):
         libpath = os.path.join('..', name)
     else:
         gcore.fatal(_("Python library '%s' not found. Probably it was not"
                       "intalled correctly.") % name)

     sys.path.append(libpath)

 add_pythonlib_to_path('rmodis')
 }}}

 I think that the Makefiles for module group (i.e. directory with GRASS
 modules written in Python and their Python packages) should work for both
 addons and when just placed in the source code.

 I think that the requirement is that the module should run without running
 make. The documentation would be broken but at least it would work.

 With this we need the naming convention for packages and module group
 directory.

-- 
Ticket URL: <https://trac.osgeo.org/grass/ticket/2132>
GRASS GIS <http://grass.osgeo.org>