[GRASS-dev] list of GRASS libs when compiling

[Glynn Clements]

Markus Neteler wrote:

> the recent changes to include/Grass.make.in seem to have
> triggered this impressive list of replicated entries:
> 
> (example NVIZ):
> gcc ... pick_vect_commands.o OBJ.i686-pc-linux-gnu/site_attr_commands.o OBJ.i686-pc-linux-gnu/site_highlight_commands.o   -lgrass_ogsf -lgrass_bitmap -lgrass_linkm  -lgrass_g3d -lgrass_gis -lgrass_datetime -lz      -lgrass_gis -lgrass_datetime -lz     -lgrass_sites -lgrass_datetime -lgrass_dbmibase -lgrass_gis -lgrass_datetime -lz      -lgrass_dbmiclient -lgrass_dbmibase -lgrass_gis -lgrass_datetime -lz      -lgrass_gis -lgrass_datetime -lz      -lgrass_gis -lgrass_datetime -lz     -lgrass_vect -lgrass_dbmibase -lgrass_gis -lgrass_datetime -lz      -lgrass_dbmiclient -lgrass_dbmibase -lgrass_gis -lgrass_datetime -lz      -lgrass_gis -lgrass_datetime -lz      -lgrass_dgl -lgrass_dig2 -lgrass_gis -lgrass_datetime -lz     -lgrass_rtree  -lgrass_gis -lgrass_datetime -lz     -lgrass_linkm -lgrass_rtree   -lgrass_vect -lgrass_dbmibase -lgrass_gis -lgrass_datetime -lz      -lgrass_dbmiclient -lgrass_dbmibase -lgrass_gis -lgrass_datetime -lz      -lgrass_gis -lgrass_datetime -lz  !
   !
>    -lgrass_dgl -lgrass_dig2 -lgrass_gis -lgrass_datetime -lz     -lgrass_rtree  -lgrass_gis -lgrass_datetime -lz     -lgrass_linkm -lgrass_rtree   \
> -lgrass_bitmap -lgrass_linkm  -lgrass_linkm -lgrass_vect -lgrass_dbmibase -lgrass_gis -lgrass_datetime -lz      -lgrass_dbmiclient -lgrass_dbmibase -lgrass_gis -lgrass_datetime -lz      -lgrass_gis -lgrass_datetime -lz      -lgrass_dgl -lgrass_dig2 -lgrass_gis -lgrass_datetime -lz     -lgrass_rtree  -lgrass_gis -lgrass_datetime -lz     -lgrass_linkm -lgrass_rtree  -lgrass_dig2 -lgrass_gis -lgrass_datetime -lz     -lgrass_rtree  -lgrass_dgl -lgrass_rtree -lgrass_linkm -lgrass_dbmiclient -lgrass_dbmibase -lgrass_gis -lgrass_datetime -lz      -lgrass_gis -lgrass_datetime -lz      -lgrass_dbmibase -lgrass_gis -lgrass_datetime -lz       -L/usr/local/lib -lgdal -lgrass_sites -lgrass_datetime -lgrass_dbmibase -lgrass_gis -lgrass_datetime -lz      -lgrass_dbmiclient -lgrass_dbmibase -lgrass_gis -lgrass_datetime -lz      -lgrass_gis -lgrass_datetime -lz      -lgrass_gis -lgrass_datetime -lz     -lgrass_vect -lgrass_dbmibase -lgrass_gis -lgrass_datetime -lz      -lgrass_dbmiclient -!
 lg!
>  rass_dbmibase -lgrass_gis -lgrass_datetime -lz      -lgrass_gis -lgrass_datetime -lz      -lgrass_dgl -lgrass_dig2 -lgrass_gis -lgrass_datetime -lz     -lgrass_rtree  -lgrass_gis -lgrass_datetime -lz     -lgrass_linkm -lgrass_rtree   -lgrass_form -lgrass_dbmibase -lgrass_gis -lgrass_datetime -lz      -lgrass_dbmiclient -lgrass_dbmibase -lgrass_gis -lgrass_datetime -lz      -lgrass_gis -lgrass_datetime -lz      -lgrass_gis -lgrass_datetime -lz      -lgrass_dbmiclient -lgrass_dbmibase -lgrass_gis -lgrass_datetime -lz      -lgrass_gis -lgrass_datetime -lz      -lgrass_dbmibase -lgrass_gis -lgrass_datetime -lz       \
> -lgrass_g3d -lgrass_gis -lgrass_datetime -lz      -lgrass_gis -lgrass_datetime -lz     -lz  \
> -lgrass_datetime  -ltk -lm  -ltcl -lm -lGLU  -lGL  -L/usr/X11R6/lib -lSM -lICE -lX11 -lXmu -lXext  -lm   -ltiff  \
>   -lm
> 
> While it doesn't harm, it looks a bit strange to me.

You can probably trim the list down a bit by removing any indirect
dependencies from the module's Makefile. Now that the *LIB variables
include all necessary dependencies, a module's Makefile only needs to
list the libraries which it uses directly.

[I've attached an SQL script which can determine this information from
the database created by tools/sql.sh.]

It could be significantly reduced by changing Grass.make.in so that
the *LIB definitions list the libraries explicitly rather than using
the $(...LIB) variables. E.g. given:

RASTERLIB     = -l$(RASTER_LIBNAME) $(PNGDRIVERLIB) $(DRIVERLIB) $(GISLIB) 
PNGDRIVERLIB  = -l$(PNGDRIVER_LIBNAME) $(DRIVERLIB) $(GISLIB) $(PNGLIB) 
DRIVERLIB     = -l$(DRIVER_LIBNAME) $(GISLIB) $(FTLIB) $(ICONVLIB) 

Substituting for $(PNGDRIVERLIB) gives:

RASTERLIB     = -l$(RASTER_LIBNAME) -l$(PNGDRIVER_LIBNAME) $(DRIVERLIB) $(GISLIB) $(PNGLIB)  $(DRIVERLIB) $(GISLIB) 
DRIVERLIB     = -l$(DRIVER_LIBNAME) $(GISLIB) $(FTLIB) $(ICONVLIB) 

and substituting for $(DRIVERLIB) (both times) gives:

RASTERLIB     = -l$(RASTER_LIBNAME) -l$(PNGDRIVER_LIBNAME) -l$(DRIVER_LIBNAME) $(GISLIB) $(FTLIB) $(ICONVLIB) $(GISLIB) $(PNGLIB) -l$(DRIVER_LIBNAME) $(GISLIB) $(FTLIB) $(ICONVLIB) $(GISLIB) 

i.e. 4 copies of $(GISLIB), and 2 of -l$(DRIVER_LIBNAME), $(FTLIB) and
$(ICONVLIB).

This could be reduced to:

RASTERLIB     = -l$(RASTER_LIBNAME) -l$(PNGDRIVER_LIBNAME) $(PNGLIB) -l$(DRIVER_LIBNAME) $(FTLIB) $(ICONVLIB) $(GISLIB)

[In practice, $(GISLIB) would also get substitued to eliminate
potential duplicates for $(DATETIMELIB), but that's not relevant to
this example.]

However, that's fragile, as anyone adding a dependency to a library
must remember to add it to any libraries which have that library as a
dependency. Given that the dependencies don't matter when using shared
libraries on Linux (or on other platforms where an existing GRASS
installation is present in the shared library path), it's likely that
this will get overlooked.

Even with that, you will still get dependencies when a library is both
a direct dependency of the module and a dependency of one of the
module's dependent libraries. The alternative is to require each
module to list all of its direct and indirect dependencies in the
correct order.

[Again, it's easy to overlook this if you're using shared libraries,
as the order doesn't matter there. We only find out when someone tries
building with static libraries and gets "undefined symbol" errors.]

If we were particularly concerned about this, it should be possible to
write a make function which will remove all copies of a particular -l
switch except for the last, and to use this function within the
rule(s) for building executables (i.e. the "cmd" and "etc" targets). 
However, that's not entirely trivial, given the fairly limited range
of built-in functions provided by make.

[This won't work if you have circular dependencies and are building
with static libraries; in that situation, you may have to list a
library more than once. I think we've eradicated all of the circular
dependencies, though.]

-- 
Glynn Clements <glynn at gclements.plus.com>

-------------- next part --------------
DROP FUNCTION concat(text, text) CASCADE ;

CREATE FUNCTION concat(text, text) RETURNS text AS $$
  SELECT $1 || ' ' || $2;
 $$ LANGUAGE SQL ;

CREATE AGGREGATE mklist (
 sfunc = concat ,
 basetype = text ,
 stype = text ,
 initcond = ''
) ;

DROP TABLE all_libs ;
CREATE TABLE all_libs AS
 SELECT DISTINCT
   substring(exp.library, '^libgrass_\(.*\)\.6\.3\.cvs\.so$') AS library ,
   symbol
  FROM shlib_exp AS exp
 UNION
 SELECT DISTINCT
   substring(exp.library, '^lib\(.*\)\.so') AS library ,
   symbol
  FROM libs AS exp
  WHERE exp.library NOT LIKE 'libgrass_%'
;

SELECT s.dir, mklist(s.lib)
 FROM (
  SELECT DISTINCT
    substring(imp.object, '^\(.*\)/OBJ.') AS dir ,
    exp.library AS lib
   FROM
    obj_imp AS imp ,
    all_libs AS exp
   WHERE imp.symbol = exp.symbol
   AND exp.library IS NOT NULL
   AND exp.library <> 'c'
   ORDER BY dir, lib
  ) AS s
 GROUP BY s.dir
 ORDER BY s.dir
;