[GRASS-dev] [grass-code I][359] GRASS fails to build with make -j3

[Glynn Clements]

Markus Neteler wrote:

> > > code I item #359, was opened at 2007-04-05 12:01
> > > Summary: GRASS fails to build with make -j3 

> > However: I'm not sure whether parallel make will actually be useful
> > with the existing Makefiles, due to the use of a shell "for" loop for
> > building subdirectories:
> > 
> > 	subdirs:
> > 		@list='$(SUBDIRS)'; \
> > 		for subdir in $$list; do \
> > 		    echo $$subdir ; \
> > 		    $(MAKE) -C $$subdir || echo $(CURDIR)/$$subdir >> $(GRASS_HOME)/error.log; \
> > 		done
> > 
> > [From include/Make/Dir.make]
> > 
> > To have the subdirectories built in parallel, you would need to use
> > something like:
> > 
> > 	.PHONY: subdirs $(SUBDIRS)
> > 
> > 	subdirs: $(SUBDIRS)
> > 
> > 	$(SUBDIRS):
> > 		$(MAKE) -C $@
> 
> I am interested to get this working but failed to get the
> second part in include/Make/Dir.make successfully running.
> 
> Any further hints? My Makefile knowledge is fairly limited
> unfortunately.

Note that this cannot be done for all directories. In particular, the
inter-dependencies between libraries mean that you cannot build the
lib/* directories in an arbitrary order. You would have to either use
the existing "for" loop in lib/Makefile, or explicitly list the
dependencies, e.g.:

	gis: datetime
	driver: gis
	pngdriver: driver gis 
	psdriver: driver gis 
	raster: driver gis pngdriver psdriver
	display: gis raster

Using the naïve parallel version for the lib directory will result in
severe failure, due to attempting to link other libraries against
libgis before libgis has been generated.

The top-level directories may as well be built sequentially, but
presumably you would want to parallelise within the raster, vector,
general etc directories. Parallelising lib is less important as most
of the subdirectories contain enough files that parallelising within
individual lib/* subdirectories would probably be sufficient.

As a first attempt, I'd suggest trying the above idiom in Dir.make,
and using the existing loop in lib/Makefile (name the target somthing
other than "subdirs" to avoid a conflict with Dir.make). Hopefully,
that should keep the number of errors down to manageable proportions
(it's a safe bet that at least some of the individual module/lib
Makefiles rely upon sequential processing).

-- 
Glynn Clements <glynn at gclements.plus.com>