[GRASS-dev] [grass-code I][359] GRASS fails to build with make -j3

[Markus Neteler]

Glynn Clements wrote:
> 
> 
> Markus Neteler wrote:
> 
>> > > code I item #359, was opened at 2007-04-05 12:01
>> > > Summary: GRASS fails to build with make -j3 
> 
>> > However: I'm not sure whether parallel make will actually be useful
>> > with the existing Makefiles, due to the use of a shell "for" loop for
>> > building subdirectories:
>> > 
>> > 	subdirs:
>> > 		@list='$(SUBDIRS)'; \
>> > 		for subdir in $$list; do \
>> > 		    echo $$subdir ; \
>> > 		    $(MAKE) -C $$subdir || echo $(CURDIR)/$$subdir >>
>> $(GRASS_HOME)/error.log; \
>> > 		done
>> > 
>> > [From include/Make/Dir.make]
>> > 
>> > To have the subdirectories built in parallel, you would need to use
>> > something like:
>> > 
>> > 	.PHONY: subdirs $(SUBDIRS)
>> > 
>> > 	subdirs: $(SUBDIRS)
>> > 
>> > 	$(SUBDIRS):
>> > 		$(MAKE) -C $@
>> 
>> I am interested to get this working but failed to get the
>> second part in include/Make/Dir.make successfully running.
>> 
>> Any further hints? My Makefile knowledge is fairly limited
>> unfortunately.
> 
> Note that this cannot be done for all directories. In particular, the
> inter-dependencies between libraries mean that you cannot build the
> lib/* directories in an arbitrary order. You would have to either use
> the existing "for" loop in lib/Makefile, or explicitly list the
> dependencies, e.g.:
> 
> 	gis: datetime
> 	driver: gis
> 	pngdriver: driver gis 
> 	psdriver: driver gis 
> 	raster: driver gis pngdriver psdriver
> 	display: gis raster
> 
> Using the naïve parallel version for the lib directory will result in
> severe failure, due to attempting to link other libraries against
> libgis before libgis has been generated.
> 

Yes, confirmed.



> The top-level directories may as well be built sequentially, but
> presumably you would want to parallelise within the raster, vector,
> general etc directories. Parallelising lib is less important as most
> of the subdirectories contain enough files that parallelising within
> individual lib/* subdirectories would probably be sufficient.
> 
> As a first attempt, I'd suggest trying the above idiom in Dir.make,
> and using the existing loop in lib/Makefile (name the target somthing
> other than "subdirs" to avoid a conflict with Dir.make). Hopefully,
> that should keep the number of errors down to manageable proportions
> (it's a safe bet that at least some of the individual module/lib
> Makefiles rely upon sequential processing).
> 

Will do. 
Concerning modules: candidates for sequential processing are certainly the
r.li suite (r.li.daemon comes first), raster/simwe/lib/, vector/v.lrs/lib/
vector/lidar/lidarlib/ and so forth. For these subdirectories some trick
must
be implemented to get them sequentially processed (which this could go in
parallel with other raster/vector programs).

Anyway, slowly we'll arrive.

Markus
-- 
View this message in context: http://www.nabble.com/-grass-code-I--359--GRASS-fails-to-build-with-make--j3-tf3530677.html#a11525174
Sent from the Grass - Dev mailing list archive at Nabble.com.