[Gdal-dev] improving gdal parallel build
alexamici at tiscali.it
Mon Aug 4 19:34:14 EDT 2003
On Monday 04 August 2003 21:49, Andrey Kiselev wrote:
> On Sun, Aug 03, 2003 at 11:22:34PM +0200, Alessandro Amici wrote:
> > i tweaked a couple of rules in the top level GNUmakefile in order to fix
> > parallel builds. 'make -j 200' on my home uni-processor now builds
> > everything ok. i will not be able to test on a real SMP machine for a
> > while, thou.
> > is there anybody else interested in parallel builds?
> At least you have one user: it's me.
then it's an important user ;)
> Parallel builds saves ~20% time on
> uniprocessor system and much more time on multi CPU one. I have access to
> the dual Xeon system and have ~2x compilation speed up.
was it really 2x?
i hardly belive it. there are still a lot of serial $(MAKE) calls into frmts
and ogrsf_frmts, from what i can see on my UP some more work is needed before
acheeving real linear scalability.
> It is real improvement, thanks for your work!
you're welcome :)
> Using -pipe option helps to save even more time (~5--15%, depending from
> system type). Of course, this is only important for those, who need
> recompile the whole library very often...
~/devel/gdal$ grep -r '$(MAKE)' . | grep GNUmakefile | grep '\.\.'
./alg/GNUmakefile: (cd ..; $(MAKE) docs)
./alg/GNUmakefile: (cd .. ; $(MAKE) force-lib)
./ogr/ogrsf_frmts/sdts/GNUmakefile: (cd ../../../frmts/iso8211; $(MAKE))
./ogr/ogrsf_frmts/sdts/GNUmakefile: (cd ../../../frmts/sdts; $(MAKE))
./ogr/GNUmakefile: (cd ..; $(MAKE) check-lib)
./apps/GNUmakefile: (cd ../gcore ; $(MAKE) )
./apps/GNUmakefile: (cd ../port ; $(MAKE) )
./gcore/GNUmakefile: (cd ..; $(MAKE) docs)
./frmts/gdb/GNUmakefile: (cd ..; $(MAKE))
each of these cross-dir make call is a potential race condition in parallel
builds. furthermore, there are lots of cross-dir dependecies that might make
the parallel build explode as well!
BEWARE! if something goes wrong you might get a make error or worse _a
silently broken library_!!
so, use parallel builds with caution and report any anomaly.
More information about the Gdal-dev