[Gdal-dev] improving gdal parallel build

[Alessandro Amici]

Andrey,

On Monday 04 August 2003 21:49, Andrey Kiselev wrote:
> On Sun, Aug 03, 2003 at 11:22:34PM +0200, Alessandro Amici wrote:
> > i tweaked a couple of rules in the top level GNUmakefile in order to fix
> > parallel builds. 'make -j 200' on my home uni-processor now builds
> > everything ok. i will not be able to test on a real SMP machine for a
> > while, thou.
> >
> > is there anybody else interested in parallel builds?
>
> Alessandro,
>
> At least you have one user: it's me.

then it's an important user ;)

> Parallel builds saves ~20% time on
> uniprocessor system and much more time on multi CPU one. I have access to
> the dual Xeon system and have ~2x compilation speed up.

was it really 2x?
i hardly belive it. there are still a lot of serial $(MAKE) calls into frmts 
and ogrsf_frmts, from what i can see on my UP some more work is needed before 
acheeving real linear scalability.

> It is real improvement, thanks for your work!

you're welcome :)

> Using -pipe option helps to save even more time (~5--15%, depending from
> system type). Of course, this is only important for those, who need
> recompile the whole library very often...

BTW
~/devel/gdal$ grep -r '$(MAKE)' . | grep GNUmakefile | grep '\.\.'
./alg/GNUmakefile:      (cd ..; $(MAKE) docs)
./alg/GNUmakefile:      (cd .. ; $(MAKE) force-lib)
./ogr/ogrsf_frmts/sdts/GNUmakefile:     (cd ../../../frmts/iso8211; $(MAKE))
./ogr/ogrsf_frmts/sdts/GNUmakefile:     (cd ../../../frmts/sdts; $(MAKE))
./ogr/GNUmakefile:      (cd ..; $(MAKE) check-lib)
./apps/GNUmakefile:     (cd ../gcore ; $(MAKE) )
./apps/GNUmakefile:     (cd ../port ; $(MAKE) )
./gcore/GNUmakefile:    (cd ..; $(MAKE) docs)
./frmts/gdb/GNUmakefile:        (cd ..; $(MAKE))

each of these cross-dir make call is a potential race condition in parallel 
builds. furthermore, there are lots of cross-dir dependecies that might make 
the parallel build explode as well!

BEWARE! if something goes wrong you might get a make error or worse _a 
silently broken library_!!

so, use parallel builds with caution and report any anomaly.

cheers,
alessandro