[postgis-devel] on parallel and recursive make

rmrodriguez at carto.com rmrodriguez at carto.com
Wed Nov 4 03:36:50 PST 2020


On Wed, Nov 4, 2020 at 11:51 AM Sandro Santilli <strk at kbt.io> wrote:
>
> Is that the postgis/ subdir ?
> I expected this behaviour, yes.
> The thing is that encoding deps in subdirs means that parallel
> builds from top-level dir may end up building the same dependency
> twice (think liblwgeom), which results in instabilities.

Not true. liblwgeom might be tried to be built twice, but the second
one won't do anything since it's already built and there weren't any
changes in the source.

>
> The change ensures the caller calls `make` from the top-level dir.

And why is that desirable? Before this change you could do a make
under the postgis/ subdirectory and if there was a change in liblwgeom
you would get an update, now things will only work if you build on the
top level directory, which used to work before too so we are losing
flexibility and quality of life without anything to show for it.

>
> > It also makes building everything in parallel unstable:
> >
> > $ ./autogen.sh && ./configure CC=clang CXX=clang++ CFLAGS="$CFLAGS"
> > CXXFLAGS="$CXXFLAGS" CPPFLAGS="$C
> > PPFLAGS" LDFLAGS="$LDFLAGS" && make -j11 check
> > [...]
> > PostGIS was built successfully. Ready to install.
> > $ echo $?
> > 2
>
> You did run 'check' here, which still has the dependencies
> encoded (due to staged-install). It works if you only run
> the "all" rule in parallel.

This was working perfectly fine before the changes. Why would I now
need to run make all in parallel, when `make check` in parallel worked
before?

> The fact that you get the "success" message last is a consequence
> of parallel build... ONE of the (multiple) thread attempting
> to build the application succeeded, but others failed.
>
> I guess a possible fix to make parallel-check work would be
> making "all" a dependency of "check", can you see if that helps
> on your side ?
>
> --strk;

I applied this patch:
=================================================
diff --git a/GNUmakefile.in b/GNUmakefile.in
index ef65ec0fe..dea69d67a 100644
--- a/GNUmakefile.in
+++ b/GNUmakefile.in
@@ -41,7 +41,7 @@ clean-local:
# TODO: drop 'test' target..
test: check

-check: check-no-trailing-blanks check-unit check-regress docs-check check-news
+check: check-no-trailing-blanks check-unit check-regress docs-check
check-news all

staged-install:
    $(MAKE) -C regress staged-install
=================================================

And I still got the same error when doing a full rebuild (./autogen.sh
&& ./configure && make -j11 check).

Parallel building had been working fine, at least for me and Travis,
for years now. If you have issues with the build system, fix them
without breaking other people's workflow.
-- 
Raúl Marín Rodríguez
carto.com


More information about the postgis-devel mailing list