<div dir="ltr">Packaging is hard. There are all sorts of sharp edges. I don't work in the windows world much at all, so I can't comment on the packaging there, but there are many package mangers that provide GDAL. And many of us who do packaging share ideas and fixes across packaging. I personally do bazel builds that are mostly focused on linux.<div><br></div><div>(please don't read anything into the order here :)</div><div><br></div><div><a href="https://conan.io/center/recipes/gdal">https://conan.io/center/recipes/gdal</a></div><div><a href="https://anaconda.org/conda-forge/gdal">https://anaconda.org/conda-forge/gdal</a></div><div><a href="https://github.com/fink/fink-distributions/blob/master/10.9-libcxx/stable/main/finkinfo/libs/gdal3.info">https://github.com/fink/fink-distributions/blob/master/10.9-libcxx/stable/main/finkinfo/libs/gdal3.info</a> for MacOSX</div><div><a href="https://ports.macports.org/port/gdal/">https://ports.macports.org/port/gdal/</a></div><div><a href="https://formulae.brew.sh/formula/gdal">https://formulae.brew.sh/formula/gdal</a></div><div><a href="https://rpmfind.net/linux/rpm2html/search.php?query=gdal">https://rpmfind.net/linux/rpm2html/search.php?query=gdal</a></div><div><a href="https://wiki.debian.org/DebianGis">https://wiki.debian.org/DebianGis</a></div><div>...</div><div><br></div><div>Or things that bring in GDAL:</div><div><br></div><div><a href="https://rasterio.readthedocs.io/en/stable/installation.html">https://rasterio.readthedocs.io/en/stable/installation.html</a> has binary wheels<br></div><div><a href="https://www.qgis.org/">https://www.qgis.org/</a></div><div>(and probably some other GIS packages bring along GDAL)</div><div>...</div><div><br></div><div>I've also collaborated on bash scripts to do custom builds of GDAL. It's a diverse ecosystem.</div><div><br></div><div>-kurt</div></div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">On Mon, Jan 6, 2025 at 3:52 PM Greg Troxel via gdal-dev <<a href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">David Klaus <<a href="mailto:dklaus@carlsonsw.com" target="_blank">dklaus@carlsonsw.com</a>> writes:<br>
<br>
> Thank you for your fast response.<br>
><br>
> Q: You didn't explain whether you are doing import/merge and carrying diffs<br>
> to the sources, and if so, you didn't give a link where others can look at<br>
> them.<br>
><br>
> A: I apologize if I misunderstand your questions. I think you are asking<br>
> whether or not anyone at our company has modified any of the source GDAL<br>
> source files or if we are maintaining our own distribution of the source<br>
> code. If this is the question, the answer is no.<br>
<br>
That makes things easier.<br>
<br>
> Q: You didn't publish a link to the scripts you are using.<br>
><br>
> A: Personally, I have no problem with sharing the scripts we currently use.<br>
> I will speak with my boss on whether it would be acceptable to share those<br>
> here.<br>
<br>
Having been at a large company, I understand. But nobody here will be<br>
able to give you advice without reading your script. And, if you want<br>
help making your build process better under nondisclosure, you'll need a<br>
consultant rather than help from the list.<br>
<br>
> Q: You didn't explain the problems you are having, just that it's hard.<br>
><br>
> A: My apologies. The problem is that modifying our current scripts in order<br>
> to produce a new build of GDAL takes a significant amount of development<br>
> time. For example, I am currently working on building Proj-9.5.1 as part of<br>
> our new GDAL build and I'm running into trouble linking "tiff" libraries.<br>
> This is not likely irreconcilable but it takes time. I see how this can be<br>
> difficult to understand without looking at our current build process. I<br>
> will discuss this with my boss.<br>
<br>
My quick guess is that you are reinventing a packaging system and that<br>
the last 10% is hard. That's a little glib, but it sounds like various<br>
paths. You might consider using a packaging system, and using a<br>
different prefix than the regulsr /usr.<br>
<br>
You may also want to be very rigorous and organized about -I and -L/-R<br>
(-R may be spelled something like -Wl,-rpath).<br>
<br>
> Q: I hope you are contributing funds to GDAL! I know that can be hard in a<br>
> corporate environment....<br>
><br>
> A: Of course I have no problem contributing my corporation's money to GDAL.<br>
> Unfortunately, that's why I don't get to make those decisions. However, I<br>
> will discuss this with my boss. Hopefully, I can persuade him,<br>
<br>
You might consider how much the company would pay annually for a<br>
proprietary license for code of equivalent breadth and quality to gdal,<br>
proj, etc. You might look at Oracle's fees, or ESRI's. And then ask<br>
management to take 10% of that and donate annually. What I have always<br>
found is that companies are ok paying huge amounts of money for<br>
proprietary licenses, but even with free software advocates inside, it's<br>
amazingly difficult to donate.<br>
<br>
> P.S. You mention the following:<br>
><br>
> "I, and probably others, have scripts that basically set up -I/-L/-R for<br>
> prereqs and run cmake, to be able to build/install-to-destdir/test various<br>
> things. These are often only 100 lines, 2/3 comments, and don't cause me<br>
> trouble."<br>
><br>
> It sounds like your solution for generating up-to-date GDAL builds is<br>
> rather streamlined. Currently we are working on updating our current build<br>
> to GDAL 3.10.0. Were you to do the same how long would you estimate this<br>
> update would take? Also, is there any chance you could share any of your<br>
> scripts so I could compare them to ours?<br>
<br>
This is not baked, and not up to date. For example it uses python 3.10<br>
for docs, and while pkgsrc can still build that, I'm on 3.12 and<br>
thinking about 3.13. It is not up to date because I haven't had to run<br>
it lately. I am the maintainer of gdal in pkgsrc, and I update there,<br>
and if that build works and passes tests, I don't need to build it<br>
outside of pkgsrc. For a while I was chasing a few portability issues,<br>
and some floating point stuff, but now there's very little trouble.<br>
<br>
But, I am not statically linking, and I am living in a world where the<br>
dependencies all come from pkgsrc. I typically build like this to check<br>
git master for portability bugs that have crept in (test ==, assuming<br>
things not required by POSIX, etc.) and to see if tests still pass (new<br>
tests are added, and floating point is hard :-).<br>
<br>
The last part is creating a distfile -- what the project would release<br>
so I can run that through packaging.<br>
<br>
Typically, for a new version, there's very little if anything I do. I'd<br>
say if things build and test with no issues, it's 15 minutes of work.<br>
If there are build/test issues, those dominate the effort and updating<br>
the script is in the noise.<br>
<br>
(I have similar scripts for autoconf projects; same concept different<br>
spelling of build commands.)<br>
<br>
#!/bin/sh<br>
<br>
VERSION=3.6.0.dev0<br>
<br>
if [ -d $HOME/bin/ccache ]; then<br>
echo "enabling ccache"<br>
ccache -z<br>
PATH=$HOME/bin/ccache:$PATH<br>
fi<br>
<br>
## <br>
<br>
PREFIX=/usr/pkg<br>
LIBDIR=${PREFIX}/lib<br>
<br>
# In theory, BSD make is ok.<br>
MAKE=make <br>
<br>
# \todo Debug poppler includes not found.<br>
<br>
(rm -rf build destdir &&<br>
mkdir build &&<br>
cd build &&<br>
cmake .. \<br>
-LH \<br>
-DCMAKE_COLOR_MAKEFILE=false \<br>
-DCMAKE_INSTALL_PREFIX=${PREFIX} \<br>
-DCMAKE_INSTALL_RPATH=${LIBDIR} \<br>
> OUT.00.cmake 2>&1 &&<br>
time ${MAKE} VERBOSE=1 > OUT.10.make 2>&1 &&<br>
(time ${MAKE} check > OUT.20.check 2>&1 || true) && \<br>
time ${MAKE} DESTDIR=../destdir install > OUT.30.install 2>&1 && \<br>
echo BUILD DONE<br>
)<br>
<br>
## Create a distfile for pkgsrc testing.<br>
<br>
# Set up tool variables.<br>
export MAKE=gmake<br>
export PYTHON=python3.10<br>
export SPHINXBUILD=sphinx-build-3.10<br>
# Remediate use of tools without search or variables.<br>
PATH=/home/gdt/bin/GDAL:$PATH<br>
<br>
(time ./mkgdaldist.sh ${VERSION} -url <a href="https://github.com/gdt/gdal" rel="noreferrer" target="_blank">https://github.com/gdt/gdal</a> > OUT.40.mkgdaldist 2>&1 &&<br>
cp -p gdal-${VERSION}.tar.xz /usr/pkgsrc/distfiles &&<br>
echo "COPIED DISTFILE")<br>
_______________________________________________<br>
gdal-dev mailing list<br>
<a href="mailto:gdal-dev@lists.osgeo.org" target="_blank">gdal-dev@lists.osgeo.org</a><br>
<a href="https://lists.osgeo.org/mailman/listinfo/gdal-dev" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a><br>
</blockquote></div>