[gdal-dev] Improving GDAL production in our release

[David Klaus]

All,

On Robert C.'s advice I am currently investigating using vcpkg to fulfill
my company's build requirements. I haven't successfully managed to build
GDAL to our specifications yet. But if I do, I will make sure to post
details to this thread,

On Mon, Jan 6, 2025 at 6:52 PM Greg Troxel <gdt at lexort.com> wrote:

> David Klaus <dklaus at carlsonsw.com> writes:
>
> > Thank you for your fast response.
> >
> > Q: You didn't explain whether you are doing import/merge and carrying
> diffs
> > to the sources, and if so, you didn't give a link where others can look
> at
> > them.
> >
> > A: I apologize if I misunderstand your questions. I think you are asking
> > whether or not anyone at our company has modified any of the source GDAL
> > source files or if we are maintaining our own distribution of the source
> > code. If this is the question, the answer is no.
>
> That makes things easier.
>
> > Q: You didn't publish a link to the scripts you are using.
> >
> > A: Personally, I have no problem with sharing the scripts we currently
> use.
> > I will speak with my boss on whether it would be acceptable to share
> those
> > here.
>
> Having been at a large company, I understand.  But nobody here will be
> able to give you advice without reading your script.  And, if you want
> help making your build process better under nondisclosure, you'll need a
> consultant rather than help from the list.
>
> > Q: You didn't explain the problems you are having, just that it's hard.
> >
> > A: My apologies. The problem is that modifying our current scripts in
> order
> > to produce a new build of GDAL takes a significant amount of development
> > time. For example, I am currently working on building Proj-9.5.1 as part
> of
> > our new GDAL build and I'm running into trouble linking "tiff" libraries.
> > This is not likely irreconcilable but it takes time. I see how this can
> be
> > difficult to understand without looking at our current build process. I
> > will discuss this with my boss.
>
> My quick guess is that you are reinventing a packaging system and that
> the last 10% is hard.  That's a little glib, but it sounds like various
> paths.  You might consider using a packaging system, and using a
> different prefix than the regulsr /usr.
>
> You may also want to be very rigorous and organized about -I and -L/-R
> (-R may be spelled something like -Wl,-rpath).
>
> > Q: I hope you are contributing funds to GDAL!  I know that can be hard
> in a
> > corporate environment....
> >
> > A: Of course I have no problem contributing my corporation's money to
> GDAL.
> > Unfortunately, that's why I don't get to make those decisions. However, I
> > will discuss this with my boss. Hopefully, I can persuade him,
>
> You might consider how much the company would pay annually for a
> proprietary license for code of equivalent breadth and quality to gdal,
> proj, etc.  You might look at Oracle's fees, or ESRI's.  And then ask
> management to take 10% of that and donate annually.  What I have always
> found is that companies are ok paying huge amounts of money for
> proprietary licenses, but even with free software advocates inside, it's
> amazingly difficult to donate.
>
> > P.S. You mention the following:
> >
> > "I, and probably others, have scripts that basically set up -I/-L/-R for
> > prereqs and run cmake, to be able to build/install-to-destdir/test
> various
> > things.   These are often only 100 lines, 2/3 comments, and don't cause
> me
> > trouble."
> >
> > It sounds like your solution for generating up-to-date GDAL builds is
> > rather streamlined. Currently we are working on updating our current
> build
> > to GDAL 3.10.0. Were you to do the same how long would you estimate this
> > update would take? Also, is there any chance you could share any of your
> > scripts so I could compare them to ours?
>
> This is not baked, and not up to date.  For example it uses python 3.10
> for docs, and while pkgsrc can still build that, I'm on 3.12 and
> thinking about 3.13.  It is not up to date because I haven't had to run
> it lately.  I am the maintainer of gdal in pkgsrc, and I update there,
> and if that build works and passes tests, I don't need to build it
> outside of pkgsrc.   For a while I was chasing a few portability issues,
> and some floating point stuff, but now there's very little trouble.
>
> But, I am not statically linking, and I am living in a world where the
> dependencies all come from pkgsrc.  I typically build like this to check
> git master for portability bugs that have crept in (test ==, assuming
> things not required by POSIX, etc.) and to see if tests still pass (new
> tests are added, and floating point is hard :-).
>
> The last part is creating a distfile -- what the project would release
> so I can run that through packaging.
>
> Typically, for a new version, there's very little if anything I do.  I'd
> say if things build and test with no issues, it's 15 minutes of work.
> If there are build/test issues, those dominate the effort and updating
> the script is in the noise.
>
> (I have similar scripts for autoconf projects; same concept different
> spelling of build commands.)
>
> #!/bin/sh
>
> VERSION=3.6.0.dev0
>
> if [ -d $HOME/bin/ccache ]; then
>     echo "enabling ccache"
>     ccache -z
>     PATH=$HOME/bin/ccache:$PATH
> fi
>
> ##
>
> PREFIX=/usr/pkg
> LIBDIR=${PREFIX}/lib
>
> # In theory, BSD make is ok.
> MAKE=make
>
> # \todo Debug poppler includes not found.
>
> (rm -rf build destdir &&
>      mkdir build &&
>      cd build &&
>      cmake .. \
>            -LH \
>            -DCMAKE_COLOR_MAKEFILE=false \
>            -DCMAKE_INSTALL_PREFIX=${PREFIX} \
>            -DCMAKE_INSTALL_RPATH=${LIBDIR} \
>            > OUT.00.cmake 2>&1 &&
>      time ${MAKE} VERBOSE=1  > OUT.10.make 2>&1 &&
>      (time ${MAKE} check > OUT.20.check 2>&1 || true) && \
>      time ${MAKE} DESTDIR=../destdir install > OUT.30.install 2>&1 && \
>      echo BUILD DONE
> )
>
> ## Create a distfile for pkgsrc testing.
>
> # Set up tool variables.
> export MAKE=gmake
> export PYTHON=python3.10
> export SPHINXBUILD=sphinx-build-3.10
> # Remediate use of tools without search or variables.
> PATH=/home/gdt/bin/GDAL:$PATH
>
> (time ./mkgdaldist.sh ${VERSION} -url
> https://github.com/gdt/gdal
> > OUT.40.mkgdaldist 2>&1 &&
>      cp -p gdal-${VERSION}.tar.xz /usr/pkgsrc/distfiles &&
>      echo "COPIED DISTFILE")
>
>

-- 
David Klaus
Carlson Software

Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20250108/31cbcf5b/attachment.htm>