[QGIS-Developer] Development environment setup

[Greg Troxel]

Tom Christian via QGIS-Developer <qgis-developer at lists.osgeo.org>
writes:

> I am interested in learning more about how QGIS developers configure
> their development environments, and I would like to get some feedback
> on an alternate approach.
>
> I recently contributed my first changes to QGIS and I found the
> development environment quite challenging. While the QGIS Developers
> Guide <https://docs.qgis.org/3.40/en/docs/developers_guide/index.html>
> contains lots of information I found most of it to be high-level,
> e.g. how to work with Git, and there was limited information on
> gathering required dependencies and setting build configurations.

I believe that INSTALL.md should be extended, and reorganized to speak
in general terms before specializing to a particular OS.

> Given the number of dependencies, the risk of conflicts with existing
> packages, and my strong preference to avoid VMs, I thought a
> containerised build environment was ideal. My work requires frequently
> switching between projects and environments. I generally cannot let
> any one project's dependencies pollute the system outside of a Conda
> environment, a container, or a VM.

I find the choices that give rise to this approach to be problematic.
Containers are a useful device for someone to choose, but if people feel
they have to use a container, that's a clue that the situation is
troubled.

I suspect you are running into packages which require an exact
dependency, rather than the proper approach of having a minimal version,
trying hard to not to raise that without cause, and adapating to any
released version >= the minimum.   Some upstreams, influenced by
language-specific packaging tools, take the approach that their package
is the most important thing in the world and therefore it's ok to insist
on exact versions.

So far, it seems that qgis does very well on not being unreasonable
about dependencies (as long as gcc 12 turns out to be ok; see below).

I'm building qgis git master on NetBSD, with dependencies provided by
pkgsrc.  So far things are almost entirely ok except for what I believe
is erroneous code in two tests.  I also build qgis 3.40 and now 3.44
releases under pkgsrc.  I have generally not had dependency issues.

> I found the CI Docker configuration
> <https://github.com/qgis/QGIS/tree/master/.docker> and thought I could
> use this to build and test QGIS. These Dockerfiles should guarantee
> all required dependencies AND this is the environment that passes or
> fails CI checks on my changes. Building QGIS using the CI Docker
> configuration was OK, but CI is not setup for running the build
> product in a container - meaning changes cannot quickly be manually /
> visually tested in this way. The project would have to be compiled
> (with the benefit of mounted ccache) and installed every time the
> container was started before the application could run. The long
> Docker commands are not currently scripted outside of GitHub Actions
> Workflows, and build configurations are provided via matrices, so
> replicating these processes locally is non-trivial.

Agreed it would be nicer if everything useful like this were hoisted to
first-class scripts that could then be used by CI, moving as much out of
CI as possible.

However, I think it's important for it to be reasonable to build without
containers.

> I created this repo
> <https://github.com/sparkgeo/QGIS-Container-Development> to better
> automate container-based development and testing of QGIS. It contains
> build / run / test scripts, .env configurations, and Docker Compose
> overrides to work with the QGIS CI Docker configuration with as few
> changes as possible. It currently requires a single small commit
> <https://github.com/sparkgeo/QGIS-EOPF/commit/64b6f6568ec24f9a064a24869b3d1fb95a6ae0b1>
> in the active QGIS branch to support this approach, which I removed
> before creating PRs for the main QGIS repo.

Thanks; will have a look.

> 1) are you installing QGIS build dependencies on "bare metal", or perhaps using a VM?

I am installing them within a system which already exists and is also
used for other purposes.  That system happens to be on bare metal.

> 2) is there a simpler approach than this for containerised builds, runs, and tests?
>   2a) if not, is there any interest in pursuing this or a similar approach to simplify containerised development environments?
>
> Ultimately I suspect the QGIS project would benefit from lower barriers to entry and the ability to better isolate build dependencies.

Perhaps, but I see the biggest issue as bringing clarity to the
situation.  As things are I found out missing dependencies from cmake
failures -- but at least that is very straightforward.  To be fair, the
things I found might be in INSTALL.md.

As an example, INSTALL.md does not say you need a C++ compiler, and it
does not say that C++20 is needed.  Beyond that, it doesn't address
which parts of C++20 is neeeed; requiring 100% confornmance is obviously
unreasonable and means, of gcc versions, only gcc 16 is acceptable.   I
have received some help in the matrix chat about this, but so far
there's no clarity on if gcc12 is good enough (it implements most of
c++20, and my take is that only odd parts aren't working in 12).