[gdal-dev] Building GDAL with rtree support
Jon Morris
Jon.Morris at jbarisk.com
Thu May 15 03:37:49 PDT 2025
Hello all,
I'm trying to build GDAL python wheels and am having some autotest fails that is preventing the build. I think most if not all of these fails are due to rtree not being available in sqlite. AFAIK I've done everything needed but the build recipe must be missing something.
The dockerfile contents are below and I run it with the command `DOCKER_BUILDKIT=0 docker build -f Dockerfile -t gdal-wheelbuilder`. It gets as far as `ctest -V` then exits with the following test fails:
46 - autotest_gdrivers (Failed)
48 - autotest_ogr (Failed)
50 - autotest_pyscripts (Failed)
Thanks,
Jon
FROM quay.io/pypa/manylinux_2_28_x86_64
# Arrow install instructions from https://arrow.apache.org/install/
RUN dnf update -y \
&& dnf groupinstall -y "Development Tools" \
&& dnf install -y epel-release \
&& dnf install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-$(cut -d: -f5 /etc/system-release-cpe | cut -d. -f1).noarch.rpm \
&& dnf install -y https://packages.apache.org/artifactory/arrow/almalinux/$(cut -d: -f5 /etc/system-release-cpe | cut -d. -f1)/apache-arrow-release-latest.rpm \
&& dnf config-manager --set-enabled epel \
&& dnf config-manager --set-enabled powertools \
&& dnf install -y arrow-devel \
&& dnf install -y arrow-dataset-devel \
&& dnf install -y parquet-devel
RUN dnf install -y \
ant \
autoconf \
automake \
bash-completion \
cmake \
ca-certificates \
ccache \
clang \
cmake \
curl-devel \
doxygen \
git \
gnupg \
gpsbabel \
lcov \
hdf5-devel \
json-devel \
json-c-devel \
libaec-devel \
libdeflate-devel \
libjpeg-devel \
libkml-devel \
libpng-devel \
libpq-devel \
libtiff-devel \
libtool \
libwebp-devel \
libxml2-devel \
libzstd-devel \
make \
minizip-devel \
numactl \
openssl-devel \
openjpeg2-devel \
pkg-config \
poppler-devel \
postgresql \
postgresql-contrib \
python3-devel \
python38-devel \
python3-lxml \
python3-numpy \
python3-pip \
python3-pytest \
python3-setuptools \
python3-virtualenv \
swig \
unzip \
wget \
zip \
zlib-devel
# Install geos
RUN mkdir -p /src \
&& cd /src \
&& curl -f -L -O https://download.osgeo.org/geos/geos-3.13.0.tar.bz2 \
&& tar jxf geos-3.13.0.tar.bz2 \
&& cd /src/geos-3.13.0 \
&& mkdir build \
&& cd build \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_TESTING=OFF -DCMAKE_BUILD_TYPE=Release .. \
&& make -j$(nproc) \
&& make install \
&& cd / \
&& rm -rf /src
# Build a SQLite version with -DSQLITE_TRUSTED_SCHEMA=0
RUN mkdir sqlite \
&& wget -q https://sqlite.org/2023/sqlite-autoconf-3430100.tar.gz -O - \
| tar xz -C sqlite --strip-components=1 \
&& cd sqlite \
&& CFLAGS="-O2 -DSQLITE_TRUSTED_SCHEMA=0 -DSQLITE_ENABLE_COLUMN_METADATA -DSQLITE_ENABLE_RTREE" ./configure --prefix=/usr/local/install-sqlite-trusted-schema-off \
&& make -j$(nproc) \
&& make -j$(nproc) install \
&& cd .. \
&& rm -rf sqlite
# Fix proj version
RUN mkdir -p /src \
&& cd /src \
&& curl -f -L -O https://download.osgeo.org/proj/proj-9.1.1.tar.gz \
&& tar xzf proj-9.1.1.tar.gz \
&& cd /src/proj-9.1.1 \
&& mkdir build \
&& cd build \
&& cmake -DEXE_SQLITE3=/usr/local/install-sqlite-trusted-schema-off/bin/sqlite3 .. \
&& cmake --build . \
&& cmake --build . --target install \
&& rm -rf /src
# Build libgeotiff (system libgeotiff pulls in system libproj as a dependency)
RUN mkdir -p /src \
&& cd /src \
&& curl -f -L -O https://github.com/OSGeo/libgeotiff/releases/download/1.7.3/libgeotiff-1.7.3.tar.gz \
&& tar xzf libgeotiff-1.7.3.tar.gz \
&& cd libgeotiff-1.7.3 \
&& mkdir build_autoconf \
&& cd build_autoconf \
&& export LD_LIBRARY_PATH=/usr/local/lib64:/usr/local/lib:/usr/lib:/lib \
&& CFLAGS="-Wall -Wextra -Werror" ../configure \
&& make -j$(nproc) \
&& make check \
&& cd .. \
&& mkdir build_cmake \
&& cd build_cmake \
&& cmake -DCMAKE_C_COMPILER_LAUNCHER=ccache -DCMAKE_C_FLAGS="-Wall -Wextra -Werror" .. \
&& make -j$(nproc) \
&& rm -rf /src
# Build freexl (spatialite dependency)
RUN mkdir -p /src \
&& cd /src \
&& curl -f -L -O https://www.gaia-gis.it/gaia-sins/freexl-2.0.0.tar.gz \
&& tar xzf freexl-2.0.0.tar.gz \
&& cd /src/freexl-2.0.0 \
&& ./configure \
&& make -j$(nproc) \
&& make install \
&& rm -rf /src
# Build spatialite so it uses correct PROJ and sqlite3
RUN mkdir -p /src \
&& cd /src \
&& curl -f -L -O https://www.gaia-gis.it/gaia-sins/libspatialite-sources/libspatialite-5.1.0.tar.gz \
&& tar xzf libspatialite-5.1.0.tar.gz \
&& cd /src/libspatialite-5.1.0 \
&& SQLITE3_CFLAGS="-I/usr/local/install-sqlite-trusted-schema-off/include/" SQLITE3_LIBS="-Wl,--enable-new-dtags,-rpath=/usr/local/install-sqlite-trusted-schema-off/lib -L/usr/local/install-sqlite-trusted-schema-off/lib -lsqlite3" ./configure --disable-rttopo \
&& make -j$(nproc) \
&& make install \
&& rm -rf /src
# configure shared libraries
RUN ldconfig -v
# Add GDAL source to container
# and checkout current branch and commit (see README.md).
# Also patch in commits https://github.com/OSGeo/gdal/commit/b36072fa1539b8ac97ea7864071d3468ac5b49ab
# and https://github.com/OSGeo/gdal/commit/222e1c111437f9fd6e43eb384f0903684735750b to get rid of
# arrow/parquet deprecation warnings
ARG GITHUB_WORKSPACE=/src
RUN mkdir -p $GITHUB_WORKSPACE \
&& cd $GITHUB_WORKSPACE \
&& git config --global user.email you at example.com<mailto:you at example.com> \
&& git config --global user.name "Your Name" \
&& git clone -b release/3.10 https://github.com/OSGeo/gdal.git superbuild \
&& cd superbuild \
&& git checkout 9b7a7c8 \
&& git cherry-pick b36072fa \
&& git cherry-pick 222e1c11
# Install test requirements
RUN python3.8 -m pip install -U pip setuptools numpy importlib_metadata \
&& python3.8 -m pip install -r $GITHUB_WORKSPACE/superbuild/autotest/requirements.txt
## This section based on build-linux-ubuntu-focal job in .github/workflows/cmake_builds.yml ##
ARG CMAKE_UNITY_BUILD=OFF
# - name: Configure
RUN set -eu \
&& cd $GITHUB_WORKSPACE/superbuild \
&& export CXXFLAGS="-march=native -O2 -Wodr -flto-odr-type-merging -Werror" \
&& export CFLAGS="-O2 -march=native -Werror" \
&& mkdir build \
&& cd build \
&& cmake .. \
-Werror=dev \
-DCMAKE_INSTALL_PREFIX=$GITHUB_WORKSPACE/install-gdal \
-DCMAKE_UNITY_BUILD=${CMAKE_UNITY_BUILD} \
-DGDAL_USE_PUBLICDECOMPWT:BOOL=ON \
-DPUBLICDECOMPWT_URL=https://github.com/rouault/PublicDecompWT \
-DWERROR_DEV_FLAG="-Werror=dev" \
-DPython_LOOKUP_VERSION=3.8 \
-DUSE_CCACHE=OFF \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=ON \
-DGDAL_USE_TIFF_INTERNAL=ON \
-DGDAL_USE_GEOTIFF_INTERNAL=ON \
-DSQLite3_INCLUDE_DIR=/usr/local/install-sqlite-trusted-schema-off/include \
-DSQLite3_LIBRARY=/usr/local/install-sqlite-trusted-schema-off/lib/libsqlite3.so \
-DBUILD_CSHARP_BINDINGS:BOOL=OFF \
&& unset CXXFLAGS \
&& unset CFLAGS
# - name: Build
RUN GIT_LFS_SKIP_SMUDGE=1 cmake --build $GITHUB_WORKSPACE/superbuild/build -- -j$(nproc)
# - name: test (with command targets)
RUN cmake --build $GITHUB_WORKSPACE/superbuild/build --target quicktest -- -j$(nproc)
# - name: test (with ctest)
RUN cd $GITHUB_WORKSPACE/superbuild/build \
&& ctest -V
# - name: install
RUN cmake --build $GITHUB_WORKSPACE/superbuild/build --target install -- -j$(nproc)
## END build-linux-ubuntu-focal job in .github/workflows/cmake_builds.yml ##
Jon Morris
Software Developer
--
e: Jon.Morris at jbarisk.com
d:+44 (0)1756 587229
t: +44 (0)1756 799919
www.jbarisk.com
All JBA Risk Management's email messages contain confidential information and are intended only for the individual(s) named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail.
Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system.
JBA Risk Management Limited is registered in England, company number 07732946, 1 Broughton Park, Old Lane North, Broughton, Skipton, North Yorkshire, BD23 3FD, England.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20250515/9b765d6c/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image958650.png
Type: image/png
Size: 7388 bytes
Desc: image958650.png
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20250515/9b765d6c/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image748490.png
Type: image/png
Size: 5172 bytes
Desc: image748490.png
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20250515/9b765d6c/attachment-0005.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image252124.png
Type: image/png
Size: 774 bytes
Desc: image252124.png
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20250515/9b765d6c/attachment-0006.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image442173.png
Type: image/png
Size: 797 bytes
Desc: image442173.png
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20250515/9b765d6c/attachment-0007.png>
More information about the gdal-dev
mailing list