[gdal-dev] An attempt to switch Vagrantfile base box from Ubuntu 22 to Debian 12, and open jdk from 11 to 17
Marc Le Bihan
mlebihan29 at gmail.com
Tue May 6 20:40:22 PDT 2025
Hello,
I'm posting this trial, this experience for information.
I'm searching to migrate few projects from Java JDK 11 to 17 (at
least) to avoid a trap where recent operating systems haven't (and
cannot anymore install) Java 11 on them. This involving upgrading some
of their dependencies, by the way.
The long explanation of my motives is here
<https://opensource.stackexchange.com/questions/15474/should-projects-with-dependencies-too-old-programming-languages-version-becomin>,
and I was especially in need of that for gdal, for my own interest of
studying it deeply.
For that, I was willing to get its convenient Vagrantfile, add
locally for my own use a GNOME to it, and then install my CLion and
others IDE there to work with it.
I encountered few troubles when generating the Vagrant VM that I
took from master, but Stack Overflow community helped me to solve them.
I've explained what happened to me here
<https://stackoverflow.com/questions/79607954/a-project-is-built-from-a-vagrantfile-and-im-switching-its-base-box-now-a-vagr>.
To resume: I've switched from Ubuntu 22 LTS Vagrant base box to a
Debian 12 one, JDK 11 to 17, add to remove mold alternate linker mention
to make everything build.
Now, everything can be built and tested, as changes allowing 96% of
ctests to run still, and for pytest ones it's 16070 passed, 1 xpassed,
268 failed. (I wasn't expecting everything could work like a charm
immediately).
Your comments are welcome.
Regards,
Marc Le Bihan
For python tests failing ones are these ones, for either Runtime Error,
Assertion Error or expected output file missing:
ogr/ogr_basic_test.py::test_ogr_basic_test_future_warning_exceptions
ogr/ogr_csv.py::test_ogr_csv_write_to_stdout
ogr/ogr_virtualogr.py::test_ogr_virtualogr_3
gcore/basic_test.py::test_basic_test_8
gcore/basic_test.py::test_basic_test_UseExceptions
gcore/basic_test.py::test_basic_test_UseExceptions_ogr_open
gcore/basic_test.py::test_basic_test_DontUseExceptions
gcore/misc.py::test_misc_quiet_errors[quiet_errors]
gcore/misc.py::test_misc_quiet_errors[quiet_warnings]
gcore/pam.py::test_pam_11
gcore/vsicurl_streaming.py::test_vsicurl_streaming_real_resource
gdrivers/bag.py::test_bag_force_opening_no_match
gdrivers/gdalhttp.py::test_http_6
gdrivers/netcdf_multidim.py::test_netcdf_multidim_cache_pamproxydb
gdrivers/wms.py::test_wms_data_via_mrf
osr/osr_basic.py::test_osr_basic_warning_exceptions
pyscripts/test_gdal2tiles.py::test_gdal2tiles_py_simple
pyscripts/test_gdal2tiles.py::test_gdal2tiles_py_zoom_option
pyscripts/test_gdal2tiles.py::test_gdal2tiles_py_resampling_option[average, near, bilinear, cubic, cubicspline, lanczos, mode, max, min, med, q1, q3]
pyscripts/test_gdal2tiles.py::test_gdal2tiles_py_xyz/te
pyscripts/test_gdal2tiles.py::test_gdal2tiles_py_invalid_srs
pyscripts/test_gdal2tiles.py::test_handle_utf8_filename
pyscripts/test_gdal2tiles.py::test_exclude_transparent_tiles
pyscripts/test_gdal2tiles.py::test_gdal2tiles_py_profile_raster
pyscripts/test_gdal2tiles.py::test_gdal2tiles_py_profile_raster_oversample
pyscripts/test_gdal2tiles.py::test_gdal2tiles_py_profile_raster_xyz
pyscripts/test_gdal2tiles.py::test_gdal2tiles_py_profile_geodetic_tmscompatible_xyz
pyscripts/test_gdal2tiles.py::test_gdal2tiles_py_mapml
pyscripts/test_gdal2tiles.py::test_gdal2tiles_excluded_values
pyscripts/test_gdal2tiles.py::test_gdal2tiles_nodata_values_pct_threshold
pyscripts/test_gdal2tiles.py::test_gdal2tiles_py_jpeg_3band_input[average-expected_stats_z00-expected_s
pyscripts/test_gdal2tiles.py::test_gdal2tiles_py_jpeg_1band_input[average-expected_stats_z140-expected_
pyscripts/test_gdal2xyz.py::test_gdal2xyz_py_2
pyscripts/test_gdal2xyz.py::test_gdal2xyz_py_3
pyscripts/test_gdal2xyz.py::test_gdal2xyz_py_srcnodata_dstnodata
pyscripts/test_gdal2xyz.py::test_gdal2xyz_py_vsistdout
pyscripts/test_gdal_calc.py::test_gdal_calc_py_1/test_g
pyscripts/test_gdal_calc.py::test_gdal_calc_py_1b/test_
pyscripts/test_gdal_calc.py::test_gdal_calc_py_1c/test_
pyscripts/test_gdal_calc.py::test_gdal_calc_py_2a/test_
pyscripts/test_gdal_calc.py::test_gdal_calc_py_2b/test_
pyscripts/test_gdal_calc.py::test_gdal_calc_py_2c/test_
pyscripts/test_gdal_calc.py::test_gdal_calc_py_3/test_g
pyscripts/test_gdal_calc.py::test_gdal_calc_py_4a/test_
pyscripts/test_gdal_calc.py::test_gdal_calc_py_4b/test_
pyscripts/test_gdal_calc.py::test_gdal_calc_py_7a[--optfile ]
pyscripts/test_gdal_calc.py::test_gdal_calc_py_7a[@]/te
pyscripts/test_gdal_calc.py::test_gdal_calc_py_7b/test_
pyscripts/test_gdal_calc.py::test_gdal_calc_py_7c/test_
pyscripts/test_gdal_calc.py::test_gdal_calc_py_7d/test_
pyscripts/test_gdal_calc.py::test_gdal_calc_py_8/test_g
pyscripts/test_gdal_calc.py::test_gdal_calc_py_10/test_
pyscripts/test_gdal_calc.py::test_gdal_calc_py_multiple_inputs_same_alpha
pyscripts/test_gdal_edit.py::test_gdal_edit_py_1[True]
pyscripts/test_gdal_edit.py::test_gdal_edit_py_1[False]
pyscripts/test_gdal_edit.py::test_gdal_edit_py_1b, 2 to 8
pyscripts/test_gdal_edit.py::test_gdal_edit_py_unsetrpc
pyscripts/test_gdal_edit.py::test_gdal_edit_py_epoch
pyscripts/test_gdal_fillnodata.py::test_gdal_fillnodata_1
pyscripts/test_gdal_fillnodata.py::test_gdal_fillnodata_2
pyscripts/test_gdal_fillnodata.py::test_gdal_fillnodata_smoothing
pyscripts/test_gdal_fillnodata.py::test_gdal_fillnodata_nearest
pyscripts/test_gdal_merge.py::test_gdal_merge_[1 to 5]/test_gd
pyscripts/test_gdal_pansharpen.py::test_gdal_pansharpen_1
pyscripts/test_gdal_pansharpen.py::test_gdal_pansharpen_2
pyscripts/test_gdal_polygonize.py::test_gdal_polygonize_[1 to 4]
pyscripts/test_gdal_polygonize.py::test_gdal_polygonize_4bis
pyscripts/test_gdal_polygonize.py::test_gdal_polygonize_minus_8
pyscripts/test_gdal_polygonize.py::test_gdal_polygonize_overwrite[geojson]
pyscripts/test_gdal_polygonize.py::test_gdal_polygonize_overwrite[gpkg]
pyscripts/test_gdal_proximity.py::test_gdal_proximity_[1 to 3]
pyscripts/test_gdal_retile.py::test_gdal_retile_[2 to 5]/test_
pyscripts/test_gdal_retile.py::test_gdal_retile_non_contigous
pyscripts/test_gdal_retile.py::test_gdal_retile_png/tes
pyscripts/test_gdal_retile.py::test_gdal_retile_rotational_geotransform
pyscripts/test_gdal_retile.py::test_gdal_retile_different_srs[32631-32632-has
pyscripts/test_gdal_retile.py::test_gdal_retile_different_srs[32631-None-has
pyscripts/test_gdal_retile.py::test_gdal_retile_different_srs[None-32631-has
pyscripts/test_gdal_sieve.py::test_gdal_sieve_1
pyscripts/test_gdal_sieve.py::test_gdal_sieve_src_without_nodata
pyscripts/test_gdal_utils_retcodes.py::test_script[input0 to 75]
pyscripts/test_gdalattachpct.py::test_gdalattachpct_basic
pyscripts/test_gdalattachpct.py::test_gdalattachpct_vrt_output p
pyscripts/test_gdalbuildvrtofvrt.py::test_gdalbuildvrtofvrt_basic an
pyscripts/test_gdalbuildvrtofvrt.py::test_gdalbuildvrtofvrt_intermediate_vrt_path
pyscripts/test_gdalbuildvrtofvrt.py::test_gdalbuildvrtofvrt_intermediate_vrt_path_specified_ovr_factors
pyscripts/test_gdalcompare.py::test_gdalcompare_same
pyscripts/test_gdalcompare.py::test_gdalcompare_different_type
pyscripts/test_gdalinfo_py.py::test_gdalinfo_py_[1 to 10]
pyscripts/test_gdallocationinfo_py.py::test_gdallocationinfo_py_1
pyscripts/test_gdallocationinfo_py.py::test_gdallocationinfo_py_3
pyscripts/test_gdallocationinfo_py.py::test_gdallocationinfo_py_4
pyscripts/test_gdallocationinfo_py.py::test_gdallocationinfo_py_6
pyscripts/test_gdallocationinfo_py.py::test_gdallocationinfo_py_wgs84
pyscripts/test_gdalmove.py::test_gdalmove_1
pyscripts/test_ogr_layer_algebra.py::test_ogr_layer_algebra_intersection
pyscripts/test_ogr_layer_algebra.py::test_ogr_layer_algebra_union
pyscripts/test_ogr_layer_algebra.py::test_ogr_layer_algebra_symdifference
pyscripts/test_ogr_layer_algebra.py::test_ogr_layer_algebra_identity
pyscripts/test_ogr_layer_algebra.py::test_ogr_layer_algebra_update
pyscripts/test_ogr_layer_algebra.py::test_ogr_layer_algebra_clip
pyscripts/test_ogr_layer_algebra.py::test_ogr_layer_algebra_erase
pyscripts/test_ogrinfo_py.py::test_ogrinfo_py_[1 to 11, 22 and 23]
pyscripts/test_ogrmerge.py::test_ogrmerge_[1 to 7]/test_ogrmer
pyscripts/test_ogrmerge.py::test_ogrmerge_7a/test_ogrme
pyscripts/test_ogrmerge.py::test_ogrmerge_7b/test_ogrme
pyscripts/test_ogrmerge.py::test_ogrmerge_7c/test_ogrme
pyscripts/test_ogrmerge.py::test_ogrmerge_[8 to 12]/test_ogrmer
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg[True-True-True-None-None-None]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg[True-False-True-None-None-None]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg[False-True-True-None-None-None]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg[False-False-True-None-None-None]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg[True-True-False-None-None-None0]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg[True-False-False-None-None-None]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg[False-True-False-None-None-None]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg[False-False-False-None-None-None]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg[True-True-False-None-None-None1]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg[True-True-False-EPSG:32631-None-None]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg[True-True-False-None-EPSG:32631-None]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg[True-True-False-None-None-EPSG:32631]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg[True-True-False-None-EPSG:32631-EPSG:4326]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg_non_spatial[True]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg_non_spatial[False]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg_curve_geom_in_generic_layer
pyscripts/test_pct.py::test_rgb2pct_1/tmp7/test_rgb2pct
pyscripts/test_pct.py::test_pct2rgb_1/test_pct2rgb_10/t
pyscripts/test_pct.py::test_rgb2pct_2/tmp8/test_rgb2pct
pyscripts/test_pct.py::test_rgb2pct_creation_option/tes
pyscripts/test_pct.py::test_rgb2pct_3/test_rgb2pct_30/t
pyscripts/test_pct.py::test_pct2rgb_4/test_pct2rgb_40/t
pyscripts/test_pct.py::test_gdalattachpct_1
pyscripts/test_validate_geoparquet.py::test_validate_geoparquet_not_parquet_file
pyscripts/test_validate_geoparquet.py::test_validate_geoparquet_ok
For ctest, the failing ones are these ones :
44 - test-virtual-memory (Failed)
59 - autotest_gdrivers (Failed)
(but I had few more failing at a previous attempt. I think : 5 of them)
Here are the changes I've experienced for this attempt on the gdal
Vagrantfile :
diff --git a/Vagrantfile b/Vagrantfile
index dd1ec6019d..ac7203d4ba 100644
--- a/Vagrantfile
+++ b/Vagrantfile
@@ -8,10 +8,11 @@ VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
# specify memory size in MiB
- vm_ram = ENV['VAGRANT_VM_RAM'] || 4096
- vm_cpu = ENV['VAGRANT_VM_CPU'] || 2
+ vm_ram = ENV['VAGRANT_VM_RAM'] || 8192
+ vm_cpu = ENV['VAGRANT_VM_CPU'] || 8
vm_ram_bytes = vm_ram * 1024 * 1024
+ config.vm.synced_folder "./", "/vagrant"
config.vm.hostname = "gdal-vagrant"
config.vm.host_name = "gdal-vagrant"
@@ -34,16 +35,25 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
# Seehttps://bugs.launchpad.net/cloud-images/+bug/1969664
# Ubuntu 22.04 no longer accepts RSA keys, which causes issues with older Vagrant
# The below location has a jammy64 image that accepts RSA keys
- config.vm.box_url ="https://people.canonical.com/~jchittum/vagrant-testing/ubuntu-jammy-vagrant-TESTING-20220419.box"
- config.vm.box = "ubuntu/jammy64"
+ config.vm.box = "generic/debian12"
+ config.vm.box_version = "4.3.12"
config.vm.provider :virtualbox do |vb|
vb.customize ["modifyvm", :id, "--memory", vm_ram]
vb.customize ["modifyvm", :id, "--cpus", vm_cpu]
vb.customize ["modifyvm", :id, "--ioapic", "on"]
vb.name = "gdal-vagrant"
+
+ # vb.gui = true
end
+ config.vm.provision "shell", inline: <<-SHELL
+ sudo apt install gnupg2
+ sh -c 'echo "debhttp://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
+ curl -fsSLhttps://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/postgresql.gpg
+ apt-get update && DEBIAN_FRONTEND=noninteractive apt-get upgrade -y
+ SHELL
+
#config.vm.provider :lxc do |lxc,ovrd|
# ovrd.vm.box = "cultuurnet/ubuntu-14.04-64-puppet"
# lxc.backingstore = 'dir'
@@ -77,15 +87,6 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
}
end
- # Unless explicitly declined, use the VM host's file system to cache
- # .deb files to avoid repeated downloads on each vagrant up
- unless File.exists?("../.no_apt_cache")
- cache_dir = "../apt-cache/#{config.vm.box}"
- FileUtils.mkdir_p(cache_dir) unless Dir.exists?(cache_dir)
- puts "Using local apt cache, #{cache_dir}"
- config.vm.synced_folder cache_dir, "/var/cache/apt/archives"
- end
-
ppaRepos = [
]
@@ -115,7 +116,7 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
"bash-completion",
"swig",
"ant",
- "openjdk-11-jdk",
+ "openjdk-17-jdk",
"mono-mcs",
"mono-runtime",
"libmono-system-drawing4.0-cil", @@ -124,8 +125,8 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do
|config| "python3-setuptools",
"python3-pip",
"postgis",
- "postgresql",
- "postgresql-postgis",
+ "postgresql-14",
+ "postgresql-14-postgis-3",
"gpsbabel",
"doxygen",
"libproj-dev", @@ -159,7 +160,7 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do
|config| "libcrypto++-dev",
"libfyba-dev",
"libkml-dev",
- "libmysqlclient-dev",
+ "libmariadb-dev",
"libogdi-dev",
"libcfitsio-dev",
"libzstd-dev", @@ -177,6 +178,7 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do
|config| "libqhull-dev",
"libjson-c-dev",
"libtiff5-dev",
+ "libgtest-dev",
];
config.ssh.forward_agent = true
@@ -191,7 +193,8 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
# install packages we need
pkg_cmd << "apt-get update -qq; "
pkg_cmd << "apt-get --no-install-recommends install -q -y " + packageList.join(" ") << " ; "
-
+ pkg_cmd << "pip install pytest filelock lxml --break-system-package; "
+
# setup environment when we log in
pkg_cmd << "echo 'CCACHE_DIR=/vagrant/ccache_vagrant' >> /etc/environment; "
pkg_cmd << "echo 'cd /vagrant/build_vagrant; source /vagrant/scripts/setdevenv.sh' >> /home/vagrant/.bashrc; "
diff --git a/scripts/vagrant/gdal.sh b/scripts/vagrant/gdal.sh
index 5a5dc6bf02..725e2712cb 100755
--- a/scripts/vagrant/gdal.sh
+++ b/scripts/vagrant/gdal.sh
@@ -26,9 +26,8 @@ cmake .. \
-GNinja \
-DCMAKE_INSTALL_PREFIX=/opt/gdal-dev \
-DUSE_CCACHE=ON \
- -DUSE_ALTERNATE_LINKER:STRING=mold \
-DCMAKE_BUILD_TYPE=Debug
ninja -j6
sudo ninja install
-PYTHON_CMD=python3 && $PYTHON_CMD -m pip install -r ../autotest/requirements.txt
+PYTHON_CMD=python3 && $PYTHON_CMD -m pip install -r ../autotest/requirements.txt --break-system-packages
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20250507/3b99c484/attachment-0001.htm>
More information about the gdal-dev
mailing list