<!DOCTYPE html>
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body>
Hello,<br>
<br>
I'm posting this trial, this experience for information.<br>
I'm searching to migrate few projects from Java JDK 11 to 17 (at
least) to avoid a trap where recent operating systems haven't (and
cannot anymore install) Java 11 on them. This involving upgrading
some of their dependencies, by the way.<br>
<br>
The long explanation of my motives <a
href="https://opensource.stackexchange.com/questions/15474/should-projects-with-dependencies-too-old-programming-languages-version-becomin">is
here</a>, and I was especially in need of that for gdal, for my
own interest of studying it deeply. <br>
For that, I was willing to get its convenient Vagrantfile, add
locally for my own use a GNOME to it, and then install my CLion and
others IDE there to work with it.<br>
<br>
I encountered few troubles when generating the Vagrant VM that I
took from master, but Stack Overflow community helped me to solve
them. I've explained what happened to me <a
href="https://stackoverflow.com/questions/79607954/a-project-is-built-from-a-vagrantfile-and-im-switching-its-base-box-now-a-vagr">here</a>.
<br>
<br>
To resume: I've switched from Ubuntu 22 LTS Vagrant base box to
a Debian 12 one, JDK 11 to 17, add to remove mold alternate linker
mention to make everything build.<br>
Now, everything can be built and tested, as changes allowing 96%
of ctests to run still, and for pytest ones it's 16070 passed, 1
xpassed, 268 failed. (I wasn't expecting everything could work like
a charm immediately).<br>
<br>
Your comments are welcome.<br>
<br>
Regards,<br>
<br>
Marc Le Bihan<br>
<br>
For python tests failing ones are these ones, for either Runtime
Error, Assertion Error or expected output file missing:<br>
<br>
<div style="background-color:#ffffff;color:#080808">
<pre
style="font-family:'Liberation Mono',monospace;font-size:13,5pt;">ogr/ogr_basic_test.py::test_ogr_basic_test_future_warning_exceptions
ogr/ogr_csv.py::test_ogr_csv_write_to_stdout
ogr/ogr_virtualogr.py::test_ogr_virtualogr_3
gcore/basic_test.py::test_basic_test_8
gcore/basic_test.py::test_basic_test_UseExceptions
gcore/basic_test.py::test_basic_test_UseExceptions_ogr_open
gcore/basic_test.py::test_basic_test_DontUseExceptions
gcore/misc.py::test_misc_quiet_errors[quiet_errors]
gcore/misc.py::test_misc_quiet_errors[quiet_warnings]
gcore/pam.py::test_pam_11
gcore/vsicurl_streaming.py::test_vsicurl_streaming_real_resource
gdrivers/bag.py::test_bag_force_opening_no_match
gdrivers/gdalhttp.py::test_http_6
gdrivers/netcdf_multidim.py::test_netcdf_multidim_cache_pamproxydb
gdrivers/wms.py::test_wms_data_via_mrf
osr/osr_basic.py::test_osr_basic_warning_exceptions
pyscripts/test_gdal2tiles.py::test_gdal2tiles_py_simple
pyscripts/test_gdal2tiles.py::test_gdal2tiles_py_zoom_option
pyscripts/test_gdal2tiles.py::test_gdal2tiles_py_resampling_option[average, near, bilinear, cubic, cubicspline, lanczos, mode, max, min, med, q1, q3]
pyscripts/test_gdal2tiles.py::test_gdal2tiles_py_xyz/te
pyscripts/test_gdal2tiles.py::test_gdal2tiles_py_invalid_srs
pyscripts/test_gdal2tiles.py::test_handle_utf8_filename
pyscripts/test_gdal2tiles.py::test_exclude_transparent_tiles
pyscripts/test_gdal2tiles.py::test_gdal2tiles_py_profile_raster
pyscripts/test_gdal2tiles.py::test_gdal2tiles_py_profile_raster_oversample
pyscripts/test_gdal2tiles.py::test_gdal2tiles_py_profile_raster_xyz
pyscripts/test_gdal2tiles.py::test_gdal2tiles_py_profile_geodetic_tmscompatible_xyz
pyscripts/test_gdal2tiles.py::test_gdal2tiles_py_mapml
pyscripts/test_gdal2tiles.py::test_gdal2tiles_excluded_values
pyscripts/test_gdal2tiles.py::test_gdal2tiles_nodata_values_pct_threshold
pyscripts/test_gdal2tiles.py::test_gdal2tiles_py_jpeg_3band_input[average-expected_stats_z00-expected_s
pyscripts/test_gdal2tiles.py::test_gdal2tiles_py_jpeg_1band_input[average-expected_stats_z140-expected_
pyscripts/test_gdal2xyz.py::test_gdal2xyz_py_2
pyscripts/test_gdal2xyz.py::test_gdal2xyz_py_3
pyscripts/test_gdal2xyz.py::test_gdal2xyz_py_srcnodata_dstnodata
pyscripts/test_gdal2xyz.py::test_gdal2xyz_py_vsistdout
pyscripts/test_gdal_calc.py::test_gdal_calc_py_1/test_g
pyscripts/test_gdal_calc.py::test_gdal_calc_py_1b/test_
pyscripts/test_gdal_calc.py::test_gdal_calc_py_1c/test_
pyscripts/test_gdal_calc.py::test_gdal_calc_py_2a/test_
pyscripts/test_gdal_calc.py::test_gdal_calc_py_2b/test_
pyscripts/test_gdal_calc.py::test_gdal_calc_py_2c/test_
pyscripts/test_gdal_calc.py::test_gdal_calc_py_3/test_g
pyscripts/test_gdal_calc.py::test_gdal_calc_py_4a/test_
pyscripts/test_gdal_calc.py::test_gdal_calc_py_4b/test_
pyscripts/test_gdal_calc.py::test_gdal_calc_py_7a[--optfile ]
pyscripts/test_gdal_calc.py::test_gdal_calc_py_7a[@]/te
pyscripts/test_gdal_calc.py::test_gdal_calc_py_7b/test_
pyscripts/test_gdal_calc.py::test_gdal_calc_py_7c/test_
pyscripts/test_gdal_calc.py::test_gdal_calc_py_7d/test_
pyscripts/test_gdal_calc.py::test_gdal_calc_py_8/test_g
pyscripts/test_gdal_calc.py::test_gdal_calc_py_10/test_
pyscripts/test_gdal_calc.py::test_gdal_calc_py_multiple_inputs_same_alpha
pyscripts/test_gdal_edit.py::test_gdal_edit_py_1[True]
pyscripts/test_gdal_edit.py::test_gdal_edit_py_1[False]
pyscripts/test_gdal_edit.py::test_gdal_edit_py_1b, 2 to 8
pyscripts/test_gdal_edit.py::test_gdal_edit_py_unsetrpc
pyscripts/test_gdal_edit.py::test_gdal_edit_py_epoch
pyscripts/test_gdal_fillnodata.py::test_gdal_fillnodata_1
pyscripts/test_gdal_fillnodata.py::test_gdal_fillnodata_2
pyscripts/test_gdal_fillnodata.py::test_gdal_fillnodata_smoothing
pyscripts/test_gdal_fillnodata.py::test_gdal_fillnodata_nearest
pyscripts/test_gdal_merge.py::test_gdal_merge_[1 to 5]/test_gd
pyscripts/test_gdal_pansharpen.py::test_gdal_pansharpen_1
pyscripts/test_gdal_pansharpen.py::test_gdal_pansharpen_2
pyscripts/test_gdal_polygonize.py::test_gdal_polygonize_[1 to 4]
pyscripts/test_gdal_polygonize.py::test_gdal_polygonize_4bis
pyscripts/test_gdal_polygonize.py::test_gdal_polygonize_minus_8
pyscripts/test_gdal_polygonize.py::test_gdal_polygonize_overwrite[geojson]
pyscripts/test_gdal_polygonize.py::test_gdal_polygonize_overwrite[gpkg]
pyscripts/test_gdal_proximity.py::test_gdal_proximity_[1 to 3]
pyscripts/test_gdal_retile.py::test_gdal_retile_[2 to 5]/test_
pyscripts/test_gdal_retile.py::test_gdal_retile_non_contigous
pyscripts/test_gdal_retile.py::test_gdal_retile_png/tes
pyscripts/test_gdal_retile.py::test_gdal_retile_rotational_geotransform
pyscripts/test_gdal_retile.py::test_gdal_retile_different_srs[32631-32632-has
pyscripts/test_gdal_retile.py::test_gdal_retile_different_srs[32631-None-has
pyscripts/test_gdal_retile.py::test_gdal_retile_different_srs[None-32631-has
pyscripts/test_gdal_sieve.py::test_gdal_sieve_1
pyscripts/test_gdal_sieve.py::test_gdal_sieve_src_without_nodata
pyscripts/test_gdal_utils_retcodes.py::test_script[input0 to 75]
pyscripts/test_gdalattachpct.py::test_gdalattachpct_basic
pyscripts/test_gdalattachpct.py::test_gdalattachpct_vrt_output p
pyscripts/test_gdalbuildvrtofvrt.py::test_gdalbuildvrtofvrt_basic an
pyscripts/test_gdalbuildvrtofvrt.py::test_gdalbuildvrtofvrt_intermediate_vrt_path
pyscripts/test_gdalbuildvrtofvrt.py::test_gdalbuildvrtofvrt_intermediate_vrt_path_specified_ovr_factors
pyscripts/test_gdalcompare.py::test_gdalcompare_same
pyscripts/test_gdalcompare.py::test_gdalcompare_different_type
pyscripts/test_gdalinfo_py.py::test_gdalinfo_py_[1 to 10]
pyscripts/test_gdallocationinfo_py.py::test_gdallocationinfo_py_1
pyscripts/test_gdallocationinfo_py.py::test_gdallocationinfo_py_3
pyscripts/test_gdallocationinfo_py.py::test_gdallocationinfo_py_4
pyscripts/test_gdallocationinfo_py.py::test_gdallocationinfo_py_6
pyscripts/test_gdallocationinfo_py.py::test_gdallocationinfo_py_wgs84
pyscripts/test_gdalmove.py::test_gdalmove_1
pyscripts/test_ogr_layer_algebra.py::test_ogr_layer_algebra_intersection
pyscripts/test_ogr_layer_algebra.py::test_ogr_layer_algebra_union
pyscripts/test_ogr_layer_algebra.py::test_ogr_layer_algebra_symdifference
pyscripts/test_ogr_layer_algebra.py::test_ogr_layer_algebra_identity
pyscripts/test_ogr_layer_algebra.py::test_ogr_layer_algebra_update
pyscripts/test_ogr_layer_algebra.py::test_ogr_layer_algebra_clip
pyscripts/test_ogr_layer_algebra.py::test_ogr_layer_algebra_erase
pyscripts/test_ogrinfo_py.py::test_ogrinfo_py_[1 to 11, 22 and 23]
pyscripts/test_ogrmerge.py::test_ogrmerge_[1 to 7]/test_ogrmer
pyscripts/test_ogrmerge.py::test_ogrmerge_7a/test_ogrme
pyscripts/test_ogrmerge.py::test_ogrmerge_7b/test_ogrme
pyscripts/test_ogrmerge.py::test_ogrmerge_7c/test_ogrme
pyscripts/test_ogrmerge.py::test_ogrmerge_[8 to 12]/test_ogrmer
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg[True-True-True-None-None-None]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg[True-False-True-None-None-None]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg[False-True-True-None-None-None]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg[False-False-True-None-None-None]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg[True-True-False-None-None-None0]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg[True-False-False-None-None-None]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg[False-True-False-None-None-None]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg[False-False-False-None-None-None]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg[True-True-False-None-None-None1]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg[True-True-False-EPSG:32631-None-None]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg[True-True-False-None-EPSG:32631-None]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg[True-True-False-None-None-EPSG:32631]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg[True-True-False-None-EPSG:32631-EPSG:4326]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg_non_spatial[True]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg_non_spatial[False]
pyscripts/test_ogrmerge.py::test_ogrmerge_gpkg_curve_geom_in_generic_layer
pyscripts/test_pct.py::test_rgb2pct_1/tmp7/test_rgb2pct
pyscripts/test_pct.py::test_pct2rgb_1/test_pct2rgb_10/t
pyscripts/test_pct.py::test_rgb2pct_2/tmp8/test_rgb2pct
pyscripts/test_pct.py::test_rgb2pct_creation_option/tes
pyscripts/test_pct.py::test_rgb2pct_3/test_rgb2pct_30/t
pyscripts/test_pct.py::test_pct2rgb_4/test_pct2rgb_40/t
pyscripts/test_pct.py::test_gdalattachpct_1
pyscripts/test_validate_geoparquet.py::test_validate_geoparquet_not_parquet_file
pyscripts/test_validate_geoparquet.py::test_validate_geoparquet_ok
</pre>
</div>
<br>
For ctest, the failing ones are these ones :<br>
<br>
44 - test-virtual-memory (Failed)<br>
59 - autotest_gdrivers (Failed)<br>
<br>
(but I had few more failing at a previous attempt. I think : 5 of
them)<br>
<br>
Here are the changes I've experienced for this attempt on the gdal
Vagrantfile :<br>
<pre wrap="" class="moz-quote-pre">
diff --git a/Vagrantfile b/Vagrantfile
index dd1ec6019d..ac7203d4ba 100644
--- a/Vagrantfile
+++ b/Vagrantfile
@@ -8,10 +8,11 @@ VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
# specify memory size in MiB
- vm_ram = ENV['VAGRANT_VM_RAM'] || 4096
- vm_cpu = ENV['VAGRANT_VM_CPU'] || 2
+ vm_ram = ENV['VAGRANT_VM_RAM'] || 8192
+ vm_cpu = ENV['VAGRANT_VM_CPU'] || 8
vm_ram_bytes = vm_ram * 1024 * 1024
+ config.vm.synced_folder "./", "/vagrant"
config.vm.hostname = "gdal-vagrant"
config.vm.host_name = "gdal-vagrant"
@@ -34,16 +35,25 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
# See <a class="moz-txt-link-freetext" href="https://bugs.launchpad.net/cloud-images/+bug/1969664">https://bugs.launchpad.net/cloud-images/+bug/1969664</a>
# Ubuntu 22.04 no longer accepts RSA keys, which causes issues with older Vagrant
# The below location has a jammy64 image that accepts RSA keys
- config.vm.box_url = <a class="moz-txt-link-rfc2396E" href="https://people.canonical.com/~jchittum/vagrant-testing/ubuntu-jammy-vagrant-TESTING-20220419.box">"https://people.canonical.com/~jchittum/vagrant-testing/ubuntu-jammy-vagrant-TESTING-20220419.box"</a>
- config.vm.box = "ubuntu/jammy64"
+ config.vm.box = "generic/debian12"
+ config.vm.box_version = "4.3.12"
config.vm.provider :virtualbox do |vb|
vb.customize ["modifyvm", :id, "--memory", vm_ram]
vb.customize ["modifyvm", :id, "--cpus", vm_cpu]
vb.customize ["modifyvm", :id, "--ioapic", "on"]
vb.name = "gdal-vagrant"
+
+ # vb.gui = true
end
+ config.vm.provision "shell", inline: <<-SHELL
+ sudo apt install gnupg2
+ sh -c 'echo "deb <a class="moz-txt-link-freetext" href="http://apt.postgresql.org/pub/repos/apt">http://apt.postgresql.org/pub/repos/apt</a> $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
+ curl -fsSL <a class="moz-txt-link-freetext" href="https://www.postgresql.org/media/keys/ACCC4CF8.asc">https://www.postgresql.org/media/keys/ACCC4CF8.asc</a> | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/postgresql.gpg
+ apt-get update && DEBIAN_FRONTEND=noninteractive apt-get upgrade -y
+ SHELL
+
#config.vm.provider :lxc do |lxc,ovrd|
# ovrd.vm.box = "cultuurnet/ubuntu-14.04-64-puppet"
# lxc.backingstore = 'dir'
@@ -77,15 +87,6 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
}
end
- # Unless explicitly declined, use the VM host's file system to cache
- # .deb files to avoid repeated downloads on each vagrant up
- unless File.exists?("../.no_apt_cache")
- cache_dir = "../apt-cache/#{config.vm.box}"
- FileUtils.mkdir_p(cache_dir) unless Dir.exists?(cache_dir)
- puts "Using local apt cache, #{cache_dir}"
- config.vm.synced_folder cache_dir, "/var/cache/apt/archives"
- end
-
ppaRepos = [
]
@@ -115,7 +116,7 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
"bash-completion",
"swig",
"ant",
- "openjdk-11-jdk",
+ "openjdk-17-jdk",
"mono-mcs",
"mono-runtime",
"libmono-system-drawing4.0-cil<a class="moz-txt-link-rfc2396E" href="mailto:,@@-124,8+125,8@@Vagrant.configure(VAGRANTFILE_API_VERSION)do|config|">",
@@ -124,8 +125,8 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
"</a>python3-setuptools",
"python3-pip",
"postgis",
- "postgresql",
- "postgresql-postgis",
+ "postgresql-14",
+ "postgresql-14-postgis-3",
"gpsbabel",
"doxygen",
"libproj-dev<a class="moz-txt-link-rfc2396E" href="mailto:,@@-159,7+160,7@@Vagrant.configure(VAGRANTFILE_API_VERSION)do|config|">",
@@ -159,7 +160,7 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
"</a>libcrypto++-dev",
"libfyba-dev",
"libkml-dev",
- "libmysqlclient-dev",
+ "libmariadb-dev",
"libogdi-dev",
"libcfitsio-dev",
"libzstd-dev<a class="moz-txt-link-rfc2396E" href="mailto:,@@-177,6+178,7@@Vagrant.configure(VAGRANTFILE_API_VERSION)do|config|">",
@@ -177,6 +178,7 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
"</a>libqhull-dev",
"libjson-c-dev",
"libtiff5-dev",
+ "libgtest-dev",
];
config.ssh.forward_agent = true
@@ -191,7 +193,8 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
# install packages we need
pkg_cmd << "apt-get update -qq; "
pkg_cmd << "apt-get --no-install-recommends install -q -y " + packageList.join(" ") << " ; "
-
+ pkg_cmd << "pip install pytest filelock lxml --break-system-package; "
+
# setup environment when we log in
pkg_cmd << "echo 'CCACHE_DIR=/vagrant/ccache_vagrant' >> /etc/environment; "
pkg_cmd << "echo 'cd /vagrant/build_vagrant; source /vagrant/scripts/setdevenv.sh' >> /home/vagrant/.bashrc; "
diff --git a/scripts/vagrant/gdal.sh b/scripts/vagrant/gdal.sh
index 5a5dc6bf02..725e2712cb 100755
--- a/scripts/vagrant/gdal.sh
+++ b/scripts/vagrant/gdal.sh
@@ -26,9 +26,8 @@ cmake .. \
-GNinja \
-DCMAKE_INSTALL_PREFIX=/opt/gdal-dev \
-DUSE_CCACHE=ON \
- -DUSE_ALTERNATE_LINKER:STRING=mold \
-DCMAKE_BUILD_TYPE=Debug
ninja -j6
sudo ninja install
-PYTHON_CMD=python3 && $PYTHON_CMD -m pip install -r ../autotest/requirements.txt
+PYTHON_CMD=python3 && $PYTHON_CMD -m pip install -r ../autotest/requirements.txt --break-system-packages
</pre>
</body>
</html>