[gdal-dev] About bug fix / patch release policy for older GDAL versions (setuptools >= 58.0 and GDAL <3.3 compatibility issues)
Momtchil Momtchev
momtchil at momtchev.com
Fri Mar 25 15:40:09 PDT 2022
If you have lots of Debian servers, what you can do is to use the Debian
experimental GDAL packages - which are usually up to date with the
latest version of GDAL - and to set up a Dockerfile to build these for
Debian stable and then to use reprepro to distribute them to your
machines. If you have never done this before, you have about 1 day of
work to set up everything. I did maintain a GDAL distribution for Debian
10 for a few years. You can find my Dockerfiles here:
https://github.com/mmomtchev/debbuild-gdal
On 25/03/2022 23:24, snehal waychal wrote:
>
> Dear Evan,
>
>
> Oh! thank you very much for a fast test release!
>
>
> I can confirm that released version works as expected. And to be sure,
> here is what/how I have tested:
>
>
> ```
>
> $ docker run -it python:3.7.13-slim-bullseye bash
>
>
> $ apt-get update && apt-get install -y libgdal-dev gcc g++
>
>
> # let's take newer, problematic setuptools
>
> $ pip install -U setuptools
>
> $ pip list
>
> PackageVersion
>
> -----------------
>
> pip22.0.4
>
> setuptools61.0.0
>
> wheel0.37.1
>
>
> # existing release from pypi.org <http://pypi.org> fails
>
>
> $ pip install gdal==3.2.2
>
>
> Collecting gdal==3.2.2
>
> Downloading GDAL-3.2.2.tar.gz (601 kB)
>
> ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 601.7/601.7 KB 8.0 MB/s
> eta 0:00:00
>
> Preparing metadata (setup.py) ... error
>
> error: subprocess-exited-with-error
>
>
> × python setup.py egg_info did not run successfully.
>
> │ exit code: 1
>
> ╰─> [6 lines of output]
>
> WARNING: numpy not available!Array support will not be enabled
>
> /usr/local/lib/python3.7/site-packages/setuptools/_distutils/dist.py:275:
> UserWarning: Unknown distribution option: 'use_2to3_fixers'
>
> warnings.warn(msg)
>
> /usr/local/lib/python3.7/site-packages/setuptools/_distutils/dist.py:275:
> UserWarning: Unknown distribution option: 'use_2to3_exclude_fixers'
>
> warnings.warn(msg)
>
> error in GDAL setup command: use_2to3 is invalid.
>
> [end of output]
>
>
> note: This error originates from a subprocess, and is likely not a
> problem with pip.
>
> error: metadata-generation-failed
>
>
> × Encountered error while generating package metadata.
>
> ╰─> See above for output.
>
>
> note: This is an issue with the package mentioned above, not pip.
>
> hint: See above for details.
>
> # installing newer one from test.pypi.org <http://test.pypi.org>
> works fine!
>
> $ pip install -i https://test.pypi.org/simple/ GDAL==3.2.2.1
>
>
> Looking in indexes: https://test.pypi.org/simple/
>
> Collecting GDAL==3.2.2.1
>
> Downloading
> https://test-files.pythonhosted.org/packages/b7/02/4356dbdabf8f4fa898fe2378c8c096df3da254508ec8bc0663920cf4b92d/GDAL-3.2.2.1.tar.gz
> (600 kB)
>
> ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 600.9/600.9 KB 8.2 MB/s
> eta 0:00:00
>
> Preparing metadata (setup.py) ... done
>
> Building wheels for collected packages: GDAL
>
> Building wheel for GDAL (setup.py) ... done
>
> Created wheel for GDAL:
> filename=GDAL-3.2.2.1-cp37-cp37m-linux_x86_64.whl size=816416
> sha256=0209dc3c56cd626137c53187c4a8fb7758587c9030cf8d479d8a2c7864fd5c77
>
> Stored in directory:
> /root/.cache/pip/wheels/4b/f2/02/0ca533c034c5cc5840d619cd7004f32d3362c04a1c1500d0a1
>
> Successfully built GDAL
>
> Installing collected packages: GDAL
>
> Successfully installed GDAL-3.2.2.1
>
> WARNING: Running pip as the 'root' user can result in broken
> permissions and conflicting behaviour with the system package
> manager. It is recommended to use a virtual environment
> instead:https://pip.pypa.io/warnings/venv
>
>
> ```
>
>
> I hope this gives us some confidence that we are on a right track! :)
>
>
> If I should test something else, please let me know! Looking forward
> for the release on the pypi.org <http://pypi.org>.
>
>
> Thank you again!!
>
>
> Regards,
>
> Snehal
>
>
>
> On Fri, Mar 25, 2022 at 1:45 PM Even Rouault
> <even.rouault at spatialys.com> wrote:
>
> ok, I missed this was on the pypi package.
>
> I've tried to make a patch over 3.2.2. For now, I've uploaded it
> only to https://test.pypi.org/project/GDAL/3.2.2.1/ . Can you test
> that and confirm that works properly ? If so, I'll push it to pypi
> official.
>
> I've tracked the changes in a patch/3.2.2.1 <http://3.2.2.1>
> branch, sitting for now in my fork:
>
> https://github.com/OSGeo/gdal/compare/v3.2.2..rouault:patch/3.2.2.1?expand=1
>
> I'll push that to OSGeo/gdal as well and tag once confirmed things
> work fine
>
> Hoping I'm not going into troubles doing this stuff outside of our
> usual release processes...
>
> Even
>
> Le 25/03/2022 à 13:13, snehal waychal a écrit :
>>
>> Dear Even, dear Bas,
>>
>>
>> Thank you very much for the quick response and highlightingthe
>> Debian/Ubuntu release policy aspects. And also about the link to
>> the ubuntugis-unstable PPA.
>>
>>
>> >/what you discuss here is all about the patch & backport policy
>> of the/
>>
>> >/Debian GDAL package. You can try to file a bug to Debian and
>> point to/
>>
>> >/the patch you'd want to see backported,/
>>
>>
>> But just to be sure I haven’t misunderstoodyour response or
>> misrepresented the original issue. The patch release I described
>> is also needed in the official releases of the GDAL **python**
>> package. If I am not mistaken, the GDAL team is taking care of
>> source distributions on https://pypi.org/project/GDAL/
>> <https://pypi.org/project/GDAL/>. I looked into released pypi
>> versions and if I am not mistaken there is no patch release for
>> v3.2.2 with fix for setuptools compatibility issue.
>>
>>
>> So, if we could make a new patch release of the/*python* package
>> of GDAL/ and push the sdist package to pypi.org <http://pypi.org>
>> (something like 3.2.2.1, as I described in the previous email)
>> then that would also solve the issue. Because as python
>> developers, we install *python* packages via pip/pipenv/poetry
>> and those package managers would pick the new patch release from
>> pypi.org <http://pypi.org>. (The GDAL library component
>> libgdal-dev v3.2.2 will still come from the debian system package
>> repo and there is no issue with that).
>>
>>
>> Hope I am not missing something obvious here. Please let me know.
>>
>>
>> Thank you again!
>>
>>
>> Regards,
>>
>> Snehal
>>
>>
>> On Fri, Mar 25, 2022 at 12:29 PM Sebastiaan Couwenberg
>> <sebastic at xs4all.nl> wrote:
>>
>> On 3/25/22 12:07, Even Rouault wrote:
>> > what you discuss here is all about the patch & backport
>> policy of the
>> > Debian GDAL package. You can try to file a bug to Debian
>> and point to
>> > the patch you'd want to see backported, but I can't promise
>> if there
>> > would be interest in their maintenance team to create an
>> updated package
>> > with it (my understanding is that even if we'd release a
>> new 3.2.x patch
>> > release, it wouldn't be packaged in LTS distributions. I'm
>> not sure how
>> > much of that is linked to Debian policy or availability of
>> people that
>> > do the work)
>>
>> Packages in Debian stable releases only get updates to fix
>> bugs of
>> severity important or higher [0]. GDAL patch releases also
>> contain
>> changes for lower severity issues, it's not worth the effort
>> to vet all
>> those changes. Any changes to packages in stable also risk
>> introducing
>> regressions which are highly undesirable in LTS releases
>> known for their
>> stability.
>>
>> People should be maintaining their own packaging repositories
>> where they
>> host packages with changes for their needs that cannot be easily
>> upstreamed to the package in the distribution itself.
>> Scratching your
>> own itch was a corner stone of Open Source that people are
>> seemingly
>> forgetting or never having known about in the first place.
>>
>> [0]
>> https://www.debian.org/doc/manuals/developers-reference/pkgs.en.html#special-case-uploads-to-the-stable-and-oldstable-distributions
>>
>> Kind Regards,
>>
>> Bas
>>
>> --
>> GPG Key ID: 4096R/6750F10AE88D4AF1
>> Fingerprint: 8182 DE41 7056 408D 6146 50D1 6750 F10A E88D 4AF1
>> _______________________________________________
>> gdal-dev mailing list
>> gdal-dev at lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>>
> --
> http://www.spatialys.com
> My software is free, but my time generally not.
>
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
--
Momtchil Momtchev<momtchil at momtchev.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220325/323bebdf/attachment-0001.html>
More information about the gdal-dev
mailing list