[gdal-dev] About bug fix / patch release policy for older GDAL versions (setuptools >= 58.0 and GDAL <3.3 compatibility issues)
snehal waychal
snehal.waychal at gmail.com
Sun Mar 27 06:37:34 PDT 2022
Thank you Momtchil for providing additional references including
debbuild-gdal! Currently we do not have such a use case but I see that it
would be useful if we want to upgrade to a newer GDAL version on an
existing distribution. If I am not mistaken, the Dockerfiles are not
updated for Bullseye and especially for Aarch64. Is there someone else now
maintaining these packages for newer Debian distribution? If so, reference
to those could be helpful if I (or someone else) will need to revisit this
in the future.
- Snehal
On Fri, Mar 25, 2022 at 11:40 PM Momtchil Momtchev <momtchil at momtchev.com>
wrote:
>
> If you have lots of Debian servers, what you can do is to use the Debian
> experimental GDAL packages - which are usually up to date with the latest
> version of GDAL - and to set up a Dockerfile to build these for Debian
> stable and then to use reprepro to distribute them to your machines. If you
> have never done this before, you have about 1 day of work to set up
> everything. I did maintain a GDAL distribution for Debian 10 for a few
> years. You can find my Dockerfiles here:
>
> https://github.com/mmomtchev/debbuild-gdal
>
>
> On 25/03/2022 23:24, snehal waychal wrote:
>
> Dear Evan,
>
>
> Oh! thank you very much for a fast test release!
>
>
> I can confirm that released version works as expected. And to be sure,
> here is what/how I have tested:
>
>
> ```
>
> $ docker run -it python:3.7.13-slim-bullseye bash
>
>
> $ apt-get update && apt-get install -y libgdal-dev gcc g++
>
>
> # let's take newer, problematic setuptools
>
> $ pip install -U setuptools
>
> $ pip list
>
> Package Version
>
> ---------- -------
>
> pip 22.0.4
>
> setuptools 61.0.0
>
> wheel 0.37.1
>
>
> # existing release from pypi.org fails
>
>
> $ pip install gdal==3.2.2
>
>
> Collecting gdal==3.2.2
>
> Downloading GDAL-3.2.2.tar.gz (601 kB)
>
> ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 601.7/601.7 KB 8.0 MB/s eta
> 0:00:00
>
> Preparing metadata (setup.py) ... error
>
> error: subprocess-exited-with-error
>
>
> × python setup.py egg_info did not run successfully.
>
> │ exit code: 1
>
> ╰─> [6 lines of output]
>
> WARNING: numpy not available! Array support will not be enabled
>
> /usr/local/lib/python3.7/site-packages/setuptools/_distutils/dist.py:275:
> UserWarning: Unknown distribution option: 'use_2to3_fixers'
>
> warnings.warn(msg)
>
> /usr/local/lib/python3.7/site-packages/setuptools/_distutils/dist.py:275:
> UserWarning: Unknown distribution option: 'use_2to3_exclude_fixers'
>
> warnings.warn(msg)
>
> error in GDAL setup command: use_2to3 is invalid.
>
> [end of output]
>
>
> note: This error originates from a subprocess, and is likely not a
> problem with pip.
>
> error: metadata-generation-failed
>
>
> × Encountered error while generating package metadata.
>
> ╰─> See above for output.
>
>
> note: This is an issue with the package mentioned above, not pip.
>
> hint: See above for details.
>
>
>
>
>
> # installing newer one from test.pypi.org works fine!
>
>
>
> $ pip install -i https://test.pypi.org/simple/ GDAL==3.2.2.1
>
>
> Looking in indexes: https://test.pypi.org/simple/
>
> Collecting GDAL==3.2.2.1
>
> Downloading
> https://test-files.pythonhosted.org/packages/b7/02/4356dbdabf8f4fa898fe2378c8c096df3da254508ec8bc0663920cf4b92d/GDAL-3.2.2.1.tar.gz
> (600 kB)
>
> ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 600.9/600.9 KB 8.2 MB/s eta
> 0:00:00
>
> Preparing metadata (setup.py) ... done
>
> Building wheels for collected packages: GDAL
>
> Building wheel for GDAL (setup.py) ... done
>
> Created wheel for GDAL:
> filename=GDAL-3.2.2.1-cp37-cp37m-linux_x86_64.whl size=816416
> sha256=0209dc3c56cd626137c53187c4a8fb7758587c9030cf8d479d8a2c7864fd5c77
>
> Stored in directory:
> /root/.cache/pip/wheels/4b/f2/02/0ca533c034c5cc5840d619cd7004f32d3362c04a1c1500d0a1
>
> Successfully built GDAL
>
> Installing collected packages: GDAL
>
> Successfully installed GDAL-3.2.2.1
>
> WARNING: Running pip as the 'root' user can result in broken permissions
> and conflicting behaviour with the system package manager. It is
> recommended to use a virtual environment instead:
> https://pip.pypa.io/warnings/venv
>
>
> ```
>
>
> I hope this gives us some confidence that we are on a right track! :)
>
>
> If I should test something else, please let me know! Looking forward for
> the release on the pypi.org.
>
>
> Thank you again!!
>
>
> Regards,
>
> Snehal
>
>
> On Fri, Mar 25, 2022 at 1:45 PM Even Rouault <even.rouault at spatialys.com>
> wrote:
>
>> ok, I missed this was on the pypi package.
>>
>> I've tried to make a patch over 3.2.2. For now, I've uploaded it only to
>> https://test.pypi.org/project/GDAL/3.2.2.1/ . Can you test that and
>> confirm that works properly ? If so, I'll push it to pypi official.
>>
>> I've tracked the changes in a patch/3.2.2.1 branch, sitting for now in
>> my fork:
>>
>>
>> https://github.com/OSGeo/gdal/compare/v3.2.2..rouault:patch/3.2.2.1?expand=1
>>
>> I'll push that to OSGeo/gdal as well and tag once confirmed things work
>> fine
>>
>> Hoping I'm not going into troubles doing this stuff outside of our usual
>> release processes...
>>
>> Even
>> Le 25/03/2022 à 13:13, snehal waychal a écrit :
>>
>> Dear Even, dear Bas,
>>
>>
>> Thank you very much for the quick response and highlighting the
>> Debian/Ubuntu release policy aspects. And also about the link to the
>> ubuntugis-unstable PPA.
>>
>>
>> >* what you discuss here is all about the patch & backport policy of
>> the *
>>
>> >* Debian GDAL package. You can try to file a bug to Debian and point
>> to *
>>
>> >* the patch you'd want to see backported,*
>>
>>
>> But just to be sure I haven’t misunderstood your response or
>> misrepresented the original issue. The patch release I described is also
>> needed in the official releases of the GDAL **python** package. If I am
>> not mistaken, the GDAL team is taking care of source distributions on
>> https://pypi.org/project/GDAL/. I looked into released pypi versions and
>> if I am not mistaken there is no patch release for v3.2.2 with fix for
>> setuptools compatibility issue.
>>
>>
>> So, if we could make a new patch release of the* *python* package of
>> GDAL* and push the sdist package to pypi.org (something like 3.2.2.1, as
>> I described in the previous email) then that would also solve the issue.
>> Because as python developers, we install *python* packages via
>> pip/pipenv/poetry and those package managers would pick the new patch
>> release from pypi.org. (The GDAL library component libgdal-dev v3.2.2
>> will still come from the debian system package repo and there is no issue
>> with that).
>>
>>
>> Hope I am not missing something obvious here. Please let me know.
>>
>>
>> Thank you again!
>>
>>
>> Regards,
>>
>> Snehal
>>
>> On Fri, Mar 25, 2022 at 12:29 PM Sebastiaan Couwenberg <
>> sebastic at xs4all.nl> wrote:
>>
>>> On 3/25/22 12:07, Even Rouault wrote:
>>> > what you discuss here is all about the patch & backport policy of the
>>> > Debian GDAL package. You can try to file a bug to Debian and point to
>>> > the patch you'd want to see backported, but I can't promise if there
>>> > would be interest in their maintenance team to create an updated
>>> package
>>> > with it (my understanding is that even if we'd release a new 3.2.x
>>> patch
>>> > release, it wouldn't be packaged in LTS distributions. I'm not sure
>>> how
>>> > much of that is linked to Debian policy or availability of people that
>>> > do the work)
>>>
>>> Packages in Debian stable releases only get updates to fix bugs of
>>> severity important or higher [0]. GDAL patch releases also contain
>>> changes for lower severity issues, it's not worth the effort to vet all
>>> those changes. Any changes to packages in stable also risk introducing
>>> regressions which are highly undesirable in LTS releases known for their
>>> stability.
>>>
>>> People should be maintaining their own packaging repositories where they
>>> host packages with changes for their needs that cannot be easily
>>> upstreamed to the package in the distribution itself. Scratching your
>>> own itch was a corner stone of Open Source that people are seemingly
>>> forgetting or never having known about in the first place.
>>>
>>> [0]
>>>
>>> https://www.debian.org/doc/manuals/developers-reference/pkgs.en.html#special-case-uploads-to-the-stable-and-oldstable-distributions
>>>
>>> Kind Regards,
>>>
>>> Bas
>>>
>>> --
>>> GPG Key ID: 4096R/6750F10AE88D4AF1
>>> Fingerprint: 8182 DE41 7056 408D 6146 50D1 6750 F10A E88D 4AF1
>>> _______________________________________________
>>> gdal-dev mailing list
>>> gdal-dev at lists.osgeo.org
>>> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>>>
>> -- http://www.spatialys.com
>> My software is free, but my time generally not.
>>
>>
> _______________________________________________
> gdal-dev mailing listgdal-dev at lists.osgeo.orghttps://lists.osgeo.org/mailman/listinfo/gdal-dev
>
> --
> Momtchil Momtchev <momtchil at momtchev.com> <momtchil at momtchev.com>
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220327/3b224bd4/attachment-0001.html>
More information about the gdal-dev
mailing list