[gdal-dev] About bug fix / patch release policy for older GDAL versions (setuptools >= 58.0 and GDAL <3.3 compatibility issues)

Fri Mar 25 15:24:40 PDT 2022

Dear Evan,

Oh! thank you very much for a fast test release!

I can confirm that released version works as expected. And to be sure, here
is what/how I have tested:

```

$ docker run -it python:3.7.13-slim-bullseye bash

$ apt-get update && apt-get install -y libgdal-dev gcc g++

# let's take newer, problematic setuptools

$ pip install -U setuptools

$ pip list

Package       Version

----------     -------

pip                 22.0.4

setuptools    61.0.0

wheel            0.37.1

# existing release from pypi.org fails

$ pip install gdal==3.2.2

Collecting gdal==3.2.2

  Downloading GDAL-3.2.2.tar.gz (601 kB)

     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 601.7/601.7 KB 8.0 MB/s eta
0:00:00

  Preparing metadata (setup.py) ... error

  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.

  │ exit code: 1

  ╰─> [6 lines of output]

      WARNING: numpy not available!  Array support will not be enabled

      /usr/local/lib/python3.7/site-packages/setuptools/_distutils/dist.py:275:
UserWarning: Unknown distribution option: 'use_2to3_fixers'

        warnings.warn(msg)

      /usr/local/lib/python3.7/site-packages/setuptools/_distutils/dist.py:275:
UserWarning: Unknown distribution option: 'use_2to3_exclude_fixers'

        warnings.warn(msg)

      error in GDAL setup command: use_2to3 is invalid.

      [end of output]

  note: This error originates from a subprocess, and is likely not a
problem with pip.

error: metadata-generation-failed

× Encountered error while generating package metadata.

╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.

hint: See above for details.

# installing newer one from test.pypi.org works fine!

$ pip install -i https://test.pypi.org/simple/ GDAL==3.2.2.1

Looking in indexes: https://test.pypi.org/simple/

Collecting GDAL==3.2.2.1

  Downloading
https://test-files.pythonhosted.org/packages/b7/02/4356dbdabf8f4fa898fe2378c8c096df3da254508ec8bc0663920cf4b92d/GDAL-3.2.2.1.tar.gz
(600 kB)

     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 600.9/600.9 KB 8.2 MB/s eta
0:00:00

  Preparing metadata (setup.py) ... done

Building wheels for collected packages: GDAL

  Building wheel for GDAL (setup.py) ... done

  Created wheel for GDAL: filename=GDAL-3.2.2.1-cp37-cp37m-linux_x86_64.whl
size=816416
sha256=0209dc3c56cd626137c53187c4a8fb7758587c9030cf8d479d8a2c7864fd5c77

  Stored in directory:
/root/.cache/pip/wheels/4b/f2/02/0ca533c034c5cc5840d619cd7004f32d3362c04a1c1500d0a1

Successfully built GDAL

Installing collected packages: GDAL

Successfully installed GDAL-3.2.2.1

WARNING: Running pip as the 'root' user can result in broken permissions
and conflicting behaviour with the system package manager. It is
recommended to use a virtual environment instead:
https://pip.pypa.io/warnings/venv

```

I hope this gives us some confidence that we are on a right track! :)

If I should test something else, please let me know! Looking forward for
the release on the pypi.org.

Thank you again!!

Regards,

Snehal

On Fri, Mar 25, 2022 at 1:45 PM Even Rouault <even.rouault at spatialys.com>
wrote:

> ok, I missed this was on the pypi package.
>
> I've tried to make a patch over 3.2.2. For now, I've uploaded it only to
> https://test.pypi.org/project/GDAL/3.2.2.1/  . Can you test that and
> confirm that works properly ? If so, I'll push it to pypi official.
>
> I've tracked the changes in a patch/3.2.2.1 branch, sitting for now in my
> fork:
>
>
> https://github.com/OSGeo/gdal/compare/v3.2.2..rouault:patch/3.2.2.1?expand=1
>
> I'll push that to OSGeo/gdal as well and tag once confirmed things work
> fine
>
> Hoping I'm not going into troubles doing this stuff outside of our usual
> release processes...
>
> Even
> Le 25/03/2022 à 13:13, snehal waychal a écrit :
>
> Dear Even, dear Bas,
>
>
> Thank you very much for the quick response and highlighting the
> Debian/Ubuntu release policy aspects. And also about the link to the
> ubuntugis-unstable PPA.
>
>
> >* what you discuss here is all about the patch & backport policy of the *
>
> >* Debian GDAL package. You can try to file a bug to Debian and point to *
>
> >* the patch you'd want to see backported,*
>
>
> But just to be sure I haven’t misunderstood your response or
> misrepresented the original issue. The patch release I described is also
> needed in the official releases of the GDAL **python** package. If I am
> not mistaken, the GDAL team is taking care of source distributions on
> https://pypi.org/project/GDAL/. I looked into released pypi versions and
> if I am not mistaken there is no patch release for v3.2.2 with fix for
> setuptools compatibility issue.
>
>
> So, if we could make a new patch release of the* *python* package of GDAL*
> and push the sdist package to pypi.org (something like 3.2.2.1, as I
> described in the previous email) then that would also solve the issue.
> Because as python developers, we install *python* packages via
> pip/pipenv/poetry and those package managers would pick the new patch
> release from pypi.org. (The GDAL library component libgdal-dev v3.2.2
> will still come from the debian system package repo and there is no issue
> with that).
>
>
> Hope I am not missing something obvious here. Please let me know.
>
>
> Thank you again!
>
>
> Regards,
>
> Snehal
>
> On Fri, Mar 25, 2022 at 12:29 PM Sebastiaan Couwenberg <sebastic at xs4all.nl>
> wrote:
>
>> On 3/25/22 12:07, Even Rouault wrote:
>> > what you discuss here is all about the patch & backport policy of the
>> > Debian GDAL package. You can try to file a bug to Debian and point to
>> > the patch you'd want to see backported, but I can't promise if there
>> > would be interest in their maintenance team to create an updated
>> package
>> > with it (my understanding is that even if we'd release a new 3.2.x
>> patch
>> > release, it wouldn't be packaged in LTS distributions. I'm not sure how
>> > much of that is linked to Debian policy or availability of people that
>> > do the work)
>>
>> Packages in Debian stable releases only get updates to fix bugs of
>> severity important or higher [0]. GDAL patch releases also contain
>> changes for lower severity issues, it's not worth the effort to vet all
>> those changes. Any changes to packages in stable also risk introducing
>> regressions which are highly undesirable in LTS releases known for their
>> stability.
>>
>> People should be maintaining their own packaging repositories where they
>> host packages with changes for their needs that cannot be easily
>> upstreamed to the package in the distribution itself. Scratching your
>> own itch was a corner stone of Open Source that people are seemingly
>> forgetting or never having known about in the first place.
>>
>> [0]
>>
>> https://www.debian.org/doc/manuals/developers-reference/pkgs.en.html#special-case-uploads-to-the-stable-and-oldstable-distributions
>>
>> Kind Regards,
>>
>> Bas
>>
>> --
>>   GPG Key ID: 4096R/6750F10AE88D4AF1
>> Fingerprint: 8182 DE41 7056 408D 6146  50D1 6750 F10A E88D 4AF1
>> _______________________________________________
>> gdal-dev mailing list
>> gdal-dev at lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>>
> -- http://www.spatialys.com
> My software is free, but my time generally not.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220325/6a57b246/attachment-0001.html>