[gdal-dev] Requiring numpy for the Python bindings

[Laurențiu Nicola]

Hi Greg,

I'd like to better understand your concerns, so I have a couple of questions and remarks:

1. do the platforms you care about package Firefox, librsvg, or any other popular software that's using Rust?
2. do you have any reasons to believe that numpy will require Rust in the future? I skimmed the existing NEPs, including the 2.0 roadmap, and there's no mention of Rust, so it's unlikely to be on the plate for the next, say, 5 years.
3. if numpy ever requires Rust, do you expect that GDAL will be unable to support both the Rust-enabled numpy, and a previous version at the same time?
4. if numpy ever requires Rust, do you expect that the platforms you care about will stop packaging and/or updating it?
5. according to Debian's popcon, numpy has about 10x more users than GDAL [1] [2], so the packagers will be under a lot of pressure to support it anyway
6. if you're more worried about the availability of an up-to-date Rust toolchain, note that a lot of "foundational" Rust libraries have relatively conservative toolchain requirements. For example, the numpy crate (unrelated to the Python package) supports on Rust 1.48, which is from November 2020 [3].

Laurentiu

[1] https://qa.debian.org/popcon.php?package=numpy
[2] https://qa.debian.org/popcon.php?package=gdal
[3] for comparison, NetBSD 9 x86_64 has Rust 1.69 or 1.70

On Tue, Dec 5, 2023, at 03:13, Greg Troxel via gdal-dev wrote:
> Even Rouault via gdal-dev <gdal-dev at lists.osgeo.org> writes:
>
>> The current situation where numpy is an optional dependency of the
>> GDAL Python bindings is quite cumbersome to deal with our setup.py's
>> setuptools . All details (a bit tricky) in
>> https://github.com/OSGeo/gdal/issues/8069 . It seems it would be
>> simpler if the bindings just required numpy, which is the confugration
>> most people using the bindings likely actually end up using anyway.
>
> That seems ok, but I think it's important to realize how heavy it is.
> numpy needs a Fortran compiler and a bunch of libs.  On NetBSD 9, that
> leads to:
>
>   Requires:
>   gcc10>=10.3.0nb2
>   python311>=3.11.0
>   lapack>=3.9.0nb1
>   cblas>=3.9.1nb1
>   lapacke>=3.9.1nb2
>
> which isn't bad compared to qgis, but it seems heavy compared to gdal.
> Still, it's just build time, and none of those are things that don't
> build.  Also, recent numpy explicitly requires gcc8, which isn't a real
> problem, but it's newer than many LTS systems will have.
>
> The big point is that numpy doesn't need rust, which seriously impairs
> portability because there's a singleton compiler available for limited
> platforms with a very difficult/recent self-hosting story.  Building the
> rust compiler requires the *immediately preceding* compiler, which seems
> unprecented, and I don't understand how people can think that's ok.  So
> I think this is ok only if gdal people have understood the numpy
> commmunity and think they would refrain from depending on rust.
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev