[QGIS-Developer] using conda for development

[Nyall Dawson]

On Sun, 2 Aug 2020 at 16:25, Ari Meyer <ari.meyer at gmail.com> wrote:
>
> Nyall, is the issue here multi-platform builds?  I have experienced problems with conda where the Windows port of a native library was faulty (I distinctly remember netcdf4/xarray last year). It seemed like Windows was the ugly stepchild -- always neglected.  I think it will always be a problem for conda, as opposed to pip, due to the native libraries.  So I think it's fair to document problems with dependency versions on particular platforms and provide workable substitutions, but to trust the latest release seems really dangerous to me.

Ari -- I think you misunderstood my argument. We're arguing for the
same things, as I also agree that having external agencies who are
completely unaware of the needs of QGIS dictate our dependencies is
dangerous!

I'm ok with this on the Linux side, because Linux distros have used
this approach forever and have very robust structures in place for
handling these. But in my experience conda and homebrew are really
just "wild west" style proving grounds where anything goes and things
are constantly breaking.

Nyall



 All of the discrepancies I noted in the above issue had differing
minor or even major versions, implying API breakages:
>
> Dependency diffs versus the downloadable Windows desktop versions 3.14.0 and 3.14.1:
> a) gdal:
> Windows desktop: 3.0.4
> conda-forge: 3.1.1
> b) qt:
> Windows desktop: 5.11.2
> conda-forge: 5.12.5
> c) sqlite:
> Windows desktop: 3.29.0
> conda-forge: 3.32.3
> d) postgresql:
> Windows desktop: 11.5
> conda-forge: 12.3
> e) qwt:
> Windows desktop: 6.3.2
> conda-forge: 7.0.0
> f) pyqt:
> Windows desktop: 5.11.3
> conda-forge: 5.12.3
>
> Given that, it would be interesting to know if one distribution was significantly more buggy than another.
>
> On Sun, Aug 2, 2020 at 12:51 AM Nyall Dawson <nyall.dawson at gmail.com> wrote:
>>
>> On Sun, 2 Aug 2020 at 11:02, Ari Meyer <ari.meyer at gmail.com> wrote:
>> >
>> > Matthias, WADR, your statement seems, at least on the surface, to indicate an incredibly fragile process.  How can you possibly maintain stability and reproducibility if the dependencies can be changed at will?
>>
>> And that's the big concern with approaches like conda. Past experience
>> with using homebrew for the mac builds suffered this very issue
>> constantly. It's only since the QGIS mac os build was seperated away
>> from the homebrew dependancy nightmare that we're seeing stability in
>> the macos builds... *
>>
>> Nyall
>>
>> * William's system packages use a different approach, so aren't
>> subject to this comment!
>>
>> >
>> > On Fri, Jul 31, 2020 at 2:27 AM Matthias Kuhn <matthias at opengis.ch> wrote:
>> >>
>> >> One thing to keep in mind is that even the main qgis installers/distributions do not have fixed dependency versions. Using downloaded macOS QGIS will have different dependencies than Windows and so will all the linux/*nix flavors.
>> >>
>> >> On 7/31/20 7:14 AM, Ari Meyer wrote:
>> >>
>> >> @Nyall, that proposal appears to be specific to plugins -- not sure if that's the best place to discuss this issue.
>> >> @Greg: I found that many of the exact dependencies were not even available through the main channels.  So if the QGIS devs compile/build against such versions, I'm not sure there's an easy way to even specify such a conda recipe, unless all those dependencies are also made available somehow.  I didn't expect that those versions would not even be there with the others for the various libraries.  Could this imply that the library developers don't want those versions to be used?  Not sure.
>> >>
>> >> As a user and developer, I just want to be sure that if I pull down conda-forge qgis version X that I will get the same effective distribution as contained in downloadable installer version X.  Right now, we are getting a very different set of dependencies.
>> >>
>> >> Thanks,
>> >> Ari
>> >>
>> >> On Thu, Jul 30, 2020 at 6:44 PM Greg Troxel <gdt at lexort.com> wrote:
>> >>>
>> >>> Nyall Dawson <nyall.dawson at gmail.com> writes:
>> >>>
>> >>> > There's a related proposal at
>> >>> > https://github.com/qgis/QGIS-Enhancement-Proposals/issues/179 --
>> >>> > probably best to keep the discussion on that page.
>> >>>
>> >>> Not really about the details of using conda but about the concept:
>> >>>
>> >>>   conda's documentation says it runs on "Windows, macOS and Linux" but
>> >>>   qgis also runs on at least NetBSD and almost certainly other BSDs.
>> >>>
>> >>>   it seems that to build something, including qgis the requirement is
>> >>>   that the things it depends on are present in some way that is found by
>> >>>   the build.  People can choose to do that however they want, and it
>> >>>   seems funny to me to impose a requirement.   If this is really "let's
>> >>>   publish a conda config file so that people who want to do that can do
>> >>>   less work", that's of course fine, but if it's more "if you don't have
>> >>>   conda then you can't build qgis", that's something else.
>> >>
>> >>
>> >> _______________________________________________
>> >> QGIS-Developer mailing list
>> >> QGIS-Developer at lists.osgeo.org
>> >> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> >> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> >
>> > _______________________________________________
>> > QGIS-Developer mailing list
>> > QGIS-Developer at lists.osgeo.org
>> > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer