[QGIS-Developer] using conda for development

Sun Aug 2 22:51:43 PDT 2020

Thanks, Nyall.  When you said:

And that's the big concern with approaches like conda. Past experience
with using homebrew for the mac builds suffered this very issue
constantly. It's only since the QGIS mac os build was seperated away
from the homebrew dependancy nightmare that we're seeing stability in
the macos builds... *

I thought you were implying that MacOS was a special case that we can't use
conda for (I'm not a Mac user, and don't have experience with homebrew).
In general, I think if we avoid following a "use the latest" policy, conda
is a stable tool for multi-platform builds.  The platform-specific problems
I ran into involved trying to use the latest and greatest.  Did you
experience problems trying conda, or just homebrew?

I guess from Matthias' comments, though, your multi-platform build
requirements are beyond the scope of what conda can deliver?
Ari

On Sun, Aug 2, 2020 at 5:28 PM Nyall Dawson <nyall.dawson at gmail.com> wrote:

> On Sun, 2 Aug 2020 at 16:25, Ari Meyer <ari.meyer at gmail.com> wrote:
> >
> > Nyall, is the issue here multi-platform builds?  I have experienced
> problems with conda where the Windows port of a native library was faulty
> (I distinctly remember netcdf4/xarray last year). It seemed like Windows
> was the ugly stepchild -- always neglected.  I think it will always be a
> problem for conda, as opposed to pip, due to the native libraries.  So I
> think it's fair to document problems with dependency versions on particular
> platforms and provide workable substitutions, but to trust the latest
> release seems really dangerous to me.
>
> Ari -- I think you misunderstood my argument. We're arguing for the
> same things, as I also agree that having external agencies who are
> completely unaware of the needs of QGIS dictate our dependencies is
> dangerous!
>
> I'm ok with this on the Linux side, because Linux distros have used
> this approach forever and have very robust structures in place for
> handling these. But in my experience conda and homebrew are really
> just "wild west" style proving grounds where anything goes and things
> are constantly breaking.
>
> Nyall
>
>
>
>  All of the discrepancies I noted in the above issue had differing
> minor or even major versions, implying API breakages:
> >
> > Dependency diffs versus the downloadable Windows desktop versions 3.14.0
> and 3.14.1:
> > a) gdal:
> > Windows desktop: 3.0.4
> > conda-forge: 3.1.1
> > b) qt:
> > Windows desktop: 5.11.2
> > conda-forge: 5.12.5
> > c) sqlite:
> > Windows desktop: 3.29.0
> > conda-forge: 3.32.3
> > d) postgresql:
> > Windows desktop: 11.5
> > conda-forge: 12.3
> > e) qwt:
> > Windows desktop: 6.3.2
> > conda-forge: 7.0.0
> > f) pyqt:
> > Windows desktop: 5.11.3
> > conda-forge: 5.12.3
> >
> > Given that, it would be interesting to know if one distribution was
> significantly more buggy than another.
> >
> > On Sun, Aug 2, 2020 at 12:51 AM Nyall Dawson <nyall.dawson at gmail.com>
> wrote:
> >>
> >> On Sun, 2 Aug 2020 at 11:02, Ari Meyer <ari.meyer at gmail.com> wrote:
> >> >
> >> > Matthias, WADR, your statement seems, at least on the surface, to
> indicate an incredibly fragile process.  How can you possibly maintain
> stability and reproducibility if the dependencies can be changed at will?
> >>
> >> And that's the big concern with approaches like conda. Past experience
> >> with using homebrew for the mac builds suffered this very issue
> >> constantly. It's only since the QGIS mac os build was seperated away
> >> from the homebrew dependancy nightmare that we're seeing stability in
> >> the macos builds... *
> >>
> >> Nyall
> >>
> >> * William's system packages use a different approach, so aren't
> >> subject to this comment!
> >>
> >> >
> >> > On Fri, Jul 31, 2020 at 2:27 AM Matthias Kuhn <matthias at opengis.ch>
> wrote:
> >> >>
> >> >> One thing to keep in mind is that even the main qgis
> installers/distributions do not have fixed dependency versions. Using
> downloaded macOS QGIS will have different dependencies than Windows and so
> will all the linux/*nix flavors.
> >> >>
> >> >> On 7/31/20 7:14 AM, Ari Meyer wrote:
> >> >>
> >> >> @Nyall, that proposal appears to be specific to plugins -- not sure
> if that's the best place to discuss this issue.
> >> >> @Greg: I found that many of the exact dependencies were not even
> available through the main channels.  So if the QGIS devs compile/build
> against such versions, I'm not sure there's an easy way to even specify
> such a conda recipe, unless all those dependencies are also made available
> somehow.  I didn't expect that those versions would not even be there with
> the others for the various libraries.  Could this imply that the library
> developers don't want those versions to be used?  Not sure.
> >> >>
> >> >> As a user and developer, I just want to be sure that if I pull down
> conda-forge qgis version X that I will get the same effective distribution
> as contained in downloadable installer version X.  Right now, we are
> getting a very different set of dependencies.
> >> >>
> >> >> Thanks,
> >> >> Ari
> >> >>
> >> >> On Thu, Jul 30, 2020 at 6:44 PM Greg Troxel <gdt at lexort.com> wrote:
> >> >>>
> >> >>> Nyall Dawson <nyall.dawson at gmail.com> writes:
> >> >>>
> >> >>> > There's a related proposal at
> >> >>> > https://github.com/qgis/QGIS-Enhancement-Proposals/issues/179 --
> >> >>> > probably best to keep the discussion on that page.
> >> >>>
> >> >>> Not really about the details of using conda but about the concept:
> >> >>>
> >> >>>   conda's documentation says it runs on "Windows, macOS and Linux"
> but
> >> >>>   qgis also runs on at least NetBSD and almost certainly other BSDs.
> >> >>>
> >> >>>   it seems that to build something, including qgis the requirement
> is
> >> >>>   that the things it depends on are present in some way that is
> found by
> >> >>>   the build.  People can choose to do that however they want, and it
> >> >>>   seems funny to me to impose a requirement.   If this is really
> "let's
> >> >>>   publish a conda config file so that people who want to do that
> can do
> >> >>>   less work", that's of course fine, but if it's more "if you don't
> have
> >> >>>   conda then you can't build qgis", that's something else.
> >> >>
> >> >>
> >> >> _______________________________________________
> >> >> QGIS-Developer mailing list
> >> >> QGIS-Developer at lists.osgeo.org
> >> >> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> >> >> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> >> >
> >> > _______________________________________________
> >> > QGIS-Developer mailing list
> >> > QGIS-Developer at lists.osgeo.org
> >> > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> >> > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20200803/4a0d35f3/attachment.html>