[Qgis-developer] Processing algorithms and API breaks

Tue Mar 8 13:00:42 PST 2016

Hi

> On 08 Mar 2016, at 09:16, Alexander Bruy <alexander.bruy at gmail.com> wrote:
> 
> Hi Tim, Nyall,
> 
> I understand your points. While new features in most cases can wait
> for next major version, do we really want to miss bugfixes, especially
> for algorithms, which depends on 3rd party backends, e.g. SAGA or
> GRASS [0]? BTW, most PR with such fixes contributed by community
> members. I'm afraid if PR will be unmerged for a long time, we will
> distract users from further contributions.

I don’t want to miss bug fixes, but I’d also like to see an architecture in place that limits how bug fixes break existing workflows. In core we manage that by making small wrappers around new methods to keep old API methods working. If a bug fix breaks something else that used to work isn’t the net gain zero? If the bug fix fixes something and doesn’t break something else then the net gain is 1 (good!).

> 
> I don't know how many users create models and scripts, but in our
> QGIS-Processing repo there are only small number of them. Maybe
> this is because nobody wants to create scripts/models because
> of unstable Processing API. But also this is can be an indicator that
> very limited number of users create scripts/models.

I think think a more likely hypothesis is that most models are probably highly specific to individual peoples workflows and not good candidates for general sharing.

Just to clarify my previous posting, I am not against doing some big overhaul to improve the situation but I would prefer to see that it :

a) happens with 3.0 when we are breaking a bunch of stuff anyway and
b) moves in a direction where the underlying architecture is able to provide a longer term stability of algorithm interfaces so that we can provide our users with some assurance that the work they do today won’t be undone tomorrow.

If the backend provider changes are causing our processing stuff to break, surely having some middleware in place to translate new backend interfaces to expose their inputs and results so that they function like the old interfaces on the user facing side of the algorithm should give us a way to deal with that?

Thanks for all the great work you guys are doing with processing, I don’t mean to detract from your efforts, only to lend a sense of user perspective to your planning.

Regards

Tim

> 
> [0] https://github.com/qgis/QGIS/pulls?utf8=%E2%9C%93&q=is%3Apr+is%3Aclosed+processing
> 
> 
> 
> 
> 2016-03-08 8:53 GMT+02:00 Paolo Cavallini <cavallini at faunalia.it>:
>> Hi Tim, Nyall,
>> while in principle I agree, I think this would be rather pointless in the
>> current situation.
>> Many algs break often anyway because of changes in the backends. My
>> suggestion:
>> " first put in place a complete set of tests
>> * only after that implement a versioning system
>> * provide clear instructions, possibly also tools, to help upgrading models
>> to new versions (only power users actually build models).
>> IMHO a current major stumbling block for users is the lack of documentation
>> for hundreds of algs, many of them quite obscure. Better address this first.
>> All the best.
>> 
>> Il 7 marzo 2016 22:37:39 CET, Nyall Dawson <nyall.dawson at gmail.com> ha
>> scritto:
>>> 
>>> On 7 March 2016 at 19:13, Alexander Bruy <alexander.bruy at gmail.com> wrote:
>>> 
>>>> 1. Postpone such changes for next major QGIS release when API break is
>>>> allowed. In this case all existing scripts/models will work. But we
>>>> should live
>>>> with bugs and without some features, even if fix already available and
>>>> keep the
>>>> code of this fixes in sync with master until next major release.
>>> 
>>> 
>>> I honestly think 1 is the only valid approach here.
>>> 
>>> I'll preface this opinion by saying that I HATE frozen API with a
>>> passion. It makes development harder, slower and more expensive, and
>>> prevents new features landing and bugs from being fixed [1]. With my
>>> totally selfish developer hat on I'd love nothing more than for us to
>>> adopt a non-frozen, do-whatever-the-heck-you-want API and with no
>>> care
>>> for compatibility with old projects ;)
>>> 
>>> BUT
>>> 
>>> The decision has been made for the project that QGIS has a frozen API
>>> between major releases, and now that processing is part of the core
>>> QGIS install it also needs to abide by that rule! Yes, it's a freaking
>>> horrible PITA, but the alternative (breaking user's scripts and
>>> models) is far worse for the project. And with my non-developer, QGIS
>>> user hat on I would hate for scripts and models which I've invested
>>> time into creating to get broken between releases.
>>> 
>>> I'd suggest following the same approach as is taken by the c++ code:
>>> make a versioned copy of the algorithm so that existing scripts/models
>>> can still use the old version BUT it's not exposed anywhere in the GUI
>>> and new models will transparently just use the new version. That's
>>> what we've done for labelling (heck... even the old label engine is
>>> STILL in QGIS on that extremely rare case that
>>> someone opens a pre 2.0
>>> project!), symbology, various composer items, etc. It results in
>>> duplicate code and fills the codebase with a lot of deprecated cruft,
>>> but that's the price of a frozen API :)
>>> 
>>> Just my 2c! ;)
>>> Nyall
>>> 
>>> 1. see https://github.com/qgis/qgis3.0_api/issues for a list of stuff
>>> we CAN'T implement/fix until API is unfrozen
>>> ________________________________
>>> 
>>> Qgis-developer mailing list
>>> Qgis-developer at lists.osgeo.org
>>> List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
>>> Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
>> 
>> 
>> --
>> Paolo Cavallini
>> www.faunalia.eu
> 
> 
> 
> -- 
> Alexander Bruy
> _______________________________________________
> Qgis-developer mailing list
> Qgis-developer at lists.osgeo.org
> List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

Tim Sutton
QGIS Project Steering Committee Member
tim at qgis.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20160308/26fb6271/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PastedGraphic-1.tiff
Type: image/tiff
Size: 9882 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20160308/26fb6271/attachment-0001.tiff>