[QGIS-Developer] Proposal: changing the behaviour of QgsFeatureRequest

Tue Apr 30 16:57:40 PDT 2019

,Greetings,
I'm going over iterators for the prototype and it would appear that the OGR
iterator uses by default the filtered fids (
https://github.com/qgis/QGIS/blob/master/src/providers/ogr/qgsogrfeatureiterator.cpp#L48
)

Though I'm not too sure about how this interacts with expression, maybe
this is done in the core of the iterator, but nonetheless part of the work
seem to already be done to some extent.

Alexis Roy-Lizotte

Le mar. 30 avr. 2019 à 08:28, Matthias Kuhn <matthias at opengis.ch> a écrit :

> Sorry, I lost you here.
>
> I think for both proposals three QgsFeatureRequests will need to be
> created, requested and iterators generated.
>
> Can you paste a minimal code sample here that demonstrates the use of your
> proposed API?
>
> Thanks Matthias
>
> On 4/30/19 1:00 PM, Alexis R.L. wrote:
>
> Hi,
>
> Well I could rework iterators to use the provided Fids as the base list to
> iterate over, this would mean that instead of checking every feature for
> each evaluation ( from what I'm seeing each evaluation goes through every
> feature and check for either the id number or if the expression is true.)
>
> If we are doing 3 evaluation on 3 distinct symbols, this mean that every
> feature will be iterated over 3 times. Because each features are checked
> one by one for each of the 3 expression to evaluate.
>
> By using Fids as the base list and incrementing the index of it, each
> feature should be evaluated only once and no Fids check would be necessary.
> (Assuming a simple legend with a depth of one.)
>
> This would drasticly speed up things when many loops are required on a
> single feature.
>
> Alexis Roy-Lizotte
>
>
> Le mar. 30 avr. 2019 à 06:30, Matthias Kuhn <matthias at opengis.ch> a
> écrit :
>
>> Hi,
>>
>> An expression `attribute = 'major_road' AND $id IN (1,2,3)` should have
>> the same result on the number of features in the iterator. Or I am missing
>> something.
>>
>> Matthias
>> On 4/30/19 12:13 PM, Alexis R.L. wrote:
>>
>> Greetings Mr.Kuhn,
>>
>> Well in my case the fact that each symbol node is evaluated add some
>> lenght to the computation, if I can make something that would only iterate
>> over a given list of Fids, (and at the same time prevent having to iterate
>> over each feature to check if that feature is within the given list of
>> feature) we would move from something that takes x by n loops to something
>> that would only take around x loop ( where x is the number of features and
>> n is the number of symbol).
>>
>> For other use case the gain might be minimal. Other similar behaviour
>> could be used with selected features (though I'm not sure if this is a
>> perfect match for this also.)
>>
>> This gain in efficiency is why I want to alter the iterator itself rather
>> than fiddling with expressions, despite the fact that this is also a
>> possibility but far less optimal for the reason mentioned above.
>>
>> I just want to know what would be the best way to alter the iteration
>> process without breaking and in a way that could be leveraged.
>>
>> Thanks.
>>
>> Alexis Roy-Lizotte
>>
>>
>> Le mar. 30 avr. 2019 à 02:31, Matthias Kuhn <matthias at opengis.ch> a
>> écrit :
>>
>>> Hi,
>>>
>>> I like the idea of a nicer API to combine several filter conditions.
>>> Currently the only possibility to perform this operation is to write a
>>> filter expression as a string or by combining expression nodes.
>>>
>>> I am not convinced however that this should be done directly on
>>> QgsFilterRequest level. It feels a bit like reimplementing the expression
>>> engine on another level again.
>>>
>>> Some questions that come to my mind:
>>>
>>>  * Are these filters always combined with AND or also OR?
>>>
>>>  * Can these filters eventually be nested?
>>>
>>>  * How should we deal with compiling expressions, should that be
>>> implemented again?
>>>
>>> An alternative approach would be to work on a nicer API for an
>>> expression builder to require 1) less string manipulation when building an
>>> expression and 2) have more introspection capabilities when manipulating an
>>> expression.
>>>
>>> Regards
>>>
>>> Matthias
>>> On 4/29/19 9:02 PM, Alexis R.L. wrote:
>>>
>>> Greetings Mr. Dawson,
>>>
>>> Thanks for the reply,
>>>
>>> The tests that I did with the method in my PR was provider agnostic but
>>> still entailed looping over every feature to do the check. Could you
>>> provide idea or methods that could enable this behaviour in an API friendly
>>> way?
>>>
>>> Should I make a new iterator entirely or should I modify existing
>>> methods and add a trigger like I tested in my PR?
>>>
>>> Thanks!
>>> Alexis Roy-Lizotte
>>>
>>>
>>> Le dim. 28 avr. 2019 à 19:46, Nyall Dawson <nyall.dawson at gmail.com> a
>>> écrit :
>>>
>>>> On Sun, 28 Apr 2019 at 07:18, Alexis R.L. <alroyliz0 at gmail.com> wrote:
>>>> >
>>>> > Greetings Everyone,
>>>> >
>>>> > While working on my PR (9648) I noticed that expression filter and a
>>>> list of feature IDs are both considered the same thing cannot be used as
>>>> two filtering criteria.
>>>> >
>>>> > I am aware that the intent was to only have one element active to
>>>> filter out features other than the rectangle(? might be wrong on that one).
>>>> >
>>>> > What I am wondering if it would be a good thing to have both co exist
>>>> (mostly for my PR as of now). A simply way to do so would be to use the
>>>> give feature ids as the list to iterate over and check for the expression
>>>> if there is one.
>>>>
>>>> I also think this would be desirable. The tricky part (well, one
>>>> tricky part -- feature iterators are low level stuff, lots of
>>>> complexity there!) is avoiding any API breakage while adding new
>>>> methods to handle your use case.
>>>>
>>>> The other (SUPER) tricky bit is that any change to feature requests
>>>> usually requires accompanying changes to every vector data provider in
>>>> order to keep consistent behaviour across all these providers. The
>>>> same request should always give the same result, regardless of the
>>>> actual underlying data provider. The consequence of this is that
>>>> you'll need to be update the postgres, mssql, oracle, OGR, memory,
>>>> spatialite, wfs, arcgis feature server, db2, and vector layer feature
>>>> iterator code. You'll also need to add many new unit tests to cover
>>>> all the changes to the different providers.
>>>>
>>>> Definitely not a trivial change! Don't let this put you off making it,
>>>> but just be aware that this change would take even a QGIS core
>>>> developer a week or more of work to implement.
>>>>
>>>> Nyall
>>>>
>>>>
>>>>
>>>> >
>>>> > I assume that such a thing might be implemented without touching the
>>>> current behaviour, otherwise one would have to remove either the expression
>>>> or the IDs in some case when we want to override the current filtering
>>>> method.
>>>> >
>>>> > Using multiple filtering method might be better than only forcing one
>>>> and would provide more flexibility.
>>>> >
>>>> > As I have a PR that is related to this (
>>>> https://github.com/qgis/QGIS/pull/9816 ) I would like to address this
>>>> point in it and would like to have the discussion occur there as to the
>>>> best way to implement such a thing without any breakage if people are not
>>>> opposed to having only one filtering method in qgsfeaturerequest.
>>>> >
>>>> > Thanks and have a nice day!
>>>> >
>>>> > Alex
>>>> > _______________________________________________
>>>> > QGIS-Developer mailing list
>>>> > QGIS-Developer at lists.osgeo.org
>>>> > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>>> > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>>>
>>>
>>> _______________________________________________
>>> QGIS-Developer mailing listQGIS-Developer at lists.osgeo.org
>>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>>
>>> --
>>> Matthias Kuhn
>>> matthias at opengis.ch
>>> +41 (0)76 435 67 63 <+41764356763>
>>> [image: OPENGIS.ch Logo] <http://www.opengis.ch>
>>>
>> --
>> Matthias Kuhn
>> matthias at opengis.ch
>> +41 (0)76 435 67 63 <+41764356763>
>> [image: OPENGIS.ch Logo] <http://www.opengis.ch>
>>
> --
> Matthias Kuhn
> matthias at opengis.ch
> +41 (0)76 435 67 63 <+41764356763>
> [image: OPENGIS.ch Logo] <http://www.opengis.ch>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20190430/7afe3a87/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 6671 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20190430/7afe3a87/attachment-0001.png>