[QGIS-Developer] Questions and problems around Python expression functions
Benjamin Jakimow
benjamin.jakimow at geo.hu-berlin.de
Thu Feb 3 07:14:00 PST 2022
Hi Johannes,
I like to answer at least one of your questions:
> - Does a qgsfunction really *always* get some `feature` (potentially
> made-up and invalid if none is available in the context, e.g. layout
> text item) and `parent`?
Yes, a qgsfunction always gets a feature. By default it's invalid, which
you can see when running the following code snippet:
from qgis.core import QgsExpressionFunction, QgsExpressionContext,
QgsExpression
class MyFunction(QgsExpressionFunction):
def __init__(self):
super().__init__('myfunction', -1, 'MyGroup', 'Help Text')
def func(self, values, context: QgsExpressionContext, parent, node):
print(context.feature().isValid())
return 42
# register function
func = MyFunction()
QgsExpression.registerFunction(func)
# use function
exp = QgsExpression('myfunction()')
context = QgsExpressionContext()
result = exp.evaluate(context)
print(f'Result {result}')
Greetings, Benjamin
On 2022-02-03 15:33, Johannes Kröger wrote:
> Hey guys!
>
> So all I wanted to do this morning was clarify the
> `referenced_columns` parameter for Python expression functions (#46161
> and #46162) and add another example function to the default template
> that uses a feature. But I ended up in some confusing and conflicting
> documentation and behaviour around them :D
>
> Docs and example updates would be something I would love to
> contribute. It's not really things I can package into issues or
> feature requests because I don't know how it is supposed to work and
> some of the magic is pretty magical so here we go. I wish we had a
> forum with proper formatting and stuff for sustainable discussions but
> well, copy to a markdown parser if you want:
>
> # qgsfunction functions without parameters that also do not use
> feature and parent
> QGIS always passes a feature (which can be an in-valid one if the
> expression is used e.g. in a context where there is no feature
> involved like
> https://gis.stackexchange.com/questions/393249/writing-custom-qgis-function-that-works-without-a-feature-given-as-input/)
> and the parent QgsExpression to expression functions. Often one might
> not need them in the function so why have them in the function
> signature.
>
> The code documentation suggests "hiding" them by using "`*args`",
> https://github.com/qgis/QGIS/blob/2d1aa68f0d044f2aced7ebeca8d2fa6b754ac970/python/core/additions/qgsfunction.py#L39-L44:
>
> So I wrote a function that needs no parameters at all and used `*args`.
>
> ```
> @qgsfunction()
> def something_independent(*args):
> # read some stuff from files
> # make some network requests
> # do something with time
> # or whatever
> return something
> ```
>
> Currently this results in an error because in
> https://github.com/qgis/QGIS/blob/2d1aa68f0d044f2aced7ebeca8d2fa6b754ac970/python/core/additions/qgsfunction.py
> inspect.getfullargspec(function).args is used (which ignores `*args`
> (->`varargs`) and `**kwargs` (->`varkw`) and is empty in the case of
> my function) and then the last argument is tried to be read from the
> empty list.
>
> Is a function like this supposed to be supported?
> I think it would be great if so and quite unintuitive if not.
>
> # How exactly is qgsfunction's args= argument supposed to work?
>
> With `args="auto"` (the default) everything works like expected, we
> can specify various arguments, then QGIS adds `feature` and `parent`
> and handles the optional `context` as well. Python does not get
> surprised with any mismatch between the function signature and what it
> receives.
>
> ```
> @qgsfunction(args="auto")
> def arghs(a, b, c, feature, parent):
> return f"{a}, {b}, {c}"
> ```
>
> `arghs(1, 2, 3)` -> "1, 2, 3"
>
> With `args=3` I expected to be able to use 3 custom arguments (in
> addition to `feature`, `parent`[, `context`].
>
> https://docs.qgis.org/3.22/en/docs/user_manual/expressions/expression.html#function-editor
> makes it sound like that.
>
> https://github.com/qgis/QGIS/blob/2d1aa68f0d044f2aced7ebeca8d2fa6b754ac970/python/core/additions/qgsfunction.py#L140
> says "`Number of parameters, set to 'auto' to accept a variable length
> of parameters.`"
>
> https://github.com/qgis/QGIS/blob/b1144173e4e5204b1fe546324c0a3498eec07053/src/gui/qgsexpressionbuilderwidget.cpp#L229-L231
> says "`Defines the number of arguments. With ``args = 'auto'`` the
> number of arguments will automatically be extracted from the
> signature. With ``args = -1``, any number of arguments are accepted.`"
>
> Sounds like what the docs say.
>
> ```
> @qgsfunction(args=3)
> def arghs(a, b, c, feature, parent):
> return f"{a}, {b}, {c}"
> ```
>
> `arghs(1, 2, 3)` -> `Eval Error: arghs() missing 2 required positional
> arguments: 'feature' and 'parent'`
>
> Hm, do I have to count `feature` and `parent` too?
>
> ```
> @qgsfunction(args=5)
> def arghs(a, b, c, feature, parent):
> return f"{a}, {b}, {c}"
> ```
>
> `arghs(1, 2, 3)` -> `Parser Errors: arghs function is called with
> wrong number of arguments. Expected 5 but got 3.`
>
> 😯 what is going on?!
>
> ```
> @qgsfunction(args=3)
> def arghs(a, b, c, *args):
> return f"{a}, {b}, {c}"
> ```
>
> `arghs(1, 2, 3)` -> `'[1, 2, 3], <qgis._core.QgsFeature object at
> 0x7f1cda4b5120>,…'`
>
> ```
> @qgsfunction(args=3)
> def arghs(a, b, c): # no *args
> return f"{a}, {b}, {c}"
> ```
>
> `arghs(1, 2, 3)` -> `'[1, 2, 3], <qgis._core.QgsFeature object at
> 0x7f1cda4b5120>,…'`
>
> ```
> @qgsfunction(args=-1) # -1
> def arghs(a, b, c):
> return f"{a}, {b}, {c}"
> ```
>
> `arghs(1, 2, 3)` -> `'[1, 2, 3], <qgis._core.QgsFeature object at
> 0x7f1cda4b5120>,…'`
>
> `arghs(1,2,3,4,5,6,7,8,9,0)` -> `'[1, 2, 3, 4, 5, 6, 7, 8, 9, 0],
> <qgis._core.QgsFeature objec…'`
>
> 🤨 wat
>
> So does `args=n` actually mean "put the n first parameter values
> before the last two (`feature` and `parent`) into a list"?
>
> # Handling and necessity of the referenced_columns parameter
> ## Docs vs code conflict
> https://github.com/qgis/QGIS/blob/2d1aa68f0d044f2aced7ebeca8d2fa6b754ac970/python/core/additions/qgsfunction.py#L146-L147
> and thus the online docs say `referenced_columns` is empty by default.
>
> https://github.com/qgis/QGIS/blob/2d1aa68f0d044f2aced7ebeca8d2fa6b754ac970/python/core/additions/qgsfunction.py#L28-L29
> however sets `referenced_columns=[QgsFeatureRequest.ALL_ATTRIBUTES]`
> as default.
>
> `QgsFeatureRequest.ALL_ATTRIBUTES` sounds reasonable as default
> (things will work, user *can* tweak if needed) **however** I wonder if
> this led to issues with expression functions being used in contexts
> where there are no features involved, e.g. layout text items.
>
> ## ... leading to a hacky workaround?
> PR #42745 was a change of the default example function template to
> include an empty list and references a case of a user at
> https://gis.stackexchange.com/questions/393249/writing-custom-qgis-function-that-works-without-a-feature-given-as-input
>
> As a user I would think "I am not doing anything with features, why
> would I need to care about any columns?" so I wonder if this is
> something that could/should be handled differently?
>
> ## Hidden Python errors
> Last but not least
> https://github.com/qgis/QGIS/blob/32c2cea54cb92bbb2243b222816c8154c2b9adf9/src/gui/qgsexpressionpreviewwidget.cpp#L102-L113
> hides actual errors if the expression happens to run in a context
> where those checks are true. Import error or referencing a variable
> that does not exist in your function? User gets the `"No feature was
> found on this layer to evaluate the expression."` message although
> that would be a less pressing issue if relevant at all ;)
>
> # This and that
> - Does a qgsfunction really *always* get some `feature` (potentially
> made-up and invalid if none is available in the context, e.g. layout
> text item) and `parent`?
> -
> https://github.com/qgis/QGIS/blob/2d1aa68f0d044f2aced7ebeca8d2fa6b754ac970/python/core/additions/qgsfunction.py#L50-L55
> is missing descriptions of the parameters.
> -
> https://github.com/qgis/QGIS/blob/2d1aa68f0d044f2aced7ebeca8d2fa6b754ac970/python/core/additions/qgsfunction.py#L28
> sets `usesgeometry=False` which also actually used and is what the
> docs say but
> https://github.com/qgis/QGIS/blob/2d1aa68f0d044f2aced7ebeca8d2fa6b754ac970/python/core/additions/qgsfunction.py#L60
> uses `usesGeometry=True` which might lead to confusion.
> - Similarly
> https://github.com/qgis/QGIS/blob/2d1aa68f0d044f2aced7ebeca8d2fa6b754ac970/python/core/additions/qgsfunction.py#L29
> uses `referenced_columns=[QgsFeatureRequest.ALL_ATTRIBUTES]` by
> default (note the "listification"!) but
> https://github.com/qgis/QGIS/blob/2d1aa68f0d044f2aced7ebeca8d2fa6b754ac970/python/core/additions/qgsfunction.py#L61
> uses `referencedColumns=QgsFeatureRequest.ALL_ATTRIBUTES`.
> https://qgis.org/api/classQgsFeatureRequest.html#a717fcc8fa42a78f0b4f30253ca1b478e
> says it is a "A special attribute that if set matches all attributes."
> so it should be correct inside the list.
> -
> https://github.com/qgis/QGIS/blob/2d1aa68f0d044f2aced7ebeca8d2fa6b754ac970/python/core/additions/qgsfunction.py#L154
> and
> https://github.com/qgis/QGIS/blob/2d1aa68f0d044f2aced7ebeca8d2fa6b754ac970/python/core/additions/qgsfunction.py#L164
> have some colons behind the decorator lines, those should be removed
> - If you made it this far, enjoy the classic
> https://www.albinoblacksheep.com/flash/llama
>
> --
> Johannes Kröger / GIS-Entwickler/-Berater
> WhereGroup GmbH
> Grevenweg 89
> 20537 Hamburg
> Germany
>
> Tel: +49 (0)228 / 90 90 38 - 36
> Fax: +49 (0)228 / 90 90 38 - 11
>
> johannes.kroeger at wheregroup.com
> www.wheregroup.com
> Geschäftsführer:
> Olaf Knopp, Peter Stamm
> Amtsgericht Bonn, HRB 9885
> -------------------------------
>
> ---------------------------------------------
> Schon gewusst?
> In unserem Blog geben wir Tipps & Tricks zu
Open-Source-GIS-Software
> und berichten aus unserem Experten-Alltag:
> https://wheregroup.com/blog/
> ---------------------------------------------
>
>
> _______________________________________________
> QGIS-Developer mailing list
> QGIS-Developer at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
--
--
Benjamin Jakimow, Doctoral Researcher
Earth Observation Lab | Geography Department | Humboldt-Universität zu
Berlin
e-mail: benjamin.jakimow at geo.hu-berlin.de
phone: +49 (0) 30 2093 6894
mobile: +49 (0) 157 5656 8477
fax: +49 (0) 30 2093 6848
mail: Unter den Linden 6 | 10099 Berlin | Germany
room: 2'222
More information about the QGIS-Developer
mailing list