[QGIS-Developer] Questions and problems around Python expression functions
Johannes Kröger (WhereGroup)
johannes.kroeger at wheregroup.com
Thu Feb 3 06:33:12 PST 2022
Hey guys!
So all I wanted to do this morning was clarify the `referenced_columns`
parameter for Python expression functions (#46161 and #46162) and add
another example function to the default template that uses a feature.
But I ended up in some confusing and conflicting documentation and
behaviour around them :D
Docs and example updates would be something I would love to contribute.
It's not really things I can package into issues or feature requests
because I don't know how it is supposed to work and some of the magic is
pretty magical so here we go. I wish we had a forum with proper
formatting and stuff for sustainable discussions but well, copy to a
markdown parser if you want:
# qgsfunction functions without parameters that also do not use feature
and parent
QGIS always passes a feature (which can be an in-valid one if the
expression is used e.g. in a context where there is no feature involved
like
https://gis.stackexchange.com/questions/393249/writing-custom-qgis-function-that-works-without-a-feature-given-as-input/)
and the parent QgsExpression to expression functions. Often one might
not need them in the function so why have them in the function signature.
The code documentation suggests "hiding" them by using "`*args`",
https://github.com/qgis/QGIS/blob/2d1aa68f0d044f2aced7ebeca8d2fa6b754ac970/python/core/additions/qgsfunction.py#L39-L44:
So I wrote a function that needs no parameters at all and used `*args`.
```
@qgsfunction()
def something_independent(*args):
# read some stuff from files
# make some network requests
# do something with time
# or whatever
return something
```
Currently this results in an error because in
https://github.com/qgis/QGIS/blob/2d1aa68f0d044f2aced7ebeca8d2fa6b754ac970/python/core/additions/qgsfunction.py
inspect.getfullargspec(function).args is used (which ignores `*args`
(->`varargs`) and `**kwargs` (->`varkw`) and is empty in the case of my
function) and then the last argument is tried to be read from the empty
list.
Is a function like this supposed to be supported?
I think it would be great if so and quite unintuitive if not.
# How exactly is qgsfunction's args= argument supposed to work?
With `args="auto"` (the default) everything works like expected, we can
specify various arguments, then QGIS adds `feature` and `parent` and
handles the optional `context` as well. Python does not get surprised
with any mismatch between the function signature and what it receives.
```
@qgsfunction(args="auto")
def arghs(a, b, c, feature, parent):
return f"{a}, {b}, {c}"
```
`arghs(1, 2, 3)` -> "1, 2, 3"
With `args=3` I expected to be able to use 3 custom arguments (in
addition to `feature`, `parent`[, `context`].
https://docs.qgis.org/3.22/en/docs/user_manual/expressions/expression.html#function-editor
makes it sound like that.
https://github.com/qgis/QGIS/blob/2d1aa68f0d044f2aced7ebeca8d2fa6b754ac970/python/core/additions/qgsfunction.py#L140
says "`Number of parameters, set to 'auto' to accept a variable length
of parameters.`"
https://github.com/qgis/QGIS/blob/b1144173e4e5204b1fe546324c0a3498eec07053/src/gui/qgsexpressionbuilderwidget.cpp#L229-L231
says "`Defines the number of arguments. With ``args = 'auto'`` the
number of arguments will automatically be extracted from the signature.
With ``args = -1``, any number of arguments are accepted.`"
Sounds like what the docs say.
```
@qgsfunction(args=3)
def arghs(a, b, c, feature, parent):
return f"{a}, {b}, {c}"
```
`arghs(1, 2, 3)` -> `Eval Error: arghs() missing 2 required positional
arguments: 'feature' and 'parent'`
Hm, do I have to count `feature` and `parent` too?
```
@qgsfunction(args=5)
def arghs(a, b, c, feature, parent):
return f"{a}, {b}, {c}"
```
`arghs(1, 2, 3)` -> `Parser Errors: arghs function is called with wrong
number of arguments. Expected 5 but got 3.`
😯 what is going on?!
```
@qgsfunction(args=3)
def arghs(a, b, c, *args):
return f"{a}, {b}, {c}"
```
`arghs(1, 2, 3)` -> `'[1, 2, 3], <qgis._core.QgsFeature object at
0x7f1cda4b5120>,…'`
```
@qgsfunction(args=3)
def arghs(a, b, c): # no *args
return f"{a}, {b}, {c}"
```
`arghs(1, 2, 3)` -> `'[1, 2, 3], <qgis._core.QgsFeature object at
0x7f1cda4b5120>,…'`
```
@qgsfunction(args=-1) # -1
def arghs(a, b, c):
return f"{a}, {b}, {c}"
```
`arghs(1, 2, 3)` -> `'[1, 2, 3], <qgis._core.QgsFeature object at
0x7f1cda4b5120>,…'`
`arghs(1,2,3,4,5,6,7,8,9,0)` -> `'[1, 2, 3, 4, 5, 6, 7, 8, 9, 0],
<qgis._core.QgsFeature objec…'`
🤨 wat
So does `args=n` actually mean "put the n first parameter values before
the last two (`feature` and `parent`) into a list"?
# Handling and necessity of the referenced_columns parameter
## Docs vs code conflict
https://github.com/qgis/QGIS/blob/2d1aa68f0d044f2aced7ebeca8d2fa6b754ac970/python/core/additions/qgsfunction.py#L146-L147
and thus the online docs say `referenced_columns` is empty by default.
https://github.com/qgis/QGIS/blob/2d1aa68f0d044f2aced7ebeca8d2fa6b754ac970/python/core/additions/qgsfunction.py#L28-L29
however sets `referenced_columns=[QgsFeatureRequest.ALL_ATTRIBUTES]` as
default.
`QgsFeatureRequest.ALL_ATTRIBUTES` sounds reasonable as default (things
will work, user *can* tweak if needed) **however** I wonder if this led
to issues with expression functions being used in contexts where there
are no features involved, e.g. layout text items.
## ... leading to a hacky workaround?
PR #42745 was a change of the default example function template to
include an empty list and references a case of a user at
https://gis.stackexchange.com/questions/393249/writing-custom-qgis-function-that-works-without-a-feature-given-as-input
As a user I would think "I am not doing anything with features, why
would I need to care about any columns?" so I wonder if this is
something that could/should be handled differently?
## Hidden Python errors
Last but not least
https://github.com/qgis/QGIS/blob/32c2cea54cb92bbb2243b222816c8154c2b9adf9/src/gui/qgsexpressionpreviewwidget.cpp#L102-L113
hides actual errors if the expression happens to run in a context where
those checks are true. Import error or referencing a variable that does
not exist in your function? User gets the `"No feature was found on this
layer to evaluate the expression."` message although that would be a
less pressing issue if relevant at all ;)
# This and that
- Does a qgsfunction really *always* get some `feature` (potentially
made-up and invalid if none is available in the context, e.g. layout
text item) and `parent`?
-
https://github.com/qgis/QGIS/blob/2d1aa68f0d044f2aced7ebeca8d2fa6b754ac970/python/core/additions/qgsfunction.py#L50-L55
is missing descriptions of the parameters.
-
https://github.com/qgis/QGIS/blob/2d1aa68f0d044f2aced7ebeca8d2fa6b754ac970/python/core/additions/qgsfunction.py#L28
sets `usesgeometry=False` which also actually used and is what the docs
say but
https://github.com/qgis/QGIS/blob/2d1aa68f0d044f2aced7ebeca8d2fa6b754ac970/python/core/additions/qgsfunction.py#L60
uses `usesGeometry=True` which might lead to confusion.
- Similarly
https://github.com/qgis/QGIS/blob/2d1aa68f0d044f2aced7ebeca8d2fa6b754ac970/python/core/additions/qgsfunction.py#L29
uses `referenced_columns=[QgsFeatureRequest.ALL_ATTRIBUTES]` by default
(note the "listification"!) but
https://github.com/qgis/QGIS/blob/2d1aa68f0d044f2aced7ebeca8d2fa6b754ac970/python/core/additions/qgsfunction.py#L61
uses `referencedColumns=QgsFeatureRequest.ALL_ATTRIBUTES`.
https://qgis.org/api/classQgsFeatureRequest.html#a717fcc8fa42a78f0b4f30253ca1b478e
says it is a "A special attribute that if set matches all attributes."
so it should be correct inside the list.
-
https://github.com/qgis/QGIS/blob/2d1aa68f0d044f2aced7ebeca8d2fa6b754ac970/python/core/additions/qgsfunction.py#L154
and
https://github.com/qgis/QGIS/blob/2d1aa68f0d044f2aced7ebeca8d2fa6b754ac970/python/core/additions/qgsfunction.py#L164
have some colons behind the decorator lines, those should be removed
- If you made it this far, enjoy the classic
https://www.albinoblacksheep.com/flash/llama
--
Johannes Kröger / GIS-Entwickler/-Berater
WhereGroup GmbH
Grevenweg 89
20537 Hamburg
Germany
Tel: +49 (0)228 / 90 90 38 - 36
Fax: +49 (0)228 / 90 90 38 - 11
johannes.kroeger at wheregroup.com
www.wheregroup.com
Geschäftsführer:
Olaf Knopp, Peter Stamm
Amtsgericht Bonn, HRB 9885
-------------------------------
---------------------------------------------
Schon gewusst?
In unserem Blog geben wir Tipps & Tricks zu
Open-Source-GIS-Software
und berichten aus unserem Experten-Alltag:
https://wheregroup.com/blog/
---------------------------------------------
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_0xBF7B268A77C202D5.asc
Type: application/pgp-keys
Size: 2476 bytes
Desc: OpenPGP public key
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20220203/422b5f58/attachment-0001.key>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 665 bytes
Desc: OpenPGP digital signature
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20220203/422b5f58/attachment-0001.sig>
More information about the QGIS-Developer
mailing list