[Qgis-developer] QgsExpression: replacement for QgsSearchString

[Martin Dobias]

Hi everyone

recently I have been trying to fix some flaws in search string
implementation and it turned out that rather than fixing them
one-by-one it will be easier to rewrite that code completely. Search
string and associated searching in attribute table was one of my first
contributions to QGIS and my first attempt to write a parser and an
evaluator. QgsExpression is my second attempt :-)

The main reason for the redesign was to bring the evaluation closer to
SQL semantics and to clean the code which started to be hard to
maintain.

QgsExpression is meant to be a replacement for QgsSearchString class.
Most of the changes are under the hood:
- fixed and simplifed the parser grammar and tree creation
- simplified lexer
- only one evaluation routine (instead of separate getValue/checkAgainst)
- correct handling of NULL values and three-value logic from SQL
(true, false, unknown)
- fixed error reporting
- easily extensible list of functions, saner evaluation of expressions
- using QVariant for return values instead of a special type
- slightly faster evaluation
- a more suitable class name
- tests included (!!!) :-)

Only few changes are visible for users:
- spaces were removed from "to int", "to real" and "to string"
functions because function names must not contain spaces
- null values are handled correctly by the functions and operators
- errors are reported when conversions between types fail

Btw. type conversions (currently supporting null, int, double, string)
are implemented in many different ways in databases. I have done some
tests on sqlite3, postgresql and mysql and many times there was no
consensus what the outcome of an evaluation should be:
- 7 > '6x' :: mysql=1, sqlite=0, postgresql=error
- 7 > '6' :: mysql=1, sqlite=0 (!), postgresql=1
- '6x' :: mysql=6 (+warning), sqlite=6, postgresql=error
- '3'+'5' :: mysql=8, sqlite=8, postgresql=error
- 3/5 :: mysql=0.6, sqlite=0, postgresql=0
- length(123) :: mysql=3, sqlite=3, postgresql=error
- 0 or 1 :: mysql=1, sqlite=1, postgresql=error
- 0 or 'x' :: mysql=0 (+warning), sqlite=0, postgresql=error

PostgreSQL is very strict about type conversions, Sqlite is not strict
at all and MySQL is somewhere in the middle. I have ended up with
these conversion rules:
- implicit conversions to desired types
- string->number conversions with invalid input return error
- arithmetic operators convert to numeric
- comparison with numeric type => numeric comparison


The code is here:
https://github.com/wonder-sk/Quantum-GIS/tree/expr

If there are no objections I will apply the changes to master in few
days. Given that I have been simultaneously writing unit tests I am
quite confident that there are not too many bugs (unit tests really
help!). In that branch I have replaced all the occurrences of search
string with expressions. So playing with search in attribute table or
doing calculation in field calculator will use the new engine.


Martin