[Qgis-developer] New Vector API

[Martin Dobias]

Hi everyone,

it seems that now it is a good time to ask for a broader review of the
work I have been doing during recent months: essentially making QGIS
vector API more flexible and more ready for introduction of threading
for rendering. That resulted in a greater refactoring of some parts of
QgsVectorLayer class and QgsVectorDataProvider implementations.
Everything is in new_vector_api branch in QGIS repository on GitHub
[1].

There are few things that are not finished, but should not take too
much work to sort out:
- disabled providers - mssql, osm, sqlanywhere, wfs - not yet updated to new API
- disabled mapserver - there are few things to update

If no serious problems will be found, I would like to merge the branch
to master and continue working on that on master branch to avoid the
possibility to drift (again) too much from the quick development that
happens on master. In short term I plan to do some polishing and
fixing bugs, then eventually start looking at the threading again.

For more details about what happened in the branch, see the text below
(long read!). A great help from early testers would be to compile the
branch, try to do some "usual" stuff with vectors and see if things
break apart (crashes? data corruption?).

Looking forward to your comments!

Regards
Martin

[1] https://github.com/qgis/Quantum-GIS/tree/new_vector_api



QGIS VECTOR API CHANGES

1. QgsFeature (changes)

a) access to attributes by name. Vector data providers and vector
layer assign pointer to the fields in QgsFeature, so it is possible to
set/get attributes by their name. This is not as efficient as with
direct access by index, but often this convenience is useful for
users.

b) attributes are stored in a QVector container rather than a QMap.
The major advantage is simplification of logic related to handling of
attributes: it's not possible to have "holes" in the array of
fields/attributes. Currently it is possible that for example layer
with three fields returns indexes 0,1,3 - but it is not so common nor
obvious, so it's a common source of errors. After refactoring there
must not be any holes, so a layer with three fields always returns
indexes 0,1,2. When iterating over layer's features, QgsFeature always
contains a vector of all layer's attributes. In case the client has
not requested attributes to be fetched, the attributes contain invalid
values.


2. QgsFields (new class)

Just like attributes within a feature are stored in a vector instead
of map, also layer's fields are now stored in a vector. QgsFields is a
new class that mimics QVector API and adds two more pieces of
functionality:

a) fast lookup of field index from name. When QgsFields is populated
with fields, it creates a map of fields that facilitates fast lookups

b) detection of field origin. When working with a vector layer, it is
sometimes useful to find out origin of the field - whether it comes
from provider, from a join or whether it is a newly added field (not
committed). In the future we could add also expression-based fields,
creating a field calculator that calculates the values on the fly.


3. QgsFeatureRequest (new class)

Class that encapsulates requests for features to a vector layer or
vector data provider. Right now in master branch, the request is
defined by arguments to select() method. That's not very flexible nor
simple to use. Feature request class allows easier extensibility in
future (support generic expression filter, native provider's SQL
filter...).

Without any customization, the request will ask for all features with
geometries and attributes - somehow better default that the current
one that does not fetch attributes if their list is not explicitly
given.

(I'm not yet completely happy with the API of this class, so it may be
changed to some degree. Suggestions are welcome.)

Examples:
- fetch all features:
    QgsFeatureRequest()
- fetch all features, only one attribute
    QgsFeatureRequest().setSubsetOfAttributes(QStringList("myfield"),
provider->fields())
- fetch all features, without geometries
    QgsFeatureRequest().setFlags(QgsFeatureRequest::NoGeometry)
- fetch only features from particular extent
    QgsFeatureRequest().setFilterRect(QgsRectangle(0,0,1,1))
- fetch only one feature
    QgsFeatureRequest().setFilterFid(45)



4. QgsFeatureIterator (new class)

The iterator class allows iteration over features of a vector layer or
a provider. It contains the usual nextFeature() method that fills
given QgsFeature class with data.

Internally, this class is a wrapper around QgsAbstractFeatureIterator
interface that is implemented by each vector data provider and by
vector layer. In theory we could use directly
QgsAbstractFeatureIterator pointers, but this wrapper allows us to use
it as a local variable, so it gets destroyed automatically when it
gets out of scope, not requiring an explicit "delete" call that would
be necessary with a pointer (easy to forget, right?)


5. Access to features

Vector layer and providers implement one important method call:
getFeatures(). It takes single argument (QgsFeatureRequest) and
returns QgsFeatureIterator. This replaces select(), nextFeature(),
rewind() and featureAtId() methods.

This approach allows users to create multiple iterators over a vector
layer (or a vector data provider). But currently that's not supported
because this needs support in the providers. At some point, we could
add support for simultaneous iterators - something that old API does
not allow at all.

When QgsFeatureIterator instance of a particular layer is closed (by
calling close() or destructed), then a new getFeatures() call may be
issued.


6. QgsVectorLayerEditBuffer (new class)

All editing functionality of vector layer has been moved out of
QgsVectorLayer into a new class that stores everything related to
editing: added features, removed features, changed geometries, changed
attribute values, added attributes, removed attributes. This class is
accessible from QgsVectorLayer, but it exists only when the layer is
in editing mode - it is deleted once features are rolled back or
committed.

The undo/redo support has been completely rewritten and greatly
simplified: instead of one undo command that gathers changes within
the edit buffer, there is now one undo command implementation for each
action. The undo commands can be grouped together with QUndoStack
macros. The new implementation ensures that the undo stack does not
get out of sync, because all editing changes are stored onto undo
stack (unlike current implementation in master branch where
begin/end-UndoCommand() methods must be called before/after edit
operations to ensure correct behavior of undo).


7. QgsVectorLayerEditUtils (new class)

Advanced editing operations that operate on vector layers (e.g. add
part) that can be composed from one or more simple edit operations
have been moved out of QgsVectorLayer and its edit buffer, so that the
vector layer API is not polluted by methods that do not need access to
its private members.


8. QgsVectorLayerCache (new class)

To speed up geometry editing, QGIS keeps a cache of geometries
currently shown in map canvas. This cache has been decoupled from
editing operations and moved to a separate class to keep things tidy.
In the future we should be able to add more general caching scheme to
allow faster rendering of vectors. Also, this might be a good place to
store index of feature ID -> row number that is calculated every time
when attribute table is opened.


9. Python

Python API has been updated to support functionality from the points
above, that is access by attribute name and support for python
iterators, e.g.:

for feature in layer.getFeatures(request):
  print feature["road_id"]