[Qgis-developer] New Vector API

[Martin Dobias]

Hi Matthias

On Tue, Jan 15, 2013 at 11:46 PM, Matthias Kuhn <matthias.kuhn at gmx.ch> wrote:
> Hi Martin,
>
> This mail concerns the vectorlayercache outlined in section 8) of your mail.
>
> I'm currently working on the attribute table which has some internal caching
> already. As I'm currently rewriting the attribute table, I moved this code
> outside and generalized it as good as possible. This work is currently
> available in my branch vectorlayercache [0]. Do you think this could be used
> instead of your geometry cache? It should be able to cache geometries of
> features (although I only tested attributes so far) and update and
> invalidate them if required.
>
> I have also the idea to create a QgsVectorDataProviderCacheProxy class which
> inherits QgsVectorDataProvider and which you can use as a proxy between your
> app and the DataProvider. I've some unfinished code here which provides such
> functionality. But for now, the cache in front of the VectorLayer should
> already help a lot.

Just out of curiosity, what are you rewriting in the attribute table -
is it just the cache or does it involve more changes?

Your idea of having a cache proxy vector data provider is interesting.
Such transparent approach would enable us to remove caching from
vector data provider implementations (e.g. wfs) and transparently add
caching to the ones where caching would be useful (e.g.
delimitedtext). I need to give it some more thoughts what impact that
would have.

The code for cache in your github repository implements cache with
feature ids as keys. That is useful for attribute table queries,
however for rendering it does not help much. In the cache we will need
also a spatial index for fast spatial queries used for map rendering,
identification and geometry editing. Without spatial index, the cache
is quite slow if the whole cache need to be traversed in order to find
features in a specified rectangle. As of now, the cache you've done is
bound to a one particular query it gets: I would be most interested in
a general cache to avoid having situations where cache would be filled
by results of a first query and then flushed and filled with results
of a following query.

The caching behavior should be ideally configurable: e.g. which
layers/providers to cache (probably we don't want to cache huge
layers), what to cache (everything / only geometries / only attributes
/ geometries and some attributes), cache size limit.

The next question is how to invalidate the cache, so that if data have
been changed in the backend, the user will not end up with a cached
version forever. In some providers I think it's possible to detect the
updates automatically, but some providers do not have any such
notifications. Maybe we should allow users to set a timeout - after
specified interval the cache would need to check again with the
backend whether the features have not been altered.

Back to your original question whether such improved cache could be
used instead of the current geometry cache: yes, it could, in case it
will include a spatial index. We are really in a need of a general
caching mechanism, so I would welcome any work in this area. The
speedup of rendering from cache should be significant. The current
caching of geometries is there just to preserve the functionality of
moveVertex() and other functions from vector layer that were using
cached geometries before.

Regards
Martin