[Qgis-developer] Performance for selection on attribute table

[Angus Carr]

For what it is worth, I frequently use several hundred MB shapefile
datasets, usually of forest inventory. The speed difference between ESRI
products and QGIS is quite apparent here, whether on spatial selections,
table selections, or whatever. I frequently turn off rendering while I play
in the table, but it is still slow compared to doing the same operations in
PostGIS or in Arc.

I'm happy to play with sample code fragments in python to run test cases on
my datasets, if that would help. I can work from shapefiles or PostGIS, as
needed.

If we need to build a test dataset to work with, then I can build something
we could release to the project.

This is a significant factor in how I use QGis, mainly driven by relatively
big data.

Cheers,
Angus Carr.

On Thu, Mar 28, 2013 at 6:47 AM, HAUBOURG <
regis.haubourg at eau-adour-garonne.fr> wrote:

> I filed a ticket for memory and reference purpose.
> http://hub.qgis.org/issues/7463
>
> Cheers
> régis
>
> -----Message d'origine-----
> De : Matthias Kuhn [mailto:matthias.kuhn at gmx.ch]
> Envoyé : jeudi 28 mars 2013 10:01
> À : HAUBOURG
> Cc : 'qgis-developer at lists.osgeo.org'
> Objet : Re: [Qgis-developer] Performance for selection on attribute table
>
> On Don 28 Mär 2013 09:49:17 CET, HAUBOURG wrote:
> > Thanks guys,
> > I also raised on list issues with redrawing triggered by selection (but
> also bad performance for "identify" highlights for big objects).  On that
> point, preventing extra refreshes (http://hub.qgis.org/issues/2596) and
> heading towards multithreading to let renderer use another process can help.
> > Anyway, we have an issue here, since datasets keep growing, and user
> won't always know about filter in layer properties. I didn't test if
> "select at id" option did change something. I didn't catch either what you
> describe with 'plugable' indexes. What could it be exactly?
>
> Plugable indexes mean, you'll be able to have a cache on a layer (or maybe
> a data provider) later on, which can have several indexes on its features
> (even written in python hopefully) which can then answer a certain
> QgsFeatureRequest (if they're able to)
>
> E.g. you can have an index on certain attributes and in case the index
> knows, that is has e.g. all the features with column1 = 5 ready (due to an
> earlier request), the backend will not be bothered. Or a spatial index
> could be integrated to speed up rendering (e.g. when panning around and
> drawing similar regions all the time).
>
> Right now, the only index available is for feature ids.
>
> Regards
>
> >
> > régis
> >
> > -----Message d'origine-----
> > De : Matthias Kuhn [mailto:matthias.kuhn at gmx.ch] Envoyé : jeudi 28
> > mars 2013 07:44 À : HAUBOURG Cc : qgis-developer at lists.osgeo.org Objet
> > : Re: [Qgis-developer] Performance for selection on attribute table
> >
> > Hi Régis
> >
> > The attribute table is not using any providers indexes. And when
> thinking about it a couple of questions arise like stable syntax across
> providers (different providers need different SQL which either needs to be
> translated from QgsExpressions or written by the user). Then a request
> needs to also include not-yet-committed features. These need to be
> evaluated as well. In case the user directly specifies sql for the provider
> and not a QgsExpression this will pose some more problems.
> >
> > A new caching mechanism has been merged recently, but I don't expect any
> performance improvements yet. The new caching mechanism nevertheless
> supports plugable indexes, which I hope can help at least in part of these
> situations as soon as they are implemented.
> >
> > The only possibility to make use of the data providers indexes that I
> know of right now (apart from making views et al) is to provide a subset
> string in the layer properties, but that's most likely not what you want.
> >
> > Also, as mentioned by Denis Rouzaud, the rendering that takes when the
> selection changes place takes its bit of the performance. I remember, that
> somebody began a list of such unnecessarily triggered renders, but can't
> find it right now. I hope that in this area some work will be done as well.
> >
> > Kind regards,
> > Matthias
> >
> > On Mit 27 Mär 2013 20:57:07 CET, Régis Haubourg wrote:
> >> Hi dev's, I was notified some performance matters when dealing with
> >> selections in attribute tables, by some users swithcing from arcgis to
> QGIS.
> >> Indexes of datasource seem to be ignored for SQL selection and search.
> >> This leads to very slow selections when dealing with large datasets.
> >> Anyone confirms?  Is it really related to indexes or another bottleneck?
> >> Is there any possible improvement ? Are providers working on
> >> selections or is that all dealing with cache?
> >>
> >> Thanks for your feedback ,
> >> Régis
> >>
> >>
> >>
> >> --
> >> View this message in context:
> >> http://osgeo-org.1560.n6.nabble.com/Performance-for-selection-on-attr
> >> i bute-table-tp5043381.html Sent from the Quantum GIS - Developer
> >> mailing list archive at Nabble.com.
> >> _______________________________________________
> >> Qgis-developer mailing list
> >> Qgis-developer at lists.osgeo.org
> >> http://lists.osgeo.org/mailman/listinfo/qgis-developer
> >
> >
> >
>
>
>
> _______________________________________________
> Qgis-developer mailing list
> Qgis-developer at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/qgis-developer
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20130328/f42d83fe/attachment-0001.html>