[QGIS-Developer] Python script to count the number of feature for each unique values

[Nyall Dawson]

On Tue, 26 Feb 2019 at 03:31, kimaidou <kimaidou at gmail.com> wrote:
>
> Hi all,
>
> I needed to gather basic stats for a layer containing multiple fields. I created the following gist to compute the number of features per each unique value for a list of given fields
> https://gist.github.com/mdouchin/a234efb7e67ebd8dae3a04cb26cf5e72

Checkout trap #3 from
http://nyalldawson.net/2016/10/speeding-up-your-pyqgis-scripts/.
Specifically, since you aren’t doing anything with the geometry, and
are only using a single attribute from the layer, by calling setFlags(
QgsFeatureRequest.NoGeometry ) and setSubsetOfAttributes() you can
tell QGIS that you don’t need the geometry, and only require a single
attribute’s value in the feature request. This will give a huge speed
boost to the feature fetching.

> I know I could use PostgreSQL to do it (see https://twitter.com/kimaidou/status/1100053546978983936 conversation), wich is much faster, but I needed a quick way to do it with Python.
> I tested virtual layers too, but my source layer has many fields (with some heavy data) and the copy/pasting into spatialite was not efficient here.
>
> The method uniqueValues(fieldIndex) of QgsVectorLayer is pretty fast compared to my iteration through the layer features. Any hint how to improve the speed of my calculation ?

Potentially this could be a new method in QgsVectorDataProvider which
sends a native query to be executed on the backend (like uniqueValues
does -- that's why it's so fast). But that would require an addition
to the c++ QgsVectorDataProvider class, and implementations in the
popular vector data providers.

Nyall

>
> Regards,
> Michaël
> _______________________________________________
> QGIS-Developer mailing list
> QGIS-Developer at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer