[gdal-dev] Faster alternative to GetFeatureCount?

[Sean Gillies]

Hi Jon,

The performance of GetFeatureCount, filters, and GetNextFeature depends a
lot on the data format. If your data is in Postgres, for example, these
operations are super fast because the database keeps the number of rows
(features) in memory and has highly optimized views of selected rows. If
your data is in a CSV file, GDAL has to read every line to count the number
of features and when getting filtered features has to read all the lines in
between the ones you want. See
https://github.com/OSGeo/gdal/blob/master/ogr/ogrsf_frmts/csv/ogrcsvlayer.cpp#L1667
.

Can you switch to a more optimized format?

On Thu, Jan 20, 2022 at 2:02 AM Jon Morris <Jon.Morris at jbarisk.com> wrote:

> I'm writing applications using the GDAL Python bindings and when I profile
> for performance, GetFeatureCount frequently comes out near the top. I'm
> often using it to check whether a spatial or attribute filter has returned
> any features and don't need the full count. When the layer contains
> millions of features, there would be a big performance improvement if we
> could exit the count early.
>
>
>
> Is there a better way of doing this? I've tried using GetNextFeature
> instead, but there must be quite a lot of overhead in that function as it
> is much slower. All I need to know is if the layer has has 0, 1 or >1
> features, I don't need the actual count. Can anyone suggest the fastest way
> of doing this in Python? I'm using GDAL 3.3.1 at the moment but could
> upgrade if there is new functionality that would help.
>
>
>
> Thanks,
>
>
>
> Jon
>
>
>
> *Jon Morris*
>
> *Software Developer*
>
>
>
>
> e:  *Jon.Morris at jbarisk.com* <Jon.Morris at jbarisk.com>
> t:  *+44 (0)1756 799919* <+44%20(0)1756%20799919>
> *www.jbarisk.com* <http://www.jbarisk.com/>
> [image: Facebook] <https://www.facebook.com/TheFloodPeople>
> [image: LinkedIn] <https://www.linkedin.com/company/jba-risk-management/>
> [image: Twitter] <https://twitter.com/JBARisk>
> [image: YouTube]
> <https://www.youtube.com/channel/UC0iatom2jYbW96voW0rlpCw>
>
>
> All JBA Risk Management's email messages contain confidential information
> and are intended only for the individual(s) named. If you are not the named
> addressee you should not disseminate, distribute or copy this e-mail.
> Please notify the sender immediately by email if you have received this
> email by mistake and delete this email from your system.
> JBA Risk Management Limited is registered in England, company number
> 07732946, 1 Broughton Park, Old Lane North, Broughton, Skipton, North
> Yorkshire, BD23 3FD, England.
>


-- 
Sean Gillies
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220120/15824040/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image918956.png
Type: image/png
Size: 9886 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220120/15824040/attachment-0005.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image215268.png
Type: image/png
Size: 651 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220120/15824040/attachment-0006.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image706640.png
Type: image/png
Size: 715 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220120/15824040/attachment-0007.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image011385.png
Type: image/png
Size: 966 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220120/15824040/attachment-0008.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image866509.png
Type: image/png
Size: 784 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220120/15824040/attachment-0009.png>