[gdal-dev] Faster alternative to GetFeatureCount?

Thu Jan 20 07:55:08 PST 2022

Hi Roman,

I'm not sure how you would determine the kill time. Each individual call to GetFeatureCount is fairly quick, but when you're doing it a lot, in a loop, they add up. Here's an example output from the Python profiler:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      195   98.689    0.506   98.690    0.506 {built-in method osgeo._ogr.Geometry_Intersection}
     4091   40.792    0.010   40.860    0.010 {built-in method osgeo._ogr.Geometry_Union}
     8358   33.862    0.004   33.862    0.004 {built-in method osgeo._ogr.Layer_GetFeatureCount}
    28653   17.059    0.001   17.115    0.001 {built-in method osgeo._ogr.Layer_GetNextFeature}

Obviously, I could be calling the feature count less often, but I was hoping I could shave a small amount of time off each call. Also obviously, my geometry operations are more costly than the feature operations, but it's worth trying to save time anywhere you can.

Jon

e: Jon.Morris at jbarisk.com
t: +44 (0)1756 799919
www.jbarisk.com
All JBA Risk Management's email messages contain confidential information and are intended only for the individual(s) named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail.
Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system. 
JBA Risk Management Limited is registered in England, company number 07732946, 1 Broughton Park, Old Lane North, Broughton, Skipton, North Yorkshire, BD23 3FD, England.
From: Roman Breitfuss-Schiffer <roman.schiffer at gmail.com>
Sent: 20 January 2022 10:38
To: Jon Morris <Jon.Morris at jbarisk.com>; gdal-dev at lists.osgeo.org
Subject: Re: [gdal-dev] Faster alternative to GetFeatureCount?

CAUTION: This email originated from outside of JBA and contains one or more links and one or more attachments. DO NOT click links or open attachments unless you recognise the sender's email address and are absolutely certain that the content is safe.
See the Phishing page on IMS on SP for more information about how to spot and report suspicious messages.

Hi Jon!

I don't know if there's an alternative to GetFeatureCount. At least I couldn't find one in the API documentation. Maybe there's a workaround I'm not aware of...

What you could do is wrap the call of GetFeatrueCount in a function which is killed after some time (2 approaches are discussed here: https://stackoverflow.com/questions/40915527/kill-function-after-a-given-amount-of-time). With that approach the question remains of getting the value of GetFeatureCount when the function gets killed. One could assume that there are more features than 1 if the function takes too long. This assumption might be a bit risky though...

Best regards
Roman
Am 20.01.2022 um 10:00 schrieb Jon Morris:
I'm writing applications using the GDAL Python bindings and when I profile for performance, GetFeatureCount frequently comes out near the top. I'm often using it to check whether a spatial or attribute filter has returned any features and don't need the full count. When the layer contains millions of features, there would be a big performance improvement if we could exit the count early.

Is there a better way of doing this? I've tried using GetNextFeature instead, but there must be quite a lot of overhead in that function as it is much slower. All I need to know is if the layer has has 0, 1 or >1 features, I don't need the actual count. Can anyone suggest the fastest way of doing this in Python? I'm using GDAL 3.3.1 at the moment but could upgrade if there is new functionality that would help.

Thanks,

Jon

Jon Morris
Software Developer

e: 
Jon.Morris at jbarisk.com<mailto:Jon.Morris at jbarisk.com>
t:
+44 (0)1756 799919<tel:+44%20(0)1756%20799919>
www.jbarisk.com<http://www.jbarisk.com/>
[cid:image001.png at 01D80E16.184A46E0]
[Facebook]<https://www.facebook.com/TheFloodPeople>
[LinkedIn]<https://www.linkedin.com/company/jba-risk-management/>
[Twitter]<https://twitter.com/JBARisk>
[YouTube]<https://www.youtube.com/channel/UC0iatom2jYbW96voW0rlpCw>

All JBA Risk Management's email messages contain confidential information and are intended only for the individual(s) named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail.
Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system.
JBA Risk Management Limited is registered in England, company number 07732946, 1 Broughton Park, Old Lane North, Broughton, Skipton, North Yorkshire, BD23 3FD, England.

_______________________________________________

gdal-dev mailing list

gdal-dev at lists.osgeo.org<mailto:gdal-dev at lists.osgeo.org>

https://lists.osgeo.org/mailman/listinfo/gdal-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220120/e12e83e7/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 9886 bytes
Desc: image001.png
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220120/e12e83e7/attachment-0010.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 651 bytes
Desc: image002.png
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220120/e12e83e7/attachment-0011.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 715 bytes
Desc: image003.png
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220120/e12e83e7/attachment-0012.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.png
Type: image/png
Size: 966 bytes
Desc: image004.png
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220120/e12e83e7/attachment-0013.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image005.png
Type: image/png
Size: 784 bytes
Desc: image005.png
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220120/e12e83e7/attachment-0014.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image244774.png
Type: image/png
Size: 9886 bytes
Desc: image244774.png
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220120/e12e83e7/attachment-0015.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image169598.png
Type: image/png
Size: 651 bytes
Desc: image169598.png
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220120/e12e83e7/attachment-0016.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image117658.png
Type: image/png
Size: 715 bytes
Desc: image117658.png
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220120/e12e83e7/attachment-0017.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image480157.png
Type: image/png
Size: 966 bytes
Desc: image480157.png
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220120/e12e83e7/attachment-0018.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image726289.png
Type: image/png
Size: 784 bytes
Desc: image726289.png
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220120/e12e83e7/attachment-0019.png>