<div dir="ltr"><div dir="ltr"><div>Hi Jon,</div><div><br></div><div>The performance of
GetFeatureCount, filters, and GetNextFeature depends a lot on the data
format. If your data is in Postgres, for example, these operations are
super fast because the database keeps the number of rows (features) in
memory and has highly optimized views of selected rows. If your data is
in a CSV file, GDAL has to read every line to count the number of
features and when getting filtered features has to read all the lines in
between the ones you want. See <a href="https://github.com/OSGeo/gdal/blob/master/ogr/ogrsf_frmts/csv/ogrcsvlayer.cpp#L1667" target="_blank">https://github.com/OSGeo/gdal/blob/master/ogr/ogrsf_frmts/csv/ogrcsvlayer.cpp#L1667</a>.</div><div><br></div><div>Can you switch to a more optimized format?</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jan 20, 2022 at 2:02 AM Jon Morris <<a href="mailto:Jon.Morris@jbarisk.com">Jon.Morris@jbarisk.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div style="overflow-wrap: break-word;" lang="EN-GB">
<div class="gmail-m_4903032386003630651WordSection1">
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;color:black">I'm writing applications using the GDAL Python bindings and when I profile for performance, GetFeatureCount frequently comes out near the top. I'm often using it to check whether
a spatial or attribute filter has returned any features and don't need the full count. When the layer contains millions of features, there would be a big performance improvement if we could exit the count early.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;color:black"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;color:black">Is there a better way of doing this? I've tried using GetNextFeature instead, but there must be quite a lot of overhead in that function as it is much slower. All I need to know is
if the layer has has 0, 1 or >1 features, I don't need the actual count. Can anyone suggest the fastest way of doing this in Python? I'm using GDAL 3.3.1 at the moment but could upgrade if there is new functionality that would help.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;color:black"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;color:black">Thanks,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;color:black"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;color:black">Jon<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;color:black"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-family:"Arial",sans-serif;color:rgb(246,161,36)">Jon Morris<u></u><u></u></span></b></p>
<p class="MsoNormal"><b><span style="font-family:"Arial",sans-serif;color:rgb(74,74,73)">Software Developer<u></u><u></u></span></b></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;color:black"><u></u> <u></u></span></p>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div dir="ltr" style="direction:ltr"><table style="width:100%" cellspacing="0" cellpadding="0" border="0"><tbody><tr style="font-size:0px"><td style="vertical-align:top" align="left"><table style="font-size:0px" cellspacing="0" cellpadding="0" border="0"><tbody><tr style="font-size:0px"><td style="padding:10px 0px 0px;vertical-align:top" align="left"><table style="font-size:0px;color:rgb(74,74,73);font-style:normal;font-weight:700;white-space:nowrap" cellspacing="0" cellpadding="0" border="0"><tbody><tr style="font-size:14.67px"><td style="padding:0px;vertical-align:top;font-family:Arial" align="left">e: <span style="font-family:remialcxesans;font-size:1px;color:rgb(255,255,255);line-height:1px"></span></td><td style="padding:0px 0px 0px 4px;vertical-align:top;color:rgb(246,161,36);font-family:Arial;font-weight:400" align="left"><a href="mailto:Jon.Morris@jbarisk.com" id="gmail-m_4903032386003630651LPlnk689713" style="text-decoration:none;color:rgb(246,161,36)" target="_blank"><b style="font-weight:400">Jon.Morris@jbarisk.com</b></a></td></tr><tr style="font-size:14.67px"><td style="padding:0px;vertical-align:top;font-family:Arial" align="left">t: </td><td style="padding:0px 0px 0px 4px;vertical-align:top;color:rgb(246,161,36);font-family:Arial;font-weight:400" align="left"><a href="tel:+44%20(0)1756%20799919" id="gmail-m_4903032386003630651LPlnk689713" style="text-decoration:none;color:rgb(246,161,36)" target="_blank"><b style="font-weight:400">+44 (0)1756 799919</b></a></td></tr><tr style="font-size:0px"><td style="padding:0px;vertical-align:top" align="left"></td><td style="padding:0px 0px 0px 4px;vertical-align:top" align="left"></td></tr></tbody></table></td></tr></tbody></table></td></tr><tr style="color:rgb(0,0,1);font-size:14.67px;font-style:normal;font-weight:400;white-space:nowrap"><td style="vertical-align:top;font-family:Arial" align="left"><a href="http://www.jbarisk.com/" id="gmail-m_4903032386003630651LPlnk689713" title="Click to visit our website" style="text-decoration:none;color:rgb(246,161,36)" target="_blank"><b style="font-weight:400">www.jbarisk.com</b></a></td></tr><tr style="font-size:0px"><td style="vertical-align:top" align="left"><table style="font-size:0px" cellspacing="0" cellpadding="0" border="0"><tbody><tr style="font-size:0px"><td style="padding:10px 0px 0px;vertical-align:top" align="left"><table style="font-size:0px" cellspacing="0" cellpadding="0" border="0"><tbody><tr style="font-size:0px"><td rowspan="3" style="padding:0px 0px 5px;vertical-align:top" align="left"><table style="width:100%;font-size:0px" cellspacing="0" cellpadding="0" border="0"><tbody><tr style="font-size:0px"><td style="vertical-align:top" align="left"><table style="font-size:0px;line-height:normal" cellspacing="0" cellpadding="0" border="0"><tbody><tr style="font-size:0px"><td style="padding:10px 0px 15px;vertical-align:top" align="left"><img src="cid:17e7828c033e03fce501" alt="" style="height: 52px; min-height: 52px; max-height: 52px; font-size: 0px;" height="52" border="0"></td></tr></tbody></table></td></tr><tr style="font-size:0px"><td style="vertical-align:top" align="left"><table style="font-size:0px" cellspacing="0" cellpadding="0" border="0"><tbody><tr style="font-size:0px"><td style="padding:0px 0px 12px;vertical-align:top" align="left"><table style="height:30px;font-size:0px" cellspacing="0" cellpadding="0" border="0"><tbody><tr style="font-size:0px"><td style="vertical-align:top" align="left"><table style="font-size:0px;line-height:normal" cellspacing="0" cellpadding="0" border="0"><tbody><tr style="font-size:0px"><td style="padding:0px 4px 5px 0px;vertical-align:top" align="left"><a href="https://www.facebook.com/TheFloodPeople" id="gmail-m_4903032386003630651LPlnk689713" style="text-decoration:none" target="_blank"><img src="cid:17e7828c0348a1f98b42" title="Facebook" alt="Facebook" style="width: 34px; min-width: 34px; max-width: 34px; font-size: 12px;" width="34" border="0"></a></td></tr></tbody></table></td><td style="vertical-align:top" align="left"><table style="font-size:0px;line-height:normal" cellspacing="0" cellpadding="0" border="0"><tbody><tr style="font-size:0px"><td style="padding:0px 4px 5px 0px;vertical-align:top" align="left"><a href="https://www.linkedin.com/company/jba-risk-management/" id="gmail-m_4903032386003630651LPlnk689713" style="text-decoration:none" target="_blank"><img src="cid:17e7828c034f2ea0beb3" title="LinkedIn" alt="LinkedIn" style="width: 34px; min-width: 34px; max-width: 34px; font-size: 12px;" width="34" border="0"></a></td></tr></tbody></table></td><td style="vertical-align:top" align="left"><table style="font-size:0px;line-height:normal" cellspacing="0" cellpadding="0" border="0"><tbody><tr style="font-size:0px"><td style="padding:0px 4px 5px 0px;vertical-align:top" align="left"><a href="https://twitter.com/JBARisk" id="gmail-m_4903032386003630651LPlnk689713" style="text-decoration:none" target="_blank"><img src="cid:17e7828c0343a2da8f64" title="Twitter" alt="Twitter" style="width: 34px; min-width: 34px; max-width: 34px; font-size: 12px;" width="34" border="0"></a></td></tr></tbody></table></td><td style="vertical-align:top" align="left"><table style="font-size:0px;line-height:normal" cellspacing="0" cellpadding="0" border="0"><tbody><tr style="font-size:0px"><td style="padding:0px 4px 5px 0px;vertical-align:top" align="left"><a href="https://www.youtube.com/channel/UC0iatom2jYbW96voW0rlpCw" id="gmail-m_4903032386003630651LPlnk689713" style="text-decoration:none" target="_blank"><img src="cid:17e7828c03425934cdc5" title="YouTube" alt="YouTube" style="width: 34px; min-width: 34px; max-width: 34px; font-size: 12px;" width="34" border="0"></a></td></tr></tbody></table></td></tr></tbody></table></td></tr></tbody></table></td></tr></tbody></table></td><td style="padding:0px"> </td></tr><tr style="font-size:0px"><td style="padding:0px"> </td></tr><tr style="font-size:0px"><td style="padding:0px"> </td></tr></tbody></table></td></tr></tbody></table></td></tr><tr style="font-size:0px"><td style="vertical-align:top" align="left"><table style="font-size:0px" cellspacing="0" cellpadding="0" border="0"><tbody><tr style="font-size:0px"><td style="padding:7px 0px 5px;border-color:rgb(74,74,73) currentcolor currentcolor;border-style:solid none none;border-width:1px medium medium;vertical-align:top" align="left"><table style="white-space:normal;color:rgb(68,68,68);font-size:14.67px;font-family:Calibri,Arial,sans-serif;font-weight:400;font-style:normal;text-align:left;line-height:12px" cellspacing="0" cellpadding="0" border="0"><tbody><tr style="font-size:9px"><td style="font-family:Arial"><span style="background-color:rgb(255,255,255)">All JBA Risk Management's email messages contain confidential information and are intended only for the individual(s) named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail.</span><br><span style="background-color:rgb(255,255,255)">Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system.</span> <br><span style="background-color:rgb(255,255,255)">JBA Risk Management Limited is registered in England, company number 07732946, 1 Broughton Park, Old Lane North, Broughton, Skipton, North Yorkshire, BD23 3FD, England</span><span style="color:rgb(0,0,1)">.</span><br></td></tr></tbody></table></td></tr></tbody></table></td></tr></tbody></table></div></div>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr" class="gmail_signature">Sean Gillies</div></div>