<div dir="ltr"><span class="gmail-im">
<p>| And only construct an OGRGeometry when it's asked for? Such as
when GetGeometryRef is called?</p></span><div>I'm wondering about a
more broad application of this. Would it be helpful to have the ability
to lazy-initialize an OGRGeometry from multiple source types such as
WKB and GEOS, initially storing only a reference to the external data in
WKB/GEOS/etc and actually materializing the geometry when required?
Then methods such as OGRGeometry::exportToWkb and
OGRGeometry::exportToGEOS could check the external data type and use it
directly if it is compatible, avoiding materialization. This would avoid
multiple conversions to/from GEOS in cases where operations are
chained, as well as allowing WKB to pass directly between input and
output drivers that support it. Relatedly, this ability could be used to
cache external-format data when it is generated for an OGRGeometry,
avoiding inefficiencies such as two conversions to GEOS when checking to see if two geometries intersect before calculating
their intersection.</div><div><br></div><div>Dan</div><div><br></div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Feb 4, 2023 at 1:55 PM Even Rouault <<a href="mailto:even.rouault@spatialys.com">even.rouault@spatialys.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>Hi Sean,</p>
<br>
<blockquote type="cite">
<div dir="ltr">
<div>but wouldn't it be possible for all OGRFeatures to carry
WKB data by default and add a method to provide it to callers?
<br>
</div>
</div>
</blockquote>
<p>My understanding of what you propose would involve massive code
rewrites in all drivers and wouldn't be desirable from a
performance point of view, because most drivers can't generate WKB
easily (PostGIS and GPKG are the exceptions rather the norm). So
either all other drivers should be modified to compose WKB at hand
(massive coding effort. Probably several weeks of effort and
significant risk of regressions). Or get it from the ExportToWkb()
method of the OGRGeometry instance they currently build, but then
you pay the price in memory and CPU time to generate WKB that
might not be consumed by users.</p>
<p>| And only construct an OGRGeometry when it's asked for? Such as
when GetGeometryRef is called?</p>
<p>Good point, we could both make GetGeometryRef() and
GetGeomFieldRef() virtual methods whose default implementation
would be the same as currently, ie. return the value of the
corresponding member variable in the base OGRFeature class stored
with SetGeometry[Directly]()/SetGeomField[Directly]()<br>
</p>
<p>And add a new virtual method:<br>
</p>
<p>virtual GByte* OGRFeature::GetWKBGeometry(int iGeomField, size_t*
pnOutSize) const<br>
</p>
<p>whose default implementation would just use
GetGeomFieldRef(iGeomField)->ExportToWkb().</p>
<p>The few drivers that can provide a more efficient implementation
(GPKG typically) would create a derived class OGRFeatureGPKG with
a specific implementation of those new virtual methods to avoid
systematic OGRGeometry instantiation. The only drawback I see is
that making GetGeometryRef() and GetGeomFieldRef() virtual would
have a slight performance impact, but probably small enough.</p>
<p><br>
</p>
<p>But fundamentally I'm wondering if RFC 92 hasn't been made mostly
out fashioned now that we have RFC 86. RFC 86 generally leads to
2x speed-up or more on real-world datasets compared to OGRFeature
iteration (as measured by the bench_ogr_c_api vs bench_ogr_batch
utilities) on drivers that have implemented it (currently Arrow,
Parquet, FlatGeoBuf, GPKG), whereas RFC 92 only applies to GPKG
& PostGIS and in the best - artificial - case only lead to 30%
speed-up.</p>
<p>Of course, adopting RFC 86 requires significant effort from GDAL
users, but the benefit is really measurable whereas with RFC 92 it
would be marginal in most scenarios. As far as I can tell, the
performance boost of RFC 86 comes mostly from saving creation
& destruction of millions of OGRFeature instances, its array
members, string attributes, geometries objects, more than the
columnar organization of the ArrowArray data structures. In the
GeoPackage driver, I've also shown that it makes it possible for
efficient multi-threading pre-fetching, totally transparent for
the user.</p>
<p>But to avoid selling false hopes, the benefit of RFC 86 in
end-to-end scenarios would probably drop significantly (at least
if looking at performance gain in percentage. The absolute
performance savings on the GDAL side would remain) if you need to
recreate individual features (QGIS' QgsFeature or MapServer'
msShape objects) from the content of ArrowArray. So this is likely
a complete shift of concepts that would be required.<br>
</p>
<p>Even<br>
</p>
<p><br>
</p>
<blockquote type="cite"><br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, Jan 31, 2023 at 4:27
AM Even Rouault <<a href="mailto:even.rouault@spatialys.com" target="_blank">even.rouault@spatialys.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi,<br>
<br>
Please find for review "RFC 92 text: WKB Only geometries" at <br>
<a href="https://github.com/OSGeo/gdal/pull/7149" rel="noreferrer" target="_blank">https://github.com/OSGeo/gdal/pull/7149</a><br>
<br>
This RFC provides shortcuts to avoid instantiation of full
OGRGeometry <br>
instances<br>
in scenarios where only the WKB representation of geometries
is needed. The<br>
hope is to save CPU time.<br>
<br>
This is something I wanted to at least experiment. I've mixed
feelings <br>
if it's something we actually want to adopt.<br>
<br>
Even<br>
<br>
-- <br>
<a href="http://www.spatialys.com" rel="noreferrer" target="_blank">http://www.spatialys.com</a><br>
My software is free, but my time generally not.<br>
<br>
_______________________________________________<br>
gdal-dev mailing list<br>
<a href="mailto:gdal-dev@lists.osgeo.org" target="_blank">gdal-dev@lists.osgeo.org</a><br>
<a href="https://lists.osgeo.org/mailman/listinfo/gdal-dev" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a><br>
</blockquote>
</div>
<br clear="all">
<br>
-- <br>
<div dir="ltr">Sean Gillies</div>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
gdal-dev mailing list
<a href="mailto:gdal-dev@lists.osgeo.org" target="_blank">gdal-dev@lists.osgeo.org</a>
<a href="https://lists.osgeo.org/mailman/listinfo/gdal-dev" target="_blank">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a>
</pre>
</blockquote>
<pre cols="72">--
<a href="http://www.spatialys.com" target="_blank">http://www.spatialys.com</a>
My software is free, but my time generally not.</pre>
</div>
_______________________________________________<br>
gdal-dev mailing list<br>
<a href="mailto:gdal-dev@lists.osgeo.org" target="_blank">gdal-dev@lists.osgeo.org</a><br>
<a href="https://lists.osgeo.org/mailman/listinfo/gdal-dev" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a><br>
</blockquote></div>