<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<br>
<blockquote type="cite"
cite="mid:CAOodmJrmdv0rFM+eFWrxGzeGNOMB3pK8F_4Srk8pfKRwZRQJ+g@mail.gmail.com">
<div dir="auto">
<div dir="auto"><br>
</div>
<div dir="auto">This RFC seems to have some things in common
with the direct access to compressed raster blocks. Both
by-pass the older raster and vector data models, yes? And
they're quite specific to certain formats. Do you anticipate
that GDAL and OGR will continue to develop more specialty APIs
in addition to the general ones?<br>
</div>
</div>
</blockquote>
<p>I have no preconceived ideas. I'd say that specialized API have
their place when they bring enough value, and don't do that at the
cost of complicating things that don't benefit from them. RFC 92
just doesn't pass my own criteria (my vote for it would be 0 in
its current state, but maybe someone will want to restart from it
and give it a second life).</p>
Even<br>
<blockquote type="cite"
cite="mid:CAOodmJrmdv0rFM+eFWrxGzeGNOMB3pK8F_4Srk8pfKRwZRQJ+g@mail.gmail.com">
<div dir="auto">
<div dir="auto"><br>
<div class="gmail_quote" dir="auto">
<div dir="ltr" class="gmail_attr">On Sat, Feb 4, 2023, 11:55
AM Even Rouault <<a
href="mailto:even.rouault@spatialys.com"
moz-do-not-send="true" class="moz-txt-link-freetext">even.rouault@spatialys.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<p>Hi Sean,</p>
<br>
<blockquote type="cite">
<div dir="ltr">
<div>but wouldn't it be possible for all OGRFeatures
to carry WKB data by default and add a method to
provide it to callers? <br>
</div>
</div>
</blockquote>
<p>My understanding of what you propose would involve
massive code rewrites in all drivers and wouldn't be
desirable from a performance point of view, because
most drivers can't generate WKB easily (PostGIS and
GPKG are the exceptions rather the norm). So either
all other drivers should be modified to compose WKB at
hand (massive coding effort. Probably several weeks of
effort and significant risk of regressions). Or get it
from the ExportToWkb() method of the OGRGeometry
instance they currently build, but then you pay the
price in memory and CPU time to generate WKB that
might not be consumed by users.</p>
<p>| And only construct an OGRGeometry when it's asked
for? Such as when GetGeometryRef is called?</p>
<p>Good point, we could both make GetGeometryRef() and
GetGeomFieldRef() virtual methods whose default
implementation would be the same as currently, ie.
return the value of the corresponding member variable
in the base OGRFeature class stored with
SetGeometry[Directly]()/SetGeomField[Directly]()<br>
</p>
<p>And add a new virtual method:<br>
</p>
<p>virtual GByte* OGRFeature::GetWKBGeometry(int
iGeomField, size_t* pnOutSize) const<br>
</p>
<p>whose default implementation would just use
GetGeomFieldRef(iGeomField)->ExportToWkb().</p>
<p>The few drivers that can provide a more efficient
implementation (GPKG typically) would create a derived
class OGRFeatureGPKG with a specific implementation of
those new virtual methods to avoid systematic
OGRGeometry instantiation. The only drawback I see is
that making GetGeometryRef() and GetGeomFieldRef()
virtual would have a slight performance impact, but
probably small enough.</p>
<p><br>
</p>
<p>But fundamentally I'm wondering if RFC 92 hasn't been
made mostly out fashioned now that we have RFC 86. RFC
86 generally leads to 2x speed-up or more on
real-world datasets compared to OGRFeature iteration
(as measured by the bench_ogr_c_api vs bench_ogr_batch
utilities) on drivers that have implemented it
(currently Arrow, Parquet, FlatGeoBuf, GPKG), whereas
RFC 92 only applies to GPKG & PostGIS and in the
best - artificial - case only lead to 30% speed-up.</p>
<p>Of course, adopting RFC 86 requires significant
effort from GDAL users, but the benefit is really
measurable whereas with RFC 92 it would be marginal in
most scenarios. As far as I can tell, the performance
boost of RFC 86 comes mostly from saving creation
& destruction of millions of OGRFeature instances,
its array members, string attributes, geometries
objects, more than the columnar organization of the
ArrowArray data structures. In the GeoPackage driver,
I've also shown that it makes it possible for
efficient multi-threading pre-fetching, totally
transparent for the user.</p>
<p>But to avoid selling false hopes, the benefit of RFC
86 in end-to-end scenarios would probably drop
significantly (at least if looking at performance gain
in percentage. The absolute performance savings on the
GDAL side would remain) if you need to recreate
individual features (QGIS' QgsFeature or MapServer'
msShape objects) from the content of ArrowArray. So
this is likely a complete shift of concepts that would
be required.<br>
</p>
<p>Even<br>
</p>
<p><br>
</p>
<blockquote type="cite"><br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, Jan 31,
2023 at 4:27 AM Even Rouault <<a
href="mailto:even.rouault@spatialys.com"
target="_blank" rel="noreferrer"
moz-do-not-send="true"
class="moz-txt-link-freetext">even.rouault@spatialys.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px
0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">Hi,<br>
<br>
Please find for review "RFC 92 text: WKB Only
geometries" at <br>
<a href="https://github.com/OSGeo/gdal/pull/7149"
rel="noreferrer noreferrer" target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">https://github.com/OSGeo/gdal/pull/7149</a><br>
<br>
This RFC provides shortcuts to avoid instantiation
of full OGRGeometry <br>
instances<br>
in scenarios where only the WKB representation of
geometries is needed. The<br>
hope is to save CPU time.<br>
<br>
This is something I wanted to at least experiment.
I've mixed feelings <br>
if it's something we actually want to adopt.<br>
<br>
Even<br>
<br>
-- <br>
<a href="http://www.spatialys.com" rel="noreferrer
noreferrer" target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">http://www.spatialys.com</a><br>
My software is free, but my time generally not.<br>
<br>
_______________________________________________<br>
gdal-dev mailing list<br>
<a href="mailto:gdal-dev@lists.osgeo.org"
target="_blank" rel="noreferrer"
moz-do-not-send="true"
class="moz-txt-link-freetext">gdal-dev@lists.osgeo.org</a><br>
<a
href="https://lists.osgeo.org/mailman/listinfo/gdal-dev"
rel="noreferrer noreferrer" target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a><br>
</blockquote>
</div>
<br clear="all">
<br>
-- <br>
<div dir="ltr">Sean Gillies</div>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
gdal-dev mailing list
<a href="mailto:gdal-dev@lists.osgeo.org" target="_blank" rel="noreferrer" moz-do-not-send="true" class="moz-txt-link-freetext">gdal-dev@lists.osgeo.org</a>
<a href="https://lists.osgeo.org/mailman/listinfo/gdal-dev" target="_blank" rel="noreferrer" moz-do-not-send="true" class="moz-txt-link-freetext">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a>
</pre>
</blockquote>
<pre cols="72">--
<a href="http://www.spatialys.com" target="_blank" rel="noreferrer" moz-do-not-send="true" class="moz-txt-link-freetext">http://www.spatialys.com</a>
My software is free, but my time generally not.</pre>
</div>
</blockquote>
</div>
</div>
</div>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
gdal-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a>
<a class="moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/gdal-dev">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a>
</pre>
</blockquote>
<pre class="moz-signature" cols="72">--
<a class="moz-txt-link-freetext" href="http://www.spatialys.com">http://www.spatialys.com</a>
My software is free, but my time generally not.</pre>
</body>
</html>