<div dir="ltr"><div>Thanks Even</div><div><br></div><div>I've spent this morning looking at the source code - you're definitely right that even though the PBF format is a similar <i>structure</i> to the JSON format, the code won't be easily adapted to support both without becoming extremely messy. So a separate driver is likely better.</div><div><br></div><div>The definitions in gpb.h look useful for this.</div><div><br></div><div>I will have to think harder at some point about how the two drivers will interact - it would be best if the new driver could fallback to ESRIJSON if the URL it has been given doesn't actually have PBF support. And of course the service and layer definition endpoints don't support PBF at all, only the query endpoints do - so the new driver will definitely need to re-use some of the code from ESRIJSON to do all that.</div><div><br></div><div>I won't be building this this week, but I'll keep you informed when I get to it.</div><div><br></div><div>Thanks</div><div>Craig</div><div><br></div></div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">On Wed, 23 Jul 2025 at 23:04, Even Rouault <<a href="mailto:even.rouault@spatialys.com">even.rouault@spatialys.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><u></u>
<div>
<p>Craig,</p>
<p>What would you be your idea: decode the PBF into JSON ? otherwise
re-using the existing code of the ESRI JSON driver is going to be
difficult as it assumes JSON everywhere.<br>
</p>
<p>For PBF reading, we have ogr/ogrsf_frmts/gpb.h used both by the
OSM PBF driver and the MVT driver. This is probably a bit tedious
to use than code generated by the protobuf compiler, but at least
this saves an extra dependency.</p>
<p>I'm a bit skeptical about the ability of a LLM to generate code
that will insert nicely into GDAL, but let's see...</p>
<p>Even<br>
</p>
<div>Le 23/07/2025 à 00:04, Craig de Stigter
via gdal-dev a écrit :<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div>Hi folks</div>
<div><br>
</div>
<div>OGR has support for ESRI FeatureService endpoints via the
<a href="https://gdal.org/en/stable/drivers/vector/esrijson.html" target="_blank">ESRI JSON</a> driver.</div>
<div>I am wondering if there would be interest in adding PBF
support to that driver, so that we could ingest the more
efficient PBF versions of those endpoints.</div>
<div><br>
</div>
<div>As far as I can tell, this is *different* and unrelated
to the OSM PBF format that GDAL already supports.</div>
<div><br>
</div>
<div>Feature servers we have come across that support PBF seem
to return query results much faster and more reliably in PBF
rather than JSON.</div>
<div><br>
</div>
<div>We have some Python code internally (not using GDAL) that
handles the PBF, but it performs poorly due to being
computation code in Python. I am investigating ways to speed
it up using other languages. As part of this I may be in a
position to convert it to C++ and contribute it to GDAL, if
there is interest. I am no C++ developer but am becoming
proficient with Claude Code :)</div>
<div><br>
</div>
<div>I would lean towards implementing the PBF support as a
by-default optimisation on the existing ESRIJSON driver, if
the server advertises support for PBF - with a config option
to force use of JSON if necessary. A separate driver would
be an alternative but seems unnecessary since the PBF
structure is quite similar to the JSON structure (the main
difference is that geometries are encoded more efficiently)</div>
<div><br>
</div>
<div>I haven't given much time towards investigating how to do
this in GDAL, but would appreciate any tips or discussion.</div>
<br>
<span class="gmail_signature_prefix">-- </span><br>
<div dir="ltr" class="gmail_signature">
<div dir="ltr">
<div style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px">Regards,</div>
<div style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px">Craig</div>
<div style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px"><br>
</div>
<div style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px">Platform
Engineer<br>
</div>
<div style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px">Koordinates</div>
<div style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px"><a href="http://koordinates.com/" style="color:rgb(17,85,204)" target="_blank">koordinates.com</a> / <a href="https://twitter.com/koordinates" style="color:rgb(17,85,204)" target="_blank">@koordinates</a></div>
</div>
</div>
</div>
</div>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
gdal-dev mailing list
<a href="mailto:gdal-dev@lists.osgeo.org" target="_blank">gdal-dev@lists.osgeo.org</a>
<a href="https://lists.osgeo.org/mailman/listinfo/gdal-dev" target="_blank">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a>
</pre>
</blockquote>
<pre cols="72">--
<a href="http://www.spatialys.com" target="_blank">http://www.spatialys.com</a>
My software is free, but my time generally not.</pre>
</div>
</blockquote></div><div><br clear="all"></div><br><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px">Regards,</div><div style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px">Craig</div><div style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px"><br></div><div style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px">Platform Engineer<br></div><div style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px">Koordinates</div><div style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px"><a href="http://koordinates.com/" style="color:rgb(17,85,204)" target="_blank">koordinates.com</a> / <a href="https://twitter.com/koordinates" style="color:rgb(17,85,204)" target="_blank">@koordinates</a></div></div></div>