<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Michael,</p>
<p>this error comes from libarrow-cpp. </p>
<p>Several potential causes:</p>
<p>- one of the parquet file is corrupted</p>
<p>- one of the parquet files is valid but uses "something" that
libarrow-cpp can't understand. I mention this because we have seen
an interoperability issues between files generated by the go
implementation of parquet and libarrow-cpp. Not clear which end is
the culprit. But the error message was different than yours<br>
</p>
<p>- there's an I/O error when getting data from S3</p>
<p>- some other bug in libarrow-cpp...<br>
</p>
<p>Perhaps run in --debug on mode to see in the traces which parquet
file cause the issue ? (assuming that the error can also be
reproduced when reading one of the component parquet files, and
not just when reading the whole dataset...)</p>
<p><a class="moz-txt-link-freetext" href="https://github.com/search?q=repo%3Aapache%2Farrow+TProtocolException%3A+Exceeded+size+limit&type=issues">https://github.com/search?q=repo%3Aapache%2Farrow+TProtocolException%3A+Exceeded+size+limit&type=issues</a>
shows a number of issues where this error message pops up</p>
<p>Even<br>
</p>
Le 16/11/2023 à 14:37, Smith, Michael ERDC-RDE-CRREL-NH CIV via
gdal-dev a écrit :<br>
<blockquote type="cite"
cite="mid:A679AA24-12B9-4DE1-B455-441697407F99@erdc.dren.mil">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="Generator"
content="Microsoft Word 15 (filtered medium)">
<style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ligatures:standardcontextual;}span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}.MsoChpDefault
{mso-style-type:export-only;
font-size:12.0pt;
font-family:"Calibri",sans-serif;}div.WordSection1
{page:WordSection1;}</style>
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt">Using
gdal3.8 (ghcr.io/osgeo/gdal:ubuntu-full-3.8.0) , got an
error I haven’t seen before:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">ReadNext()
failed: Couldn't deserialize thrift: TProtocolException:
Exceeded size limit<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Deserializing
page header failed.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">This
happened at 92%<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Command:
ogr2ogr -f gpkg /data/overturemaps_2023_11_14.gpkg
/vsis3/overturemaps-us-west-2/release/2023-11-14-alpha.0/theme=buildings/
theme=buildings -progress -NLT MULTIPOLYGON<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">So anyone
know what this means and what caused it and any workarounds?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Mike<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;mso-ligatures:none">-- <o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span
style="font-size:11.0pt;mso-ligatures:none">Michael Smith<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;mso-ligatures:none">US Army Corps
of Engineers<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;mso-ligatures:none">Remote
Sensing/GIS Center</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
gdal-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a>
<a class="moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/gdal-dev">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a>
</pre>
</blockquote>
<pre class="moz-signature" cols="72">--
<a class="moz-txt-link-freetext" href="http://www.spatialys.com">http://www.spatialys.com</a>
My software is free, but my time generally not.</pre>
</body>
</html>