<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hi Michael,</p>
<p>This error comes very deep inside libparquet, actually in
libthrift. The more natural assumption would be that it would be
due to a corrupted Parquet file. If you disabled multithreading (<span
style="font-size:11.0pt">GDAL_NUM_THREADS=1) and enabled --debug
on, perhaps this would happen on the same file ? I was wondering
also if that couldn't come from RAM exhaustion, but that doesn't
seem likely (</span>did you monitor the RAM consumption?). It
would be interesting to see if that would also occur after
fetching locally the files under <span style="font-size:11.0pt">overturemaps-us-west-2/release/2024-07-22.0/theme=buildings
and running the conversion from the local files<br>
</span></p>
<p>Even<br>
</p>
<div class="moz-cite-prefix">Le 30/07/2024 à 20:37, Michael Smith
via gdal-dev a écrit :<br>
</div>
<blockquote type="cite"
cite="mid:A717A0A7-75F2-4064-935C-A5AA8589F294@gmail.com">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="Generator"
content="Microsoft Word 15 (filtered medium)">
<style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ligatures:standardcontextual;}span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}.MsoChpDefault
{mso-style-type:export-only;
font-size:12.0pt;}div.WordSection1
{page:WordSection1;}</style>
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt">Hi all,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">I was
converting the overture maps parquet data to geopackage and
got this error using gdal master (via conda).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">GDAL_NUM_THREADS=ALL_CPUS
CPL_TMPDIR=/data2 ogr2ogr -f gpkg
/data/overture_buildings.gpkg
/vsis3/overturemaps-us-west-2/release/2024-07-22.0/theme=buildings
"theme=buildings" -progress -nlt PROMOTE_TO_MULTI -nln
buildings -skipfailures<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">ERROR 1:
ReadNext() failed: Couldn't deserialize thrift:
TProtocolException: Exceeded size limit<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Not sure
what size limit it exceeded? I’ve had this happen several
times but at very different points in the process. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Ideas?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Mike<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;mso-ligatures:none">-- <o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span
style="font-size:11.0pt;mso-ligatures:none">Michael Smith<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;mso-ligatures:none">US Army Corps
of Engineers<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;mso-ligatures:none">Remote
Sensing/GIS Center</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
gdal-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a>
<a class="moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/gdal-dev">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a>
</pre>
</blockquote>
<pre class="moz-signature" cols="72">--
<a class="moz-txt-link-freetext" href="http://www.spatialys.com">http://www.spatialys.com</a>
My software is free, but my time generally not.</pre>
</body>
</html>