<!DOCTYPE html>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <p>Hi Michael,</p>
    <p>This error comes very deep inside libparquet, actually in
      libthrift. The more natural assumption would be that it would be
      due to a corrupted Parquet file. If you disabled multithreading (<span
        style="font-size:11.0pt">GDAL_NUM_THREADS=1) and enabled --debug
        on, perhaps this would happen on the same file ? I was wondering
        also if that couldn't come from RAM exhaustion, but that doesn't
        seem likely (</span>did you monitor the RAM consumption?). It
      would be interesting to see if that would also occur after
      fetching locally the files under <span style="font-size:11.0pt">overturemaps-us-west-2/release/2024-07-22.0/theme=buildings
        and running the conversion from the local files<br>
      </span></p>
    <p>Even<br>
    </p>
    <div class="moz-cite-prefix">Le 30/07/2024 à 20:37, Michael Smith
      via gdal-dev a écrit :<br>
    </div>
    <blockquote type="cite"
      cite="mid:A717A0A7-75F2-4064-935C-A5AA8589F294@gmail.com">
      <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
      <meta name="Generator"
        content="Microsoft Word 15 (filtered medium)">
      <style>@font-face
        {font-family:"Cambria Math";
        panose-1:2 4 5 3 5 4 6 3 2 4;}@font-face
        {font-family:Calibri;
        panose-1:2 15 5 2 2 2 4 3 2 4;}p.MsoNormal, li.MsoNormal, div.MsoNormal
        {margin:0in;
        font-size:12.0pt;
        font-family:"Calibri",sans-serif;
        mso-ligatures:standardcontextual;}span.EmailStyle17
        {mso-style-type:personal-compose;
        font-family:"Calibri",sans-serif;
        color:windowtext;}.MsoChpDefault
        {mso-style-type:export-only;
        font-size:12.0pt;}div.WordSection1
        {page:WordSection1;}</style>
      <div class="WordSection1">
        <p class="MsoNormal"><span style="font-size:11.0pt">Hi all,<o:p></o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt">I was
            converting the overture maps parquet data to geopackage and
            got this error using gdal master (via conda).<o:p></o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt">GDAL_NUM_THREADS=ALL_CPUS
            CPL_TMPDIR=/data2 ogr2ogr -f gpkg
            /data/overture_buildings.gpkg
            /vsis3/overturemaps-us-west-2/release/2024-07-22.0/theme=buildings
            "theme=buildings"  -progress  -nlt PROMOTE_TO_MULTI -nln
            buildings -skipfailures<o:p></o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt">ERROR 1:
            ReadNext() failed: Couldn't deserialize thrift:
            TProtocolException: Exceeded size limit<o:p></o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt">Not sure
            what size limit it exceeded? I’ve had this happen several
            times but at very different points in the process. <o:p></o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt">Ideas?<o:p></o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt">Mike<o:p></o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
        <p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
        <p class="MsoNormal"><span
            style="font-size:11.0pt;mso-ligatures:none">-- <o:p></o:p></span></p>
        <div>
          <p class="MsoNormal"><span
              style="font-size:11.0pt;mso-ligatures:none">Michael Smith<o:p></o:p></span></p>
          <p class="MsoNormal"><span
              style="font-size:11.0pt;mso-ligatures:none">US Army Corps
              of Engineers<o:p></o:p></span></p>
          <p class="MsoNormal"><span
              style="font-size:11.0pt;mso-ligatures:none">Remote
              Sensing/GIS Center</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
        </div>
        <p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
      </div>
      <br>
      <fieldset class="moz-mime-attachment-header"></fieldset>
      <pre class="moz-quote-pre" wrap="">_______________________________________________
gdal-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a>
<a class="moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/gdal-dev">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a>
</pre>
    </blockquote>
    <pre class="moz-signature" cols="72">-- 
<a class="moz-txt-link-freetext" href="http://www.spatialys.com">http://www.spatialys.com</a>
My software is free, but my time generally not.</pre>
  </body>
</html>