<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Michael,</p>
<p>are you sure this is the *same* schema ? From the error messages
(which comes from libarrow-compute itself), it would seem there's
a mix of timestamp in microsecond and timestamp in nanosecond, and
that the algorithm in libarrow tries to homogenize things and an
overflow occur. If 237718454400000000 is a timestamp in
microseconds, that corresponds to January 1st 9503... I doubt
the SQLite dialect plays any role in that.<br>
</p>
<p>Even<br>
</p>
<div class="moz-cite-prefix">Le 26/07/2025 à 20:56, Michael Smith
via gdal-dev a écrit :<br>
</div>
<blockquote type="cite"
cite="mid:2BF8CE04-F9CE-4525-B5A9-18C31B26EDB7@gmail.com">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="Generator"
content="Microsoft Word 15 (filtered medium)">
<style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}@font-face
{font-family:Aptos;
panose-1:2 11 0 4 2 2 2 2 2 4;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ligatures:standardcontextual;}span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}.MsoChpDefault
{mso-style-type:export-only;}div.WordSection1
{page:WordSection1;}</style>
<div class="WordSection1">
<p class="MsoNormal">I have a collection of parquet files all
with the same schema, different stac collections written using
geopandas to parquet. <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">When I query at the cli or in python for a
directory of parquert files using sql I get timestamp casting
errors<br>
<br>
<o:p></o:p></p>
<p class="MsoNormal">gdal vector info -i
PARQUET:s3://mybucket/stac/mds/rasters/ --sql "select * from
'rasters' where st_intersects(geometry,
st_geomfromtext('POLYGON ((-68.00948853933728
17.7602787370086, -64.99052950907739 17.7602787370086,
-64.99052950907739 18.6509945435268, -68.00948853933728
18.6509945435268, -68.00948853933728 17.7602787370086))'))"
--dialect sqlite -f text<o:p></o:p></p>
<p class="MsoNormal">INFO: Open of
`PARQUET:s3://grid-dev-publiclidar/stac/mds/rasters/'<o:p></o:p></p>
<p class="MsoNormal"> using driver `Parquet' successful.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Layer name: SELECT<o:p></o:p></p>
<p class="MsoNormal">Geometry: Polygon<o:p></o:p></p>
<p class="MsoNormal">ERROR 1: ReadNext() failed: Casting from
timestamp[us, tz=UTC] to timestamp[ns, tz=UTC] would result in
out of bounds timestamp: 237718454400000000<o:p></o:p></p>
<p class="MsoNormal">Feature Count: 34<o:p></o:p></p>
<p class="MsoNormal">ERROR 1: ReadNext() failed: Casting from
timestamp[us, tz=UTC] to timestamp[ns, tz=UTC] would result in
out of bounds timestamp: 237718454400000000<o:p></o:p></p>
<p class="MsoNormal">Extent: (-180.000000, -90.000000) -
(180.000000, 83.999167)<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">If I do it file by file for all the parquet
in a directory, I don’t get an error. <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Is this a bug or a problem with sqlite
dialect and parquet?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal"><span
style="font-family:"Aptos",sans-serif;mso-ligatures:none">-- <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-family:"Aptos",sans-serif;mso-ligatures:none">Michael
Smith<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-family:"Aptos",sans-serif;mso-ligatures:none">RSGIS
Center – ERDC CRREL NH<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-family:"Aptos",sans-serif;mso-ligatures:none">US
Army Corps</span><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
gdal-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a>
<a class="moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/gdal-dev">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a>
</pre>
</blockquote>
<pre class="moz-signature" cols="72">--
<a class="moz-txt-link-freetext" href="http://www.spatialys.com">http://www.spatialys.com</a>
My software is free, but my time generally not.</pre>
</body>
</html>