<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Moises,</p>
<p>the reason is that autodetection by default only considers the
content of the first megabyte of the file, and in the first
megabyte, only integer values that fit on 32 bit integers are
present, hence this data type is selected. But later in the file,
there are values that require a 64 bit integer.<br>
</p>
<p>You can add -oo AUTODETECT_SIZE_LIMIT=0 to mean unlimited size
limit. Unfortunately I just noticed this got broken in 3.5.1,
where AUTODETECT_SIZE_LIMIT=0 would disable any autodetection.
This will be fixed per <a class="moz-txt-link-freetext" href="https://github.com/OSGeo/gdal/pull/6087">https://github.com/OSGeo/gdal/pull/6087</a>. So
if you've 3.5.1, use AUTODETECT_SIZE_LIMIT=-1 instead. Or a large
value in bytes not greater than 2 billion. In the previously
mentioned fix, I've also added a warning if integer values that
don't fit on 32 bit are set.<br>
</p>
<p>Even<br>
</p>
<div class="moz-cite-prefix">Le 18/07/2022 à 15:54, Moises Calzado
via gdal-dev a écrit :<br>
</div>
<blockquote type="cite"
cite="mid:CAObRW4SY1gan0p3sjvP2qH9z7FX1ROsZffbq=dtU-SDfhAsqCQ@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">
<div class="gmail_chip gmail_drive_chip"
style="width:396px;height:18px;max-height:18px;background-color:rgb(245,245,245);padding:5px;font-family:arial;font-weight:bold;font-size:13px;border:1px
solid rgb(221,221,221);line-height:1"><a
href="https://drive.google.com/file/d/11moLc5Shd78iTCKzsHxCKL33xSFHaVva/view?usp=drive_web"
target="_blank"
style="display:inline-block;max-width:366px;overflow:hidden;text-overflow:ellipsis;white-space:nowrap;text-decoration-line:none;padding:1px
0px;border:none" aria-label="test.csv"
moz-do-not-send="true"><img style="vertical-align: bottom;
border: none;"
src="https://ssl.gstatic.com/docs/doclist/images/icon_10_generic_list.png"
moz-do-not-send="true"> <span dir="ltr"
style="vertical-align:bottom;text-decoration:none">test.csv</span></a><img
src="//ssl.gstatic.com/ui/v1/icons/common/x_8px.png"
style="opacity: 0.55; cursor: pointer; float: right;
position: relative; top: -1px; display: none;"
moz-do-not-send="true"></div>
Hello everyone!
<div><br>
</div>
<div>I'm dealing with some issues with ogr2ogr trying to convert
my data to a CSV file guessing the data types. I'm using the
following command:<br>
<br>
<pre style="white-space:pre-wrap;overflow:visible;font-family:monospace,monospace;font-size:13px;line-height:18.2px;margin-top:12px;margin-bottom:14px;padding:0px"><code style="font-family:monospace,monospace;display:block;overflow:visible;padding:14px 18px;border-radius:2px;line-height:18.2px;border:none;background:none">ogr2ogr -f CSV -skipfailures -makevalid /vsistdout/ CSV:test.<span style="background-color:initial">csv -oo AUTODETECT_TYPE=YES -</span>lco<span style="background-color:initial"> CREATE_CSVT=YES > test.csv</span></code><div style="opacity:0"><button id="gmail-m_-8846465243904577121m_-8070599243595118999m_-2561339975320245687m_-3685824568019928664gmail-disable-code-block-wrapping" style="font-family:"Open Sans",Helvetica,Arial,sans-serif;font-style:inherit;font-variant:inherit;font-weight:700;font-stretch:inherit;font-size:12px;line-height:16px;margin:0px;overflow:visible;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial;border-radius:0px 0px 3px 3px;outline:none;padding:3px 8px"><span style="display:inline-block;font-variant-numeric:normal;font-variant-east-asian:normal;font-weight:normal;font-stretch:normal;font-size:14px;line-height:1;font-family:FontAwesome;opacity:0.6;padding:0px;width:13px"></span></button></div></pre>
When I add the AUTODETECT_TYPE=YES flag, ogr2ogr is applying
some weird transformations on the "uprn" column. If you check
the generated output and the original file, you'll notice that
some integers have changed into a negative number. Do you know
if I'm doing something wrong, or if it's a bug? </div>
<div><br>
</div>
<div>Thanks so much for your help,</div>
<div>Regards!</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
-- <br>
<div dir="ltr" class="gmail_signature"
data-smartmail="gmail_signature"> <strong>Moises Calzado</strong>
<p style="margin-bottom:4px;margin-top:4px"> Support Engineer
</p>
<p style="margin-bottom:20px;margin-top:4px"> (US) +1 917 463
3232 | (ES) +34 911 165 823 | <a
href="mailto:mcalzado@carto.com" target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">mcalzado@carto.com</a>
</p>
<a href="https://www.carto.com/" rel="nofollow"
target="_blank" moz-do-not-send="true"> <img
src="https://storage.googleapis.com/carto-it-files/signature/default-signature.png"
moz-do-not-send="true" width="496" height="46"> </a> </div>
</div>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
gdal-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a>
<a class="moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/gdal-dev">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a>
</pre>
</blockquote>
<pre class="moz-signature" cols="72">--
<a class="moz-txt-link-freetext" href="http://www.spatialys.com">http://www.spatialys.com</a>
My software is free, but my time generally not.</pre>
</body>
</html>