<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.Shkpostityyli22
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 2.0cm 70.85pt 2.0cm;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="FI" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Hi,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Did you ever notice that the result from GDAL 3.5.1 is wrong as well? You csv file has coordinates with a comma as a decimal separator and enclosed between double quotes<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">my_file.csv<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">=========<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">CD_MES,store_id,date_id,ticket_id,hour_id,lat,lon<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">202112,101,04/12/21,1,11,"19,38358","-99,183735"<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">202112,101,25/12/21,31,15,"19,38066","-99,183405"<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">The result that you considered right cuts the decimals<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">"POINT (-99 19)","202112","101",04/12/21,"1","11"<br>
</span><span lang="EN-US" style="mso-fareast-language:EN-US">and I think that it is worse result that not getting coordinates at all.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Editing the csv file to present coordinates as numbers with point as a decimal separator gives correct result<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">my_file.csv<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">=========<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">CD_MES,store_id,date_id,ticket_id,hour_id,lat,lon<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">202112,101,04/12/21,1,11,19.38358,-99.183735<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">202112,101,25/12/21,31,15,19.38066,-99.183405<br>
ogrinfo<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">======<br>
"POINT (-99.183735 19.38358)","202112","101",04/12/21,"1","11"<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">"POINT (-99.183405 19.38066)","202112","101",25/12/21,"31","15"<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">-Jukka Rahkonen-<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b>Lähettäjä:</b> gdal-dev <gdal-dev-bounces@lists.osgeo.org>
<b>Puolesta </b>Rahkonen Jukka<br>
<b>Lähetetty:</b> maanantai 21. marraskuuta 2022 13.07<br>
<b>Vastaanottaja:</b> Moises Calzado <mcalzado@carto.com><br>
<b>Kopio:</b> 'gdal-dev@lists.osgeo.org' (gdal-dev@lists.osgeo.org) <gdal-dev@lists.osgeo.org><br>
<b>Aihe:</b> Re: [gdal-dev] Latest GDAL version not transforming lat/lng into geom column<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Hi,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">I could reproduce with OSGeo4W version GDAL 3.6.0, released 2022/11/06. I have also a bit older version GDAL 3.6.0dev from gisinternals.com, probably installed 2022-05-16, and that
version finds the geometries. So if it is not because of differences in build systems then something has happened after mid of May 2022.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">-Jukka Rahkonen-<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US">Lähettäjä:</span></b><span lang="EN-US"> Moises Calzado <<a href="mailto:mcalzado@carto.com">mcalzado@carto.com</a>>
<br>
<b>Lähetetty:</b> maanantai 21. marraskuuta 2022 11.50<br>
<b>Vastaanottaja:</b> Rahkonen Jukka <<a href="mailto:jukka.rahkonen@maanmittauslaitos.fi">jukka.rahkonen@maanmittauslaitos.fi</a>><br>
<b>Aihe:</b> Re: [gdal-dev] Latest GDAL version not transforming lat/lng into geom column<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal">Hi Jukka,<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Thanks so much for taking a look at this.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">This is the result when running the command in the 3.5.1 version:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<p class="MsoNormal">GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AXIS["Latitude",NORTH],AXIS["Longitude",EAST],AUTHORITY["EPSG","4326"]]<br>
WKT,String,String,String,String,String<br>
geom,CD_MES,store_id,date_id,ticket_id,hour_id<br>
"POINT (-99 19)","202112","101",04/12/21,"1","11"<br>
"POINT (-99 19)","202112","101",25/12/21,"31","15"<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">And this is what I obtained in the 3.6.0 version:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<p class="MsoNormal">GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AXIS["Latitude",NORTH],AXIS["Longitude",EAST],AUTHORITY["EPSG","4326"]]<br>
WKT,String,String,String,String,String<br>
geom,CD_MES,store_id,date_id,ticket_id,hour_id<br>
,"202112","101",04/12/21,"1","11"<br>
,"202112","101",25/12/21,"31","15"<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">That's the file that I've used for testing: <a href="https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstorage.googleapis.com%2Fimport-testing-stream%2Fmy_file.csv&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7C7597a8d011e749c72a0a08dacbb0844e%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C638046256280278841%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=XU3u4Oy0HUhuyLS%2FBuDD6XSWoNzVqNYuv57zudr29yU%3D&reserved=0">https://storage.googleapis.com/import-testing-stream/my_file.csv</a> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">El lun, 21 nov 2022 a las 10:28, Rahkonen Jukka (<<a href="mailto:jukka.rahkonen@maanmittauslaitos.fi">jukka.rahkonen@maanmittauslaitos.fi</a>>) escribió:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Hi,<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Please add a one line test file “my_file.csv” so we can try to reproduce. Confirm first that it fails with your GDAL version.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">-Jukka Rahkonen-</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> </span><o:p></o:p></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>Lähettäjä:</b> gdal-dev <<a href="mailto:gdal-dev-bounces@lists.osgeo.org" target="_blank">gdal-dev-bounces@lists.osgeo.org</a>>
<b>Puolesta </b>Moises Calzado via gdal-dev<br>
<b>Lähetetty:</b> maanantai 21. marraskuuta 2022 11.21<br>
<b>Vastaanottaja:</b> <a href="mailto:gdal-dev@lists.osgeo.org" target="_blank">gdal-dev@lists.osgeo.org</a><br>
<b>Aihe:</b> [gdal-dev] Latest GDAL version not transforming lat/lng into geom column<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Hello everyone,<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">We've just updated to the latest GDAL version (v3.6.0) and it seems that something is not working correctly when trying to obtain a geom column from a CSV containing latitudes and
longitudes.<br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">This is the command that is being used: <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">ogr2ogr -f CSV -skipfailures -makevalid /vsistdout/ CSV:my_file.csv -simplify 0.00001 -dim XY -t_srs EPSG:4326 -lco GEOMETRY=AS_WKT -lco GEOMETRY_NAME=geom -lco CREATE_CSVT=YES
-s_srs EPSG:4326 -oo KEEP_GEOM_COLUMNS=NO -oo X_POSSIBLE_NAMES=point_longitude,longitude,longitud,lon,Lon,Longitude,longitudedecimal,decimallongitude,decimallong,lng,long,Lng -oo Y_POSSIBLE_NAMES=latitude,latitud,lati,lat,Latitude,decimallat,decimallatitude,latitudedecimal,point_latitude
-oo GEOM_POSSIBLE_NAMES=geom,Geom,geometry,the_geom,wkt,wkb,wkt_geometry,wkb_geometry -oo EMPTY_STRING_AS_NULL=YES -oo QUOTED_FIELDS_AS_STRING=NO -lco PRECISION=NO -oo AUTODETECT_SIZE_LIMIT=500000<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">The file that is being processed contains a column called lat and another one called lon, and when I execute the same process on a docker container running the version 3.5.1 of
GDAL it works like a charm. We've also tried to execute this process on the 3.5.3 version, and it also fails. Is that expected?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Looking forward to your response,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Regards!<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">--
<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><strong><span style="font-family:"Calibri",sans-serif">Moises Calzado</span></strong>
<o:p></o:p></p>
<p style="margin-bottom:3.0pt">Support Engineer <o:p></o:p></p>
<p style="margin-bottom:15.0pt">(US) +1 917 463 3232 | (ES) +34 911 165 823 | <a href="mailto:mcalzado@carto.com" target="_blank">
mcalzado@carto.com</a> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><a href="https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.carto.com%2F&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7C7597a8d011e749c72a0a08dacbb0844e%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C638046256280278841%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=1A8HxLvT6%2FV0p9wIm6AReJJebpOPu3t%2Br6WRm6mRhN8%3D&reserved=0" target="_blank"><span style="border:solid windowtext 1.0pt;padding:0cm;text-decoration:none"><img border="0" width="496" height="46" style="width:5.1666in;height:.4791in" id="m_1910996925908232162_x005f_x0000_i1025" src="cid:image001.jpg@01D8FDAB.0635E560" alt="Lähettäjä poisti kuvan."></span></a><o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">-- <o:p></o:p></p>
<div>
<p class="MsoNormal"><strong><span style="font-family:"Calibri",sans-serif">Moises Calzado</span></strong>
<o:p></o:p></p>
<p style="mso-margin-top-alt:3.0pt;margin-right:0cm;margin-bottom:3.0pt;margin-left:0cm">
Support Engineer <o:p></o:p></p>
<p style="mso-margin-top-alt:3.0pt;margin-right:0cm;margin-bottom:15.0pt;margin-left:0cm">
(US) +1 917 463 3232 | (ES) +34 911 165 823 | <a href="mailto:mcalzado@carto.com" target="_blank">
mcalzado@carto.com</a> <o:p></o:p></p>
<p class="MsoNormal"><a href="https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.carto.com%2F&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7C7597a8d011e749c72a0a08dacbb0844e%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C638046256280278841%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=1A8HxLvT6%2FV0p9wIm6AReJJebpOPu3t%2Br6WRm6mRhN8%3D&reserved=0" target="_blank"><span style="border:solid windowtext 1.0pt;padding:0cm;text-decoration:none"><img border="0" width="496" height="46" style="width:5.1666in;height:.4791in" id="Kuva_x0020_2" src="cid:image001.jpg@01D8FDAB.0635E560" alt="Lähettäjä poisti kuvan."></span></a><o:p></o:p></p>
</div>
</div>
</body>
</html>