<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Ah of course. And the ILI2 driver does ingest the whole file into
memory</p>
<p>ili2c is not used by the driver. It is just a preparation step to
create the .imd file. So that wouldn't help here. But your
alternatives are good hints. Could be worth to extend
<a class="moz-txt-link-freetext" href="https://gdal.org/drivers/vector/ili.html">https://gdal.org/drivers/vector/ili.html</a> to mention them,
especially if they can process files of arbitrary files with
modest RAM usage<br>
</p>
<p>Even<br>
</p>
<div class="moz-cite-prefix">Le 17/05/2022 à 11:58, Andreas Neumann
a écrit :<br>
</div>
<blockquote type="cite"
cite="mid:087548b8ff29df589f4f98a04186b379@carto.net">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<p>Hi,</p>
<p>Most likely, Frando is talking about the "Interlis" format - a
format common in Switzerland (and also Colombia, by export from
Switzerland).</p>
<p>see <a href="https://gdal.org/drivers/vector/ili.html"
moz-do-not-send="true" class="moz-txt-link-freetext">https://gdal.org/drivers/vector/ili.html</a></p>
<p>Since this involves Java ili2c, you should set the
corresponding XmX memory setting for the Java compiler, which is
memory limited by default.</p>
<p>There is also an alternative XTF import export tool available,
called il2db (ili2pg, ili2ora, ili2gpkg, ili2fgdb). See <a
href="https://github.com/claeis/ili2db/blob/master/docs/ili2db.rst"
moz-do-not-send="true" class="moz-txt-link-freetext">https://github.com/claeis/ili2db/blob/master/docs/ili2db.rst</a></p>
<p>I generally recommend to use ili2db versions rather than the
ogr interlis drive. The ili2db is way more tested and flexible
with loads of options.</p>
<p>Greetings,</p>
<p>Andreas</p>
<p id="reply-intro">On 2022-05-17 11:53, Joaquim Manuel Freire
Luís wrote:</p>
<blockquote type="cite" style="padding: 0 0.4em; border-left:
#1010ff 2px solid; margin: 0">
<div id="replybody1">
<style type="text/css">@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}@font-face
{font-family:Georgia;
panose-1:2 4 5 2 5 4 5 2 3 3;}#replybody1 p.v1MsoNormal, #replybody1 li.v1MsoNormal, #replybody1 div.v1MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}#replybody1 a:link, #replybody1 span.v1MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}#replybody1 pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
font-size:10.0pt;
font-family:"Courier New";
mso-fareast-language:PT;}#replybody1 span.v1HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;
mso-fareast-language:EN-US;}#replybody1 span.v1EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}#replybody1 .v1MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}#replybody1 div.v1WordSection1
{page: unset;}</style>
<div style="word-wrap: break-word;" lang="PT">
<div class="v1WordSection1">
<p class="v1MsoNormal"><span>xtf is the extension by a
format of sidescan sonar data.</span></p>
<p class="v1MsoNormal"><span> </span></p>
<p class="v1MsoNormal"><span>This one was picked randomly
from a google search </span></p>
<p class="v1MsoNormal"><span><a
href="https://www.usna.edu/Users/oceano/pguth/md_help/html/sidescan.htm"
target="_blank" rel="noopener noreferrer"
moz-do-not-send="true" class="moz-txt-link-freetext">https://www.usna.edu/Users/oceano/pguth/md_help/html/sidescan.htm</a></span></p>
<p class="v1MsoNormal"><span> </span></p>
<div>
<div style="border: none; border-top: solid #E1E1E1
1.0pt; padding: 3.0pt 0in 0in 0in;">
<p class="v1MsoNormal"><strong><span
style="mso-fareast-language: PT;">From:</span></strong><span
style="mso-fareast-language: PT;"> gdal-dev
<a class="moz-txt-link-rfc2396E" href="mailto:gdal-dev-bounces@lists.osgeo.org"><gdal-dev-bounces@lists.osgeo.org></a> <strong>On
Behalf Of </strong>Even Rouault<br>
<strong>Sent:</strong> Tuesday, May 17, 2022 10:43
AM<br>
<strong>To:</strong> Gonzalez Velasquez, Frando
Alexis <a class="moz-txt-link-rfc2396E" href="mailto:fagonzalezv@indracompany.com"><fagonzalezv@indracompany.com></a>;
<a class="moz-txt-link-abbreviated" href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a><br>
<strong>Cc:</strong> Serrano Granados, Yency
<a class="moz-txt-link-rfc2396E" href="mailto:yserrano@indracompany.com"><yserrano@indracompany.com></a><br>
<strong>Subject:</strong> Re: [gdal-dev] Ogrinfo ,
ogr2ogr file size limits</span></p>
</div>
</div>
<p class="v1MsoNormal"> </p>
<p>Frando,<span style="mso-fareast-language: PT;"></span></p>
<p>The .xtf extension doens't ring a bell to me. Which OGR
driver recognizes such file ? (remove the -q option and
look at the beginning of the report of ogrinfo), and
what is your GDAL version ?</p>
<p>Most OGR drivers work in streaming mode and can handle
arbitrary big files, but a few ones have no other choice
than ingesting the whole file, in which case there's no
workaround apart splitting the file with other tools or
buy more RAM</p>
<p>Even</p>
<div>
<p class="v1MsoNormal">Le 17/05/2022 à 04:21, Gonzalez
Velasquez, Frando Alexis a écrit :</p>
</div>
<blockquote style="margin-top: 5.0pt; margin-bottom:
5.0pt;">
<p class="v1MsoNormal"><span>Hi dev team!</span></p>
<p class="v1MsoNormal"><span> </span></p>
<p class="v1MsoNormal"><span>I am writing to inquire
about big files problem,</span></p>
<p class="v1MsoNormal"><span> </span></p>
<p class="v1MsoNormal"><span>We are using the ogrinfo
and ogr2ogr executables over small files with
success (1 gb max) but with 10 gb files we are
experiencing memory problems and and it does not
work, is there a file size limit? And if there is a
size limit, is there any strategy to handle this
problema and process big files with success ? </span></p>
<p class="v1MsoNormal"><span> </span></p>
<p class="v1MsoNormal"><span>For example:</span></p>
<p class="v1MsoNormal"><span> </span></p>
<p class="v1MsoNormal"><strong><em><span>ogrinfo -q -so
C:\\tmp\\file.xtf </span> </em></strong></p>
<p class="v1MsoNormal"><span> </span></p>
<p class="v1MsoNormal"><span>When file is 1Gb success ;
but when the file is 10gb size the process fails.</span></p>
<p class="v1MsoNormal"><span> </span></p>
<p class="v1MsoNormal"><span> </span></p>
<p class="v1MsoNormal"><span>Any advice would be
appreciated, thanks.</span></p>
<p class="v1MsoNormal"><span> </span></p>
<p class="v1MsoNormal"><span> </span></p>
<p class="v1MsoNormal"><span> </span></p>
<p class="v1MsoNormal"><span style="color: #004254;
mso-fareast-language: ES-CO;"><img
id="v1Imagen_x0020_10" style="width: .75in;
height: .4458in;"
src="cid:part1.BM0QDAvb.KZQRP7Sj@spatialys.com"
alt="minsait-indra" class="" width="72"
height="43" border="0"></span></p>
<p class="v1MsoNormal" style="line-height: 12.0pt;"><span
style="font-size: 10.0pt; font-family:
'Georgia',serif; color: #004254;
mso-fareast-language: ES-CO;"> </span></p>
<p class="v1MsoNormal" style="line-height: 12.0pt;"><span
style="font-size: 10.0pt; font-family:
'Georgia',serif; color: #004254;
mso-fareast-language: ES-CO;">FRANDO GONZÁLEZ</span></p>
<p class="v1MsoNormal" style="mso-line-height-alt:
8.25pt;"><span style="font-size: 9.0pt; font-family:
'Arial',sans-serif; color: #999999;
mso-fareast-language: ES-CO;">Senior Systems
Engineer</span></p>
<p class="v1MsoNormal" style="mso-line-height-alt:
8.25pt;"><span style="font-size: 9.0pt; font-family:
'Arial',sans-serif; color: #999999;
mso-fareast-language: ES-CO;"><a
href="mailto:fagonzalezv@indracompany.com"
rel="noreferrer" moz-do-not-send="true"><span
style="color: #0563c1;">fagonzalezv@indracompany.com</span></a>
</span></p>
<p class="v1MsoNormal" style="line-height: 8.25pt;"><span
style="font-size: 8.0pt; font-family:
'Arial',sans-serif; color: #004254;
mso-fareast-language: ES-CO;"> </span></p>
<p class="v1MsoNormal" style="line-height: 8.25pt;"><span
style="font-size: 8.0pt; font-family:
'Arial',sans-serif; color: #004254;
mso-fareast-language: ES-CO;">Calle 93 16 25</span></p>
<p class="v1MsoNormal" style="line-height: 8.25pt;"><span
style="font-size: 8.0pt; font-family:
'Arial',sans-serif; color: #004254;
mso-fareast-language: ES;">Bogotá, Colombia</span></p>
<p class="v1MsoNormal" style="line-height: 8.25pt;"><span
style="font-size: 8.0pt; font-family:
'Arial',sans-serif; color: #004254;
mso-fareast-language: ES;">T +57 1 646 36 00</span></p>
<p class="v1MsoNormal" style="line-height: 8.25pt;"><span
style="font-size: 8.0pt; font-family:
'Arial',sans-serif; color: #004254;
mso-fareast-language: ES;">M +57 3006604908</span></p>
<p class="v1MsoNormal" style="line-height: 8.25pt;"><span
style="font-size: 8.0pt; font-family:
'Arial',sans-serif; color: #004254;
mso-fareast-language: ES-CO;"> </span></p>
<p class="v1MsoNormal" style="line-height: 8.25pt;"><span
style="font-size: 8.0pt; font-family:
'Arial',sans-serif; color: #004254;
mso-fareast-language: ES-CO;"><a
href="https://eur04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.minsait.com%2F&data=04%7C01%7Cfagonzalezv%40indracompany.com%7C6c419c97a7134841c0da08d9f7ce4282%7C7808e00514894374954bd3b08f193920%7C0%7C0%7C637813287514711623%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=0Jr%2BkI3f3Uh3g%2FG5C5yElQX9pHauhEi9ds6w%2FPOxLtM%3D&reserved=0"
target="_blank" rel="noopener noreferrer"
moz-do-not-send="true"><span style="color:
#999999;">minsait.com</span></a></span></p>
<p class="v1MsoNormal"><span style="font-size: 8.0pt;
font-family: 'Arial',sans-serif; color: #004254;
mso-fareast-language: ES-CO;"><a
href="https://eur04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.indracompany.com%2F&data=04%7C01%7Cfagonzalezv%40indracompany.com%7C6c419c97a7134841c0da08d9f7ce4282%7C7808e00514894374954bd3b08f193920%7C0%7C0%7C637813287514711623%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=W2xsw4RMlZ8TUymKMTE5zqosvKN2LV5QhwvGmAxgZM0%3D&reserved=0"
target="_blank" rel="noopener noreferrer"
moz-do-not-send="true"><span style="color:
#999999;">indracompany.com</span></a></span></p>
<p class="v1MsoNormal" style="line-height: 8.25pt;"><span
style="font-size: 8.0pt; font-family:
'Arial',sans-serif; color: #004254;
mso-fareast-language: ES-CO;"> </span></p>
<p class="v1MsoNormal"> </p>
<p class="v1MsoNormal"><span
style="mso-fareast-language: PT;"> </span></p>
<div class="v1MsoNormal" style="text-align: center;"
align="center">
<hr width="100%" size="2" align="center"></div>
<p class="v1MsoNormal"><span style="font-size: 7.5pt;
font-family: 'Arial',sans-serif; color: gray;
mso-fareast-language: PT;"><br>
Este correo electrónico y, en su caso, cualquier
fichero anexo al mismo, contiene información de
carácter confidencial exclusivamente dirigida a su
destinatario o destinatarios. Si no es vd. el
destinatario indicado, queda notificado que la
lectura, utilización, divulgación y/o copia sin
autorización está prohibida en virtud de la
legislación vigente. En el caso de haber recibido
este correo electrónico por error, se ruega
notificar inmediatamente esta circunstancia mediante
reenvío a la dirección electrónica del remitente.<br>
Evite imprimir este mensaje si no es estrictamente
necesario.<br>
<br>
This email and any file attached to it (when
applicable) contain(s) confidential information that
is exclusively addressed to its recipient(s). If you
are not the indicated recipient, you are informed
that reading, using, disseminating and/or copying it
without authorisation is forbidden in accordance
with the legislation in effect. If you have received
this email by mistake, please immediately notify the
sender of the situation by resending it to their
email address.<br>
Avoid printing this message if it is not absolutely
necessary.<br>
</span><span style="mso-fareast-language: PT;"><br>
<br>
</span></p>
<pre>_______________________________________________</pre>
<pre>gdal-dev mailing list</pre>
<pre><a href="mailto:gdal-dev@lists.osgeo.org" rel="noreferrer" moz-do-not-send="true" class="moz-txt-link-freetext">gdal-dev@lists.osgeo.org</a></pre>
<pre><a href="https://lists.osgeo.org/mailman/listinfo/gdal-dev" target="_blank" rel="noopener noreferrer" moz-do-not-send="true" class="moz-txt-link-freetext">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a></pre>
</blockquote>
<pre>-- </pre>
<pre><a href="http://www.spatialys.com" target="_blank" rel="noopener noreferrer" moz-do-not-send="true" class="moz-txt-link-freetext">http://www.spatialys.com</a></pre>
<pre>My software is free, but my time generally not.</pre>
</div>
</div>
</div>
<br>
<div class="pre" style="margin: 0; padding: 0; font-family:
monospace">_______________________________________________<br>
gdal-dev mailing list<br>
<a href="mailto:gdal-dev@lists.osgeo.org"
moz-do-not-send="true" class="moz-txt-link-freetext">gdal-dev@lists.osgeo.org</a><br>
<a href="https://lists.osgeo.org/mailman/listinfo/gdal-dev"
target="_blank" rel="noopener noreferrer"
moz-do-not-send="true" class="moz-txt-link-freetext">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a></div>
</blockquote>
<p><br>
</p>
</blockquote>
<pre class="moz-signature" cols="72">--
<a class="moz-txt-link-freetext" href="http://www.spatialys.com">http://www.spatialys.com</a>
My software is free, but my time generally not.</pre>
</body>
</html>