[gdal-dev] Ogrinfo , ogr2ogr file size limits

Tue May 17 02:58:46 PDT 2022

Hi,

Most likely, Frando is talking about the "Interlis" format - a format 
common in Switzerland (and also Colombia, by export from Switzerland).

see https://gdal.org/drivers/vector/ili.html

Since this involves Java ili2c, you should set the corresponding XmX 
memory setting for the Java compiler, which is memory limited by 
default.

There is also an alternative XTF import export tool available, called 
il2db (ili2pg, ili2ora, ili2gpkg, ili2fgdb). See 
https://github.com/claeis/ili2db/blob/master/docs/ili2db.rst

I generally recommend to use ili2db versions rather than the ogr 
interlis drive. The ili2db is way more tested and flexible with loads of 
options.

Greetings,

Andreas

On 2022-05-17 11:53, Joaquim Manuel Freire Luís wrote:

> xtf is the extension by a format of sidescan sonar data.
> 
> This one was picked randomly from a google search
> 
> https://www.usna.edu/Users/oceano/pguth/md_help/html/sidescan.htm
> 
> From: gdal-dev <gdal-dev-bounces at lists.osgeo.org> On Behalf Of Even 
> Rouault
> Sent: Tuesday, May 17, 2022 10:43 AM
> To: Gonzalez Velasquez, Frando Alexis <fagonzalezv at indracompany.com>; 
> gdal-dev at lists.osgeo.org
> Cc: Serrano Granados, Yency <yserrano at indracompany.com>
> Subject: Re: [gdal-dev] Ogrinfo , ogr2ogr file size limits
> 
> Frando,
> 
> The .xtf extension doens't ring a bell to me. Which OGR driver 
> recognizes such file ? (remove the -q option and look at the beginning 
> of the report of ogrinfo), and what is your GDAL version ?
> 
> Most OGR drivers work in streaming mode and can handle arbitrary big 
> files, but a few ones have no other choice than ingesting the whole 
> file, in which case there's no workaround apart splitting the file with 
> other tools or buy more RAM
> 
> Even
> 
> Le 17/05/2022 à 04:21, Gonzalez Velasquez, Frando Alexis a écrit :
> 
>> Hi dev team!
>> 
>> I am writing to inquire about big files problem,
>> 
>> We are using the ogrinfo and ogr2ogr executables over small files with 
>> success (1 gb max)  but with 10 gb files we are experiencing memory 
>> problems and and it does not work, is there a file size limit? And if 
>> there is a size limit, is there any strategy to handle this problema 
>> and process big files with success ?
>> 
>> For example:
>> 
>> _ogrinfo -q -so C:\\tmp\\file.xtf _
>> 
>> When file is 1Gb success ; but when  the file is 10gb size the process 
>> fails.
>> 
>> Any advice would be appreciated, thanks.
>> 
>> FRANDO GONZÁLEZ
>> 
>> Senior Systems Engineer
>> 
>> fagonzalezv at indracompany.com
>> 
>> Calle 93 16 25
>> 
>> Bogotá, Colombia
>> 
>> T +57 1 646 36 00
>> 
>> M +57 3006604908
>> 
>> minsait.com [1]
>> 
>> indracompany.com [2]
>> 
>> -------------------------
>> 
>> Este correo electrónico y, en su caso, cualquier fichero anexo al 
>> mismo, contiene información de carácter confidencial exclusivamente 
>> dirigida a su destinatario o destinatarios. Si no es vd. el 
>> destinatario indicado, queda notificado que la lectura, utilización, 
>> divulgación y/o copia sin autorización está prohibida en virtud de la 
>> legislación vigente. En el caso de haber recibido este correo 
>> electrónico por error, se ruega notificar inmediatamente esta 
>> circunstancia mediante reenvío a la dirección electrónica del 
>> remitente.
>> Evite imprimir este mensaje si no es estrictamente necesario.
>> 
>> This email and any file attached to it (when applicable) contain(s) 
>> confidential information that is exclusively addressed to its 
>> recipient(s). If you are not the indicated recipient, you are informed 
>> that reading, using, disseminating and/or copying it without 
>> authorisation is forbidden in accordance with the legislation in 
>> effect. If you have received this email by mistake, please immediately 
>> notify the sender of the situation by resending it to their email 
>> address.
>> Avoid printing this message if it is not absolutely necessary.
>> 
>> _______________________________________________
>> 
>> gdal-dev mailing list
>> 
>> gdal-dev at lists.osgeo.org
>> 
>> https://lists.osgeo.org/mailman/listinfo/gdal-dev
> 
> --
> 
> http://www.spatialys.com
> 
> My software is free, but my time generally not.
> 
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev

Links:
------
[1] 
https://eur04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.minsait.com%2F&data=04%7C01%7Cfagonzalezv%40indracompany.com%7C6c419c97a7134841c0da08d9f7ce4282%7C7808e00514894374954bd3b08f193920%7C0%7C0%7C637813287514711623%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=0Jr%2BkI3f3Uh3g%2FG5C5yElQX9pHauhEi9ds6w%2FPOxLtM%3D&reserved=0
[2] 
https://eur04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.indracompany.com%2F&data=04%7C01%7Cfagonzalezv%40indracompany.com%7C6c419c97a7134841c0da08d9f7ce4282%7C7808e00514894374954bd3b08f193920%7C0%7C0%7C637813287514711623%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=W2xsw4RMlZ8TUymKMTE5zqosvKN2LV5QhwvGmAxgZM0%3D&reserved=0
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220517/46541e93/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 1476 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220517/46541e93/attachment.png>