[gdal-dev] gdal vector concat fails with `Too many open files` but ogrmerge works fine

[Even Rouault]

Robin,

this was an oversight of the implementation of gdal vector concat. Fixed 
per https://github.com/OSGeo/gdal/pull/12856

Apart from using ogrmerge, a workaround is to increate the limit of open 
files with "ulimit -n some_number"

Even

Le 04/08/2025 à 13:19, Robin Wilson via gdal-dev a écrit :
> Hi,
>
> I’ve just started experimenting with the new gdal command line tools.
>
> I’ve tried the following command to merge a few thousand GeoJSON files 
> into one GeoPackage:
>
> gdal vector concat --mode=single gb-postcodes-v5/units/*.geojson 
> all_unit_postcodes.gpkg
>
> I get thousands of errors like:
>
> ERROR 4: gb-postcodes-v5/units/KY4.geojson: Too many open files
>
> Switching back to ogrmerge, I find that this command works fine:
>
> ogrmerge -single -progress -o all_unit_postcodes_v2.gpkg 
> gb-postcodes-v5/units/*.geojson
>
> Has there been a change which means that gdal vector concat tries to 
> open all the files in parallel, and then runs out of spare file 
> handles? Is there any way around this for the new gdal vector concat 
> command?
>
> Best regards,
>
> Robin
>
> Dr Robin Wilson
> www.rtwilson.com
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev

-- 
http://www.spatialys.com
My software is free, but my time generally not.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20250804/3565c982/attachment.htm>