[gdal-dev] Does writing GeoJSON need to be so slow?

[Rahkonen Jukka]

Hi,

I was comparing some alternative scenarios for data exports, and I was a bit surprised when I noticed that GeoJSON output from ogr2ogr is really slow.
I used these lake polygons as test data https://wwwd3.ymparisto.fi/d3/gis_data/spesific/ranta10jarvet.zip and I tested on Windows with GDAL 3.11.0dev-181b6b9991, released 2024/11/21.

I was thinking that maybe it is slow to write JSON just because it is text based format so I made tests also with other text formats (GML, MapInfo MIF, and CSV). My commands and timings:

ogr2ogr -f geojson lakes.json jarvi10.shp --config cpl_debug on --config cpl_timestamp on
220 sec - 1000 features/sec
ogr2ogr -f "mapinfo file" lakes.mif jarvi10.shp --config cpl_debug on --config cpl_timestamp on
110 sec - 2000 features/sec
ogr2ogr -f gml lakes.gml jarvi10.shp --config cpl_debug on --config cpl_timestamp on
92 sec - 2300 features/sec
ogr2ogr -f csv lakes.csv jarvi10.shp -lco geometry=as_wkt --config cpl_debug on --config cpl_timestamp on
77 sec - 2800 featurs/sec

Then I pondered if I know any other tools for exporting GeoJSON, and SpatiaLite came into my mind. ExportGeoJSON https://www.gaia-gis.it/gaia-sins/spatialite-sql-5.1.0.html from GeoPackage into GeoJSON file was 4 times faster than ogr2ogr.
select exportgeojson('vgpkg_jarvi10','geom','c:\data\jarvet\fromspatialite.json');
54 sec - 4000 features/sec

For calibrating the speedometer, I converted data also from shapefile into GeoPackage
ogr2ogr -f gpkg lakes.gpkg jarvi10.shp --config cpl_debug on --config cpl_timestamp on
12 sec - 18000 features/sec

I made also a couple of tests with geojsonseq output but I did not notice much difference. Does writing GeoJSON require some tricks that other formats do not require, or why it is so slow?

-Jukka Rahkonen-
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20241128/35f6c648/attachment.htm>