[gdal-dev] geojson size matters?

[Even Rouault]

Le samedi 01 octobre 2016 21:47:59, Jaak Laineste a écrit :
> I have a big geojson (4.83G) and try to convert it using ogr2ogr (GDAL
> 2.1.1, released 2016/07/07, Mac), but after 6 minutes of processing it
> terminates with:
> 
> ERROR 1: GeoJSON parsing error: continue (at offset -1)

This would seem to suggest something invalid in the GeoJSON, but this might be 
an artifact of an out of memory situation.

> ERROR 4: Failed to read GeoJSON data
> FAILURE:
> Unable to open datasource `geonames.geojson’ with the following drivers.
> …
> 
> Also I was not able to open same file with QGIS (it tried to consume 50+G
> RAM and crashed). The file is here:
> https://dl.dropboxusercontent.com/u/3573333/tmp/geonames.geojson.bz2
> 
> I have similar smaller file from same proces, with size just below 2G and
> that one works fine. So it looks like some 2G or 4G size issue.
> 
> Is there maximum size for readable GeoJSON files?

If you have lots of RAM, no. But the current reader typically requires an 
amout of RAM that is at least 10 times or more than the file size.

Known as https://trac.osgeo.org/gdal/ticket/6540

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com