[gdal-dev] ogr2ogr : limit memory usage

[Even Rouault]

On vendredi 13 octobre 2017 12:43:29 CEST Even Rouault wrote:
> On vendredi 13 octobre 2017 10:06:56 CEST Matthieu Lefort wrote:
> > Hi list,
> > I am using ogr2ogr to load big geojson files (about 1 Go each) into
> > postgresql database. On my MacBook, (GDAL 1.11.3, released 2015/09/16),
> > although it lasts about 20 minutes for each file, this works fine. I tried
> > to do the same thing on a VPS (Debian Stretch 9.2, 8 Go RAM) but the
> > process consumes all RAM in 20 seconds before stopping (out of memory).
> > Install was made with apt-get install gdal-bin. (GDAL 2.1.2, released
> > 2016/10/24)
> > 
> > Is there a way to control memory usage or to avoid at least using more
> > memory than the available one ?
> 
> Matthieu,
> 
> no, this is a known issue ( see https://trac.osgeo.org/gdal/ticket/6540 )

Hi,

I've modified the GeoJSON driver in trunk to be able to ingest arbitrary large 
files with neglectable RAM consumption. Testing appreciated as this is a non 
trivial change.

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com