[gdal-dev] ogr2ogr : limit memory usage

[Kurt Schwehr]

Added some small fuzzer corpus test cases that cause me OOM on a 2GB
instance:

https://trac.osgeo.org/gdal/ticket/6540#comment:5

On Tue, Oct 17, 2017 at 3:05 AM, Even Rouault <even.rouault at spatialys.com>
wrote:

> On vendredi 13 octobre 2017 12:43:29 CEST Even Rouault wrote:
> > On vendredi 13 octobre 2017 10:06:56 CEST Matthieu Lefort wrote:
> > > Hi list,
> > > I am using ogr2ogr to load big geojson files (about 1 Go each) into
> > > postgresql database. On my MacBook, (GDAL 1.11.3, released 2015/09/16),
> > > although it lasts about 20 minutes for each file, this works fine. I
> tried
> > > to do the same thing on a VPS (Debian Stretch 9.2, 8 Go RAM) but the
> > > process consumes all RAM in 20 seconds before stopping (out of memory).
> > > Install was made with apt-get install gdal-bin. (GDAL 2.1.2, released
> > > 2016/10/24)
> > >
> > > Is there a way to control memory usage or to avoid at least using more
> > > memory than the available one ?
> >
> > Matthieu,
> >
> > no, this is a known issue ( see https://trac.osgeo.org/gdal/ticket/6540
> )
>
> Hi,
>
> I've modified the GeoJSON driver in trunk to be able to ingest arbitrary
> large
> files with neglectable RAM consumption. Testing appreciated as this is a
> non
> trivial change.
>
> Even
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev




-- 
--
http://schwehr.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20171017/e7670cee/attachment-0001.html>