<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">OK, I’ll try it. Thanks for theses informations.<div class="">Matthieu</div><div class=""><br class=""><div><blockquote type="cite" class=""><div class="">Le 13 oct. 2017 à 12:43, Even Rouault <<a href="mailto:even.rouault@spatialys.com" class="">even.rouault@spatialys.com</a>> a écrit :</div><br class="Apple-interchange-newline"><div class=""><div style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class="">On vendredi 13 octobre 2017 10:06:56 CEST Matthieu Lefort wrote:</div><div style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class="">> Hi list,</div><div style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class="">> I am using ogr2ogr to load big geojson files (about 1 Go each) into</div><div style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class="">> postgresql database. On my MacBook, (GDAL 1.11.3, released 2015/09/16),</div><div style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class="">> although it lasts about 20 minutes for each file, this works fine. I tried</div><div style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class="">> to do the same thing on a VPS (Debian Stretch 9.2, 8 Go RAM) but the</div><div style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class="">> process consumes all RAM in 20 seconds before stopping (out of memory).</div><div style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class="">> Install was made with apt-get install gdal-bin. (GDAL 2.1.2, released</div><div style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class="">> 2016/10/24)</div><div style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class="">> </div><div style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class="">> Is there a way to control memory usage or to avoid at least using more</div><div style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class="">> memory than the available one ?</div><p style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class=""> </p><div style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class="">Matthieu,</div><p style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class=""> </p><div style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class="">no, this is a known issue ( see <a href="https://trac.osgeo.org/gdal/ticket/6540" class="">https://trac.osgeo.org/gdal/ticket/6540</a> ) which would require substantial changes in the geojson driver to be adressed. It needs to ingest the whole file in memory currently, parse it completely, and store the resulting features in RAM. So typically, you need 10 to 20 times more RAM than the file size (I wouldn't expect the memory usage to be substantially different between GDAL 1.11 and 2.1, but there has been indeed a small change to use the MEM driver underneath)</div><p style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class=""> </p><div style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class="">So for now, your best chance is to try splitting the features[] array into smaller files.</div><p style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class=""> </p><div style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class=""><a href="https://www.google.fr/search?q=splitting+geojson+files" class="">https://www.google.fr/search?q=splitting+geojson+files</a> points to various hints on how to split geojson files.</div><p style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class=""> </p><div style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class="">Even</div><p style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class=""> </p><p style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class=""> </p><div style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class="">-- </div><div style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class="">Spatialys - Geospatial professional services</div><div style="white-space: pre-wrap; font-family: 'Sans Serif'; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px;" class=""><a href="http://www.spatialys.com/" class="">http://www.spatialys.com</a></div></div></blockquote></div><br class=""></div></body></html>