<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Angus,</p>
<p>there has been a recent extra fix that landed in GDAL 3.6.2 that
might possibly help: <a class="moz-txt-link-freetext" href="https://github.com/OSGeo/gdal/pull/6926">https://github.com/OSGeo/gdal/pull/6926</a></p>
<p>Even</p>
<div class="moz-cite-prefix">Le 25/01/2023 à 01:36, Angus Dickey a
écrit :<br>
</div>
<blockquote type="cite"
cite="mid:CABADqykJDO05X35BZsyn2BpZRoBd3_knvmrxUc-hbXDNLp_XbQ@mail.gmail.com">
<div dir="ltr">Hi all,
<div><br>
</div>
<div>I am running into an issue where GDAL is overestimating the
amount of physical memory it has leading to it locking up the
OS by taking 100% of the memory. Here is an example program
that illustrates the issue:<br>
<br>
#include <stdio.h><br>
#include "gdal.h"<br>
<br>
int main(void) {<br>
printf("GDAL version is %s\n",
GDALVersionInfo("RELEASE_NAME"));<br>
printf("GDAL thinks is has %lld bytes of physical
memory\n", CPLGetPhysicalRAM());<br>
printf("GDAL thinks it has %lld bytes of usable physical
memory\n", CPLGetUsablePhysicalRAM());<br>
return 0;<br>
}<br>
</div>
<div><br>
</div>
<div>When this is compiled with GDAL 3.5.1 on Ubuntu 22.04 we
get:<br>
</div>
<div><br>
</div>
<div>$ ./get_gdal_memory <br>
GDAL version is 3.5.1<br>
GDAL thinks is has 811526475776 bytes of physical memory<br>
GDAL thinks it has 811526475776 bytes of usable physical
memory<br>
<br>
Which is not consistent with the actual available memory:</div>
<div><br>
$ free -h<br>
total used free shared
buff/cache available<br>
Mem: 2.0Gi 148Mi 1.2Gi 0.0Ki
639Mi 1.8Gi<br>
Swap: 256Mi 0B 256Mi<br>
</div>
<div><br>
</div>
<div>So GDAL thinks it has 755GB of memory but it only has 2GB,
this causes issues with the raster read cache and maybe
elsewhere. I suspect this is happening because it is running
in a <a href="https://linuxcontainers.org/"
moz-do-not-send="true">Linux container</a> and GDAL is
getting the total physical memory of the host, not the
container. The strange thing is Linux containers use cgroups
for memory restrictions and the API docs <a
href="https://gdal.org/api/cpl.html#_CPPv417CPLGetPhysicalRAMv"
moz-do-not-send="true">mention it was fixed in GDAL 2.4.0</a> but
I am still seeing the issue in 3.5.1.</div>
<div><br>
</div>
<div>Any help or insight would be appreciated; I am happy to
provide any additional information or testing.</div>
<div><br>
</div>
<div>Thanks,</div>
<div><br>
</div>
<div>Angus</div>
</div>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
gdal-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a>
<a class="moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/gdal-dev">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a>
</pre>
</blockquote>
<pre class="moz-signature" cols="72">--
<a class="moz-txt-link-freetext" href="http://www.spatialys.com">http://www.spatialys.com</a>
My software is free, but my time generally not.</pre>
</body>
</html>