<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Aptos;
panose-1:2 11 0 4 2 2 2 2 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
font-size:10.0pt;
font-family:"Courier New";}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;}
span.EmailStyle22
{mso-style-type:personal-reply;
font-family:"Aptos",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif">I’ve used mimalloc successfully in the past, worth a look if a drop in replacement for new / delete / malloc / free is desirable. Do note that its performance is usually uniformly
superior to glibc / msvc but there are unintuitive performance cliffs. Given the block nature of most gdal raster workloads, I don’t expect them to surface, but fyi.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif">Our allocators only call VAlloc when necessary – we don’t issue a call 1:1 when a user would’ve used malloc. The allocator has an internal state that knows when to call the
underlying OS functions. So in this case, if a user asks for 4kb, VAlloc would map in 64kb, and the next time a user asks for 4kb (or any size that would fit w/ alignment), we don’t ask VAlloc for memory, we issue a pointer bump (or something along those
lines). Naturally this is more complicated in a multithreaded context. What we’ve done there is have a per-thread allocator so there’s no contention between threads in user-space. Devil in the details, tho.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-family:"Calibri",sans-serif;color:black">From:
</span></b><span style="font-family:"Calibri",sans-serif;color:black">Even Rouault <even.rouault@spatialys.com><br>
<b>Date: </b>Thursday, March 21, 2024 at 9:59 AM<br>
<b>To: </b>"Meyer, Jesse R. (GSFC-618.0)[SCIENCE SYSTEMS AND APPLICATIONS INC]" <jesse.r.meyer@nasa.gov>, Abel Pau <a.pau@creaf.uab.cat>, "gdal-dev@lists.osgeo.org" <gdal-dev@lists.osgeo.org><br>
<b>Subject: </b>Re: [gdal-dev] [EXTERNAL] [BULK] Re: Experience with slowness of free() on Windows with lots of allocations?<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Aptos",sans-serif"><o:p> </o:p></span></p>
</div>
<table class="MsoNormalTable" border="1" cellspacing="0" cellpadding="0" align="left" style="border:solid black 1.5pt">
<tbody>
<tr>
<td width="100%" style="width:100.0%;border:none;background:#FFEB9C;padding:3.75pt 3.75pt 3.75pt 3.75pt">
<p class="MsoNormal" style="mso-element:frame;mso-element-frame-hspace:2.25pt;mso-element-wrap:around;mso-element-anchor-vertical:paragraph;mso-element-anchor-horizontal:column;mso-height-rule:exactly">
<b><span style="font-size:10.0pt;color:black">CAUTION:</span></b><span style="color:black">
</span><span style="font-size:10.0pt;color:black">This email originated from outside of NASA. Please take care when clicking links or opening attachments. Use the "Report Message" button to report suspicious messages to the NASA SOC.</span><span style="color:black">
</span><o:p></o:p></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
<br>
<o:p></o:p></p>
<div>
<p>I've played with VirtualAlloc(NULL, SINGLE_ALLOC_SIZE, MEM_COMMIT | MEM_RESERVE, PAGE_READWRITE), and it does avoid the performance issue. However I see that VitualAlloc() allocates by chunks of 64 kB, so depending on the size of a block, it might cause
significant waste of RAM, so that can't be used as a direct replacement of malloc().<o:p></o:p></p>
<p>My inclination would be to perhaps have an optional config option like GDAL_BLOCK_CACHE_USE_PRIVATE_HEAP that could be set, and when doing so it would use HeapCreate(0, 0, GDAL_CACHEMAX) to create a heap only used by the block cache. Not ideal, since that
would reserve the whole GDAL_CACHEMAX (but for a large enough processing, you'll end up consuming it), but it has the advantage of not being extremely intrusive either... and could be easily ditched/replaced by something better in the future.<o:p></o:p></p>
<p>Regarding tcmalloc, I've had to use it on Linux too, but only on scenarios involving multithreading where it helps reducing RAM fragmentation: cf
<a href="https://gdal.org/user/multithreading.html#ram-fragmentation-and-multi-threading">
https://gdal.org/user/multithreading.html#ram-fragmentation-and-multi-threading</a> . I've just tried quickly to use it on Windows to test it on the scenario, but didn't really manage to make it work. Even building it was challenging. Actually I tried
<a href="https://github.com/gperftools/gperftools">https://github.com/gperftools/gperftools</a> and I had to build from master since the latest tagged version doesn't build with CMake on Windows. But then nothing happens when linking tcmalloc_minimal.lib against
my toy app. I probably missed something.<o:p></o:p></p>
<p>Anyway I don't really think we can force tcmalloc to be used in GDAL, as a library. Unless there would be a way to have its allocator to be optionnaly used at places that we control (ie explicitly call tc_malloc / tc_free), and not replace the default malloc
/ free etc, which might be undesirable when GDAL is just a component of a larger application.<o:p></o:p></p>
<p>Disabling entirely the block cache (or setting it to a minimum value) is only a workable option for uncompressed formats, or if you use per-band blocks (INTERLEAVE=BAND in GTiff language) and not one block for all bands (INTERLEAVE=PIXEL), otherwise you'll
pay multiple time the decompression.<o:p></o:p></p>
<div>
<p class="MsoNormal">Le 21/03/2024 à 14:38, Meyer, Jesse R. (GSFC-618.0)[SCIENCE SYSTEMS AND APPLICATIONS INC] via gdal-dev a écrit :<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif">+1. We use a variety of hand-rolled VirtualAlloc based (for basic tasks, a simple pointer bump, and for more elaborate needs, a ‘buddy’) allocators, some of which try to be
smart about memory usage via de-committing regions. In our work, we tend to disable the GDAL cache entirely and rely on the file system’s file cache instead, which is a simplification we can make but is surely untenable in general here.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Aptos",sans-serif"> </span><o:p></o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span lang="CA" style="font-family:"Calibri",sans-serif;color:black">From:
</span></b><span lang="CA" style="font-family:"Calibri",sans-serif;color:black">gdal-dev
<a href="mailto:gdal-dev-bounces@lists.osgeo.org"><gdal-dev-bounces@lists.osgeo.org></a> on behalf of Abel Pau via gdal-dev
<a href="mailto:gdal-dev@lists.osgeo.org"><gdal-dev@lists.osgeo.org></a><br>
<b>Reply-To: </b>Abel Pau <a href="mailto:a.pau@creaf.uab.cat"><a.pau@creaf.uab.cat></a><br>
<b>Date: </b>Thursday, March 21, 2024 at 4:51 AM<br>
<b>To: </b><a href="mailto:gdal-dev@lists.osgeo.org">"gdal-dev@lists.osgeo.org"</a>
<a href="mailto:gdal-dev@lists.osgeo.org"><gdal-dev@lists.osgeo.org></a><br>
<b>Subject: </b>[EXTERNAL] [BULK] Re: [gdal-dev] Experience with slowness of free() on Windows with lots of allocations?</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span lang="CA" style="font-family:"Aptos",sans-serif"> </span><o:p></o:p></p>
</div>
<table class="MsoNormalTable" border="1" cellspacing="0" cellpadding="0" align="left" style="border:solid black 1.5pt">
<tbody>
<tr>
<td width="100%" style="width:100.0%;border:none;background:#FFEB9C;padding:3.75pt 3.75pt 3.75pt 3.75pt">
<p class="MsoNormal" style="mso-element:frame;mso-element-frame-hspace:2.25pt;mso-element-wrap:around;mso-element-anchor-vertical:paragraph;mso-element-anchor-horizontal:column;mso-height-rule:exactly">
<b><span style="font-size:10.0pt;font-family:"Aptos",sans-serif;color:black">CAUTION:</span></b><span style="font-family:"Aptos",sans-serif;color:black">
</span><span style="font-size:10.0pt;font-family:"Aptos",sans-serif;color:black">This email originated from outside of NASA. Please take care when clicking links or opening attachments. Use the "Report Message" button to report suspicious messages to the
NASA SOC.</span><span style="font-family:"Aptos",sans-serif;color:black"> </span>
<o:p></o:p></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span lang="CA" style="font-family:"Aptos",sans-serif"><br>
<br>
<br>
</span><o:p></o:p></p>
<div>
<p class="MsoNormal"><span lang="ES" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Hi Even,</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="ES" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="ES" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">you’re right. We also know that.
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">When programming the driver I took it in consideration. Our solution is not rely on windows to make a good job with memory and we try to reuse as memory as possible instead
of use calloc/free freely.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">For instance, in the driver, for each feature I have to get or write the coordinates. I could do it every time I have to, so lots of times: create memory for
reading, and then put them on the feature, and then free... so many times. What I do? When opening the layer I create some memory blocs of 250 Mb (due to the format itself) and I use that created memory to manage whatever I need. And when closing, I free it.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">While doing that I observed that sometimes I have to use GDAL code that doesn’t take it in consideration (</span><span lang="CA" style="font-size:9.5pt;font-family:Consolas;color:#6F008A">CPLRecode()</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">
for instance). Perhaps it could be improves as well.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Thanks for noticing that.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><b><span lang="ES" style="font-size:11.0pt;font-family:"Calibri",sans-serif">De:</span></b><span lang="ES" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> gdal-dev
<a href="mailto:gdal-dev-bounces@lists.osgeo.org"><gdal-dev-bounces@lists.osgeo.org></a>
<b>En nombre de </b>Javier Jimenez Shaw via gdal-dev<br>
<b>Enviado el:</b> dijous, 21 de març de 2024 8:27<br>
<b>Para:</b> Even Rouault <a href="mailto:even.rouault@spatialys.com"><even.rouault@spatialys.com></a><br>
<b>CC:</b> gdal dev <a href="mailto:gdal-dev@lists.osgeo.org"><gdal-dev@lists.osgeo.org></a><br>
<b>Asunto:</b> Re: [gdal-dev] Experience with slowness of free() on Windows with lots of allocations?</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="CA"> </span><o:p></o:p></p>
<div>
<p class="MsoNormal"><span lang="CA">In my company we confirmed that "Windows heap allocation mechanism sucks."</span><o:p></o:p></p>
<div>
<p class="MsoNormal"><span lang="CA">Closing the application after using gtiff driver can take many seconds due to memory deallocations.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span lang="CA"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span lang="CA">One workaround was to use tcmalloc. I will ask my colleagues more details next week.</span><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><span lang="CA"> </span><o:p></o:p></p>
<div>
<div>
<p class="MsoNormal"><span lang="CA">On Thu, 21 Mar 2024, 01:55 Even Rouault via gdal-dev, <<a href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a>> wrote:</span><o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<p class="MsoNormal"><span lang="CA">Hi,<br>
<br>
while investigating <br>
<a href="https://github.com/OSGeo/gdal/issues/9510#issuecomment-2010950408" target="_blank">https://github.com/OSGeo/gdal/issues/9510#issuecomment-2010950408</a>, I've
<br>
come to the conclusion that the Windows heap allocation mechanism sucks. <br>
Basically if you allocate a lot of heap regions of modest size with <br>
malloc()/new[], the time spent when freeing them all with corresponding <br>
free()/delete[] is excruciatingly slow (like ~ 10 seconds for ~ 80,000 <br>
allocations). The slowness is clearly quadratic with the number of <br>
allocations. You only start noticing it with ~ 30,000 allocations. And <br>
interestingly, another condition for that slowness is that each <br>
individual allocation much be strictly greater than 4096 * 4 bytes. At <br>
exactly that value, perf is acceptable, but add one extra byte, and it <br>
suddenly drops. I suspect that there must be a threshold from which <br>
malloc() starts using VirtualAlloc() instead of the heap, which must <br>
involve slow system calls, instead of a user-land allocation mechanism.<br>
<br>
Anyone has already hit that and found solutions? The only potential idea <br>
I found until now would be to use a private heap with HeapCreate() with <br>
a fixed maximum size, which is a bit problematic to adopt by default, <br>
basically that would mean that the size of GDAL_CACHEMAX would be <br>
consumed as soon as one use the block cache.<br>
<br>
Even<br>
<br>
-- <br>
<a href="http://www.spatialys.com/" target="_blank">http://www.spatialys.com</a><br>
My software is free, but my time generally not.<br>
<br>
_______________________________________________<br>
gdal-dev mailing list<br>
<a href="mailto:gdal-dev@lists.osgeo.org" target="_blank">gdal-dev@lists.osgeo.org</a><br>
<a href="https://lists.osgeo.org/mailman/listinfo/gdal-dev" target="_blank">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a></span><o:p></o:p></p>
</blockquote>
</div>
</div>
<p class="MsoNormal"><span style="font-family:"Aptos",sans-serif"><br>
<br>
<o:p></o:p></span></p>
<pre>_______________________________________________<o:p></o:p></pre>
<pre>gdal-dev mailing list<o:p></o:p></pre>
<pre><a href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a><o:p></o:p></pre>
<pre><a href="https://lists.osgeo.org/mailman/listinfo/gdal-dev">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a><o:p></o:p></pre>
</blockquote>
<pre>-- <o:p></o:p></pre>
<pre><a href="http://www.spatialys.com/">http://www.spatialys.com</a><o:p></o:p></pre>
<pre>My software is free, but my time generally not.<o:p></o:p></pre>
</div>
</div>
</body>
</html>