<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p><br>
</p>
<p>Can you check the CPU usage of your process? Normally, if you
don't specify <i>GDAL_NUM_THREADS</i>, GDAL will use as many
threads as there are CPU cores. Also try reducing to =1 to see if
it slows down.<br>
</p>
<p><br>
</p>
<div class="moz-cite-prefix">On 02/03/2022 08:21, Szymon Piskula (C)
via gdal-dev wrote:<br>
</div>
<blockquote type="cite"
cite="mid:LO0P123MB6301A5650FCE4FE13375072E9D039@LO0P123MB6301.GBRP123.PROD.OUTLOOK.COM">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}div.WordSection1
{page:WordSection1;}</style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal">Hello,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I am trying to speed up my MVT generation
with GDAL. The GDAL documentation says that some of the work
can be multi-threaded by use of the GDAL_NUM_THREADS
parameter. From my understanding by default there should be as
many threads in the pool as many cores there are on the
machine. Since I run on Azure VM, I tried scaling up and down
between 4 and 16 core but my MVT test data always processes in
10minutes, regardless of the VM spec. I tried running ogr2ogr
both with setting --config GDAL_NUM_THREADS 2 (to try force
slow down) or help speed up with --config GDAL_NUM_THREADS 12
by hinting to use more threads. Still my processing time is
nearly fixed and does not change. I also tried setting the
environmental variable GDAL_NUM_THREADS 12 but no change with
performance. Does anyone know please how well the GDAL MVT
actually multi-threads? Can this help speed things up in any
significant way? Or is there a different way to control the
multi-threading ? The full command I run (on Windows) is :<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">ogr2ogr --debug ON --config
GDAL_NUM_THREADS 12 -f MVT F:\Task_3 -dsco MINZOOM=16 -dsco
MAXZOOM=17 -dsco MAX_SIZE=1000000 -dsco EXTENT=8192 -dsco
COMPRESS=NO PG:<constring> tablea, tableb<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">My GDAL Version is: <o:p></o:p></p>
<p class="MsoNormal">GDAL 3.4.1, released 2021/12/27<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I can see that the tiles are written to the
.temp.db SQLite, then are written to the local disk folder, so
I wonder if it’s the SQLite bit that is the bottleneck ?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thank you<o:p></o:p></p>
</div>
<br>
<br>
This email and any attachments are intended only for the intended
recipient and may contain sensitive information. If you are not
the intended recipient, please immediately delete this email and
inform the sender.<br>
<br>
OS email communications may be monitored to ensure the secure and
effective operation of our systems and for other lawful purposes.
Subject to contract: No rights are to be derived from any proposal
contained in this email until a written agreement containing all
necessary terms is executed between the relevant parties.<br>
<br>
Thank you for your cooperation.<br>
<br>
Ordnance Survey Limited (Company Registration number 09121572)<br>
Registered Office: Explorer House<br>
Adanac Drive<br>
Southampton SO16 0AS<br>
Tel: 03456 050505<br>
<a href="http://www.os.uk" moz-do-not-send="true"
class="moz-txt-link-freetext">http://www.os.uk</a>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
gdal-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a>
<a class="moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/gdal-dev">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a>
</pre>
</blockquote>
<pre class="moz-signature" cols="72">--
Momtchil Momtchev <a class="moz-txt-link-rfc2396E" href="mailto:momtchil@momtchev.com"><momtchil@momtchev.com></a>
</pre>
</body>
</html>