<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
span.Shkpostityyli19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="FI" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Hi,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">There are some performance hints in
<a href="https://gdal.org/drivers/vector/sqlite.html#target-drivers-vector-sqlite-performance-hints">
https://gdal.org/drivers/vector/sqlite.html#target-drivers-vector-sqlite-performance-hints</a> but everything in SQLite is stored into one single file that must have a proper organization all the time and that inevitably sets some limits. Ogr2ogr is made to
use automatically the most obvious tricks for increasing the performance, like creating a new table first and indexing it afterwards. You may find other hints from SQLite forums, for example this looks interesting
<a href="https://stackoverflow.com/questions/1711631/improve-insert-per-second-performance-of-sqlite">
https://stackoverflow.com/questions/1711631/improve-insert-per-second-performance-of-sqlite</a> but some tricks like “without_rowid” cannot be used for GeoPackage because the GeoPackage standard does not allow that.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">If would help to evaluate if the speed that you have now is reasonable or not if you could provide some metrics like inserted features or megabytes per second.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">-Jukka Rahkonen-<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b>Lähettäjä:</b> gdal-dev <gdal-dev-bounces@lists.osgeo.org>
<b>Puolesta </b>Dennis Burgess<br>
<b>Lähetetty:</b> torstai 20. lokakuuta 2022 15.07<br>
<b>Vastaanottaja:</b> gdal-dev@lists.osgeo.org<br>
<b>Aihe:</b> [gdal-dev] Tips on Speeding up OGR2OGR GeoPackage Creation<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span lang="EN-US">We have a large table in MSSQL, this table we extract data from and drop it into a GeoPackage for further processing. However, the extraction can take some time, its only using 5% of the CPU available, and pulling the
data is fairly quick in debug mode, its really its generation and index creation that is slow. Any tips on speeding this up? We have SSDs and NVME drives, 10gig networking etc.. So. It should be as quick as it can be, but the 5% CPU is what I think the
killer is. On this server 24 cores, so 5% would be right at one core.. Is there no multi-threading or other options here?
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><b><span lang="EN-US" style="font-size:10.0pt;color:#1F497D"><img border="0" width="112" height="32" style="width:1.1666in;height:.3333in" id="x_x_x_Picture_x0020_1" src="cid:image001.png@01D8E49D.E5D835A0" alt="LTI-Full_175px"></span></b><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><b><span lang="EN-US" style="font-size:10.0pt;color:#1F497D">Dennis Burgess, Mikrotik Certified Trainer<br>
MTCNA, MTCRE, MTCWE, MTCTCE, MTCINE, MTCSE, HE IPv6 Sage, Cambium ePMP Certified </span>
</b><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:10.0pt;color:#1F497D">Author of "Learn RouterOS- Second Edition”
</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><b><span lang="EN-US" style="font-size:10.0pt;color:#1F497D">Link Technologies, Inc</span></b><span lang="EN-US" style="font-size:10.0pt;color:#1F497D"> -- Mikrotik & WISP Support Services
</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><b><span lang="EN-US" style="font-size:10.0pt;color:#1F497D">Office</span></b><span lang="EN-US" style="font-size:10.0pt;color:#1F497D">: 314-735-0270 Website:
</span><span lang="EN-US"><a href="https://eur06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.linktechs.net%2F&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7Cdefc186c65ee4c75330708dab2977a8d%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C1%7C638018660925600304%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=J1OWaQUH7p7F%2FpaPKRKFPoPwGwRadewZKRv%2FlRYV%2BUw%3D&reserved=0"><span style="font-size:10.0pt;color:blue">http://www.linktechs.net</span></a></span><span lang="EN-US" style="font-size:10.0pt;color:#1F497D">
<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:10.0pt;color:#1F497D">Need to Automate MikroTik Backups:
<a href="https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcloud.linktechs.net%2F&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7Cdefc186c65ee4c75330708dab2977a8d%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C1%7C638018660925600304%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=B5Mke%2F1KVgT8wTbp2tVo2eoyvE60qhDyzqdEpZCASCM%3D&reserved=0">
https://cloud.linktechs.net</a> </span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:10.0pt;color:#1F497D">Create Wireless Coverage’s with
</span><span lang="EN-US"><a href="https://eur06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.towercoverage.com%2F&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7Cdefc186c65ee4c75330708dab2977a8d%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C1%7C638018660925600304%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=E9o9%2BFk9OXa6LWRCm50qpZGWLne6ecTPOJ9ZWwmc6eQ%3D&reserved=0"><span style="font-size:10.0pt;color:blue">www.towercoverage.com</span></a></span><span lang="EN-US" style="font-size:10.0pt;color:#1F497D">
</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
</body>
</html>