<!DOCTYPE html><html><head><title></title><style type="text/css">#qt p.qt-MsoNormal{margin-top:0in;margin-right:0in;margin-bottom:0in;margin-left:0in;font-size:11pt;font-family:Calibri, sans-serif;}
#qt a:link{color:blue;text-decoration-line:underline;text-decoration-thickness:initial;text-decoration-style:initial;text-decoration-color:initial;}
#qt p.qt-qt-msonormal1{margin-top:0in;margin-right:0in;margin-bottom:0in;margin-left:0in;font-size:11pt;font-family:Calibri, sans-serif;}
p.MsoNormal,p.MsoNoSpacing{margin:0}</style></head><body><div style="font-family:Arial;">It is single-threaded (AFAIK), but as you said, loading the data doesn't take too long, so you won't win more than that by pipelining it. And SQLite doesn't support concurrent writes, not even in WAL mode, so there's not much use for threads there.</div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">I would have expected OGR_SQLITE_SYNCHRONOUS=NO to make a slight difference, but probably less than disabling the spatial index.</div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">Laurentiu</div><div style="font-family:Arial;"><br></div><div>On Fri, Oct 21, 2022, at 15:14, Dennis Burgess wrote:<br></div><blockquote type="cite" id="qt" style="overflow-wrap:break-word;"><div class="qt-WordSection1"><p class="qt-MsoNormal"><span style="color:rgb(31, 73, 125);"><span class="size" style="font-size:12pt;">Yep, tried all of those items it does help if I don’t create the index and then index it later, but we are not talking a huge amount of time. Guess it’s a single threaded app?</span></span><br></p><p class="qt-MsoNormal"><span style="color:rgb(31, 73, 125);"><span class="size" style="font-size:12pt;"> </span></span><br></p><div><p class="qt-MsoNormal"><span style="color:rgb(31, 73, 125);"> </span><br></p><p class="qt-MsoNormal" style=""><b><span style="color:rgb(31, 73, 125);"><span class="size" style="font-size:10pt;"><img width="112" height="32" style="width:1.1666in;height:0.3333in;" id="qt-Picture_x0020_2" src="cid:image001.png@01D8E51C.B981DBD0" alt="LTI-Full_175px"></span></span></b><span style="color:rgb(31, 73, 125);"></span><br></p><p class="qt-MsoNormal" style=""><b><span style="color:rgb(31, 73, 125);"><span class="size" style="font-size:10pt;">Dennis Burgess</span></span></b><br></p><p class="qt-MsoNormal" style=""><b><span style="color:rgb(31, 73, 125);"><span class="size" style="font-size:10pt;"><br></span></span></b><span style="color:rgb(31, 73, 125);"><span class="size" style="font-size:10pt;">Author of "Learn RouterOS- Second Edition” </span></span><span style="color:rgb(31, 73, 125);"></span></p><p class="qt-MsoNormal" style=""><b><span style="color:rgb(31, 73, 125);"><span class="size" style="font-size:10pt;">Link Technologies, Inc</span></span></b><span style="color:rgb(31, 73, 125);"><span class="size" style="font-size:10pt;"> -- Mikrotik & WISP Support Services </span></span><span style="color:rgb(31, 73, 125);"></span><br></p><p class="qt-MsoNormal" style=""><b><span style="color:rgb(31, 73, 125);"><span class="size" style="font-size:10pt;">Office</span></span></b><span style="color:rgb(31, 73, 125);"><span class="size" style="font-size:10pt;">: 314-735-0270 Website: </span></span><span style="color:rgb(31, 73, 125);"><a href="http://www.linktechs.net/"><span style=""><span class="size" style="font-size:10pt;">http://www.linktechs.net</span></span></a></span><span style="color:rgb(31, 73, 125);"><span class="size" style="font-size:10pt;"> </span></span><span style="color:rgb(31, 73, 125);"></span><br></p><p class="qt-MsoNormal" style=""><span style="color:rgb(31, 73, 125);"><span class="size" style="font-size:10pt;">Create Wireless Coverage’s with </span></span><span style="color:rgb(31, 73, 125);"><a href="http://www.towercoverage.com"><span style=""><span class="size" style="font-size:10pt;">www.towercoverage.com</span></span></a></span><span style="color:rgb(31, 73, 125);"><span class="size" style="font-size:10pt;"></span></span><br></p><p class="qt-MsoNormal" style=""><span style="color:rgb(31, 73, 125);"><span class="size" style="font-size:10pt;">Need MikroTik Cloud Management: <a href="https://cloud.linktechs.net"><span style="color:rgb(5, 99, 193);">https://cloud.linktechs.net</span></a></span></span><br></p></div><p class="qt-MsoNormal"><span style="color:rgb(31, 73, 125);"><span class="size" style="font-size:12pt;"> </span></span><br></p><div><div style="border-right-width:initial;border-bottom-width:initial;border-left-width:initial;border-right-style:none;border-bottom-style:none;border-left-style:none;border-right-color:initial;border-bottom-color:initial;border-left-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-top-width:1pt;border-top-style:solid;border-top-color:rgb(225, 225, 225);padding-top:3pt;padding-right:0in;padding-bottom:0in;padding-left:0in;"><p class="qt-MsoNormal"></p><div><b>From:</b> gdal-dev <gdal-dev-bounces@lists.osgeo.org> <b> On Behalf Of </b>Lauren?iu Nicola via gdal-dev<br></div><div> <b>Sent:</b> Thursday, October 20, 2022 8:09 AM<br></div><div> <b>To:</b> gdal-dev@lists.osgeo.org<br></div><div> <b>Subject:</b> Re: [gdal-dev] Tips on Speeding up OGR2OGR GeoPackage Creation<br></div><p></p></div></div><p class="qt-MsoNormal"> <br></p><div><p class="qt-MsoNormal"><span style=""><span class="font" style="font-family:Arial, sans-serif;">Hi,</span></span><br></p></div><div><p class="qt-MsoNormal"><span style=""><span class="font" style="font-family:Arial, sans-serif;"> </span></span><br></p></div><div><p class="qt-MsoNormal"><span style=""><span class="font" style="font-family:Arial, sans-serif;"><a href="https://gdal.org/drivers/vector/gpkg.html#configuration-options">https://gdal.org/drivers/vector/gpkg.html#configuration-options</a> has a couple of options that can help with performance.
See also <a href="https://gdal.org/drivers/vector/sqlite.html#target-drivers-vector-sqlite-performance-hints"> https://gdal.org/drivers/vector/sqlite.html#target-drivers-vector-sqlite-performance-hints</a>.</span></span><br></p></div><div><p class="qt-MsoNormal"><span style=""><span class="font" style="font-family:Arial, sans-serif;"> </span></span><br></p></div><div><p class="qt-MsoNormal"><span style=""><span class="font" style="font-family:Arial, sans-serif;">But I don't recall them helping that much. YMMV.</span></span><br></p></div><div><p class="qt-MsoNormal"><span style=""><span class="font" style="font-family:Arial, sans-serif;"> </span></span><br></p></div><div><p class="qt-MsoNormal"><span style=""><span class="font" style="font-family:Arial, sans-serif;">Regards,</span></span><br></p></div><div><p class="qt-MsoNormal"><span style=""><span class="font" style="font-family:Arial, sans-serif;">Laurentiu</span></span><br></p></div><div><p class="qt-MsoNormal"><span style=""><span class="font" style="font-family:Arial, sans-serif;"> </span></span><br></p></div><div><p class="qt-MsoNormal">On Thu, Oct 20, 2022, at 15:06, Dennis Burgess wrote:<br></p></div><blockquote style="margin-top:5pt;margin-bottom:5pt;overflow-wrap:break-word;" id="qt-qt"><div><p class="qt-qt-msonormal1">We have a large table in MSSQL, this table we extract data from and drop it into a GeoPackage for further processing. However, the extraction can take some time, its only using 5% of the CPU available, and pulling the data is fairly
quick in debug mode, its really its generation and index creation that is slow. Any tips on speeding this up? We have SSDs and NVME drives, 10gig networking etc.. So. It should be as quick as it can be, but the 5% CPU is what I think the killer is. On
this server 24 cores, so 5% would be right at one core.. Is there no multi-threading or other options here? <br></p><p class="qt-qt-msonormal1"> <br></p><p class="qt-qt-msonormal1"> <br></p><p class="qt-qt-msonormal1"><b><span style="color:rgb(31, 73, 125);"><span class="size" style="font-size:10pt;"><img border="0" width="112" height="32" style="width:1.1666in;height:0.3333in;" id="qt-qt-x_x_x_Picture_x0020_1" src="cid:image001.png@01D8E51C.B981DBD0" alt="LTI-Full_175px"></span></span></b><br></p><p class="qt-qt-msonormal1"><span class="qt-size"><b><span style="color:rgb(31, 73, 125);"><span class="size" style="font-size:10pt;">Dennis Burgess, Mikrotik Certified Trainer</span></span></b></span><b><span style="color:rgb(31, 73, 125);"><span class="size" style="font-size:10pt;"><br> <span class="qt-size">MTCNA, MTCRE, MTCWE, MTCTCE, MTCINE, MTCSE, HE IPv6 Sage, Cambium ePMP Certified</span></span></span></b></p><p class="qt-qt-msonormal1"><span class="qt-size"><span style="color:rgb(31, 73, 125);"><span class="size" style="font-size:10pt;">Author of "Learn RouterOS- Second Edition”</span></span></span><br></p><p class="qt-qt-msonormal1"><span class="qt-size"><b><span style="color:rgb(31, 73, 125);"><span class="size" style="font-size:10pt;">Link Technologies, Inc</span></span></b></span><span class="qt-size"><span style="color:rgb(31, 73, 125);"><span class="size" style="font-size:10pt;"> -- Mikrotik & WISP Support Services</span></span></span><br></p><p class="qt-qt-msonormal1"><span class="qt-size"><b><span style="color:rgb(31, 73, 125);"><span class="size" style="font-size:10pt;">Office</span></span></b></span><span class="qt-size"><span style="color:rgb(31, 73, 125);"><span class="size" style="font-size:10pt;">: 314-735-0270 Website: </span></span></span><a href="http://www.linktechs.net/"><span class="qt-size"><span style=""><span class="size" style="font-size:10pt;">http://www.linktechs.net</span></span></span></a><br></p><p class="qt-qt-msonormal1"><span class="qt-size"><span style="color:rgb(31, 73, 125);"><span class="size" style="font-size:10pt;">Need to Automate MikroTik Backups: <a href="https://cloud.linktechs.net"><span style="color:rgb(5, 99, 193);">https://cloud.linktechs.net</span></a></span></span></span><br></p><p class="qt-qt-msonormal1"><span class="qt-size"><span style="color:rgb(31, 73, 125);"><span class="size" style="font-size:10pt;">Create Wireless Coverage’s with </span></span></span><a href="http://www.towercoverage.com"><span class="qt-size"><span style=""><span class="size" style="font-size:10pt;">www.towercoverage.com</span></span></span></a><br></p><p class="qt-qt-msonormal1"> <br></p></div><div><p class="qt-MsoNormal">_______________________________________________<br></p></div><div><p class="qt-MsoNormal">gdal-dev mailing list<br></p></div><div><p class="qt-MsoNormal"><a href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a><br></p></div><div><p class="qt-MsoNormal"><a href="https://lists.osgeo.org/mailman/listinfo/gdal-dev">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a><br></p></div><div><p class="qt-MsoNormal"> <br></p></div></blockquote><div><p class="qt-MsoNormal"><span style=""><span class="font" style="font-family:Arial, sans-serif;"> </span></span><br></p></div></div></blockquote></body></html>