<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.Shkpostityyli18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 2.0cm 70.85pt 2.0cm;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="FI" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Hi,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">I tested with this environmental variable and command<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">set OGR_MVT_REMOVE_TEMP_FILE=NO<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">ogr2ogr -f mvt -dsco temporary_db=c:\data\mtk\temp\temp.db out.mvt mtkmaasto.gpkg rakennus --debug on<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Once ogr2ogr stopped I had the temp.db file left in the directory. It seems to be a simple SQLite database. Without setting the environmental variable the temporary database was immediately
deleted. During the process the file explorer only shows that the file exists but with file size of 0 bytes. I believe that this is normal behavior with SQLite.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">-Jukka Rahkonen-<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b>Lähettäjä:</b> gdal-dev <gdal-dev-bounces@lists.osgeo.org>
<b>Puolesta </b>Leo Fuhrmann via gdal-dev<br>
<b>Lähetetty:</b> perjantai 6. lokakuuta 2023 16.08<br>
<b>Vastaanottaja:</b> gdal-dev@lists.osgeo.org<br>
<b>Aihe:</b> [gdal-dev] Questions regarding temporary database file (ogr2ogr with MVT driver)<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Hi,<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I’m creating vector tiles from a PostGIS database using ogr2ogr with the MVT driver. The region is quite large and I’m creating tiles for levels 0-15, so the process takes quite some time (hours). The tiles are written directly to a S3
storage using the /vsis3/ virtual file system (see: <a href="https://gdal.org/user/virtual_file_systems.html#vsis3-aws-s3-files">https://gdal.org/user/virtual_file_systems.html#vsis3-aws-s3-files</a>).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Looking at the logs, I can see that, firstly, a “temporary database” file (*.temp.db) is created. Some things are notable:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">- this step seems to be CPU intensive compared to the creation of the actual pbf files<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">- the temporary database file is not being written to the output file directory (S3 storage in this case) as stated in the documentation<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">- the dataset creation option `TEMPORARY_DB` (see: <a href="https://gdal.org/drivers/vector/mvt.html#dataset-creation-options">https://gdal.org/drivers/vector/mvt.html#dataset-creation-options</a>) seems to be ignored (I tried writing the
file to a local directory but no such file shows up)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">- my guess is that the file is only kept in memory (which would be /vsimem/ if I’m correct)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Is it possible to skip the step of first creating the temp db and directly create tiles from PostGIS or any other data source? Where/how exactly is the database file stored? Any ideas on optimizing the process?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I’m looking forward to some discussion on this topic. I will happily try to contribute where I can.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Regards,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Leo<o:p></o:p></p>
</div>
</div>
</body>
</html>