<div dir="ltr">Hi Matthias,<div><br></div><div>May I add another pain here?</div><div><br></div><div>Storing gpkg in the cloud means syncing even after the file was only opened in QGIS and nothing has changed inside - it was only opened to view something.</div><div>On very large gpkg files this is also not really nice.</div><div>(Maybe I am just using gpkg wrong, but at least this happens on my installation)</div><div><br></div><div>regards</div><div>Werner</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, May 8, 2020 at 11:31 AM Matthias Kuhn <<a href="mailto:matthias@opengis.ch">matthias@opengis.ch</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>Hi list,</p>
<p>I wondered about the state of GeoPackage. Personally, cince it
has been introduced to qgis and evenmore since it has been
selected as the default format, I have never grown to fully and
completely.</p>
<p>I do not want to trigger a evangelical discussion here. I'd like
to see where we are and what we can reasonably do to have a
default file format which can be recommended with no bad feelings.<br>
</p>
<p><br>
</p>
<p>Here follow a couple of observations over the years, some of them
properties of the specs I believe:</p>
<p><br>
</p>
<p><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr">* The fid requirement</span></p>
<p><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"> I sometimes want my features to be identified by
uuids or others. They also tend to accumulate if derived
datasets are created (through processing etc). If I need some
pseudo stable primary key there is a rowid builtin into sqlite,
we don't need a second one.<br>
</span></p>
<p><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"> Possible mitigation: alter the ogr implementation.
possibly alter the standard (required?)<br>
</span></p>
<p><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"></span><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr">* </span>The
modification on r/o open</span></p>
<p><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"> Has caused too much pain on git.</span></p>
<p style="margin:0px;text-indent:0px"><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"> Possible mitigation: a)
switch to journal mode=delete (not an easy option because of </span><a href="https://issues.qgis.org/issues/15351" target="_blank">https://issues.qgis.org/issues/15351</a>)
b) only switch to wal mode when layers are put into edit mode (I
have strong doubts this is a safe thing to do)<br>
</p>
<p><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"></span></p>
<p><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"></span><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr">* </span>The
network share freeze</span></p>
<p><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"> Our default file should play nicely with (windows)
network shares. It's clear to everyone that we can't expect
concurrent writes. But it should "just work" for concurrent read
by many.</span></p>
<p><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"> Possible mitigation: switch to journal mode=delete
for network shares (we are looking into this)<br>
</span></p>
<p><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"></span><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr">* </span>The
wal file appearing next to the file</span></p>
<p><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"> It is confusing to newcomers and looks almost like a
sidecar file. I would care less if it was put into some system
cache folder instead of just into my data folder. Or at least if
it was a hidden file.<br>
</span></p>
<p><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"> Possible mitigation: switch to journal mode=delete (</span><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr">not an easy option because
of </span><a href="https://issues.qgis.org/issues/15351" target="_blank">https://issues.qgis.org/issues/15351</a>)</span></p>
<p><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"></span><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr">* </span>The
couple of corrupted files I have received over the years which
could only be repaired by a command line "dump contents as sql
and execute into new file"</span></p>
<p><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"> I have not found a way to reproduce this. Some of
them were produced by older qgis versions making it easy to
violate foreign key constraints and hard to recover. This has
been fixed.</span></p>
<p><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"> Possible mitigation: offer a "repair" option in
qgis. Through processing or "on the fly" upon detection.<br>
</span></p>
<p>*<span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"> </span>Default value
magic replace values on insert (with no possibility to
pre-evaluate them)</span></p>
<p><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"> E.g. a global sequence like on postgres would be
nice. Can be worked around through default values in qgis
though.<br>
</span></p>
<p><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"> Possible mitigation: a)add it as a feature to
sqlite. b) use qgis default values. c) live with it.<br>
</span></p>
<p>*<span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"> </span>The requirement
for a single geometry column per table</span></p>
<p><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"> I just don't see a good reason to forbid that</span></p>
<p><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"> Possible mitigation: a) alter the standard. b)
ignore the standard and patch the ogr implementation.<br>
</span></p>
<p><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"><br>
</span></p>
<p><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr">I wonder how others feel about these topics.</span></p>
<p><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr"><br>
</span></p>
<p><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr">- Are there more pain points I forgot to list?<br>
</span></p>
<p><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr">- Do you see more approaches to mitigate these
problems?<br>
</span></p>
<p><span id="gmail-m_-5280934158331864378:ws.co" style="text-align:left" dir="ltr">- Is someone already working on these issues?<br>
</span></p>
<p><br>
</p>
<p>It would be great to have a standard file format that we can
fully trust. Let's make a reality check if GeoPackage can be this
format.</p>
<p>Best regards<br>
</p>
<div>-- <br>
<div>
<div> <span style="text-align:left;color:rgb(0,0,0);font-family:Verdana,sans-serif;font-size:10pt">Matthias Kuhn</span><br>
<a href="mailto:matthias@opengis.ch" target="_blank"> <span style="text-align:left;color:rgb(0,0,0);font-family:Verdana,sans-serif;font-size:8pt">matthias@opengis.ch</span>
</a><br>
<span style="text-align:left;color:rgb(0,0,0);font-family:Verdana,sans-serif;font-size:8pt"><a href="tel:+41764356763" target="_blank">+41 (0)76 435 67 63</a></span><br>
<div> <a href="http://www.opengis.ch" target="_blank"> <img src="cid:171f3a1964dcb971f161" alt="OPENGIS.ch Logo" width="200" height="80"></a> </div>
</div>
</div>
</div>
</div>
_______________________________________________<br>
QGIS-Developer mailing list<br>
<a href="mailto:QGIS-Developer@lists.osgeo.org" target="_blank">QGIS-Developer@lists.osgeo.org</a><br>
List info: <a href="https://lists.osgeo.org/mailman/listinfo/qgis-developer" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/qgis-developer</a><br>
Unsubscribe: <a href="https://lists.osgeo.org/mailman/listinfo/qgis-developer" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/qgis-developer</a></blockquote></div>