[Qgis-user] How to Prevent GeoPackage file from updating modified date when viewed in a Qgis project when it is not being edited

[Even Rouault]

On lundi 9 avril 2018 15:24:41 CEST rjwillson at gmail.com wrote:
> I would really like to use and promote GeoPackage as the go to data format
> in my organization but I am having the following problem that I haven't been
> able to solve.
> 
> 
> 
> When ever someone opens a Q template project that has a layer with a
> GeoPackage data source, the GeoPackage file (not its contents) will be
> altered and it will show that it has been updated in the Windows file
> manager. I have been told that this happens because the write ahead logging
> has been triggered (creation of *.wal and *.shm) files by simply viewing the
> contents of the GeoPackage in QGIS and/or in Intramaps Roam (works on top
> of QGIS for tablets).
> 
> 
> 
> This is problematic because most users in our organization will then look at
> the windows modified date time stamp and conclude that the file's contents
> have been modified. Additionally, all of the GIS layers are stored in
> Dropbox and subsequently Dropbox concludes that the file has been modified
> and it re-uploads a new copy of the GeoPackage. When it is a large
> GeoPackage > 1 GB in size this is quite problematic and causes a cascade of
> uploads and downloads.
> 
> 
> 
> Over the last few months I have tried to suppress this modification for the
> large GeoPackages in our system in the following ways: (1) make the layer
> read only in the QGIS project (no effect); (2) make the Dropbox folder read
> only for all users except the Dropbox administrator (this is effective for
> the large base layers); (3) change the properties of the file in windows to
> read only (effective for the Dropbox administrator); and (4) add a user
> variable to the windows system environment variables: OGR_SQLITE_JOURNAL =
> DELETE (effective for QGIS but not for Intramaps Roam)
> 
> 
> 
> Combining 2, 3, and 4 allows me to use GeoPackage for layers I can ensure
> are placed in the read-only Dropbox folders.
> 
> 
> 
> It is likely that my organization will never need the simultaneous editing
> of datasets that is facilitated by Write-Ahead Logging, but I would really
> like to be able to replace shapefiles and File geodatabases with GeoPackages
> in all of our projects going forward.
> 
> 
> 
> Therefore, is there a simpler way to suppress the write ahead logging
> behaviour that causes the files to believe they have been updated for all
> GeoPackages created in QGIS or used in a QGIS project?

My memory is already fading about the details, but the reason for the current 
behaviour is that if you open a GeoPackage database in read-only non-WAL mode 
and someone else try to edit it concurrently, unpleasant locking will occur. 
Actually due to how QGIS works, locking could occur even in the same session 
(since background rendering threads can open read-only connections while the 
main connection is writing data). See https://issues.qgis.org/issues/15351

So the solution was:

commit b6b8759efbeb833d0d3dbf6df008087701361ad3
Author: Even Rouault <even.rouault at spatialys.com>
Date:   Thu Sep 22 23:23:00 2016 +0200

    Fix database locking when editing GeoPackage
    
    Concurrent read and write can lock a GeoPackage database given
    the default journaling mode of SQLite (delete). Use WAL when
    possible to avoid that.
    
    Fixes #15351

I understand your use case as well, but I don't have a solution in mind that 
would fit all. 
I don't remember if I tried scenarios where one connection would open in 
DELETE mode and another one would try to turn to WAL on. 
A quick try shows that the a connection that opens in DELETE mode doesn't see 
the change to WAL done by another one. And WAL documentation doesn't describe 
this scenario, so it is quite unknown what would happen. So the current 
behaviour is to be on the safe side and always turn on WAL on opening.

Or perhaps have some option in a dialog box somewhere! "I promise that I will 
not edit this file in any concurrent session, or in this session, so you can 
safely open it in DELETE mode" ? Or spend more time digging again into the 
details of sqlite locking to see if we could make a clever use of it, and/or 
possibly improving it in sqlite itself (rather involved!)

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com