[QGIS-Developer] The curious case of GeoPackage on exFAT

[Bo Victor Thomsen]

Hi Jonathan -

Thanks for the "heads-up" regarding wal files. The technical problem is 
using the xFAT file system  - when xFAT gets corrupted, it will partly 
be write-protected. This will inhibit QGIS in making the 2 wal files (or 
updating or deleting them).

But QGIS has an issue here: When the situation occurs, QGIS will quietly 
tell you about in the relatively obscure log system. And it will show 
you an *empty* GeoPackage - no layers. That is not smart. It will upset 
users (It certainly did upset me :-) . They will believe, that their 
precious layers in the GeoPackage has disappeared. However, the layers 
is perfectly alright. If you copy the GeoPackage to another filesystem, 
QGIS will happily open the layers from the copy.

IMHO, a better approach would to have a big fat error message informing 
users that they have a problem with the underlying file system. (I plan 
to write a feature request about this later today)


-- 

Med venlig hilsen / Kind regards

Bo Victor Thomsen


Den 18-05-2020 kl. 17:49 skrev Jonathan Moules:
>
> Hi Bo,
>
> Looking at the WAL docs for SQLite (https://www.sqlite.org/wal.html) 
> (what geopackage is built from) I see this:
>
> "The WAL file exists for as long as any database connection has the 
> database open. Usually, the WAL file is deleted automatically when the 
> last connection to the database closes. However, if the last process 
> to have the database open exits without cleanly shutting down the 
> database connection, or if the SQLITE_FCNTL_PERSIST_WAL file control 
> is used, then the WAL file might be retained on disk after all 
> connections to the database have been closed."
>
> So it sounds like whatever the last process is to touch the GeoPackage 
> may not be closing the connection cleanly on exFat.
>
> Cheers,
>
> Jonathan
>
> On 2020-05-11 13:10, Bo Victor Thomsen wrote:
>>
>> Hi all -
>>
>> I have a strange problem. I'm have 3 different disk on my windows 
>> based system on Mac hardware
>>
>>  1. My system drive. Formatted to NTFS.
>>  2. A flash-drive. Formatted to FAT32.
>>  3. A data drive. Formatted to exFAT. The last is my primary data
>>     drive and is shared between my Windows partition and my Mac
>>     partition on my MacBook Pro. Hence the use of exFAT.
>>
>> I have a QGIS plugin, which copies a template of a GeoPackage file to 
>> "where-ever the user wants it placed" and afterward make some content 
>> changes in the copy using the PyQT QSQL module with the QSPATIALITE 
>> driver
>>
>> This work if the Geopackage  file is copied to either disk no 1 
>> (NTFS) or disk no 2 (FAT32). However, it doesn't work if the file is 
>> copied to disk no 3 (exFAT). The process leaves the WAL files even 
>> after the database is closed properly.
>>
>> And even more strange: If I reformat the flash-drive to exFAT and 
>> repeat the experiment using the reformatted drive it too works 
>> without a hitch.
>>
>> The normal "divide et impera" method tells me that my exFat data disk 
>> is bork'ed. However this error *only* occurs with the QGIS/GeoPackage 
>> creation/modification scenario. Everything else is working OK.
>>
>> The disk is not shared on the network. Has anyone experienced the 
>> same type of problems ? And have a solution ?? Just asking before I 
>> begin to clean up / reformat my 100 GB data disk
>>
>> System setup: MacBook Pro 2014 / Windows 8.1 OS /QGIS 3.10.5 (the 
>> same problem occurs with 3.10.0 , 3.10.2 ...3.12.2)
>>
>>
>> _______________________________________________
>> QGIS-Developer mailing list
>> QGIS-Developer at lists.osgeo.org
>> List info:https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> Unsubscribe:https://lists.osgeo.org/mailman/listinfo/qgis-developer

-- 
Med venlig hilsen / Kind regards

Bo Victor Thomsen

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20200520/a79771c0/attachment-0001.html>