[Qgis-user] [EXTERNAL] Re: Geopackage slow on NAS if not read-only

Thu Mar 16 02:52:15 PDT 2023

Long story short: gpkg is bad idea for network drives as SMB/NAS etc

Sources:
Write-Ahead Logging (sqlite.org)<https://www.sqlite.org/wal.html#advantages>
writing gpkg and sqlite on samba shares fails * Issue #628 * r-spatial/sf * GitHub<https://github.com/r-spatial/sf/issues/628>

_________________________________

Jarosław Sadowski
Kierownik Zespołu ds. Ochrony Środowiska | Biuro Strategii i Planowania, Projektowania i Inżynierii Podprogramu Kolejowego
Environmental Protection Team Leader | Railway Subprogramme Strategy & Planning, Design & Engineering Department
e: jaroslaw.sadowski at cpk.pl<mailto:jaroslaw.sadowski at cpk.pl>
m: +48 532 720 230

[cid:transparency_ce4012c8-87e7-4198-a48f-d42a38680681.png]

Centralny Port Komunikacyjny Sp. z o.o. z siedzibą w Warszawie, Aleje Jerozolimskie 142B, 02-305 Warszawa; nr KRS 0000759991, Sąd Rejonowy dla m.st. Warszawy, XII Wydział Gospodarczy Krajowego Rejestru Sądowego; NIP 701-08-94-497; REGON 381918620; kapitał zakładowy 1.277.500.000,00 zł
Administratorem danych osobowych przekazanych przez Panią/Pana m.in. w korespondencji mailowej jest Centralny Port Komunikacyjny Sp. z o.o. z siedzibą w Warszawie. Przetwarzamy dane osobowe zgodnie z przepisami ogólnego rozporządzenia o ochronie danych (RODO), więcej informacji na ten temat znajduje się w zakładce Polityka Prywatności oraz w Klauzuli informacyjnej na stronie internetowej www.cpk.pl<http://www.cpk.pl/>.
Treści zawarte w niniejszej wiadomości i załącznikach do niej stanowią Tajemnicę Przedsiębiorstwa w rozumieniu ustawy z dnia 16 kwietnia 1993 r. o zwalczaniu nieuczciwej konkurencji. Jeśli otrzymałeś tę wiadomość przez pomyłkę, bezzwłocznie skontaktuj się z nadawcą wiadomości oraz usuń jej treść.
Centralny Port Komunikacyjny Sp. z o.o. with headquarters in Warsaw, Aleje Jerozolimskie 142B, 02-305 Warsaw; KRS No. 0000759991, District Court for the Capital City of Warsaw, 12th Commercial Department of the National Court Register; NIP 701-08-94-497; REGON 381918620; share capital of PLN 1.277.500.000,00.
The personal data controller of the personal data provided by you, among others, in the e-mail correspondence, is Centralny Port Komunikacyjny Sp. z o. o. based in Warsaw. The personal data is processed by us in accordance with the provisions of the General Data Protection Regulation (GDPR), for further information please read the Privacy Policy tab and the Information Clause on the www.cpk.pl<http://www.cpk.pl/> website.
The content of this message and its attachments constitute the Business Secret within the meaning of the Act of 16 April 1993 on combating unfair competition. If you received this message by mistake, contact the sender of the message immediately and delete its content
From: QGIS-User <qgis-user-bounces at lists.osgeo.org> On Behalf Of Árni Geirsson via QGIS-User
Sent: Thursday, March 16, 2023 10:18 AM
To: jhubbslist at att.net
Cc: qgis-user at lists.osgeo.org
Subject: [EXTERNAL] Re: [Qgis-user] Geopackage slow on NAS if not read-only

UWAGA: Ta wiadomość pochodzi spoza CPK Sp. z o.o. Proszę pomóż zadbać o bezpieczeństwo naszej organizacji. Zastanów się, zanim otworzysz link lub załącznik.

Yes, as far as I know, the NAS unit uses SMB as the file sharing protocol. I used another NAS from QNAP before this one, also using SMB and with the same problem. I thought pretty much all of the Linux based NAS units were using SMB and if that is the problem, it should be widespread, but I don't see any signs of that. Is SMB a problem for geopackage?
I did a quick test in QGIS: A dataset of 178.000 line features is rendered in about 1 second from a read only geopackage. When I remove the read only flag, it is rendered in about 5 seconds.
If I store the geopackage on the local hard drive, the problem disappears, read-write or read-only does not matter.
I have had suspicions about SMB being part of the problem but I don't know enough about file access deep down in the operating system to understand it.

Árni Geirsson

On Thu, 16 Mar 2023 at 01:45, jhubbslist--- via QGIS-User <qgis-user at lists.osgeo.org<mailto:qgis-user at lists.osgeo.org>> wrote:
Árni -

Are you using SMB/CIFS to access this NAS, and are you using wifi or Ethernet to connect to it?

- Jeff

On 3/15/23 3:30 PM, Árni Geirsson via QGIS-User wrote:
Hello all QGIS and geopackage users.
I store my geopackages on a Synology RackStation NAS unit, like all other documents that are kept on a shared drive in the office. For larger datasets, the rendering is very slow, unless I open the properties dialog for the file in Windows and check the read only box. After that, the features are rendered blazingly fast. Nothing else is changed to see the dramatic difference in the rendering speed. Luckily, I don't need to edit many of the larger datasets, such as road networks and elevation contours and the geopackage can be kept read only. Shapefiles are not affected.
What explains this and does anyone know how to solve the problem?
Do other users experience this?

Árni Geirsson

_______________________________________________

QGIS-User mailing list

QGIS-User at lists.osgeo.org<mailto:QGIS-User at lists.osgeo.org>

List info: https://lists.osgeo.org/mailman/listinfo/qgis-user<https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fqgis-user&data=05%7C01%7Cjaroslaw.sadowski%40cpk.pl%7C11269f199d644f13364608db25ff69c5%7Cfa798250ca0b4a1bb47381cff4a1752b%7C1%7C0%7C638145551176051605%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4NfOOZrB2k%2B7Q3V0QHNXIvEOTYHFWp4D10C1MzYTTQQ%3D&reserved=0>

Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user<https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fqgis-user&data=05%7C01%7Cjaroslaw.sadowski%40cpk.pl%7C11269f199d644f13364608db25ff69c5%7Cfa798250ca0b4a1bb47381cff4a1752b%7C1%7C0%7C638145551176051605%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4NfOOZrB2k%2B7Q3V0QHNXIvEOTYHFWp4D10C1MzYTTQQ%3D&reserved=0>

_______________________________________________
QGIS-User mailing list
QGIS-User at lists.osgeo.org<mailto:QGIS-User at lists.osgeo.org>
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user<https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fqgis-user&data=05%7C01%7Cjaroslaw.sadowski%40cpk.pl%7C11269f199d644f13364608db25ff69c5%7Cfa798250ca0b4a1bb47381cff4a1752b%7C1%7C0%7C638145551176051605%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4NfOOZrB2k%2B7Q3V0QHNXIvEOTYHFWp4D10C1MzYTTQQ%3D&reserved=0>
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user<https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fqgis-user&data=05%7C01%7Cjaroslaw.sadowski%40cpk.pl%7C11269f199d644f13364608db25ff69c5%7Cfa798250ca0b4a1bb47381cff4a1752b%7C1%7C0%7C638145551176051605%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4NfOOZrB2k%2B7Q3V0QHNXIvEOTYHFWp4D10C1MzYTTQQ%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20230316/1b623757/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: transparency_ce4012c8-87e7-4198-a48f-d42a38680681.png
Type: image/png
Size: 2458 bytes
Desc: transparency_ce4012c8-87e7-4198-a48f-d42a38680681.png
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20230316/1b623757/attachment.png>