[Qgis-user] [EXTERNAL] Re: Geopackage slow on NAS if not read-only

Thu Mar 16 04:30:21 PDT 2023

Thank you Jarosław. Isn't it strange that this was discussed 5 years ago
and SMB file sharing is very common? Would a linux based NAS be able to use
another protocol? What are my options for file based data sharing in QGIS?
Abandoning geopackages is not a realistic option for me, but I could get a
different kind of NAS unit, if that helps, what kind then? What amazes me
is how little I see this discussed. There was a message in this thread this
morning from Thomas Struller, but I am not sure it is about the same root
problem, maybe Thomas can elaborate.
Should this perhaps be discussed in another forum, closer to the
development of geopackage/sqlite?

Árni Geirsson

On Thu, 16 Mar 2023 at 09:52, Sadowski Jarosław <jaroslaw.sadowski at cpk.pl>
wrote:

> Long story short: gpkg is bad idea for network drives as SMB/NAS etc
>
>
>
> Sources:
> Write-Ahead Logging (sqlite.org)
> <https://www.sqlite.org/wal.html#advantages>
>
> writing gpkg and sqlite on samba shares fails · Issue #628 · r-spatial/sf
> · GitHub <https://github.com/r-spatial/sf/issues/628>
>
>
>
>
>
>
> *_________________________________*
>
> *Jarosław Sadowski*
> Kierownik Zespołu ds. Ochrony Środowiska | *Biuro Strategii i Planowania,
> Projektowania i Inżynierii Podprogramu Kolejowego*
> *Environmental Protection Team Leader **| Railway Subprogramme Strategy &
> Planning, Design & Engineering Department*
> e: jaroslaw.sadowski at cpk.pl
> m: +48 532 720 230
>
> Centralny Port Komunikacyjny Sp. z o.o. z siedzibą w Warszawie, Aleje
> Jerozolimskie 142B, 02-305 Warszawa; nr KRS 0000759991, Sąd Rejonowy dla
> m.st. Warszawy, XII Wydział Gospodarczy Krajowego Rejestru Sądowego; NIP
> 701-08-94-497; REGON 381918620; kapitał zakładowy 1.277.500.000,00 zł
> Administratorem danych osobowych przekazanych przez Panią/Pana m.in. w
> korespondencji mailowej jest Centralny Port Komunikacyjny Sp. z o.o. z
> siedzibą w Warszawie. Przetwarzamy dane osobowe zgodnie z przepisami
> ogólnego rozporządzenia o ochronie danych (RODO), więcej informacji na ten
> temat znajduje się w zakładce Polityka Prywatności oraz w Klauzuli
> informacyjnej na stronie internetowej www.cpk.pl.
> Treści zawarte w niniejszej wiadomości i załącznikach do niej stanowią
> Tajemnicę Przedsiębiorstwa w rozumieniu ustawy z dnia 16 kwietnia 1993 r. o
> zwalczaniu nieuczciwej konkurencji. Jeśli otrzymałeś tę wiadomość przez
> pomyłkę, bezzwłocznie skontaktuj się z nadawcą wiadomości oraz usuń jej
> treść.
> Centralny Port Komunikacyjny Sp. z o.o. with headquarters in Warsaw, Aleje
> Jerozolimskie 142B, 02-305 Warsaw; KRS No. 0000759991, District Court for
> the Capital City of Warsaw, 12th Commercial Department of the National
> Court Register; NIP 701-08-94-497; REGON 381918620; share capital of PLN
> 1.277.500.000,00.
> The personal data controller of the personal data provided by you, among
> others, in the e-mail correspondence, is Centralny Port Komunikacyjny Sp. z
> o. o. based in Warsaw. The personal data is processed by us in accordance
> with the provisions of the General Data Protection Regulation (GDPR), for
> further information please read the Privacy Policy tab and the Information
> Clause on the www.cpk.pl website.
> The content of this message and its attachments constitute the Business
> Secret within the meaning of the Act of 16 April 1993 on combating unfair
> competition. If you received this message by mistake, contact the sender of
> the message immediately and delete its content
>
> *From:* QGIS-User <qgis-user-bounces at lists.osgeo.org> *On Behalf Of *Árni
> Geirsson via QGIS-User
> *Sent:* Thursday, March 16, 2023 10:18 AM
> *To:* jhubbslist at att.net
> *Cc:* qgis-user at lists.osgeo.org
> *Subject:* [EXTERNAL] Re: [Qgis-user] Geopackage slow on NAS if not
> read-only
>
>
>
> UWAGA: Ta wiadomość pochodzi spoza CPK Sp. z o.o. Proszę pomóż zadbać o
> bezpieczeństwo naszej organizacji. Zastanów się, zanim otworzysz link lub
> załącznik.
>
>
>
> Yes, as far as I know, the NAS unit uses SMB as the file sharing protocol.
> I used another NAS from QNAP before this one, also using SMB and with the
> same problem. I thought pretty much all of the Linux based NAS units were
> using SMB and if that is the problem, it should be widespread, but I don't
> see any signs of that. Is SMB a problem for geopackage?
>
> I did a quick test in QGIS: A dataset of 178.000 line features is rendered
> in about 1 second from a read only geopackage. When I remove the read only
> flag, it is rendered in about 5 seconds.
>
> If I store the geopackage on the local hard drive, the problem disappears,
> read-write or read-only does not matter.
>
> I have had suspicions about SMB being part of the problem but I don't know
> enough about file access deep down in the operating system to understand it.
>
>
>
> Árni Geirsson
>
>
>
>
>
>
>
> On Thu, 16 Mar 2023 at 01:45, jhubbslist--- via QGIS-User <
> qgis-user at lists.osgeo.org> wrote:
>
> Árni -
>
>
>
> Are you using SMB/CIFS to access this NAS, and are you using wifi or
> Ethernet to connect to it?
>
>
>
> - Jeff
>
>
>
> On 3/15/23 3:30 PM, Árni Geirsson via QGIS-User wrote:
>
> Hello all QGIS and geopackage users.
>
> I store my geopackages on a Synology RackStation NAS unit, like all other
> documents that are kept on a shared drive in the office. For larger
> datasets, the rendering is very slow, unless I open the properties dialog
> for the file in Windows and check the read only box. After that, the
> features are rendered blazingly fast. Nothing else is changed to see the
> dramatic difference in the rendering speed. Luckily, I don't need to edit
> many of the larger datasets, such as road networks and elevation contours
> and the geopackage can be kept read only. Shapefiles are not affected.
>
> What explains this and does anyone know how to solve the problem?
>
> Do other users experience this?
>
>
>
> Árni Geirsson
>
>
>
> _______________________________________________
>
> QGIS-User mailing list
>
> QGIS-User at lists.osgeo.org
>
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user <https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fqgis-user&data=05%7C01%7Cjaroslaw.sadowski%40cpk.pl%7C11269f199d644f13364608db25ff69c5%7Cfa798250ca0b4a1bb47381cff4a1752b%7C1%7C0%7C638145551176051605%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4NfOOZrB2k%2B7Q3V0QHNXIvEOTYHFWp4D10C1MzYTTQQ%3D&reserved=0>
>
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user <https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fqgis-user&data=05%7C01%7Cjaroslaw.sadowski%40cpk.pl%7C11269f199d644f13364608db25ff69c5%7Cfa798250ca0b4a1bb47381cff4a1752b%7C1%7C0%7C638145551176051605%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4NfOOZrB2k%2B7Q3V0QHNXIvEOTYHFWp4D10C1MzYTTQQ%3D&reserved=0>
>
>
>
> _______________________________________________
> QGIS-User mailing list
> QGIS-User at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> <https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fqgis-user&data=05%7C01%7Cjaroslaw.sadowski%40cpk.pl%7C11269f199d644f13364608db25ff69c5%7Cfa798250ca0b4a1bb47381cff4a1752b%7C1%7C0%7C638145551176051605%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4NfOOZrB2k%2B7Q3V0QHNXIvEOTYHFWp4D10C1MzYTTQQ%3D&reserved=0>
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
> <https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fqgis-user&data=05%7C01%7Cjaroslaw.sadowski%40cpk.pl%7C11269f199d644f13364608db25ff69c5%7Cfa798250ca0b4a1bb47381cff4a1752b%7C1%7C0%7C638145551176051605%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4NfOOZrB2k%2B7Q3V0QHNXIvEOTYHFWp4D10C1MzYTTQQ%3D&reserved=0>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20230316/dd91d223/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: transparency_ce4012c8-87e7-4198-a48f-d42a38680681.png
Type: image/png
Size: 2458 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20230316/dd91d223/attachment-0001.png>