[gdal-dev] (no subject)

Mon Sep 19 06:31:50 PDT 2022

Could you write a server side script that runs the ogrinfo command with r/w rights to the upload directory?

-Jukka Rahkonen-

Lähettäjä: Moises Calzado <mcalzado at carto.com>
Lähetetty: maanantai 19. syyskuuta 2022 16.00
Vastaanottaja: Rahkonen Jukka <jukka.rahkonen at maanmittauslaitos.fi>
Aihe: Re: [gdal-dev] (no subject)

Oh no! We use ogrinfo to check the file type, as these files are uploaded by other users into a GCS bucket.
We don't apply any change to these files, as we just check them with ogrinfo to perform other operations afterwards, so that's why we can't change the journal mode before executing the ogrinfo command.
If there's no way that we can complete this without modifying it, we'll think about an alternative solution!

Thanks so much for your help!

El lun, 19 sept 2022 a las 14:07, Rahkonen Jukka (<jukka.rahkonen at maanmittauslaitos.fi<mailto:jukka.rahkonen at maanmittauslaitos.fi>>) escribió:
Hi,

What do you mean by processing? Http users cannot edit anything in any case. Do you edit the GPKG file locally at the same time? "Immutable" could be a solution if GDAL can support it somehow, but if the database is edited at the same it does not feel right. Taken from the SQLite documentation:
"immutable: The immutable parameter is a boolean query parameter that indicates that the database file is stored on read-only media. When immutable is set, SQLite assumes that the database file cannot be changed, even by a process with higher privilege, and so the database is opened read-only and all locking and change detection is disabled. Caution: Setting the immutable property on a database file that does in fact change can result in incorrect query results and/or SQLITE_CORRUPT errors. See also: SQLITE_IOCAP_IMMUTABLE."

Do you have some good reason not to change the pragma?

-Jukka Rahkonen-

Lähettäjä: Moises Calzado <mcalzado at carto.com<mailto:mcalzado at carto.com>>
Lähetetty: maanantai 19. syyskuuta 2022 14.46
Vastaanottaja: Rahkonen Jukka <jukka.rahkonen at maanmittauslaitos.fi<mailto:jukka.rahkonen at maanmittauslaitos.fi>>
Aihe: Re: [gdal-dev] (no subject)

Hi Jukka,

Thanks so much for taking a look at this! We already tried to change the journal mode, and it works as expected, but we would like to know if there is any way of processing this files without changing anything on the file.

El lun, 19 sept 2022 a las 13:25, Rahkonen Jukka (<jukka.rahkonen at maanmittauslaitos.fi<mailto:jukka.rahkonen at maanmittauslaitos.fi>>) escribió:
Hi,

Read https://www.sqlite.org/wal.html<https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.sqlite.org%2Fwal.html&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7C92f8891853344ee1f57008da9a3ee90a%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C637991892247543363%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=pa6FZF5AI0XqoyrfkqiqhWhsbfpzBMopKuk%2FSWB9kE4%3D&reserved=0>. I suggest to change the journal mode into the default mode before inserting the GPKG file into server. How to: Execute "PRAGMA journal_mode=DELETE". I tested that it will now work:

INFO: Open of `/vsicurl/http://localhost:8060/SuomenHallinnollisetKuntajakopohjaisetAluejaot_2022_10k.gpkg'
      using driver `GPKG' successful.
1: Aluehallintovirasto
2: Valtakunta
3: Maakunta
4: Kunta

If the GPKG file in WAL mode is already placed on the server it might be possible to open it somehow as "immutable". I do not know how to do that with GDAL.

-Jukka Rahkonen-

Lähettäjä: gdal-dev <gdal-dev-bounces at lists.osgeo.org<mailto:gdal-dev-bounces at lists.osgeo.org>> Puolesta Moises Calzado via gdal-dev
Lähetetty: maanantai 19. syyskuuta 2022 14.08
Vastaanottaja: gdal-dev at lists.osgeo.org<mailto:gdal-dev at lists.osgeo.org>
Aihe: [gdal-dev] (no subject)

Hello everyone,

We're performing some tests with ogrinfo trying to read GPKG files, and we're facing some issues executing the command with remote GPKG files in WAL mode. As can be checked in the following command output, ogr fails while reading the file content:

ERROR: Error: Command failed: ogrinfo /vsicurl/MY_FILE_URL has GPKG application_id, but non conformant file extension ERROR 1: unable to open database file: this file is a WAL-enabled database. It cannot be opened because it is presumably read-only or in a read-only directory. ERROR 1: Only read-only mode is supported for /vsicurl

What is strange here is that the same file works correctly if we execute the command using the downloaded file.

Is there any way we can execute the command using a remote file?

Thanks so much in advance,
Kind regards!
--
Moises Calzado

Support Engineer

(US) +1 917 463 3232 | (ES) +34 911 165 823 | mcalzado at carto.com<mailto:mcalzado at carto.com>
[https://storage.googleapis.com/carto-it-files/signature/SDSC-2022-NY_Signature-mini.jpg]<https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fspatial-data-science-conference.com%2F2022%2Fnewyork%2F&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7C92f8891853344ee1f57008da9a3ee90a%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C637991892247543363%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=lmr8XyGPICyCHyat8AVJMRQsSMvODOLuoQiIwkpJRxw%3D&reserved=0>

--
Moises Calzado

Support Engineer

(US) +1 917 463 3232 | (ES) +34 911 165 823 | mcalzado at carto.com<mailto:mcalzado at carto.com>
[https://storage.googleapis.com/carto-it-files/signature/SDSC-2022-NY_Signature-mini.jpg]<https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fspatial-data-science-conference.com%2F2022%2Fnewyork%2F&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7C92f8891853344ee1f57008da9a3ee90a%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C637991892247699606%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=8BVavG0DX3ruJ%2BxvP3c0rNuF8FowsmjzmKC2Huf7u9w%3D&reserved=0>

--
Moises Calzado

Support Engineer

(US) +1 917 463 3232 | (ES) +34 911 165 823 | mcalzado at carto.com<mailto:mcalzado at carto.com>
[https://storage.googleapis.com/carto-it-files/signature/SDSC-2022-NY_Signature-mini.jpg]<https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fspatial-data-science-conference.com%2F2022%2Fnewyork%2F&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7C92f8891853344ee1f57008da9a3ee90a%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C637991892247699606%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=8BVavG0DX3ruJ%2BxvP3c0rNuF8FowsmjzmKC2Huf7u9w%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220919/98d40f43/attachment-0001.htm>