[gdal-dev] netCDF with virtual file system (/vsicurl/) not working

[Stefan Blumentrath]

Thanks so much for clarifying, Even.
Very much appreciated.

Cheers,
Stefan

From: Even Rouault <even.rouault at spatialys.com>
Sent: onsdag 2. februar 2022 11:15
To: Stefan Blumentrath <Stefan.Blumentrath at nina.no>; Rahkonen Jukka (MML) <jukka.rahkonen at maanmittauslaitos.fi>; gdal-dev <gdal-dev at lists.osgeo.org>
Subject: Re: [gdal-dev] netCDF with virtual file system (/vsicurl/) not working


The issue with userfaultfd not working with modern Linux kernels was fixed in 3.4.0 per https://github.com/OSGeo/gdal/pull/4508/commits/f861fd3fc6826ab753e0155f15507494d54f7ecb<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgdal%2Fpull%2F4508%2Fcommits%2Ff861fd3fc6826ab753e0155f15507494d54f7ecb&data=04%7C01%7CStefan.Blumentrath%40nina.no%7C695dd2841dda4047724608d9e634e2f5%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637793937096379254%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ZwQ1uLLPdq46e6s5om%2FK5hTFndj7mzppoWX78BRT8Xc%3D&reserved=0>

/vsicurl/ on a netCDF file, with the netCDF driver, cannot work on non-Linux systems. The HDF5 driver will then trigger.
Le 02/02/2022 à 10:01, Stefan Blumentrath a écrit :
Hi,

And thanks for your swift reply, Jukka.
I tried different versions on Linux (Ubuntu: ubuntugis-stable and ubuntugis-unstable):
3.3.2, 3.4.0
Fedora:
3.3.2

On Windows the latest OSGeo4W provided version:
3.4.1 (and there it does not work)

On my Fedora box however, I now compiled GDAL 3.4.1, there it seems to work again...

Cheers
Stefan

From: Rahkonen Jukka (MML) <jukka.rahkonen at maanmittauslaitos.fi><mailto:jukka.rahkonen at maanmittauslaitos.fi>
Sent: onsdag 2. februar 2022 09:49
To: Stefan Blumentrath <Stefan.Blumentrath at nina.no><mailto:Stefan.Blumentrath at nina.no>; gdal-dev <gdal-dev at lists.osgeo.org><mailto:gdal-dev at lists.osgeo.org>
Subject: Re: netCDF with virtual file system (/vsicurl/) not working

Hi,

What is your GDAL version? Check with "gdalinfo --version"

-Jukka Rahkonen-

Lähettäjä: gdal-dev <gdal-dev-bounces at lists.osgeo.org<mailto:gdal-dev-bounces at lists.osgeo.org>> Puolesta Stefan Blumentrath
Lähetetty: keskiviikko 2. helmikuuta 2022 10.36
Vastaanottaja: gdal-dev <gdal-dev at lists.osgeo.org<mailto:gdal-dev at lists.osgeo.org>>
Aihe: [gdal-dev] netCDF with virtual file system (/vsicurl/) not working

Hi,

I have been using GDAL to access online data sources in netCDF successfully.
With Ubuntu 20.04, this unfortunately no longer works with the netCDF driver.

Here is a reproducible example:
gdalinfo -if netCDF /vsicurl/https://nbstds.met.no/thredds/fileServer/NBS/S2A/2021/02/28/S2A_MSIL1C_20210228T103021_N0202_R108_T35WPU_20210228T201033_DTERRENGDATA.nc<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnbstds.met.no%2Fthredds%2FfileServer%2FNBS%2FS2A%2F2021%2F02%2F28%2FS2A_MSIL1C_20210228T103021_N0202_R108_T35WPU_20210228T201033_DTERRENGDATA.nc&data=04%7C01%7CStefan.Blumentrath%40nina.no%7C695dd2841dda4047724608d9e634e2f5%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637793937096379254%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=JeCeCXg8GaE%2BbEqcfCZtSqSyhYenGCJGEXeSgRcj1hE%3D&reserved=0>

The original error message (before working around the HDF5 driver) has been that permissions for the userfaultfd system call were denied. But if I specify the netCDF driver, the format is not recognized. Downloading the dataset and opening It locally works.

I saw some possibly relevant changes:
https://github.com/OSGeo/gdal/pull/4508<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgdal%2Fpull%2F4508&data=04%7C01%7CStefan.Blumentrath%40nina.no%7C695dd2841dda4047724608d9e634e2f5%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637793937096379254%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=kC9KX9p8z8Li5%2B2Bz0n9mnIqVe7PGXZvJWmnk3nxwTs%3D&reserved=0>
https://github.com/OSGeo/gdal/issues/2412<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgdal%2Fissues%2F2412&data=04%7C01%7CStefan.Blumentrath%40nina.no%7C695dd2841dda4047724608d9e634e2f5%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637793937096379254%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=0%2BaNjv%2FzsERgvjae4m6w7OmLwyyeI8hwmWweTjfk%2FQA%3D&reserved=0>

Any hint on how to get that functionality back would be very much appreciated!

Cheers
Stefan



_______________________________________________

gdal-dev mailing list

gdal-dev at lists.osgeo.org<mailto:gdal-dev at lists.osgeo.org>

https://lists.osgeo.org/mailman/listinfo/gdal-dev<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fgdal-dev&data=04%7C01%7CStefan.Blumentrath%40nina.no%7C695dd2841dda4047724608d9e634e2f5%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637793937096379254%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=u3923jDQ0jKgi0RyTPRBpgBa%2FDhBqUGLC3lL9vF9UsE%3D&reserved=0>

--

http://www.spatialys.com<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.spatialys.com%2F&data=04%7C01%7CStefan.Blumentrath%40nina.no%7C695dd2841dda4047724608d9e634e2f5%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637793937096379254%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=%2B6jmR1Ho4RRW%2F9odzAzLd2S%2FGQ6uZx9zOQjKKwaY3L0%3D&reserved=0>

My software is free, but my time generally not.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220202/2efa2f22/attachment.html>