[gdal-dev] NetCDF on S3?

[duncfw at gmail.com]

Hello,
I'm trying to open a NetCDF file from S3 with Gdal and I'm hitting some
problems and I'm not sure whether they are limitations of the driver or I'm
just not using it right.

I have two NetCDF files, one is (small) sample file from the Copernicus
File Danger Index, the other is one month of data (1.5GB) from the ECMWF
Reanalysis 2m Temperature variable. The first file opens file from my local
file system and using the /vsis3 virtual filesystem but the second one
doesn't.

With CPL_DEBUG=ON, successfully opening the file with gdalinfo looks like
this:

GNM: GNMRegisterAllInternal
GNM: RegisterGNMFile
GNM: RegisterGNMdatabase
GDAL_netCDF: driver detected file type=2, libnetcdf detected type=2
GDAL_netCDF: dim_count = 3
GDAL_netCDF: var_count = 4
GDAL_netCDF:
=====
SetProjectionFromVar( 3)
GDAL_netCDF: bIsGdalFile=0 bIsGdalCfFile=0 bBottomUp=1
GDAL_netCDF: set bBottomUp = 0 from Y axis
GDAL_netCDF: xdim: 1440 nSpacingBegin: 250 nSpacingMiddle: 250 nSpacingLast: 250
GDAL_netCDF: ydim: 721 nSpacingBegin: -250 nSpacingMiddle: -250
nSpacingLast: -250
GDAL_netCDF: SetGeoTransform(-0.125000,0.250000,0.000000,90.125000,0.000000,-0.250000)
GDAL_netCDF: bGotGeogCS=0 bGotCfSRS=0 bGotCfGT=1 bGotGdalSRS=0 bGotGdalGT=0
GDAL_netCDF: netcdf type=3 gdal type=3 signedByte=1
...

The same file when uploaded to S3 outputs the following using the
vsis3 virtual filesystem:

GNM: GNMRegisterAllInternal
GNM: RegisterGNMFile
GNM: RegisterGNMdatabase
HTTP: libcurl/7.69.1 OpenSSL/1.1.1g zlib/1.2.5 libssh2/1.8.2
HTTP: GDAL was built against curl 7.64.1, but is running against 7.69.1.
S3: Switching to region eu-west-1
VSICURL: Downloading 0-16383
(https://some-s3-bucket.s3.amazonaws.com/era5-reanalysis/2m_temperature/2020-01.nc)...
VSICURL: Got response_code=206
ERROR 4: `/vsis3/some-s3-bucket/era5-reanalysis/2m_temperature/2020-01.nc'
not recognized as a supported file format.
gdalinfo failed - unable to open
'/vsis3/some-s3-bucket/era5-reanalysis/2m_temperature/2020-01.nc'.


Is it clear from this output whether there is an issue with the 2020-01.nc file?

Thanks,

Duncan Walker
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20210112/46b3c105/attachment.html>