[gdal-dev] Reading NetCDF on S3 with /vsis3/

[Scott Staniewicz]

Are there any known limitations that prevent NetCDF datasets from being
read using `/vsis3/`?

I've created some NetCDF4 datasets with h5netcdf, and they are readable
locally and recognized by gdal. But when I put them on S3, `gdalinfo` only
recognizes them with the HDF5 driver, and fails when I force the use with
NETCDF:/vsis3/my-bucket/my-file.nc:/dataset

I'm wondering if there was some regression, because the command mentioned
here no longer works for me:
https://github.com/conda-forge/gdal-feedstock/issues/376#issuecomment-1320872404

$ gdalinfo --version
GDAL 3.7.2, released 2023/09/05

$ gdalinfo /vsicurl/
http://download.osgeo.org/gdal/data/netcdf/utm-big-chunks.nc
Driver: HDF5Image/HDF5 Dataset
Files: /vsicurl/http://download.osgeo.org/gdal/data/netcdf/utm-big-chunks.nc
Size is 3072, 3072
Metadata:
  CDI=Climate Data Interface version 1.6.0 (http://code.zmaw.de/projects/cdi
)
  CDO=Climate Data Operators version 1.6.0 (http://code.zmaw.de/projects/cdo
)
  Conventions=CF-1.5
  GDAL=GDAL 1.10.0, released 2013/04/13
  history=Tue Apr 16 16:07:46 2013: cdo -f nc4 -z zip -k auto copy utmbig.nc
utmbig2.nc
Tue Apr 16 16:06:59 2013: GDAL Create( utmbig.nc, ... )
Corner Coordinates:
Upper Left  (    0.0,    0.0)
Lower Left  (    0.0, 3072.0)
Upper Right ( 3072.0,    0.0)
Lower Right ( 3072.0, 3072.0)
Center      ( 1536.0, 1536.0)
Band 1 Block=1536x1536 Type=Byte, ColorInterp=Undefined
  Metadata:
    long_name=GDAL Band Number 1
    _FillValue=0


When I upload one of the Netcdf test datasets from gdal (e.g.
https://github.com/OSGeo/gdal/blob/master/autotest/gdrivers/data/netcdf/longitude_latitude.nc)
to my s3 bucket, it fails to read it:


$ aws s3 cp longitude_latitude.nc s3://my-bucket/
upload: longitude_latitude.nc to s3://my-bucket/longitude_latitude.nc

l$ gdalinfo /vsis3/my-bucket/longitude_latitude.nc
ERROR 4: `/vsis3/my-bucket/longitude_latitude.nc' not recognized as a
supported file format.
gdalinfo failed - unable to open '/vsis3/my-bucket/longitude_latitude.nc'.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20231202/288cc65d/attachment.htm>