[gdal-dev] netCDF chunk fetch failed

[Julien Demaria]

Kurt,

I tried the latest version of NetCDF’s nccopy utility on your files and it also failed.  Same with an old NetCDF v4.2/HDF5 v1.8.9 version.
So the files seem wrong/corrupted for an unknow reason, or created with a specific NetCDF/HDF5 version which include a compatibility break.
Did you get answer from NOAA?
I’m not sure if GDAL should handle this case (skip failed chunks and set them to nodata).

Best,
Julien


De : gdal-dev <gdal-dev-bounces at lists.osgeo.org> De la part de Kurt Schwehr
Envoyé : mardi 12 janvier 2021 00:52
À : gdal dev <gdal-dev at lists.osgeo.org>
Objet : [gdal-dev] netCDF chunk fetch failed

Hi all,

In a build of gdal 3.2.0, I'm seeing this error: "netCDF chunk fetch failed" for 2 out of 5 subdatasets.  Is there an easy way to handle this so the bad data is returned as the nodata value?  And also, any idea what is actually wrong with the netcdf files?

I am in contact with the NOAA folks who made this data.

Thanks!
-kurt



gsutil cp gs://gcp-public-data-goes-16/ABI-L2-FDCF/2017/263/21/OR_ABI-L2-FDCF-M3_G16_s20172632115407_e20172632126173_c20172632126283.nc .
gsutil cp gs://gcp-public-data-goes-16/ABI-L2-FDCF/2018/123/12/OR_ABI-L2-FDCF-M3_G16_s20181231215382_e20181231226149_c20181231226258.nc .
gsutil cp gs://gcp-public-data-goes-16/ABI-L2-FDCF/2018/324/18/OR_ABI-L2-FDCF-M3_G16_s20183241845341_e20183241856108_c20183241856213.nc .
gsutil cp gs://gcp-public-data-goes-16/ABI-L2-FDCF/2020/247/16/OR_ABI-L2-FDCF-M6_G16_s20202471650186_e20202471659494_c20202471700318.nc .


for subdataset in Area Temp Mask Power DQF; do
    echo $subdataset
    (gdalinfo -stats NETCDF:OR_ABI-L2-FDCF-M3_G16_s20183241845341_e20183241856108_c20183241856213.nc:$subdataset | grep 'Band 1');
done

Area
Warning 1: netCDFDataset::valid_range: min > max:
  min: 0.000000
  max: -6.000000

ERROR 1: netCDF chunk fetch failed: #-101 (NetCDF: HDF error)
ERROR 1: NETCDF:OR_ABI-L2-FDCF-M3_G16_s20183241845341_e20183241856108_c20183241856213.nc:Area, band 1: IReadBlock failed at X offset 19, Y offset 10: netCDF chunk fetch failed: #-101 (NetCDF: HDF error)
Band 1 Block=226x226 Type=Int16, ColorInterp=Undefined
Temp
Warning 1: netCDFDataset::valid_range: min > max:
  min: 0.000000
  max: -6.000000
Band 1 Block=226x226 Type=Int16, ColorInterp=Undefined
Mask
ERROR 1: netCDF chunk fetch failed: #-101 (NetCDF: HDF error)
ERROR 1: NETCDF:OR_ABI-L2-FDCF-M3_G16_s20183241845341_e20183241856108_c20183241856213.nc:Mask, band 1: IReadBlock failed at X offset 4, Y offset 11: netCDF chunk fetch failed: #-101 (NetCDF: HDF error)
Band 1 Block=226x226 Type=Int16, ColorInterp=Undefined
Power
Band 1 Block=226x226 Type=Float32, ColorInterp=Undefined
DQF
Band 1 Block=226x226 Type=Byte, ColorInterp=Undefined
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20210112/568333bc/attachment.html>