[gdal-dev] gdal_translate shifted values

[Laurențiu Nicola]

It's actually pretty easy to test:

import numpy as np
import xarray as xr
from osgeo import gdal

xarray_ds = xr.load_dataset("st4_pr.2017092016.01h", engine="cfgrib")
# GDAL uses 9999 as NODATA
xarray_data = np.nan_to_num(xarray_ds.tp.data, nan=9999.0)
gdal_ds = gdal.Open("st4_pr.2017092016.01h")
gdal_data = gdal_ds.GetRasterBand(1).ReadAsArray()

>>> gdal_data[80]
array([9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   ,   40.5  ,   41.125,   41.75 ,   42.5  ,
         43.   ,   26.5  ,   26.875,   27.   ,   29.875,   49.875,
         50.375,   41.5  ,   41.375,    0.   ,    0.   ,    0.   ,
          0.   ,    0.   ,    0.   ,    0.   ,    0.   ,    0.   ,
          0.   ,    0.   ,    0.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
          0.   ,    0.   ,    0.   ,    0.   ,    0.   ,    0.   ,
          0.   ,    0.   ,    0.   ,    0.   ,    0.   ,    0.   ,
          0.   ,    0.   ,    0.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   ])
>>> xarray_data[80]
array([9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   ,   13.75 ,   14.25 ,   14.75 ,
         19.125,   32.5  ,   30.125,   27.125,   25.5  ,   22.25 ,
         22.375,   20.25 ,   18.375,   16.625,   15.25 , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   , 9999.   , 9999.   , 9999.   , 9999.   , 9999.   ,
       9999.   ], dtype=float32)

The values are obviously different.

I also tried to convert to NetCDF and open them in QGIS, but they look almost the same (not identical) to me after fixing up the coordinates, so there's probably more at play.

Again, comparing the gdal_translate output with the GRIB data as read by GDAL can't show any issues.

Laurentiu

On Thu, Feb 1, 2024, at 13:24, Laurențiu Nicola via gdal-dev wrote:
> Hi Jukka,
> 
> If GDAL shuffled around the pixels in that file, you wouldn't be able to notice it by comparing the GRIB and the TIFF, because both would be shuffled.
> 
> My suggestion was to convert the GRIB to a NC or even CSV using xarray, then comparing the GDAL output against that.
> 
> Laurentiu
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240201/65267d0d/attachment-0001.htm>