[gdal-dev] Zarr BLOCKSIZE in gdalmdimtranslate with variables of varying dimension

[Simon Lyngby Kokkendorff]

Hi list,

  gdalmdimtranslate is a very handy tool e.g. to translate netCDF
files to Zarr as it can handle huge files that xarray just chokes on.
   I have a question regarding handling output blocksizes for the Zarr
format for files with variables of varying dimensions though. For
example I need to translate a netCDF file with time, z, lat, lon
dimensions and variables which are either 4D (time, z, lat, lon), 3D
(time, lat, lon) and 2D (lat, lon). I would like to remap chunksizes /
blocksizes to better suit my use case. However, it seems that I can
only specify blocksizes as e.g. -co ARRAY:BLOCKSIZE 2, 2, 128, 128 -
which will throw an error and skip the variables that don't  have 4
dimensions. Also, the output doesn't contain the dimension variables
in this case. Likewise, for 4D variables if I only specify 3
blocksizes.
  Have tried this with gdal versions 3.8.4 and 3.4.1 with similar results.
   So the question is if there is any way to specify blocksizes e.g.
by dimension name? Or if the approach should rather be to just create
3 output Zarr folders - one for each variable dimensionality (2D, 3D
and 4D)? I can of course manually "merge" these folders afterwards.

Best regards,
Simon