[gdal-dev] Slow GDALComputeRasterMinMax on nc grids

[Even Rouault]

On vendredi 3 janvier 2020 14:02:18 CET Jeff McKenna wrote:
> Hi Joaquim,
> 
> I have tested your file on Windows 10 here, with the command:
> 
>     gdalinfo grav_29_img.nc -mm
> 
> which takes about 5 seconds to execute fully.  I am using MS4W 4.0.2
> (GDAL 2.4.3 , NetCDF 4.7.3 )

Jeff,

Do you build netCDF with a custom value of the CHUNK_CACHE_SIZE / chunk-cache-
size setting ?

With the default (4 MB), runtime on my machine is about 7 minutes. But if I 
use --with-chunk-cache-size=67108864 as in
https://trac.osgeo.org/gdal/ticket/5291#comment:26 , then it goes down to
9 sec.

So this is a quite old known issue with the netCDF driver with HDF5 chunking & 
compression. Due to the netCDF driver exposing rasters with a north origin, 
and netCDF Y origin being at south, there's a mismatch between GDAL blocks and 
netCDF chunks. So for the sake of simplicity, the netCDF driver exposes one 
single line as the GDAL block size. It could/should be improved to make a 
better use of netCDF chunks, so as not being too much dependent of the quite 
pessimistic default of the netCDF chunk_cache_size.

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com