[gdal-dev] Slow GDALComputeRasterMinMax on nc grids

[Joaquim Manuel Freire Luís]

Nope, rebuilt netCDF but same timings.


-----Original Message-----
From: gdal-dev <gdal-dev-bounces at lists.osgeo.org> On Behalf Of Joaquim Manuel Freire Luís
Sent: Friday, January 3, 2020 6:26 PM
To: Jeff McKenna <jmckenna at gatewaygeomatics.com>; gdal-dev at lists.osgeo.org
Subject: Re: [gdal-dev] Slow GDALComputeRasterMinMax on nc grids

Hi Jeff,

Thanks for testing this. Sure it's useful. It also reminded me that I have a OSGeo4W with GDAL 2.4.1 but don't know which netCDF version, and using it took ... over 6 minutes.
My Windows netCDF lib is a bit old by now (I build it from source). It might be it.


Hi Joaquim,

I have tested your file on Windows 10 here, with the command:

    gdalinfo grav_29_img.nc -mm

which takes about 5 seconds to execute fully.  I am using MS4W 4.0.2 (GDAL 2.4.3 , NetCDF 4.7.3 )

I haven't tested using your wrappers though, so imagine that my feedback is not very useful.

PS. but thanks for sharing the larger grid file, which is useful for testing.

Wishing you a prosperous 2020.

-jeff



--
Jeff McKenna
MapServer Consulting and Training Services https://gatewaygeomatics.com/



On 2020-01-03 1:27 PM, Joaquim Manuel Freire Luís wrote:
> Hi Even,
> 
> New Year, new mysteries. I’m having quite strange slow times in 
> computing the min/max of netCDF files.
> 
> In GMT we can read files via GDAL by appending =gd to the file name. 
> So both of these do a similar job
> 
> grdinfo grav_29_img.nc=gd
> 
> and
> 
> gdalinfo grav_29_img.nc -mm
> 
> and it takes around 28 sec in a new laptop running Windows and master 
> GDAL. However in WSL, same laptop but GDAL 2.2.3, they take about 5:30 
> MINUTES to run. On a OSX it takes about 3:30 minutes (not me who run 
> this, and a GDAL 3.0.3 MacPorts)
> 
> On Windows for files of similar size it run faster when data is of 
> type float (the grid in this example is a short int).
> 
> Now, perhaps the weirdest thing is that if I run the GMT command via 
> our Julia wrapper it only takes ~8 sec, whilst the same command via 
> the Matlab wrapper took 1:18 min
> 
> The ~8 sec time is close to what I get with a pure GMT (i.e. not using 
> GDAL to read the nc file  (grdinfo grav_29_img.nc -M))
> 
>  From the GMT side all happens in the call
> 
>          GDALComputeRasterMinMax(hBand, false, adfMinMax);
> 
> And I guess that the same holds from the pure GDAL call.
> 
> If you want to test with the grid used in these testings, it’s here
> 
> ftp://ftp.soest.hawaii.edu
> <ftp://ftp.soest.hawaii.edu.pwessel>/pwessel/grav_29_img.nc
> <http://grav_29_img.nc>
> 
> Happy new year
> 
> Joaquim
> 
> 
_______________________________________________
gdal-dev mailing list
gdal-dev at lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________
gdal-dev mailing list
gdal-dev at lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev