[gdal-dev] slow netCDF read times

[Pablo Rozas Larraondo]

Hello,

I've come across some NetCDF4 files where GDAL is taking a surprisingly
long time to read data from them. For example this is an example public
file containing precipitation data:

ftp://ftp.chg.ucsb.edu/pub/org/chg/products/CHIRPS-2.0/
global_dekad/netcdf/chirps-v2.0.2015.dekads.nc

If I use GDAL to read a small top left block (500x500) from one of its time
bands, it takes approximately 1 minute on my computer. Source code is
available here:

https://gist.github.com/monkeybutter/769a24bcf87682171eb87ac05c9347c5

The equivalent operation is completed in less than a second using the
NetCDF library and even reading the whole file takes around 6 seconds with
the same library.

I've tried to profile the GDAL program to get more insight and understand
what's causing the overhead with not much success. All I know is that the
deflate function is using 96% of the resources. I also guess that the way
this file is chunked has something to do with its performance. Can anyone
suggest any idea for better understanding what's happening here?

Thank you for your help,
Pablo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20161122/87556c73/attachment.html>