[gdal-dev] Geoid read performance

[Vautour, André (INT)]

Hi all,

I am using GDAL to read some geoid files in order to do some vertical datum transformations. The transformation engine I am using is done point by point transformations and doing a grid lookup for each point, so this means when transforming say a million points, it is doing a two million grid lookups (one for the source, and one for the destination).

After a performance profiling run, it became clear that much of the slowdown is because of mutexing. Most of the Geoid formats are based on the RawDataset, which is mutexing on every read for two different reasons. While one is about the accessing of the block cache, which for obvious reasons cannot be avoided, but, the biggest bottleneck was the CPLGetConfigOption("GDAL_ONE_BIG_READ") call in RawRasterBand::CanUseDirectIO(). I am wondering if it would make sense to move that call to the constructor and store the result for future use? Or, would we expect that setting to change during the lifetime of the raster, and the raster to react dynamically to those option changes?

Since most geoids are really small grids, I opted to try to copy the geoid to a MEMDataset raster. That had the benefit of avoiding the CPLGetConfigOption() bottleneck and also avoiding the block cache. That being said, the memory raster is always set to an access mode of GA_Update. That means that reading will also try to acquire a mutex in GDALDataset::EnterReadWrite. Would it make sense to be able to either specify the access mode of a MEMDataset to GA_ReadOnly or to add a SetAccess() method to it similar to RawRasterBand so that it can be changed to read-only after the initial copy was done?

Keep in mind that I am willing to make and contribute the necessary changes. I just want to get a feel as to what would make sense at the general level if any such changes are required. Also, do you have any other suggestions on how to avoid such mutexes when reading what is essentially static data?

Kind regards,
André Vautour
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20201027/1d9b0251/attachment.html>