[gdal-dev] Multithreading issue on pixel-based intensive access (GDAL 2.1.0)

[Even Rouault]

Javier,

Your use case looks like a legit use case of the API. I'm not aware of bugs 
regarding to locking of the raster block cache, but that doesn't mean that 
they aren't any. Could you prepare a self contained source code + dataset that 
would reproduce the issue ?

As far as I can see, GDALRasterBlock::Touch_unlocked() should only be invoked 
inside a mutex, so I wouldn't expect races.

We have a test program that stress tests the block cache, but at a higher 
level (RasterIO) : 
https://svn.osgeo.org/gdal/trunk/autotest/cpp/testblockcache.cpp
It is run in our continuous integration and works fine.

Even

> Hi guys, I'm experiencing an issue when running some simple tests
> involving multithreading. I would like to know if it is something I am
> not doing correctly with GDAL or it is a GDAL business dealing with
> threads and block cache.
> 
> To make long story short, just imagine two threads accessing different
> raster dataset (different files) and looping through all of its pixels
> to read its value. So, for each pixel I do something like:
> 
> // Lock.
> GDALRasterBlock* poBlock = m_poRB->GetLockedBlockRef(blockXIndex,
> blockYIndex);
> void* pData = poBlock->GetDataRef();
> 
> // Access value.
> T value = ((T*)pData)[blockOffset];
> 
> // Unlock.
> poBlock->DropLock();
> 
> 
> I know this is a very aggressive way to access the pixel value as I'm
> locking/unlocking in a pixel basis instead of a block basis (I mean
> here, locking the block just once and reading all of its pixels, and
> then unlocking), and also it may decrease performance. But I'm limited
> by some requirements in my application legacy code and there is not much
> I can do.
> 
> The problem with this implementation is that I get an assertion failed
> in gdalrasterblock.cpp line 800, which is in summary, the moment where a
> 'touched' block is moved on top of the LRU block cache list. It seems
> that a race condition may be happening between threads.
> 
> So my question is: this way of accessing pixels with so many block
> locks/unlocks is fully supported by GDAL as thread-safe? Or is it a bug
> in GDAL's block cache management?
> 
> Thank you so much for your time and help.
> 
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com