[gdal-dev] GeoTIFF and concurrent block reads

[Laurențiu Nicola]

Hi,

On Thu, Sep 21, 2023, at 17:02, Even Rouault wrote:
> Laurentiu,
> 
>> 
>> GDAL 3.6 added support for multi-threaded reading using PRead, but I couldn't spot ReadBlock using the same code path.
> If you read one single block at a time, the multi-threaded optimization cannot kick in, since the elementary decoding unit is a block. You must call RasterIO() with a window intersecting several block.
> 
Of course, but I'm asking if it's worth calling ReadBlock on multiple threads (if it always takes a lock, it's not, and I should use RasterIO instead).
>> 
>> (*) I think these can also be implemented at VSI level by wrapping a file handle and turning Seek + Read into a PRead (when supported), even for drivers that don't explicitly support PRead.
> The multi-threaded optimization also works for VSI file systems not supported PRead(), but doing what you mention above: doing Seek+Read under a lock. This is of course less efficient.
> 
I meant something slightly different here: if a file supports PRead, but a driver doesn't (maybe it's using some external library like libtiff, which has _tiffReadProc and _tiffSeekProc), threading could still be supported with a hack. The driver could ask the library to open the same file multiple times, wrapping the same VSI handle, but passing in a seekProc that only keeps track of the current offset. So the library could seek and read as it pleases, but GDAL would turn those into PRead calls. The backing file handle would never see a seek, just PRead.

Laurentiu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20230921/cfee4bbb/attachment.htm>