[gdal-dev] Block cache and VRT sources

[Sean Gillies]

Thanks, Even. I think I've got my head around it. Since I've never written
a driver, I've never fully learned the difference between IReadBlock and
IRasterIO or where one is used instead of the other. But I'm learning that
now.

On Fri, Apr 19, 2024 at 9:35 AM Even Rouault <even.rouault at spatialys.com>
wrote:

>
> Le 19/04/2024 à 17:28, Sean Gillies a écrit :
>
> Even,
>
> Does the shared attribute of a VRT sourceFilename element not affect
> caching at all?
>
> If shared is set to 0, then one GDALDataset per VRTSource will be opened.
> This has little benefit.
>
> The scope of sharing was initially greater since it extented to
> potentially several GDALDataset instance of VRT, but this wasn't safe in
> multithreaded scenarios where you would read one VRT in a thread, another
> one in another thread, but both would point to let's same the same TIFF
> GDALDataset. Hence the scope of sharing was reduced to a single GDALDataset
> VRT instance to be safe by default.
>
> Is the cache avoided so that potentially stale data isn't propagated, or
> for other reasons?
>
> Are you reacting to "if it is triggered. VRTSource::IRasterIO() calls
> IRasterIO() on the source band, which doesn't necessarily always trigger
> block-based reading" ?  I just meant that calling IRasterIO() on a
> GDALDataset/GDALRasterBand doesn't necessarily cause the block cache to
> trigger. This is dependent of the driver and the parameters of the request.
> And this isn't specific to use it from a VRT.
>
>
> On Fri, Apr 19, 2024 at 9:09 AM Even Rouault <even.rouault at spatialys.com>
> wrote:
>
>> Sean,
>>
>> Within a given GDALDataset opened on a VRT, if the VRT references several
>> times the same source, only one GDALDataset will be opened for it, so you
>> may benefit from the block cache mechanism (if it is triggered.
>> VRTSource::IRasterIO() calls IRasterIO() on the source band, which doesn't
>> necessarily always trigger block-based reading).  But if you open another
>> VRT (or the same one), it will not share the same GDALDataset for sources
>> that may be common with the first one, so no re-use of existing block
>> cache. For network sources, the I/O cache at the /vsicurl/ level works
>> however on filenames, not VSIFILE* instances, so you will save network reads
>>
>> Even
>> Le 19/04/2024 à 16:48, Sean Gillies via gdal-dev a écrit :
>>
>> Happy Friday, folks!
>>
>> Are the source rasters of a VRT added to the block cache such that
>> different VRTs using the same source can avoid reads from disk or the
>> network? Or is it intended that the VSI cache covers this need instead?
>>
>> Thanks,
>>
>>

-- 
Sean Gillies
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240419/d328cd8a/attachment.htm>