[gdal-dev] RFC 47 Status.

[Tomaka, Jacek]

Even, 

That makes sense to me. 
I will try to test your mutexing changes to see if it improves our use cases. Maybe your locking improvements will solve all the problems we are seeing. 
Out of curiosity, the second value test on the slide, was it with sleep of 125 ms in case of contention?

Regards.
Jacek Tomaka


-----Original Message-----
From: Even Rouault [mailto:even.rouault at spatialys.com] 
Sent: Thursday, 7 May 2015 8:45 PM
To: Tomaka, Jacek
Cc: gdal-dev at lists.osgeo.org
Subject: Re: [gdal-dev] RFC 47 Status.

Le jeudi 07 mai 2015 14:20:42, Tomaka, Jacek a écrit :
> Even,
> 
> I am not sure I understand what you are saying about 2.0 beta. Does it 
> mean that it is too late for inclusion or it did not make it to the 
> beta but can still make it to 2.0?

Now that we are in beta, I wouldn't normally expect new RFCs to be implemented for 2.0 final that are not already in the beta.

> Given that the functionality is enabled by setting 
> GDAL_DATASET_CACHING=YES it should not be too destabilizing?

It still implies changing core classes. Anyway, the implementation isn't ready enough (2 different implementations, no tests etc...) and the RFC not finalized to expose the final solution. Somehow, for me, it goes a bit too far currently and should be rectricted only on adding the possibility of a per-dataset block cache. Further work on multithreaded access to a same dataset should be the subject of another RFC.

> 
> I agree that per-dataset block cache is huge improvement. Especially 
> in scenarios when different threads are reading different files/datasets.
> 
> If there is chance for it to still be included in 2.0 please let me 
> know what needs to be done to make it happen.

Well, someone would need to champion it & very quickly. But I don't see a compelling reason to absolutely put it in GDAL 2.0. I'd note I've done subtle improvements related to the scope of the mutex of the block cache (e.g. no longer making it a recursive mutex, but a regular one) during the 2.0 cycle that already reduce lock contention in some use cases (see slide 39 of
https://docs.google.com/presentation/d/1kHOoZfnqcWCR1S3I49RWoddG2tGxCXxXkjSjANJ2Gh0/pub?start=false&loop=false&delayms=3000#slide=id.g7b40ab322_227
). Further work like RFC 47 could go into 2.1

As far as I'm concerned, I'll revisit in the coming weeks (for 2.1) another older RFC in the same area of raster blocks (https://trac.osgeo.org/gdal/wiki/rfc26_blockcache, to deal with WMS/WMTS datasets that have dimensions nearing 1 billion x 1 billion pixels and thus a huge block address space), but that's a bit orthogonal to the per-dataset vs global block cache.

Even

> 
> Regards.
> Jacek Tomaka
> 
> -----Original Message-----
> From: Even Rouault [mailto:even.rouault at spatialys.com]
> Sent: Thursday, 7 May 2015 8:05 PM
> To: gdal-dev at lists.osgeo.org
> Cc: Tomaka, Jacek
> Subject: Re: [gdal-dev] RFC 47 Status.
> 
> Jacek,
> 
> > What is the current status of RFC 47?
> 
> Pending. IMHO the per-dataset block cache vs global block cache looks good.
> I'm less convinced by the attempts to provide even limited multi-threaded
> access to a same GDALDataset object.
> 
> > Is per dataset caching going to make it to GDAL 2.0?
> 
> No, not that we are now in beta phase.
> 
> Even
> 
> --
> Spatialys - Geospatial professional services http://www.spatialys.com

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com