[gdal-dev] RFC 47 Status.

[Even Rouault]

Le jeudi 07 mai 2015 17:18:34, Tomaka, Jacek a écrit :
> >I was wondering why you mention 125 ms, but a grep shows that there's such
> >a delay in the Windows implementation of CPLAcquireMutex() when the
> >critical section fails to be acquired, so I guess you >must think to that
> >?
> 
> Yes. I would not have come up with such a number myself ;) It is not prime
> ;)
> 
> >Well, the tests in my slides have only be done on Linux where there's no
> >such sleep (just rely on pthread implementation).
> 
> That is not a good news for Windows. But I will test it...

The changes done should logically not degrade performance on Windows, and 
hopefully improve it a bit. Your feedback is welcome.

> 
> Regards.
> Jacek Tomaka
> 
> -----Original Message-----
> From: Even Rouault [mailto:even.rouault at spatialys.com]
> Sent: Thursday, 7 May 2015 9:20 PM
> To: Tomaka, Jacek
> Cc: gdal-dev at lists.osgeo.org
> Subject: Re: [gdal-dev] RFC 47 Status.
> 
> Le jeudi 07 mai 2015 15:13:51, Tomaka, Jacek a écrit :
> > Even,
> > 
> > That makes sense to me.
> > I will try to test your mutexing changes to see if it improves our use
> > cases. Maybe your locking improvements will solve all the problems we
> > are seeing. Out of curiosity, the second value test on the slide, was
> > it with sleep of 125 ms in case of contention?
> 
> I was wondering why you mention 125 ms, but a grep shows that there's such
> a delay in the Windows implementation of CPLAcquireMutex() when the
> critical section fails to be acquired, so I guess you must think to that ?
> Well, the tests in my slides have only be done on Linux where there's no
> such sleep (just rely on pthread implementation).
> 
> > Regards.
> > Jacek Tomaka
> > 
> > 
> > -----Original Message-----
> > From: Even Rouault [mailto:even.rouault at spatialys.com]
> > Sent: Thursday, 7 May 2015 8:45 PM
> > To: Tomaka, Jacek
> > Cc: gdal-dev at lists.osgeo.org
> > Subject: Re: [gdal-dev] RFC 47 Status.
> > 
> > Le jeudi 07 mai 2015 14:20:42, Tomaka, Jacek a écrit :
> > > Even,
> > > 
> > > I am not sure I understand what you are saying about 2.0 beta. Does it
> > > mean that it is too late for inclusion or it did not make it to the
> > > beta but can still make it to 2.0?
> > 
> > Now that we are in beta, I wouldn't normally expect new RFCs to be
> > implemented for 2.0 final that are not already in the beta.
> > 
> > > Given that the functionality is enabled by setting
> > > GDAL_DATASET_CACHING=YES it should not be too destabilizing?
> > 
> > It still implies changing core classes. Anyway, the implementation isn't
> > ready enough (2 different implementations, no tests etc...) and the RFC
> > not finalized to expose the final solution. Somehow, for me, it goes a
> > bit too far currently and should be rectricted only on adding the
> > possibility of a per-dataset block cache. Further work on multithreaded
> > access to a same dataset should be the subject of another RFC.
> > 
> > > I agree that per-dataset block cache is huge improvement. Especially
> > > in scenarios when different threads are reading different
> > > files/datasets.
> > > 
> > > If there is chance for it to still be included in 2.0 please let me
> > > know what needs to be done to make it happen.
> > 
> > Well, someone would need to champion it & very quickly. But I don't see a
> > compelling reason to absolutely put it in GDAL 2.0. I'd note I've done
> > subtle improvements related to the scope of the mutex of the block cache
> > (e.g. no longer making it a recursive mutex, but a regular one) during
> > the 2.0 cycle that already reduce lock contention in some use cases (see
> > slide 39 of
> > https://docs.google.com/presentation/d/1kHOoZfnqcWCR1S3I49RWoddG2tGxCXxXk
> > j
> > SjANJ2Gh0/pub?start=false&loop=false&delayms=3000#slide=id.g7b40ab322_22
> > 7 ). Further work like RFC 47 could go into 2.1
> > 
> > As far as I'm concerned, I'll revisit in the coming weeks (for 2.1)
> > another older RFC in the same area of raster blocks
> > (https://trac.osgeo.org/gdal/wiki/rfc26_blockcache, to deal with WMS/WMTS
> > datasets that have dimensions nearing 1 billion x 1 billion pixels and
> > thus a huge block address space), but that's a bit orthogonal to the
> > per-dataset vs global block cache.
> > 
> > Even
> > 
> > > Regards.
> > > Jacek Tomaka
> > > 
> > > -----Original Message-----
> > > From: Even Rouault [mailto:even.rouault at spatialys.com]
> > > Sent: Thursday, 7 May 2015 8:05 PM
> > > To: gdal-dev at lists.osgeo.org
> > > Cc: Tomaka, Jacek
> > > Subject: Re: [gdal-dev] RFC 47 Status.
> > > 
> > > Jacek,
> > > 
> > > > What is the current status of RFC 47?
> > > 
> > > Pending. IMHO the per-dataset block cache vs global block cache looks
> > > good. I'm less convinced by the attempts to provide even limited
> > > multi-threaded access to a same GDALDataset object.
> > > 
> > > > Is per dataset caching going to make it to GDAL 2.0?
> > > 
> > > No, not that we are now in beta phase.
> > > 
> > > Even
> > > 
> > > --
> > > Spatialys - Geospatial professional services http://www.spatialys.com

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com