[gdal-dev] RFC 47 and Threading

[Jeff Lacoste]

Thanks Blake for the detailed response. I did not realize that I did not do
a reply all in my previous email I sent.


On Thu, Aug 21, 2014 at 3:59 PM, Blake Thompson <flippmoke at gmail.com> wrote:

>
>
>
> On Thu, Aug 21, 2014 at 11:36 AM, Jeff Lacoste <jefflacostegdal at gmail.com>
> wrote:
>
>> Hi,
>>
>> Improving the thread safety of GDAL is a big improvement. I know this
>> proposal is not claiming to 'fix' gdal thread safety but adress at least
>> the cache safety. This is said, may be to help
>> clarify more the proposal, we can state what the change would address and
>> (may be more important) not address. Just to make it more specific about
>> gain of caching per dataset instead of global one.
>>
>
>  Updated the RFC to hopefully reflect this better reflect some of this,
> please send more questions as you have them.
>
>
>
>>  Would this mean for ex. that batch translating datasets would gain from
>> this and be done in a parallel way ? Since we can avoid avoid trashing a
>> global trash.
>> So instead of translation x datasets in a sequential manner now (with the
>> proposed changes) this can be done in parallel ?
>>
>
> Currently it is possible to do batch translating of datasets, but it is
> not efficient. The reason for this is that there is a point where each
> thread is attempting to add or remove gdalrasterblocks from the global
> cache. Once the global cache size is reached all the threads are often
> blocked for extended periods as each one attempts to clear the cache. The
> worst part is that this also prevents simple reading from the cache during
> this period, because other blocks can not efficiently use
> GDALRasterBand::TryGetLockedBlockRef, to pull existing blocks from the
> cache.
>

--> I thought that this was not possible using current trunk GDAL because
of the global cache. At least the writing side using multiple threads can
cause issues.
See this response on a older topic I found in GDAL dev list:
http://lists.osgeo.org/pipermail/gdal-dev/2013-January/035215.html
The response there was for a similar question as the one i posted about
supporting of batch translating of dataset with this RFC .

>
> The simple and first fix I made while doing this was simply to make it
> possible to operate with a per dataset cache (this was not selected as per
> band due to possible deadlock issues). Doing this allows for each dataset
> to have its own lock for its cache and the performance increases
> dramatically for operating on different dataset in parallel.
>
>
>>
>> If yes, what would the cache flushing strategy once the cach max is
>> reached  ? For ex. 5 running threads converting 5 dataset. We reach the max
>> cache while the 5 threads are executing, this mean that threads will be
>> blocked from executing as no cache is available until it has been released
>> by other threads ?
>>
>
>
>> Jeff Lacoste
>>
>
> In the case I described above each dataset has its own max cache.
> Therefore, reaching the maximum cache can only occur in a single dataset.
> Therefore if the maximum cache was reached on that cache, it would flush
> and not prevent other dataset and threads from operating on their own
> caches. However, this means that the person writing the code MUST be aware
> of the total cache sizes of all their dataset (I know this isn't ideal for
> many people).
>

--> Ok, I was under the wrong impression that in addition of  a cache per
dataset there is still a cap of global cache that oversee the total cache
used by GDAL.

>
> So I decided to take it a step farther and wanted to make it possible for
> a dataset using a driver such as memory to be able to do something such as:
> 5 running threads translating 1 dataset, 5 times. Also I wanted to be able
> to use one global cache and make it so that, the 5 threads translating 5
> datasets would not lock as often with a global cache. To achieve this a
> separation of concerns had to occur, and this lead to the development of
> multiple mutexes.
>
> Three different data structure are at risk during threading within the
> cache:
>
> #1 - The Linked List of the cache and size of this linked list (this
> allows the cache to flush the least recently used GDALRasterBlock)
> #2 - The cache block array of a GDALRasterBand (this allows the
> GDALRasterBand to find its GDALRasterBlocks)
> #3 - The data stored within the GDALRasterBlock (this is the actual data
> stored in the block)
>
> If you are limiting the scope of our support of threading to only allow 1
> thread to access any 1 dataset at a time #3 requires no protection.
> However, in any cache that is shared by more then 1 dataset (global cache),
> not only must the linked list be protected by the list global cache, but
> you must protect the cache block array in the raster bands. Since currently
> when items are removed from the cache, it simply removes blocks until it is
> below the cache limit, all raster bands can not read from their cache block
> arrays until all flushing has completed.
>
> Therefore, during my design I decided that each portion should be
> protected by its own mutex. In order to not require the linked list (LRU)
> mutex from being locked for extended periods, I mark blocks for deletion
> when it is to removed from the cache and it will be removed at the earliest
> safe period, which unless another the block is about to be used by another
> thread, will be right away. Otherwise it will occur once the other thread
> is done using that block.
>
> Therefore, this also means the mutex protecting the global cache in my
> code is also locked less. Therefore, even with out using a per dataset
> cache, the example of using 5 threads to translate 5 datasets should still
> be faster then the current configuration.
>

--> Is this mean that in addition of the per dataset cache once can still
use the global cache (only not per dataset one) and have the scenario of
translating multiple datasets in a parallel way
works ok (without threading issues due to current implementation global
cache).

So we are dealing here with two possible solutions (per dataset cache and
improved global cache) ? or they are complimentary ?
or the global cache improvement is done to only support the use case of 'x
running thread translating a dataset x times' ? Any other use cases this
solve ?

>
> TLDR; There are benefits in my design outside of just a non global cache
> for datasets.
>
> Blake Thompson
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20140822/d06a2da5/attachment.html>