[gdal-dev] Best practices for concurrent writes with GDAL (was RasterIO in paralel)

[Antonio Valentino]

Hi Francis,

Il giorno Wed, 9 Feb 2011 09:38:49 +1100
Francis Markham <fmarkham at gmail.com> ha scritto:

> Is there a document anywhere specifying the best practices for
> parallel writes to a GDAL raster?  I have an embarrasingly parallel
> problem that I would like to parallelise with MPI, but I'm not sure
> what I am allowed to do in parallel.  I would like to assign blocks
> exclusively to worker threads to read and write concurrently, but I
> am unsure how I might do this safely.
> 
> Any suggestions would be greatly appreciated.

Are you sure it is the best way to parallelize your problem?
If you write tho the disk operation are in some way serialized.
It is a common schema to have several worker processes/thread for CPU
bound computations and a single IO processes/thread that collects all
results and write them to disk.

The IO process/thread int this case can also be designed to implement
some optimization or caching mechanism that you could not apply at
worker process level.

With this schema you can bypass completely the problem of concurrent
writes without loss of performance IMHO.


regards

-- 
Antonio Valentino