[gdal-dev] does gdal support multiple simultaneous writers to raster

[Even Rouault]

Hi,

This is an intersting topic, with many "intersecting" issues to deal with at
different levels.

First, are you confident that in the use cases you imagine that I/O access won't
be the limiting factor, in which case serialization of I/O could be acceptable
and this would just require an API with a dataset level mutex.

There are several places where parallel write should be addressed :
- The GDAL core mechanisms that deal with the block cache
- Each GDAL driver where parallel write would be supported. I guess that GDAL
drivers should advertize a specific capability
- The low-level library used by the driver. In the case of GDAL, libtiff

And finally, as Frank underlined, there are intrinsic limitations due to the
format itself. For a compressed TIFF, at some point, you have to serialize the
writing of the tile, because you cannot kown in advance the size of the
compressed data, or at least have some coordination of the writers so that a
"next offset available" is properly synchronized between them. The compression
itself could be serialized.

I'm not sure however if what Jan mentionned, different process, writing the same
dataset is doable.