[gdal-dev] Best practices for concurrent writes with GDAL (was RasterIO in paralel)

[Francis Markham]

Is there a document anywhere specifying the best practices for parallel
writes to a GDAL raster?  I have an embarrasingly parallel problem that I
would like to parallelise with MPI, but I'm not sure what I am allowed to do
in parallel.  I would like to assign blocks exclusively to worker threads to
read and write concurrently, but I am unsure how I might do this safely.

Any suggestions would be greatly appreciated.

Kind Regards,

Francis Markham

On 9 February 2011 09:14, Even Rouault <even.rouault at mines-paris.org> wrote:

> Le mardi 08 février 2011 23:07:36, Stefano Moratto a écrit :
> > Thanks for the suggestion,
> >
> > I have to read png from openstreetmap. I'm trying to render an A4 paper
> > size sheet so I have a lot of data to move.
> > It may be better to divide the area to be read in blocks and to give to
> > each thread a block. Foreach  block a thread should read all the bands.
> > Probabilly I should allign the block to the tile's boundary
>
> Unfortunately the PNG format is such that it is not possible to seek to an
> arbitrary line without decoding the data of all previous scanlines, so this
> approach won't work. The same would be true for JPEG or GIF. If you can
> fetch
> the data as TIFF, you could however try that approach.
>
> >
> > Stefano
> >
> > > Le mardi 08 février 2011 22:41:50, Stefano Moratto a écrit :
> > > > Hello,
> > > >
> > > >           given an opened GDAL dataset, is it possibile to perform
> > > >
> > > > gdalrasterio calls for each band in a dedicated thread per band in a
> > > > thead-safe way?
> > > > E.g. I have a raster image with 3 band (r,g,b). Since I have 4 core I
> > >
> > > would
> > >
> > > > like to I run a thread for a band r, one for band g and one for band
> g.
> > >
> > > This has been discussed recently. See
> > > http://lists.osgeo.org/pipermail/gdal-
> > > dev/2011-January/027567.html and the following messages.
> > >
> > > (I'd note that if you process a pixel interleaved image, there are
> > > optimizations in the GTiff driver to make the fetching of other bands
> > > almost
> > > 'free' if you read block by block, so there's probably little to gain,
> > > but the
> > > results of your experiments are welcome)
> > >
> > > > Regards,
> > > > Stefano
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/gdal-dev/attachments/20110209/bd467a7d/attachment.html