[gdal-dev] does gdal support multiple simultaneous writers to raster

[Kennedy, Paul]

hi,

I would be prepared to contribute to such a facility in GDAL if it
permitted multiple simultaneous writes (to different bands / tiles)

 

pk

 

 

 

 

 

From: Jan Hartmann [mailto:j.l.h.hartmann at uva.nl] 
Sent: Friday, 11 January 2013 5:43 PM
To: Frank Warmerdam
Cc: Kennedy, Paul; gdal dev
Subject: Re: [gdal-dev] does gdal support multiple simultaneous writers
to raster

 

I'v tried this in a Cloud environment with multiple VMs writing to the
same shared file location. I didn't use tiles, so the results where
negative. It would be a great asset for Cloud environments, however. Is
it in principle possible to build such a facility into GDAL, and if yes,
what kind of funding would it require?

Jan

On 01/11/2013 02:32 AM, Frank Warmerdam wrote:

	Paul,
	 
	Generally speaking - no - GDAL does not now support
multi-threaded
	writing.  Part of this is due to the way stuff gets
write-flushed out
	of the block cache and partly it is just that none of GDAL
supports
	multiple threading on a single dataset.
	 
	This statement is based on the assumption it is all in one
process.
	 
	If you want to run different processes with the output to a
	pre-existing "fixed location" format like uncompressed GeoTIFF
with
	tiles that is all preinitialized to some real value then you
could
	certainly use one output file for many writers as long as case
is
	taken to not have multiple processes write to the same tile.
But GDAL
	isn't doing anything to make this easy.
	 
	Best regards,
	Frank
	 
	On Thu, Jan 10, 2013 at 4:12 PM, Kennedy, Paul
<P.Kennedy at fugro.com.au> <mailto:P.Kennedy at fugro.com.au>  wrote:

		I am looking to parallelize a long duration gridding
process, and was
		wondering if GDAL can support multiple simultaneous
writers from processess?
		 
		 
		 
		I am not sure if any of the underlying raster formats
support this?
		 
		 
		 
		thanks in advance.
		 
		 
		 
		regards
		 
		pk
		 
		 
		 
		 
		 
		 
		_______________________________________________
		gdal-dev mailing list
		gdal-dev at lists.osgeo.org
		http://lists.osgeo.org/mailman/listinfo/gdal-dev

	 
	 
	 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20130111/10a4f22c/attachment.html>