[gdal-dev] parallel processing

[Lucena, Ivan]

The HDFGroup's Parallel-HDF5 library is another good example of the use 
of MPI-IO (Message Passing Interface) to implements parallel file 
access, see:
http://hdf.ncsa.uiuc.edu/Parallel_HDF/PHDF5/ph5-status.html

James McManus wrote:
> Frank Warmerdam wrote:
>> James McManus wrote:
>>> Has implementing parallel process for GDAL and OGR been considered?
>>
>> James,
>>
>> There has not been much work on parallel processing within GDAL but
>> some applications do use GDAL in multiple threads at a time and there
>> has been considerable work to make aspects of GDAL thread safe.  In
>> particular modern GDAL should support a "many readers" situation fairly
>> well now as long as each of the readers is operating on distinct
>> GDALDataset objects.
>>
>> For various reasons you can't have multiple readers active on a common
>> gdal dataset, and when GDAL is writing there should really only be
>> one thread active at a time.
>>
>> Best regards,
> Frank,
> /
> /That is basically what I am looking for. I would like to take advantage 
> of my additional cores and CPU when I import data using v.in.ogr.  As 
> you know large vector files take a long time to read, so it would be 
> great if I could employ additional threads.  I searched the web to see 
> if any other IO software was doing this. I found that netcdf has a 
> library which uses MPI-IO:
> 
> http://www-unix.mcs.anl.gov/parallel-netcdf/
> 
> Reading spaghetti shape files into GRASS requires building of topology 
> among other things. Multi-thread processing would speed this up 
> dramatically. With the proliferation of multi-thread processors for PC's 
> it seems like something that would eventually have to be incorporated 
> into GDAL/OGR.
> 
> Jim
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
> 
>