[gdal-dev] CUDA PyCUDA and GDAL

[Sean Hills]

On Wed, Nov 18, 2009 at 7:19 PM, Frank Warmerdam <warmerdam at pobox.com>wrote:

> Shaun Kolomeitz wrote:
>
>> Thanks Seth,
>>
>> It makes sense that the slowest part of the whole equation would be the
>> disk operations, and there must be quite a number of disk reads/writes
>> when processing imagery. Currently we use Raid Arrays that push data
>> through at a rate of 300MB/s, granted if these were SSDs in Raid0 we
>> could push beyond 1GB/s. Currently to process (mosaic) an 80GB image it
>> takes several days to complete. This is also on 32bit hardware, and I
>> suspect is only single threaded so we're limited to 3GB RAM. From what I
>> understood the most optimal caching size in GDAL is 500MB, using a 512M
>> window (unless that has changed).
>> If you can easily lay your hands on your GSoC application than that
>> would be great. We are discussing what might be possible with a very
>> talented coder, who eats these types of "challenges" for breakfast !
>> Perhaps a better approach would be to use something like a grid
>> computing approach then like Condor to break up the processing ?
>>
>
> Shaun,
>
> I suspect that there is a gross issue with how the warping is being done,
> and that it could be sped up substantially. On my two year old consumer
> grade desktop I'm able to warp/reproject around 30GB/hour with gdalwarp.
>
> I don't want to dissuade you from investigating CUDA options, but
> you might be able to get ten fold improvement by hiring an experienced
> gdalwarp developer for a few hours to investigate your particular
> situation.
>
> Best regards,
> --
>
> ---------------------------------------+--------------------------------------
> I set the clouds in motion - turn up   | Frank Warmerdam,
> warmerdam at pobox.com
> light and sound - activate the windows | http://pobox.com/~warmerdam
> and watch the world go round - Rush    | Geospatial Programmer for Rent
>
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>

Along these same lines,  I'm looking to CUDA or Condor cluster to aid in
gdal_retile.  Current projects are taking from 3 weeks to 2 months to
process.  Will this process lend itself well to the parallel processing
environment?

-- 
Sean
Villagis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/gdal-dev/attachments/20091118/af8a1de1/attachment.html