<div><br></div><div><br><div class="gmail_quote">On Wed, Nov 18, 2009 at 7:19 PM, Frank Warmerdam <span dir="ltr"><<a href="mailto:warmerdam@pobox.com">warmerdam@pobox.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div class="im">Shaun Kolomeitz wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Thanks Seth,<br>
<br>
It makes sense that the slowest part of the whole equation would be the<br>
disk operations, and there must be quite a number of disk reads/writes<br>
when processing imagery. Currently we use Raid Arrays that push data<br>
through at a rate of 300MB/s, granted if these were SSDs in Raid0 we<br>
could push beyond 1GB/s. Currently to process (mosaic) an 80GB image it<br>
takes several days to complete. This is also on 32bit hardware, and I<br>
suspect is only single threaded so we're limited to 3GB RAM. From what I<br>
understood the most optimal caching size in GDAL is 500MB, using a 512M<br>
window (unless that has changed).<br>
If you can easily lay your hands on your GSoC application than that<br>
would be great. We are discussing what might be possible with a very<br>
talented coder, who eats these types of "challenges" for breakfast !<br>
Perhaps a better approach would be to use something like a grid<br>
computing approach then like Condor to break up the processing ?<br>
</blockquote>
<br></div>
Shaun,<br>
<br>
I suspect that there is a gross issue with how the warping is being done,<br>
and that it could be sped up substantially. On my two year old consumer<br>
grade desktop I'm able to warp/reproject around 30GB/hour with gdalwarp.<br>
<br>
I don't want to dissuade you from investigating CUDA options, but<br>
you might be able to get ten fold improvement by hiring an experienced<br>
gdalwarp developer for a few hours to investigate your particular<br>
situation.<br>
<br>
Best regards,<br><font color="#888888">
-- <br>
---------------------------------------+--------------------------------------<br>
I set the clouds in motion - turn up | Frank Warmerdam, <a href="mailto:warmerdam@pobox.com" target="_blank">warmerdam@pobox.com</a><br>
light and sound - activate the windows | <a href="http://pobox.com/~warmerdam" target="_blank">http://pobox.com/~warmerdam</a><br>
and watch the world go round - Rush | Geospatial Programmer for Rent</font><div><div></div><div class="h5"><br>
<br>
_______________________________________________<br>
gdal-dev mailing list<br>
<a href="mailto:gdal-dev@lists.osgeo.org" target="_blank">gdal-dev@lists.osgeo.org</a><br>
<a href="http://lists.osgeo.org/mailman/listinfo/gdal-dev" target="_blank">http://lists.osgeo.org/mailman/listinfo/gdal-dev</a><br>
</div></div></blockquote></div><br>Along these same lines, I'm looking to CUDA or Condor cluster to aid in gdal_retile. Current projects are taking from 3 weeks to 2 months to process. Will this process lend itself well to the parallel processing environment?<br>
<br>-- <br>Sean<br>Villagis<br>
</div>