Opps, forgot to send this to mailing list as well...<br><br><div class="gmail_quote">On Tue, Mar 30, 2010 at 8:32 PM, Jordan Neumeyer <span dir="ltr"><<a href="mailto:jordan.neumeyer@mines.sdsmt.edu">jordan.neumeyer@mines.sdsmt.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">Hi Glynn,<br><br><div class="gmail_quote"><div class="im">On Tue, Mar 30, 2010 at 8:29 AM, Glynn Clements <span dir="ltr"><<a href="mailto:glynn@gclements.plus.com" target="_blank">glynn@gclements.plus.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div><br>
Jordan Neumeyer wrote:<br>
<br>
> Just kind of my thought process about how I would try to go about<br>
> parallelizing a module.<br>
<br>
</div>The main issue with parallelising raster input is that the library<br>
keeps a copy of the current row's data, so that consecutive reads of<br>
the same row (as happen when upsampling) don't re-read the data.<br>
<br>
For concurrent access to a single map, you would need to either keep<br>
one row per thread, or abandon caching. Also, you would need to use<br>
pread() rather than fseek()+read().<br></blockquote><div> </div></div><div>It sounds like you're talking about parallelism in I/O from a file or database. Neither of which is my intent or goal for this project. I will parallelize things after they have already been read into memory, and tasks are processor intensive. I wouldn't want parallelize any I/O, but if I were to optimize I/O. I would make all operations I/O asynchronous, which is can mimic parallelism in a sense. Queuing up the chunks of data and then processing them as resources become available. <br>
</div><div class="im"><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
It's more straightfoward to read multiple maps concurrently. In 7.0,<br>
this case should be thread-safe.<br>
<br>
Alternatively, you could have one thread for reading, one for writing,<br>
and multiple worker threads for the actual processing. However, unless<br>
the processing is complex, I/O will be the bottleneck.<font color="#888888"><br></font></blockquote></div><div><br>I/O is generally a bottleneck anyway. Something always tends to be waiting on another.<br>
</div><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;"><font color="#888888">
--<div class="im"><br>
Glynn Clements <<a href="mailto:glynn@gclements.plus.com" target="_blank">glynn@gclements.plus.com</a>></div></font> <br></blockquote></div><br>
</blockquote></div><br>