Hi Collin, Markus,<br><br>Thanks a lot for the script and tips, I really appreciate it. That's definitely a much bigger chunk to chew on than I'd planned, but it's good to know that. It seems like most of the stuff would be fairly simple to implement - the program I'm working with is built modularly. I've got a couple of questions that would give me some clarity here on the way though, if you don't mind:<br>
<br>I notice in your script that you use the multiprocessing library? Are there any advantages in doing so? It looks to me like you'd be able to accomplish the same things without using the library so I'd be interested to know why you chose it.<br>
<br>What do you mean, r.sun can do multithreading? I've heard that r.sun uses multithreading in GRASS 7, but is that implemented in GRASS 6? Or are you talking about "poor man's multithreading," like on the GRASS wiki?<br>
<br>I haven't noticed any problems with running r.horizon on GRASS 6.4.2, the angles looked correct in test data I've used thus far. The maps are all assigned different names later on and the results look fine, but a lot of them were missing. I was puzzled for a while because test data worked and my actual data didn't, but I did a little debugging today and think it's due to the large size of my study area (~36 km², 0.5m resolution). If I spatially partition the area and then stitch everything back together, I hope it works - tests on smaller regions have worked correctly thus far but I'll need to wait a while to see the real results.<br>
<br>To make a long story short, I'm going to try using the tricks in Markus' link to get the processes to coexist peacefully and then partition the area spatially so that it doesn't kill my memory. Hopefully I'll get better results then. In the meantime you've been a great help :)<br>
<br>Best,<br>Daniel<br clear="all">
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"><span style="font-size:13px;font-family:arial,sans-serif"></span></p><p>--<br></p><p>B.Sc. Daniel Lee<br>
Geschäftsführung für Forschung und Entwicklung<br>ISIS - International Solar Information Solutions GbR<br>Vertreten durch: Daniel Lee, Nepomuk Reinhard und Nils Räder<br></p><p>Softwarecenter 3<br>35037 Marburg<br>Festnetz: <a value="+4964213796256" style="color:rgb(28,81,168)">+49 6421 379 6256</a><br>
Mobil: <a value="+4917661277269" style="color:rgb(28,81,168)">+49 176 6127 7269</a><br>E-Mail: <a href="mailto:Lee@isi-solutions.org" style="color:rgb(28,81,168)" target="_blank">Lee@isi-solutions.org</a><br>Web: <a href="http://www.isi-solutions.org/" style="color:rgb(28,81,168)" target="_blank">http://www.isi-solutions.org</a></p>
<p></p><br>
<br><br><div class="gmail_quote">Am 17. April 2012 21:21 schrieb Collin Bode <span dir="ltr"><<a href="mailto:collin@berkeley.edu" target="_blank">collin@berkeley.edu</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">Lee,<div><br></div><div>I tried the same thing, trying to integrate a full workflow in Python for r.sun (see attached code). I am using a 2x2 meter lidar elevation map with about 120 million cells, so automation is pretty necessary. The whole thing takes 3 weeks to run using 8 CPU cores and 12GB ram. </div>
<div><br></div><div>However, r.horizon does not work properly on Grass 6.4.2. I can only get it to run on one core in sequence. I cannot get it to accept a single angle and run properly. The output is always for angle 0, both in name and output data. Bug? I was planning to post this to the dev list if no one responds here.</div>
<div><br></div><div>It would be worthwhile to find out if your version is simply overwriting the previous, similarly named map. If it is, and the maps are actually different (<a href="http://r.info" target="_blank">r.info</a>), then you can change your code to run r.horizon for a single angle, then rename the map. Something like this:</div>
<div><br></div><div><div> for azimuth in range(0,361,step_size):</div><div> grass.run_command("r.horizon", elevin=dem, \</div><div> direction=angle, maxdistance=maxdistance, \</div>
<div> horizon=prefhor, dist=dist, overwrite = 1)</div><div> az = str(azimuth).zfill(3)</div><div> tempmap = prefhor+'_0'</div><div> horizonmap = prefhor+'_'+az</div>
<div> cmd = tempmap+","+horizonmap</div><div> grass.run_command("g.rename",rast=cmd) </div></div><div><br></div><div>r.sun does work in multiple cores. <span><font color="#888888"><br>
<div><br></div><div>
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:auto;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:12px;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:12px;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:12px;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<div>Collin Bode</div><div>Project Manager, Desktop Watershed Integrated Program</div><div>National Center for Earth-surface Dynamics</div><div>University of California, Berkeley</div></div></span></div></span></div></span></div>
</span></span><span style="font-size:12px"><a href="http://www.nced.umn.edu" target="_blank">http://www.nced.umn.edu</a></span><span style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:auto;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:12px;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:12px;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:12px;white-space:normal;font-family:Helvetica;word-spacing:0px"><a href="http://ib.berkeley.edu/labs/power" target="_blank">http://ib.berkeley.edu/labs/power</a></span></div>
<div style="word-wrap:break-word"><span style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:12px;white-space:normal;font-family:Helvetica;word-spacing:0px"><a href="http://angelo.berkeley.edu" target="_blank">http://angelo.berkeley.edu</a></span></div>
<div style="word-wrap:break-word"><br></div></span></div></span></div></span></span></div></font></span></div></div><br><div style="word-wrap:break-word"><div><div>
</div>
<br><div><div>On Apr 1, 2012, at 5:10 AM, Daniel Lee wrote:</div><br><blockquote type="cite">Hi list,<br><br>I've got a large area I'm running r.horizon on. Since the process takes quite a while but it only runs on one processor core and I've got plenty of RAM, I've decided to script it in Python and use run_command and start_command in order to create multiple horizon maps in parallel. Another reason I'm doing it that way is that I only need certain horizons, so I don't want GRASS to do all 360°.<br>
<br>The problem is, when I run the script, all of the maps that I want made seem to be made and I can see in top that there are several instances of r.horizon. But only the instance to be finished last is actually saved. Here's the output:<br>
<br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">Calculating map 1 of 1 (angle 0.000000, raster map <horizon0_0>)<br> 100%<br>Calculating map 1 of 1 (angle 12.500000, raster map <horizon12.5_0>)<br>
Calculating map 1 of 1 (angle 7.500000, raster map <horizon7.5_0>)<br>Calculating map 1 of 1 (angle 42.500000, raster map <horizon42.5_0>)<br>Calculating map 1 of 1 (angle 150.000000, raster map <horizon150.0_0>)<br>
100%<br>Calculating map 1 of 1 (angle 162.500000, raster map <horizon162.5_0>)<br> 100%<br>Calculating map 1 of 1 (angle 175.000000, raster map <horizon175.0_0>)<br>Calculating map 1 of 1 (angle 167.500000, raster map <horizon167.5_0>)<br>
Calculating map 1 of 1 (angle 197.500000, raster map <horizon197.5_0>)<br>Calculating map 1 of 1 (angle 205.000000, raster map <horizon205.0_0>)<br>Calculating map 1 of 1 (angle 225.000000, raster map <horizon225.0_0>)<br>
100%<br>Calculating map 1 of 1 (angle 257.500000, raster map <horizon257.5_0>)<br>Calculating map 1 of 1 (angle 262.500000, raster map <horizon262.5_0>)<br> 100%<br>Calculating map 1 of 1 (angle 275.000000, raster map <horizon275.0_0>)<br>
100%<br>Calculating map 1 of 1 (angle 295.000000, raster map <horizon295.0_0>)<br>Calculating map 1 of 1 (angle 300.000000, raster map <horizon300.0_0>)<br> 100%<br>Calculating map 1 of 1 (angle 322.500000, raster map <horizon322.5_0>)<br>
Calculating map 1 of 1 (angle 325.000000, raster map <horizon325.0_0>)<br> 100%<br>Calculating map 1 of 1 (angle 337.500000, raster map <horizon337.5_0>)<br>Calculating map 1 of 1 (angle 335.000000, raster map <horizon335.0_0>)<br>
Calculating map 1 of 1 (angle 345.000000, raster map <horizon345.0_0>)<br>Calculating map 1 of 1 (angle 350.000000, raster map <horizon350.0_0>)<br> 100%<br>Calculating map 1 of 1 (angle 352.500000, raster map <horizon352.5_0>)<br>
100%<br>Calculating map 1 of 1 (angle 355.000000, raster map <horizon355.0_0>)<br> 100%<br>Calculating map 1 of 1 (angle 357.500000, raster map <horizon357.5_0>)<br> 100%<br>Calculating map 1 of 1 (angle 257.500000, raster map <horizon257.5_0>)</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">Calculating map 1 of 1 (angle 262.500000, raster map <horizon262.5_0>)</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> 100%</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">Calculating map 1 of 1 (angle 275.000000, raster map <horizon275.0_0>)</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> 100%</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">Calculating map 1 of 1 (angle 295.000000, raster map <horizon295.0_0>)</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">Calculating map 1 of 1 (angle 300.000000, raster map <horizon300.0_0>)</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> 100%</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">Calculating map 1 of 1 (angle 322.500000, raster map <horizon322.5_0>)</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">Calculating map 1 of 1 (angle 325.000000, raster map <horizon325.0_0>)</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> 100%</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">Calculating map 1 of 1 (angle 337.500000, raster map <horizon337.5_0>)</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">Calculating map 1 of 1 (angle 335.000000, raster map <horizon335.0_0>)</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">Calculating map 1 of 1 (angle 345.000000, raster map <horizon345.0_0>)</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">Calculating map 1 of 1 (angle 350.000000, raster map <horizon350.0_0>)</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> 100%</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">Calculating map 1 of 1 (angle 352.500000, raster map <horizon352.5_0>)</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> 100%</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">Calculating map 1 of 1 (angle 355.000000, raster map <horizon355.0_0>)</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> 100%</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">Calculating map 1 of 1 (angle 357.500000, raster map <horizon357.5_0>)</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> 100%</span><br><br>But when I do g.list rast at the end of it all I only have these maps:<br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">raster files available in mapset <Cloppenburg>:</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">dom_aspect horizon162.5_0 horizon325.0_0</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">dom_complete horizon225.0_0 horizon350.0_0</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">dom_slope horizon262.5_0 horizon352.5_0</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">horizon0_0 horizon275.0_0 horizon355.0_0</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">horizon150.0_0 horizon300.0_0 horizon357.5_0</span><br><br>As you can see, the only horizon maps saved are the last ones made. Does anybody know why this is the case and, if so, what would be a good workaround? Finding the horizons serially on only one processor means that my computer does a lot of sitting around twiddling thumbs. Thanks a bunch!<br>
<br>Daniel<br clear="all"><p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"><span style="font-size:13px;font-family:arial,sans-serif"></span></p><p>--<br></p><p>B.Sc. Daniel Lee<br>
Geschäftsführung für Forschung und Entwicklung<br>ISIS - International Solar Information Solutions GbR<br>Vertreten durch: Daniel Lee, Nepomuk Reinhard und Nils Räder<br></p><p>Deutschhausstr. 10<br>35037 Marburg<br>Festnetz: <a value="+4964213796256" style="color:rgb(28,81,168)">+49 6421 379 6256</a><br>
Mobil: <a value="+4917661277269" style="color:rgb(28,81,168)">+49 176 6127 7269</a><br>E-Mail: <a href="mailto:Lee@isi-solutions.org" style="color:rgb(28,81,168)" target="_blank">Lee@isi-solutions.org</a><br>Web: <a href="http://www.isi-solutions.org/" style="color:rgb(28,81,168)" target="_blank">http://www.isi-solutions.org</a></p>
<p>ISIS wird gefördert durch die Bundesrepublik Deutschland, Zuwendungsgeber: Bundesministerium für Wirtschaft und Technologie aufgrund eines Beschlusses des Deutschen Bundestages, sowie durch die Europäische Union, Zuwendungsgeber: Europäischer Sozialfonds.<br>
Zusätzliche Unterstützung erhält ISIS von dem Entrepreneurship Cluster Mittelhessen, der Universität Marburg, dem Laboratory for Climatology and Remote Sensing und dem GIS-Lab Marburg.</p><div><br></div>
_______________________________________________<br>grass-user mailing list<br><a href="mailto:grass-user@lists.osgeo.org" target="_blank">grass-user@lists.osgeo.org</a><br><a href="http://lists.osgeo.org/mailman/listinfo/grass-user" target="_blank">http://lists.osgeo.org/mailman/listinfo/grass-user</a><br>
</blockquote></div><br></div></div><br>_______________________________________________<br>
grass-user mailing list<br>
<a href="mailto:grass-user@lists.osgeo.org" target="_blank">grass-user@lists.osgeo.org</a><br>
<a href="http://lists.osgeo.org/mailman/listinfo/grass-user" target="_blank">http://lists.osgeo.org/mailman/listinfo/grass-user</a><br>
<br></blockquote></div><br>