<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Jun 30, 2014 at 5:21 AM, Annalisa Minelli <span dir="ltr"><<a href="mailto:annagrass6@gmail.com" target="_blank">annagrass6@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi all,<div>I'm attempting to parallelize a job in a python script using multiprocess library in grass70.</div>
<div>I had a look at the following links: <a href="http://grasswiki.osgeo.org/wiki/Parallel_GRASS_jobs" target="_blank">http://grasswiki.osgeo.org/wiki/Parallel_GRASS_jobs</a></div>
<div>and <a href="http://grasswiki.osgeo.org/wiki/Parallelizing_Scripts" target="_blank">http://grasswiki.osgeo.org/wiki/Parallelizing_Scripts</a>.</div><div><br></div><div>I would like to work in the same location but in different mapsets because my jobs touch the region settings, but I don't know how to set separate mapset for separate jobs.</div>
<div><br></div><div>Since now I discovered that this processes, if run in the same mapset, clean all the environmental variables (GISDBASE, LOCATION, MAPSET) so then GRASS does not start anymore and I have to restore the .grass70/rc file..</div>
<div><br></div><div>can anyone hint me on how to set different mapsets for different jobs?</div><div><br></div></div></blockquote><div><br></div><div>First, look at the PyGRASS GridModule [1] whether this can help you.<br>
<br>For general case, there is unfortunately no API. From what I understand, you have to create a file "gisrc" somewhere and then do something like env = copy(os.environ) and change GISRC there to your custom "gisrc". Then you the change the mapset and region by standard GRASS means but you must pass `env` parameter to all command/module calls (env is used by Python subprocess to set environment just for one process).<br>
</div><div><br></div><div>Note that GISRC, GISBASE and LOCATION are (system) environmental variables while GISDBASE, LOCATION_NAME and MAPSET are GRASS GIS session/environment variables and are stored in "gisrc" file. I don't have an idea what LOCATION variable is for (it contains full path to the mapset).<br>
</div><div><br></div><div>I would be glad to hear what others think about this.<br></div><div><br></div><div>You can of course read source code of GridModule, rendering in wxGUI, g.gui.animation, or the following snipped but I don't say that it will be easy to understand and there might be a lot of imperfections.<br>
<br></div><div>Vaclav<br></div><div><br><div class="" id="LC90"><span class=""> # we rely on the tmp dir having enough space for our map</span></div><div class="" id="LC91"> <span class="">tgt_gisdbase</span> <span class="">=</span> <span class="">tempfile</span><span class="">.</span><span class="">mkdtemp</span><span class="">()</span></div>
<div class="" id="LC92"> <span class=""># this is not needed if we use mkdtemp but why not</span></div><div class="" id="LC93"> <span class="">tgt_location</span> <span class="">=</span> <span class="">'r.out.png.proj_location_</span><span class="">%s</span><span class="">'</span> <span class="">%</span> <span class="">epsg_code</span></div>
<div class="" id="LC94"> <span class=""># because we are using PERMANENT we don't have to create mapset explicitly</span></div><div class="" id="LC95"> <span class="">tgt_mapset_name</span> <span class="">=</span> <span class="">'PERMANENT'</span></div>
<div class="" id="LC96"><br></div><div class="" id="LC97"> <span class="">src_mapset</span> <span class="">=</span> <span class="">Mapset</span><span class="">(</span><span class="">src_mapset_name</span><span class="">)</span></div>
<div class="" id="LC98"><br></div><div class="" id="LC99"> <span class=""># get source (old) and set target (new) GISRC enviromental variable</span></div><div class="" id="LC100"> <span class=""># TODO: set environ only for child processes could be enough and it would</span></div>
<div class="" id="LC101"> <span class=""># enable (?) parallel runs</span></div><div class="" id="LC102"> <span class="">src_gisrc</span> <span class="">=</span> <span class="">os</span><span class="">.</span><span class="">environ</span><span class="">[</span><span class="">'GISRC'</span><span class="">]</span></div>
<div class="" id="LC103"> <span class="">tgt_gisrc</span> <span class="">=</span> <span class="">gsetup</span><span class="">.</span><span class="">write_gisrc</span><span class="">(</span><span class="">tgt_gisdbase</span><span class="">,</span></div>
<div class="" id="LC104"> <span class="">tgt_location</span><span class="">,</span> <span class="">tgt_mapset_name</span><span class="">)<br></span></div><div class="" id="LC104"><span class=""> # we should use a copy and pass it but then it would not be possible to use create_location<br>
</span></div><div class="" id="LC105"> <span class="">os</span><span class="">.</span><span class="">environ</span><span class="">[</span><span class="">'GISRC'</span><span class="">]</span> <span class="">=</span> <span class="">tgt_gisrc</span></div>
<div class="" id="LC106"> <span class="">if</span> <span class="">os</span><span class="">.</span><span class="">environ</span><span class="">.</span><span class="">get</span><span class="">(</span><span class="">'WIND_OVERRIDE'</span><span class="">):</span></div>
<div class="" id="LC107"> <span class="">old_temp_region</span> <span class="">=</span> <span class="">os</span><span class="">.</span><span class="">environ</span><span class="">[</span><span class="">'WIND_OVERRIDE'</span><span class="">]</span></div>
<div class="" id="LC108"> <span class="">del</span> <span class="">os</span><span class="">.</span><span class="">environ</span><span class="">[</span><span class="">'WIND_OVERRIDE'</span><span class="">]</span></div>
<div class="" id="LC109"> <span class="">else</span><span class="">:</span></div><div class="" id="LC110"> <span class="">old_temp_region</span> <span class="">=</span> <span class="">None</span></div><div class="" id="LC111">
<span class=""># these lines looks good but anyway when developing the module</span></div><div class="" id="LC112"> <span class=""># switching location seemed fragile and on some errors (while running</span></div>
<div class="" id="LC113">
<span class=""># unfinished module) location was switched in the command line</span></div><div class="" id="LC114"><br></div><div class="" id="LC115"> <span class="">try</span><span class="">:</span></div><div class="" id="LC116">
<span class=""># the function itself is not safe for other (backgroud) processes</span></div><div class="" id="LC117"> <span class=""># (e.g. GUI), however we already switched GISRC for us</span></div><div class="" id="LC118">
<span class=""># and child processes, so we don't influece others</span></div><div class="" id="LC119"> <span class="">gcore</span><span class="">.</span><span class="">create_location</span><span class="">(</span><span class="">dbase</span><span class="">=</span><span class="">tgt_gisdbase</span><span class="">,</span></div>
<div class="" id="LC120"> <span class="">location</span><span class="">=</span><span class="">tgt_location</span><span class="">,</span></div><div class="" id="LC121"> <span class="">epsg</span><span class="">=</span><span class="">epsg_code</span><span class="">,</span></div>
<div class="" id="LC122"> <span class="">datum</span><span class="">=</span><span class="">None</span><span class="">,</span></div><div class="" id="LC123"> <span class="">datum_trans</span><span class="">=</span><span class="">None</span><span class="">)</span></div>
<div class="" id="LC124"><br></div><div class="" id="LC125"> <span class=""># Mapset object cannot be created if the real mapset does not exists</span></div><div class="" id="LC126"> <span class="">tgt_mapset</span> <span class="">=</span> <span class="">Mapset</span><span class="">(</span><span class="">gisdbase</span><span class="">=</span><span class="">tgt_gisdbase</span><span class="">,</span> <span class="">location</span><span class="">=</span><span class="">tgt_location</span><span class="">,</span></div>
<div class="" id="LC127"> <span class="">mapset</span><span class="">=</span><span class="">tgt_mapset_name</span><span class="">)</span></div><div class="" id="LC128"> <span class=""># set the current mapset in the library</span></div>
<div class="" id="LC129"> <span class=""># we actually don't need to switch when only calling modules</span></div><div class="" id="LC130"> <span class=""># (right GISRC is enough for them)</span></div>
<div class="" id="LC131">
<span class="">tgt_mapset</span><span class="">.</span><span class="">current</span><span class="">()</span></div><div class="" id="LC132">...<br></div><br> <br><br>[1] <a href="http://grass.osgeo.org/grass71/manuals/pygrass/modules_grid.html">http://grass.osgeo.org/grass71/manuals/pygrass/modules_grid.html</a><br>
<br><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div></div><div><br></div><div>All the best,</div><div>Annalisa</div></div>
<br>_______________________________________________<br>
grass-dev mailing list<br>
<a href="mailto:grass-dev@lists.osgeo.org">grass-dev@lists.osgeo.org</a><br>
<a href="http://lists.osgeo.org/mailman/listinfo/grass-dev" target="_blank">http://lists.osgeo.org/mailman/listinfo/grass-dev</a><br></blockquote></div><br></div></div>