<div dir="ltr"><div>Hi Moritz, <br></div><div><br></div><div>thanks for coming back to this topic :)</div><div><br></div><div>Here, the original region settings:</div><div><br></div><div><span id="gmail-:5y.co" class="gmail-tL8wMe gmail-EMoHub" style="text-align:left" dir="ltr">g.region -p<br>projection: 1 (UTM)<br>zone: -21<br>datum: wgs84<br>ellipsoid: wgs84<br>north: 7168182<br>south: 7157384<br>west: 737450<br>east: 750236<br>nsres: 0.5<br>ewres: 0.5<br>rows: 21596<br>cols: 25572<br>cells: 552252912</span></div><div><span id="gmail-:5y.co" class="gmail-tL8wMe gmail-EMoHub" style="text-align:left" dir="ltr"><br></span></div><div><span id="gmail-:5y.co" class="gmail-tL8wMe gmail-EMoHub" style="text-align:left" dir="ltr">Because of the error we reduced it by ~half, but still got the same error...<br></span></div><div><span id="gmail-:5y.co" class="gmail-tL8wMe gmail-EMoHub" style="text-align:left" dir="ltr"><br></span></div><div><span id="gmail-:5y.co" class="gmail-tL8wMe gmail-EMoHub" style="text-align:left" dir="ltr">[..]<br>rows: 20478<br>cols: 13553<br>cells: 277538334</span></div><div><span id="gmail-:5y.co" class="gmail-tL8wMe gmail-EMoHub" style="text-align:left" dir="ltr"><br></span></div><div><span id="gmail-:5y.co" class="gmail-tL8wMe gmail-EMoHub" style="text-align:left" dir="ltr">In the end, we solved it by extracting the stats per tile (cutlines irregular tiles) and then patching...</span></div><div><span id="gmail-:5y.co" class="gmail-tL8wMe gmail-EMoHub" style="text-align:left" dir="ltr"><br></span></div><div><span id="gmail-:5y.co" class="gmail-tL8wMe gmail-EMoHub" style="text-align:left" dir="ltr">best,</span></div><div><span id="gmail-:5y.co" class="gmail-tL8wMe gmail-EMoHub" style="text-align:left" dir="ltr">Vero<br></span></div><div><span id="gmail-:5y.co" class="gmail-tL8wMe gmail-EMoHub" style="text-align:left" dir="ltr"><br></span></div><div><span id="gmail-:5y.co" class="gmail-tL8wMe gmail-EMoHub" style="text-align:left" dir="ltr"><span id="gmail-:es.co" class="gmail-tL8wMe gmail-EMoHub" style="text-align:left" dir="ltr"></span></span></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">El jue., 5 dic. 2019 a las 9:35, Moritz Lennert (<<a href="mailto:mlennert@club.worldonline.be">mlennert@club.worldonline.be</a>>) escribió:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 3/12/19 13:26, Veronica Andreo wrote:<br>
> Hi Moritz,<br>
> <br>
> Thanks for your answer :)<br>
> <br>
> In smaller subsets (per cutlines tiles), it works just fine.<br>
<br>
With grass78 using python 3 ? Then probably it's not a python version issue.<br>
<br>
What is the region size ? When it is calculating the raster statistics <br>
it uses r.univar with extended statistics (-e flag), if the region size <br>
is too big, actually you are right that this might lead to memory errors <br>
if it tries to run 4 instances in parallel.<br>
<br>
Unfortunately, r.univar does not have a memory option allowing to limit <br>
memory usage. Maybe i.segment.stats should check the region size and <br>
available memory and bail out if there's not enough memory. Not sure how <br>
to calculate the needed memory size, though (especially since r.univar's <br>
-t flag is also set).<br>
<br>
You could also try to reduce parallelization, i.e. run it with two <br>
processes only. It will obviously be slower.<br>
<br>
Moritz<br>
<br>
<br>
> <br>
> El mar., 3 dic. 2019 a las 12:41, Moritz Lennert <br>
> (<<a href="mailto:mlennert@club.worldonline.be" target="_blank">mlennert@club.worldonline.be</a> <mailto:<a href="mailto:mlennert@club.worldonline.be" target="_blank">mlennert@club.worldonline.be</a>>>) <br>
> escribió:<br>
> <br>
> Hi Vero,<br>
> <br>
> Le Mon, 2 Dec 2019 13:43:30 +0100,<br>
> Veronica Andreo <<a href="mailto:veroandreo@gmail.com" target="_blank">veroandreo@gmail.com</a> <mailto:<a href="mailto:veroandreo@gmail.com" target="_blank">veroandreo@gmail.com</a>>><br>
> a écrit :<br>
> <br>
> > Hi,<br>
> ><br>
> > We are trying to use i.segment.stats for a map with 800000+ segments<br>
> > and already in two different laptops, we get:<br>
> ><br>
> > bands=`g.list rast pat=IGUAZU_IMG_* sep=,`<br>
> ><br>
> RASTER_STATS=(min,max,range,mean,stddev,median,first_quart,third_quart,perc_90)<br>
> > AREA_STATS=(area,perimeter,compact_circle,compact_square,fd)<br>
> ><br>
> > i.segment.stats -rc \<br>
> > map=segments_full_region \<br>
> > rasters=$bands \<br>
> > raster_statistics=$RASTER_STATS \<br>
> > area_measures=$AREA_STATS \<br>
> > vectormap=segs_stats_map \<br>
> > processes=4<br>
> > Calculating geometry statistics...<br>
> > Calculating statistics for raster maps...<br>
> > Exception in thread Thread-3:<br>
> > Traceback (most recent call last):<br>
> > File "/usr/lib/python3.6/threading.py", line 916, in<br>
> > _bootstrap_inner self.run()<br>
> > File "/usr/lib/python3.6/threading.py", line 864, in run<br>
> > self._target(*self._args, **self._kwargs)<br>
> > File "/usr/lib/python3.6/multiprocessing/pool.py", line 463, in<br>
> > _handle_results<br>
> > task = get()<br>
> > File "/usr/lib/python3.6/multiprocessing/connection.py", line 251,<br>
> > in recv return _ForkingPickler.loads(buf.getbuffer())<br>
> > TypeError: __init__() missing 3 required positional arguments:<br>
> > 'module', 'code', and 'returncode'<br>
> ><br>
> > Does it have to do with memory? I used the module a month ago with<br>
> > 500000+ segments and it worked just fine...<br>
> <br>
> I don't think memory is the issue, but I find the error message<br>
> pretty cryptic, so wouldn't exclude altogether. Could it be some<br>
> difference between Python 2 and 3 in the multiprocessing module ? Would<br>
> it be possible for you try running it in Python 2 ?<br>
> <br>
> Maybe you could also try to run it on a smaller subset of the<br>
> segmentation result ?<br>
> <br>
> Moritz<br>
> <br>
> <br>
> -- <br>
> Département Géosciences, Environnement et Société Université Libre de<br>
> Bruxelles Bureau: S.DB.6.138<br>
> CP 130/03<br>
> Av. F.D. Roosevelt 50<br>
> 1050 Bruxelles<br>
> Belgique<br>
> <br>
> tél. + 32 2 650.68.12 / 68.11 (secr.)<br>
> fax + 32 2 650.68.30<br>
> <br>
<br>
<br>
</blockquote></div>