[GRASS-user] error in remove mask

[Markus Metz]

On Thu, May 25, 2017 at 5:24 PM, Giuseppe Amatulli <
giuseppe.amatulli at gmail.com> wrote:
>
> Hi Markus,
> in reality the computation beyond the mask problem is very complex, I
will try to summarize.
> I have the UNIT3753 mask sit on the PERMANENT and I create 1000 location
in a multicore environment (HPC).
> Each location use the UNIT3753-mask from the PERMANENT. This action
produce that the  cell_misc/UNIT3753/reclassed_to is written simultaneously
from different cores.

This is the problem. If several processes try to edit the same file at
once, this can cause data corruption. The reclassed_to file holds the names
of the reclassed raster maps based on that raster map. Instead of r.mask,
you can use r.mapcalc to create specific MASKs for each process on each
compute node. This should avoid these problems, as long as each process on
each compute node uses its own unique mapset.

Markus M (running GRASS on an HPC system every day)

> I think that this action creates a  reclassed_to file very large and
probably also with different permission which was not be able to be
removed.
>
> I solved by manually  remove the reclassed_to file, so now the process is
running smoothly.
>
> A potential solution would be:
> 1) for each location  g.copy raster=PERMANENT,UNIT3753 , and than apply
the r.mask using the UNIT3753 in the location and not in the PERMANENT.
> This would work but data duplication will be there.
> 2) a second solution would be if  the reclassed_to can have a unique name
(e.g. reclassed_to$$) associate to the location.
>
> Any thought?
> Thank you
> Best Regards
> Giuseppe
> p.s. in few weeks I will be in Berlin so we may chat in person.
>
>
>
>
>
>
> On 24 May 2017 at 18:29, Markus Neteler <neteler at osgeo.org> wrote:
>>
>> On Wed, May 24, 2017 at 11:40 PM, Giuseppe Amatulli
>> <giuseppe.amatulli at gmail.com> wrote:
>> > Hi Nikos
>> > I just sort out.
>> > the file /PERMANENT/cell_misc/UNIT3753/reclassed_to was very big
(2.5g) and
>> > it  was not be able to be removed with the r.remove.
>>
>> That size is not an issue at all (the EU DEM is 25GB or more for
>> example)... the problem must be elsewhere. Happy to help if possible.
>> BTW: I have compiled GRASS 7.2 also for RHEL via EPEL, see the download
page.
>>
>> Markus
>>
>>
>> --
>> Markus Neteler, PhD
>> http://www.mundialis.de - free data with free software
>> http://grass.osgeo.org
>> http://courses.neteler.org/blog
>
>
>
>
> --
> Giuseppe Amatulli, Ph.D.
>
> Research scientist at
> Yale School of Forestry & Environmental Studies
> Yale Center for Research Computing
> Center for Science and Social Science Information
> New Haven, 06511
> Teaching: http://spatial-ecology.org
> Work:  https://environment.yale.edu/profile/giuseppe-amatulli/
>
> _______________________________________________
> grass-user mailing list
> grass-user at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-user
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20170528/b13326a4/attachment.html>