[GRASS-user] error in remove mask

[Giuseppe Amatulli]

Thanks,
I implemented accordingly and seams to work fine.
Thank you.
Best
Giuseppe

On 28 May 2017 at 17:18, Markus Metz <markus.metz.giswork at gmail.com> wrote:

>
>
> On Thu, May 25, 2017 at 5:24 PM, Giuseppe Amatulli <
> giuseppe.amatulli at gmail.com> wrote:
> >
> > Hi Markus,
> > in reality the computation beyond the mask problem is very complex, I
> will try to summarize.
> > I have the UNIT3753 mask sit on the PERMANENT and I create 1000 location
> in a multicore environment (HPC).
> > Each location use the UNIT3753-mask from the PERMANENT. This action
> produce that the  cell_misc/UNIT3753/reclassed_to is written
> simultaneously from different cores.
>
> This is the problem. If several processes try to edit the same file at
> once, this can cause data corruption. The reclassed_to file holds the names
> of the reclassed raster maps based on that raster map. Instead of r.mask,
> you can use r.mapcalc to create specific MASKs for each process on each
> compute node. This should avoid these problems, as long as each process on
> each compute node uses its own unique mapset.
>
> Markus M (running GRASS on an HPC system every day)
>
> > I think that this action creates a  reclassed_to file very large and
> probably also with different permission which was not be able to be
> removed.
> >
> > I solved by manually  remove the reclassed_to file, so now the process
> is running smoothly.
> >
> > A potential solution would be:
> > 1) for each location  g.copy raster=PERMANENT,UNIT3753 , and than apply
> the r.mask using the UNIT3753 in the location and not in the PERMANENT.
> > This would work but data duplication will be there.
> > 2) a second solution would be if  the reclassed_to can have a unique
> name (e.g. reclassed_to$$) associate to the location.
> >
> > Any thought?
> > Thank you
> > Best Regards
> > Giuseppe
> > p.s. in few weeks I will be in Berlin so we may chat in person.
> >
> >
> >
> >
> >
> >
> > On 24 May 2017 at 18:29, Markus Neteler <neteler at osgeo.org> wrote:
> >>
> >> On Wed, May 24, 2017 at 11:40 PM, Giuseppe Amatulli
> >> <giuseppe.amatulli at gmail.com> wrote:
> >> > Hi Nikos
> >> > I just sort out.
> >> > the file /PERMANENT/cell_misc/UNIT3753/reclassed_to was very big
> (2.5g) and
> >> > it  was not be able to be removed with the r.remove.
> >>
> >> That size is not an issue at all (the EU DEM is 25GB or more for
> >> example)... the problem must be elsewhere. Happy to help if possible.
> >> BTW: I have compiled GRASS 7.2 also for RHEL via EPEL, see the download
> page.
> >>
> >> Markus
> >>
> >>
> >> --
> >> Markus Neteler, PhD
> >> http://www.mundialis.de - free data with free software
> >> http://grass.osgeo.org
> >> http://courses.neteler.org/blog
> >
> >
> >
> >
> > --
> > Giuseppe Amatulli, Ph.D.
> >
> > Research scientist at
> > Yale School of Forestry & Environmental Studies
> > Yale Center for Research Computing
> > Center for Science and Social Science Information
> > New Haven, 06511
> > Teaching: http://spatial-ecology.org
> > Work:  https://environment.yale.edu/profile/giuseppe-amatulli/
> >
> > _______________________________________________
> > grass-user mailing list
> > grass-user at lists.osgeo.org
> > https://lists.osgeo.org/mailman/listinfo/grass-user
>
>


-- 
Giuseppe Amatulli, Ph.D.

Research scientist at
Yale School of Forestry & Environmental Studies
Yale Center for Research Computing
Center for Science and Social Science Information
New Haven, 06511
Teaching: http://spatial-ecology.org
Work:  https://environment.yale.edu/profile/giuseppe-amatulli/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20170601/557fe4e0/attachment.html>