Dear Nick,
<br><br>I am working on SRTM rasters of resolution 1km, 500m, 250m and
recently 90m using GRASS for long and had countered the problem being
faced by you.
<br><br>My workaround is given below and as a by-product you may get a very nice coastline vector/raster also.
<br><br>1. I have converted all the nulls with r.mapcalc to -160 using the formula C=if(A, if(isnull(B),-160,B))
<br>Where,
<br>A: region of interest
<br>B: original srtm raster with nulls for sea and holes
<br>C: Output map in which nulls have been converted to -160.
<br><br>(Why -160, because in my map, as from my human memory, the
lowest value was -70. I just chose a value lower than this value. We
will need it later on to convert back them as holes for r.fillnulls to
work upon.)
<br><br>A sample from the history file of the raster is produced below:
<br><br>Sun Aug 1 17:49:10 2010
<br>test5
<br>PERMANENT
<br>pks
<br>raster
<br><br><br>generated by r.mapcalc
<br>if(test_nL_region@PERMANENT, if(isnull(ib2@PERMANENT), -160, ib2@PERMANENT))
<br><br><br>A: test_nL_region
<br><br>This raster map is a region of my interest out the big srtm
raster converted from a vector. Grass has a command , which converts the
current region to a vector file.
<br>Again, the sample from the history file of the raster is produced below:
<br><br>ed Jul 28 10:09:42 2010
<br>test_nL_region
<br>PERMANENT
<br>pks
<br>raster
<br>Vector Map: nL_region@PERMANENT in mapset PERMANENT
<br>Original scale from vector map: 1:1
<br>generated by v.to.rast
<br>v.to.rast input="nL_region@PERMANENT" output="test_nL_region" use="c\
<br>at" type="point,line,area" layer=1 value=1 rows=4096
<br><br>nL_region:
<br>Again, the sample from the history file of the vector is produced below:
<br>COMMAND: v.in.region output="nL_region" type="area" cat=1
<br>GISDBASE: /home/pks/renamed_grassdata
<br>LOCATION: newLocation MAPSET: PERMANENT USER: pks DATE: Wed Jul 28 09:53:43 2010
<br><br>So, now we have -160s all around , i.e. the sea and holes on land.
<br><br>Now use the r.reclass.area feature to differentiate sea from holes on land. A sample from my recent work is given below:
<br><br>Tue Nov 8 13:55:47 2011
<br>bi_reclassed3_area.recl
<br>work2
<br>rdcgis1
<br>reclass
<br>Reclassified map based on:
<br> Map [bi_reclassed3.clump.bi_reclassed3_area] in mapset [work2]
<br>generated by r.reclass
<br><br>I am using GRASS7 and in the history given above , all the parameters are not available.
<br>In my case , I used 1 billion hectare as "greater than area"
parameter to bring out the sea , land , and holes in land in different
categories.
<br><br>Thereafter, use mask and pick up the landmass with holes in land , which you want to fill.
<br>Just remember to convert back the -160 to null again.
<br><br>I felt that r.fillnulls works best when the nulls have values around them.
<br><br>By the way, CGIAR provides holes-filled-SRTM-raster.You may like
to read their literature for understanding the algorithms used to fill
the nulls. It's good and useful.
<br><br>Try this link for downloading clean rasters of your area of interest:
<br><br><a href="http://srtm.csi.cgiar.org/SELECTION/inputCoord.asp" target="_top" rel="nofollow">http://srtm.csi.cgiar.org/SELECTION/inputCoord.asp</a><br><br>Hope, it helps.
<br><br>Some of my observations are listed below:
<br>1. If the system lacks RAM, then increase patience.
<br>2. If the system lacks hard-disk space, don't go for GIS.
<br>3. Learn python.
<br><br><div class="gmail_quote">On Fri, Dec 2, 2011 at 5:37 AM, <span dir="ltr"><<a href="mailto:grass-user-request@lists.osgeo.org">grass-user-request@lists.osgeo.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Send grass-user mailing list submissions to<br>
<a href="mailto:grass-user@lists.osgeo.org">grass-user@lists.osgeo.org</a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="http://lists.osgeo.org/mailman/listinfo/grass-user" target="_blank">http://lists.osgeo.org/mailman/listinfo/grass-user</a><br>
or, via email, send a message with subject or body 'help' to<br>
<a href="mailto:grass-user-request@lists.osgeo.org">grass-user-request@lists.osgeo.org</a><br>
<br>
You can reach the person managing the list at<br>
<a href="mailto:grass-user-owner@lists.osgeo.org">grass-user-owner@lists.osgeo.org</a><br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of grass-user digest..."<br>
<br>
<br>
Today's Topics:<br>
<br>
1. Re: r.fillnulls for large area (Nick Jachowski)<br>
2. topology model resume (and some proposals) (G. Allegri)<br>
3. Re: [GRASS-dev] Should v.in.ogr clean topology by default ?<br>
(Hamish)<br>
4. Re: r.fillnulls for large area (Hamish)<br>
5. Re: r.fillnulls for large area (Marcello Gorini)<br>
<br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Fri, 2 Dec 2011 00:07:07 +0700<br>
From: Nick Jachowski <<a href="mailto:njachowski@gmail.com">njachowski@gmail.com</a>><br>
Subject: Re: [GRASS-user] r.fillnulls for large area<br>
To: Hamish <<a href="mailto:hamish_b@yahoo.com">hamish_b@yahoo.com</a>><br>
Cc: <a href="mailto:grass-user@lists.osgeo.org">grass-user@lists.osgeo.org</a><br>
Message-ID:<br>
<<a href="mailto:CAPFL4TFhef32rraoch_%2B29xet8xWm6-B4sWYtva1qKi4GDC7qQ@mail.gmail.com">CAPFL4TFhef32rraoch_+29xet8xWm6-B4sWYtva1qKi4GDC7qQ@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="iso-8859-1"<br>
<br>
Hi Hamish,<br>
<br>
Thanks for the advice to fill the ocean with zeros. It worked better, and<br>
didn't get "killed", but it still didn't work...This time it made it<br>
further than before, but it got "abondoned" - see the output below...<br>
<br>
Maybe I need to split my region into subregions and run r.fillnulls on each<br>
subregion then patch them together. If anybody has a better idea I would be<br>
grateful to know.<br>
<br>
Thanks!<br>
Nick<br>
<br>
GRASS 6.4.1 (seasia_ll):~ > r.fillnulls in=srtmm0 out=srtmm0f<br>
Locating and isolating NULL areas...<br>
100%<br>
Reading input raster map <r_fillnulls_18931@PERMANENT>...<br>
100%<br>
Finding buffer zones...<br>
100%<br>
Writing output raster map <r_fillnulls_18931.buf>...<br>
100%<br>
100%<br>
Creating interpolation points...<br>
Extracting points...<br>
100%<br>
Building topology for vector map <vecttmp_fillnulls_18931>...<br>
Registering primitives...<br>
13428068 primitives registered<br>
13428068 vertices registered<br>
Building areas...<br>
100%<br>
0 areas built<br>
0 isles built<br>
Attaching islands...<br>
Attaching centroids...<br>
100%<br>
Number of nodes: 13428068<br>
Number of primitives: 13428068<br>
Number of points: 13428068<br>
Number of lines: 0<br>
Number of boundaries: 0<br>
Number of centroids: 0<br>
Number of areas: 0<br>
Number of isles: 0<br>
ERROR: /usr/lib/grass64/scripts/r.fillnulls abandoned. Removing temporary<br>
maps, restoring user mask if needed:<br>
Removing raster <MASK><br>
Removing raster <r_fillnulls_18931><br>
Removing raster <r_fillnulls_18931.buf><br>
Removing raster <r_fillnulls_18931_filled><br>
WARNING: Raster map <r_fillnulls_18931_filled> not found<br>
WARNING: <r_fillnulls_18931_filled> nothing removed<br>
Removing vector <vecttmp_fillnulls_18931><br>
WARNING: Table <vecttmp_fillnulls_18931> linked to vector map<br>
<vecttmp_fillnulls_18931> does not exist<br>
WARNING: raster <usermask_mask.18931> not found<br>
<br>
<br>
On Thu, Dec 1, 2011 at 8:36 AM, Hamish <<a href="mailto:hamish_b@yahoo.com">hamish_b@yahoo.com</a>> wrote:<br>
<br>
> Nick wrote:<br>
> > I am having trouble filling the null areas in the SRTM (3 second<br>
> > resolution) DEM for Southeast Asia. I think the reason is because the<br>
> > region I am trying fill is too large. The region is almost a billion<br>
> > cells - see g.region -p output below - but about 40% of that region is<br>
> > ocean areas with no DEM data. I'm wondering if maybe the problem is that<br>
> > r.fillnulls is trying to fill the ocean areas.<br>
><br>
> it will try that. a good first step is to fill in the ocean with 0s.<br>
> I was just doing this yesterday with srtm data actually.<br>
> GRASS> r.mapcalc "zero = 0"<br>
><br>
> then r.patch with your srtm data listed first and the zeros listed last.<br>
><br>
> but be careful you don't fill in zeros where there are holes in the data<br>
> on land. if you have a vector coastline use "v.to.rast use=val val=1" to<br>
> create a land mask, then invert with:<br>
> r.mapcalc "sea.mask = if(isnull(land.mask), 1, null())"<br>
><br>
> and use the sea.mask as the background fill-in for the r.patch step.<br>
><br>
><br>
> > r.fillnulls gets to the point where it says "Building areas..." then<br>
> > it says "Killed" and prints an error message.<br>
><br>
> often "Killed" happens when the computer ran out of memory.<br>
> not interpolating over the ocean may make the thing go much faster and<br>
> need much less memory, as it has much less to do.<br>
><br>
><br>
> > I am using GRASS 6.4.1. Eventually I plan on using the null-filled<br>
> > DEM to delineate the watersheds in Southeast Asia.<br>
><br>
> start thinking about your watershed threshold size now :)<br>
><br>
><br>
> good luck,<br>
> Hamish<br>
><br>
><br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <a href="http://lists.osgeo.org/pipermail/grass-user/attachments/20111202/2a3d50ef/attachment-0001.html" target="_blank">http://lists.osgeo.org/pipermail/grass-user/attachments/20111202/2a3d50ef/attachment-0001.html</a><br>
<br>
------------------------------<br>
<br>
Message: 2<br>
Date: Thu, 1 Dec 2011 18:33:01 +0100<br>
From: "G. Allegri" <<a href="mailto:giohappy@gmail.com">giohappy@gmail.com</a>><br>
Subject: [GRASS-user] topology model resume (and some proposals)<br>
To: <a href="mailto:grass-user@lists.osgeo.org">grass-user@lists.osgeo.org</a><br>
Message-ID:<br>
<CAB4g1=<a href="mailto:xqBcQOjoZ49n0adS0HGsEokccwtpS4nNtotzfMgKLvSQ@mail.gmail.com">xqBcQOjoZ49n0adS0HGsEokccwtpS4nNtotzfMgKLvSQ@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="iso-8859-1"<br>
<br>
I resume (first as a repeat to myself) what I've learned from the various<br>
email on the topic<br>
<br>
Vectors can be:<br>
<br>
LEVEL 1:<br>
- no topology -> very limited use<br>
LEVEL 2:<br>
- unclean topology -> limited use<br>
- clean topology -> full support<br>
<br>
I previously thought that LEVEL 2 was only possible for clean topologies,<br>
and I was wrong...<br>
<br>
At the moment there isn't a tool to list the the uncorrect geometries from<br>
a topological point of view. v.build only checks some constraints, not all.<br>
The proposal is to extend it to check against all the rules that are<br>
required to consider a geometry topologically correct (an extended flag to<br>
v defaul.build maybe).<br>
<br>
v.in.ogr builds and cleans (by default). It would be useful to have the<br>
"clean" phase available to be launched independently. I mean, something<br>
like an "automatic" flag for v.clean, that would operate the same cleaning<br>
as during the import of a vector.<br>
<br>
Conclusions: the topological correctness isn't a constraint for the vector<br>
topology data structure. GRASS haven't all the topology rules hard-coded<br>
(... or yes?). Most of thems (all?) are defined inside the code of v.build<br>
and v.clean, but I suppose that there isn't an autonomous<br>
library/functionality that provide the semantics of a "correct topology".<br>
Am I wrong?<br>
<br>
Thanks everyone for the support ;)<br>
giovanni<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <a href="http://lists.osgeo.org/pipermail/grass-user/attachments/20111201/48197e88/attachment-0001.html" target="_blank">http://lists.osgeo.org/pipermail/grass-user/attachments/20111201/48197e88/attachment-0001.html</a><br>
<br>
------------------------------<br>
<br>
Message: 3<br>
Date: Thu, 1 Dec 2011 13:01:38 -0800 (PST)<br>
From: Hamish <<a href="mailto:hamish_b@yahoo.com">hamish_b@yahoo.com</a>><br>
Subject: [GRASS-user] Re: [GRASS-dev] Should v.in.ogr clean topology<br>
by default ?<br>
To: <a href="mailto:grass-user@lists.osgeo.org">grass-user@lists.osgeo.org</a><br>
Message-ID:<br>
<<a href="mailto:1322773298.49418.YahooMailClassic@web110005.mail.gq1.yahoo.com">1322773298.49418.YahooMailClassic@web110005.mail.gq1.yahoo.com</a>><br>
Content-Type: text/plain; charset=us-ascii<br>
<br>
[cc'd from grass-dev ML]<br>
<br>
Hi,<br>
<br>
a couple of points--<br>
<br>
* Radim said many times: v.in.ogr's cleaning and v.clean are not<br>
the same thing. (with a number of !! added) He explained it and<br>
understood it always better than I ever could, so check in the<br>
mailing list archives for details.<br>
<br>
* GRASS is a topological GIS. so when data is imported into GRASS<br>
it is natural and desirable that it gets converted into the<br>
family way at that time.<br>
<br>
* In defense of the option to allow non topological imports:<br>
in the past I have imported a shapefile created by leading<br>
proprietary non-topological GIS. this file was a string of<br>
buffers around points with unique ID numbers. the idea was to<br>
run (the predecessor to) v.rast.stats on each buffer to collect<br>
some neighborhood context stats for each ID point, which could<br>
be loaded into a table. wherever the line of points was not<br>
exactly straight the buffers overlapped a little and grass's<br>
cleaning would treat it as a ven diagram and create a new<br>
centroid in the overlapping sliver. no good for extracting the<br>
buffer area where='NODE_ID = 1234'.<br>
also you might consider a vector area map with multiple over-<br>
lapping home ranges for different territorial animals. not 3d,<br>
but still merging/flattening overlapping areas is not appropriate.<br>
<br>
<br>
for things like neighboring land parcels, political lines, or<br>
land/sea boundaries non-topological is a nightmare, but for<br>
other tasks it can occasionally be very useful.<br>
<br>
<br>
Hamish<br>
<br>
<br>
------------------------------<br>
<br>
Message: 4<br>
Date: Thu, 1 Dec 2011 13:16:45 -0800 (PST)<br>
From: Hamish <<a href="mailto:hamish_b@yahoo.com">hamish_b@yahoo.com</a>><br>
Subject: Re: [GRASS-user] r.fillnulls for large area<br>
To: Nick Jachowski <<a href="mailto:njachowski@gmail.com">njachowski@gmail.com</a>><br>
Cc: <a href="mailto:grass-user@lists.osgeo.org">grass-user@lists.osgeo.org</a><br>
Message-ID:<br>
<<a href="mailto:1322774205.78253.YahooMailClassic@web110013.mail.gq1.yahoo.com">1322774205.78253.YahooMailClassic@web110013.mail.gq1.yahoo.com</a>><br>
Content-Type: text/plain; charset=iso-8859-1<br>
<br>
Nick wrote:<br>
> Thanks for the advice to fill the ocean with<br>
> zeros. It worked better, and didn't get "killed",<br>
> but it still didn't work...This time it made it<br>
> further than before, but it got "abondoned" - see<br>
> the output below...<br>
<br>
try editing the r.fillnulls script, on line 165<br>
replace "r.to.vect input=..." with<br>
"r.to.vect -b input=...". I think it would work,<br>
but I'm not sure if it would make it slower to run<br>
v.surf.rst or not. (?)<br>
<br>
> Maybe I need to split my region into subregions<br>
> and run r.fillnulls on each subregion then patch<br>
> them together.<br>
<br>
yes, that should work well.<br>
<br>
> Number of nodes: 13428068<br>
<br>
building topology for anything more that 3-5 million<br>
cells will consume a large amount of memory due to<br>
the cumulative effect of the small, but real,<br>
memory requirement of each individual feature.<br>
maybe if you had 16gb RAM 13 million would be ok?<br>
<br>
<br>
Hamish<br>
<br>
<br>
------------------------------<br>
<br>
Message: 5<br>
Date: Thu, 1 Dec 2011 20:04:38 -0200<br>
From: Marcello Gorini <<a href="mailto:gorini@gmail.com">gorini@gmail.com</a>><br>
Subject: Re: [GRASS-user] r.fillnulls for large area<br>
To: Hamish <<a href="mailto:hamish_b@yahoo.com">hamish_b@yahoo.com</a>><br>
Cc: <a href="mailto:grass-user@lists.osgeo.org">grass-user@lists.osgeo.org</a><br>
Message-ID:<br>
<CAEvwxEJtmTd_dEfowfDcpfk7trjAWC=<a href="mailto:OzC7xjMZKQyPO4zp3Fg@mail.gmail.com">OzC7xjMZKQyPO4zp3Fg@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="iso-8859-1"<br>
<br>
Nick wrote:<br>
<br>
<br>
<br>
> > Maybe I need to split my region into subregions<br>
> > and run r.fillnulls on each subregion then patch<br>
> > them together.<br>
><br>
<br>
Hamish:<br>
<br>
> yes, that should work well.<br>
<br>
<br>
I always wanted to create a script to do that. Since (I think) r.fill.nulls<br>
identifies the null locations and creates small buffers around them, I<br>
believe that if we combined something like the region handling of, say, the<br>
r.roughness script, with r.fill.nulls, that would really speed up matters.<br>
<br>
Don't you think?<br>
<br>
Cheers,<br>
Marcello.<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <a href="http://lists.osgeo.org/pipermail/grass-user/attachments/20111201/02e8ea3f/attachment.html" target="_blank">http://lists.osgeo.org/pipermail/grass-user/attachments/20111201/02e8ea3f/attachment.html</a><br>
<br>
------------------------------<br>
<br>
_______________________________________________<br>
grass-user mailing list<br>
<a href="mailto:grass-user@lists.osgeo.org">grass-user@lists.osgeo.org</a><br>
<a href="http://lists.osgeo.org/mailman/listinfo/grass-user" target="_blank">http://lists.osgeo.org/mailman/listinfo/grass-user</a><br>
<br>
<br>
End of grass-user Digest, Vol 68, Issue 2<br>
*****************************************<br>
</blockquote></div><br>