[GRASS-user] r.terraflow on massive DEM

Thu Jan 15 02:56:14 PST 2015

THANKS!

I was downloading the SRTM data from earthexplorer and I choose the dataset
that appeared under Digita Elevation/SRTM/SRTM 1 Arc-Second Global.
The srtmgl1 dataset is "hidden" under NASA LPDAAC Collections / Nasa SRTM
(SRTM 3) Collections / NASA SRTM3 SRTMGL1.

Back to downloading the data...

Cheers and thanks once again
Daniel

On Thu, Jan 15, 2015 at 5:59 AM, Markus Metz <markus.metz.giswork at gmail.com>
wrote:

> On Wed, Jan 14, 2015 at 11:48 AM, Daniel Victoria
> <daniel.victoria at gmail.com> wrote:
> > Hi Markus,
> >
> > I was using r.terraflow because I had converted my voids to negative
> values
> > while the areas outside the country where set to null.
> > But the data is proving to be too big and I'll just skip the gap-filling
> for
> > now. Might do it later on, filling the gaps in the import process.
> >
> > The SRTM v3 you pointed out is very good but it's the 3-arcsec
> resolution.
> > I'm working with the 1-arcsec res.
>
> Then you could try the gap-filled 1 arc sec version of SRTM v3:
>
> https://lpdaac.usgs.gov/products/measures_products_table/srtmgl1
>
> Markus M
>
> >
> > Cheers and thanks
> > Daniel
> >
> > On Tue, Jan 13, 2015 at 5:40 PM, Markus Metz <
> markus.metz.giswork at gmail.com>
> > wrote:
> >>
> >> On Tue, Jan 13, 2015 at 7:52 PM, Daniel Victoria
> >> <daniel.victoria at gmail.com> wrote:
> >> >
> >> > Now, the thing is, my main interest is to fill the voids in the SRTM
> >> > data.
> >> > So I though about using r.terraflow to get the filled elevation.
> Should
> >> > I
> >> > continue on that path or would it be better to use r.fill.null on a
> >> > large
> >> > dataset?
> >>
> >> r.terraflow does not fill voids, it fills sinks. Use SRTM v3 [0], this
> >> is gap-filled. Using r.fill.null on SRTM v2 will not give you the same
> >> quality. r.terraflow has a limit on the number of rows and columns
> >> (max 32,767 each). r.watershed also has a limit on the number of rows
> >> and columns, but that is max 2,147,483,647 each. In any case, you
> >> should use GRASS 7 for such large datasets.
> >>
> >> Markus M
> >>
> >> [0] https://lpdaac.usgs.gov/products/measures_products_table/srtmgl3
> >>
> >> >
> >> > Thanks
> >> > Daniel
> >> >
> >> > On Tue, Jan 13, 2015 at 1:46 PM, Daniel Victoria
> >> > <daniel.victoria at gmail.com>
> >> > wrote:
> >> >>
> >> >> Charlie,
> >> >>
> >> >> I just downloaded some SRTM 1arc sec. from EarthExplorer. The data is
> >> >> supplied in 3 different file types, GeoTIFF, DTED or BIL and they are
> >> >> all in
> >> >> Integer values (Int16). No floating point elevation values.
> >> >>
> >> >> Cheers
> >> >> Daniel
> >> >>
> >> >> On Tue, Jan 13, 2015 at 12:12 PM, Charlie Shobe <
> chsh5846 at colorado.edu>
> >> >> wrote:
> >> >>>
> >> >>> Hi Daniel,
> >> >>>
> >> >>> I believe that SRTM data does in fact provide floating point
> elevation
> >> >>> values, so you may want to try working with the DEM as type FCELL
> from
> >> >>> the
> >> >>> very beginning. I don't know if this will help solve your problem,
> but
> >> >>> the
> >> >>> last time I brought in SRTM (1 arc second) data it was type FCELL
> and
> >> >>> contained several decimal places of precision.
> >> >>>
> >> >>> Good luck,
> >> >>>
> >> >>> Charlie
> >> >>>
> >> >>>
> >> >>> On Jan 13, 2015, at 4:00 AM, Daniel Victoria
> >> >>> <daniel.victoria at gmail.com>
> >> >>> wrote:
> >> >>>
> >> >>> Stephan, I'll give r.watershed a try and let it run for a couple of
> >> >>> days.
> >> >>> Thanks
> >> >>>
> >> >>> Thayer, I used r.recode because the person that sent me the data
> >> >>> messed
> >> >>> up the null values. So in order to fix that I did:
> >> >>> 1) use r.external to bring the data to Grass
> >> >>> 2) fix null values with r.recode, which was faster than r.null
> >> >>>
> >> >>> But I gave up on that path and since imported the data (r.in.gdal)
> and
> >> >>> fixed the null values with r.null.
> >> >>> I'm also using integer values (CELL type) because from what I heard,
> >> >>> SRTM
> >> >>> does not provide floating point data.
> >> >>>
> >> >>> Now I tried to run r.terraflow but I got a type error which asked me
> >> >>> to
> >> >>> use r.terraflow.short. Since I don't have that command in my grass
> >> >>> instalatin, I converted the SRTM data to float. But that was to no
> >> >>> avail
> >> >>> since it's now giving me a dimensions type overflow error and asking
> >> >>> me to
> >> >>> change the dimension_type and recompile.
> >> >>>
> >> >>> I'll give r.watershed a try. If that does not work, I'll see if I
> can
> >> >>> recompile.
> >> >>>
> >> >>> Thanks
> >> >>> Daniel
> >> >>>
> >> >>> On Mon, Jan 12, 2015 at 9:22 PM, Thayer Young <thayeray at yahoo.com>
> >> >>> wrote:
> >> >>>>
> >> >>>> With regards to the size of DEM, if you can get the command to run
> it
> >> >>>> may take several days to finish, this is more than twice the size
> of
> >> >>>> the
> >> >>>> Washington State DEM that took the authors 33 hours in 2003 (Arge,
> >> >>>> Chase,
> >> >>>> Halpin, Toma, Urban, Vitter, Wickremesinghe  Efficient Flow
> >> >>>> Computation on
> >> >>>> Massive Grid Terrain Datasets Geoinformatica Volume 7 Issue 4,
> >> >>>> December 2003
> >> >>>> Pages 283 - 313 ).  My mid-2014 computer runs at about twice the
> >> >>>> speed
> >> >>>> quoted in the paper.
> >> >>>>
> >> >>>> Did you r.recode it to make it smaller?  In flat areas you need the
> >> >>>> data
> >> >>>> contained after the decimal point to allow the flow a chance to
> make
> >> >>>> it down
> >> >>>> hill, otherwise you can get streams going no where near where they
> do
> >> >>>> in
> >> >>>> real life.  I have never tried running r.terraflow on an integer
> >> >>>> raster, I
> >> >>>> know that it works on decimal rasters.
> >> >>>>
> >> >>>> -Thayer
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>> ________________________________
> >> >>>>
> >> >>>> Date: Mon, 12 Jan 2015 09:59:40 -0200
> >> >>>> From: Daniel Victoria <daniel.victoria at gmail.com>
> >> >>>> To: grass <grass-user at lists.osgeo.org>
> >> >>>> Subject: [GRASS-user] r.terraflow on massive DEM
> >> >>>> Message-ID:
> >> >>>>
> >> >>>> <CA+irsJjf7xhWgU968D+4+xc2YGJ=_
> n4xJtP0_WeU1ZpoRTt9ng at mail.gmail.com>
> >> >>>> Content-Type: text/plain; charset="utf-8"
> >> >>>>
> >> >>>>
> >> >>>> Hi list,
> >> >>>>
> >> >>>> I'm trying to run r.terraflow on a very large DEM but I wander if
> >> >>>> it's
> >> >>>> __too large__.
> >> >>>>
> >> >>>> The region dimensions are:
> >> >>>> ncol=141114
> >> >>>> nrow=140487
> >> >>>> Data type = CELL
> >> >>>>
> >> >>>> The map is actually a reclass map of a raster that I imported using
> >> >>>> r.external.
> >> >>>>
> >> >>>> When I run r.terraflow I get:
> >> >>>>
> >> >>>> r.terraflow elevation=srtm_brasil at PERMANENT filled=srtm_fill
> >> >>>> direction=flowdir swatershed=sink accumulation=flowacc tci=srtm_tci
> >> >>>> directory=E:\terraflow_temp
> >> >>>>
> >> >>>> WARNING: raster srtm_brasil is of type CELL_TYPE --you should use
> >> >>>> r.terraflow.short
> >> >>>> ERROR: [nrows=140487, ncols=141114] dimension_type overflow --
> change
> >> >>>> dimension_type and recompile
> >> >>>> (Mon Jan 12 09:57:18 2015) Command finished (0
> >> >>>> sec)
> >> >>>>
> >> >>>> However, I don't have an r.terraflow.short command
> >> >>>> I'm running Grass 7.0.0beta4 from OsGeo4W
> >> >>>>
> >> >>>> Cheers
> >> >>>> Daniel
> >> >>>> -------------- next part --------------
> >> >>>> An HTML attachment was scrubbed...
> >> >>>> URL:
> >> >>>>
> >> >>>> <
> http://lists.osgeo.org/pipermail/grass-user/attachments/20150112/a27c139f/attachment-0001.html
> >
> >> >>>>
> >> >>>> ------------------------------
> >> >>>>
> >> >>>> _______________________________________________
> >> >>>> grass-user mailing list
> >> >>>> grass-user at lists.osgeo.org
> >> >>>> http://lists.osgeo.org/mailman/listinfo/grass-user
> >> >>>>
> >> >>>> End of grass-user Digest, Vol 105, Issue 16
> >> >>>> *******************************************
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>> _______________________________________________
> >> >>>> grass-user mailing list
> >> >>>> grass-user at lists.osgeo.org
> >> >>>> http://lists.osgeo.org/mailman/listinfo/grass-user
> >> >>>
> >> >>>
> >> >>> _______________________________________________
> >> >>> grass-user mailing list
> >> >>> grass-user at lists.osgeo.org
> >> >>> http://lists.osgeo.org/mailman/listinfo/grass-user
> >> >>
> >> >>
> >> >
> >> >
> >> > _______________________________________________
> >> > grass-user mailing list
> >> > grass-user at lists.osgeo.org
> >> > http://lists.osgeo.org/mailman/listinfo/grass-user
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20150115/9b7b7a81/attachment-0001.html>