[GRASS-user] r.terraflow on massive DEM

[Markus Metz]

On Wed, Jan 14, 2015 at 11:48 AM, Daniel Victoria
<daniel.victoria at gmail.com> wrote:
> Hi Markus,
>
> I was using r.terraflow because I had converted my voids to negative values
> while the areas outside the country where set to null.
> But the data is proving to be too big and I'll just skip the gap-filling for
> now. Might do it later on, filling the gaps in the import process.
>
> The SRTM v3 you pointed out is very good but it's the 3-arcsec resolution.
> I'm working with the 1-arcsec res.

Then you could try the gap-filled 1 arc sec version of SRTM v3:

https://lpdaac.usgs.gov/products/measures_products_table/srtmgl1

Markus M

>
> Cheers and thanks
> Daniel
>
> On Tue, Jan 13, 2015 at 5:40 PM, Markus Metz <markus.metz.giswork at gmail.com>
> wrote:
>>
>> On Tue, Jan 13, 2015 at 7:52 PM, Daniel Victoria
>> <daniel.victoria at gmail.com> wrote:
>> >
>> > Now, the thing is, my main interest is to fill the voids in the SRTM
>> > data.
>> > So I though about using r.terraflow to get the filled elevation. Should
>> > I
>> > continue on that path or would it be better to use r.fill.null on a
>> > large
>> > dataset?
>>
>> r.terraflow does not fill voids, it fills sinks. Use SRTM v3 [0], this
>> is gap-filled. Using r.fill.null on SRTM v2 will not give you the same
>> quality. r.terraflow has a limit on the number of rows and columns
>> (max 32,767 each). r.watershed also has a limit on the number of rows
>> and columns, but that is max 2,147,483,647 each. In any case, you
>> should use GRASS 7 for such large datasets.
>>
>> Markus M
>>
>> [0] https://lpdaac.usgs.gov/products/measures_products_table/srtmgl3
>>
>> >
>> > Thanks
>> > Daniel
>> >
>> > On Tue, Jan 13, 2015 at 1:46 PM, Daniel Victoria
>> > <daniel.victoria at gmail.com>
>> > wrote:
>> >>
>> >> Charlie,
>> >>
>> >> I just downloaded some SRTM 1arc sec. from EarthExplorer. The data is
>> >> supplied in 3 different file types, GeoTIFF, DTED or BIL and they are
>> >> all in
>> >> Integer values (Int16). No floating point elevation values.
>> >>
>> >> Cheers
>> >> Daniel
>> >>
>> >> On Tue, Jan 13, 2015 at 12:12 PM, Charlie Shobe <chsh5846 at colorado.edu>
>> >> wrote:
>> >>>
>> >>> Hi Daniel,
>> >>>
>> >>> I believe that SRTM data does in fact provide floating point elevation
>> >>> values, so you may want to try working with the DEM as type FCELL from
>> >>> the
>> >>> very beginning. I don't know if this will help solve your problem, but
>> >>> the
>> >>> last time I brought in SRTM (1 arc second) data it was type FCELL and
>> >>> contained several decimal places of precision.
>> >>>
>> >>> Good luck,
>> >>>
>> >>> Charlie
>> >>>
>> >>>
>> >>> On Jan 13, 2015, at 4:00 AM, Daniel Victoria
>> >>> <daniel.victoria at gmail.com>
>> >>> wrote:
>> >>>
>> >>> Stephan, I'll give r.watershed a try and let it run for a couple of
>> >>> days.
>> >>> Thanks
>> >>>
>> >>> Thayer, I used r.recode because the person that sent me the data
>> >>> messed
>> >>> up the null values. So in order to fix that I did:
>> >>> 1) use r.external to bring the data to Grass
>> >>> 2) fix null values with r.recode, which was faster than r.null
>> >>>
>> >>> But I gave up on that path and since imported the data (r.in.gdal) and
>> >>> fixed the null values with r.null.
>> >>> I'm also using integer values (CELL type) because from what I heard,
>> >>> SRTM
>> >>> does not provide floating point data.
>> >>>
>> >>> Now I tried to run r.terraflow but I got a type error which asked me
>> >>> to
>> >>> use r.terraflow.short. Since I don't have that command in my grass
>> >>> instalatin, I converted the SRTM data to float. But that was to no
>> >>> avail
>> >>> since it's now giving me a dimensions type overflow error and asking
>> >>> me to
>> >>> change the dimension_type and recompile.
>> >>>
>> >>> I'll give r.watershed a try. If that does not work, I'll see if I can
>> >>> recompile.
>> >>>
>> >>> Thanks
>> >>> Daniel
>> >>>
>> >>> On Mon, Jan 12, 2015 at 9:22 PM, Thayer Young <thayeray at yahoo.com>
>> >>> wrote:
>> >>>>
>> >>>> With regards to the size of DEM, if you can get the command to run it
>> >>>> may take several days to finish, this is more than twice the size of
>> >>>> the
>> >>>> Washington State DEM that took the authors 33 hours in 2003 (Arge,
>> >>>> Chase,
>> >>>> Halpin, Toma, Urban, Vitter, Wickremesinghe  Efficient Flow
>> >>>> Computation on
>> >>>> Massive Grid Terrain Datasets Geoinformatica Volume 7 Issue 4,
>> >>>> December 2003
>> >>>> Pages 283 - 313 ).  My mid-2014 computer runs at about twice the
>> >>>> speed
>> >>>> quoted in the paper.
>> >>>>
>> >>>> Did you r.recode it to make it smaller?  In flat areas you need the
>> >>>> data
>> >>>> contained after the decimal point to allow the flow a chance to make
>> >>>> it down
>> >>>> hill, otherwise you can get streams going no where near where they do
>> >>>> in
>> >>>> real life.  I have never tried running r.terraflow on an integer
>> >>>> raster, I
>> >>>> know that it works on decimal rasters.
>> >>>>
>> >>>> -Thayer
>> >>>>
>> >>>>
>> >>>>
>> >>>> ________________________________
>> >>>>
>> >>>> Date: Mon, 12 Jan 2015 09:59:40 -0200
>> >>>> From: Daniel Victoria <daniel.victoria at gmail.com>
>> >>>> To: grass <grass-user at lists.osgeo.org>
>> >>>> Subject: [GRASS-user] r.terraflow on massive DEM
>> >>>> Message-ID:
>> >>>>
>> >>>> <CA+irsJjf7xhWgU968D+4+xc2YGJ=_n4xJtP0_WeU1ZpoRTt9ng at mail.gmail.com>
>> >>>> Content-Type: text/plain; charset="utf-8"
>> >>>>
>> >>>>
>> >>>> Hi list,
>> >>>>
>> >>>> I'm trying to run r.terraflow on a very large DEM but I wander if
>> >>>> it's
>> >>>> __too large__.
>> >>>>
>> >>>> The region dimensions are:
>> >>>> ncol=141114
>> >>>> nrow=140487
>> >>>> Data type = CELL
>> >>>>
>> >>>> The map is actually a reclass map of a raster that I imported using
>> >>>> r.external.
>> >>>>
>> >>>> When I run r.terraflow I get:
>> >>>>
>> >>>> r.terraflow elevation=srtm_brasil at PERMANENT filled=srtm_fill
>> >>>> direction=flowdir swatershed=sink accumulation=flowacc tci=srtm_tci
>> >>>> directory=E:\terraflow_temp
>> >>>>
>> >>>> WARNING: raster srtm_brasil is of type CELL_TYPE --you should use
>> >>>> r.terraflow.short
>> >>>> ERROR: [nrows=140487, ncols=141114] dimension_type overflow -- change
>> >>>> dimension_type and recompile
>> >>>> (Mon Jan 12 09:57:18 2015) Command finished (0
>> >>>> sec)
>> >>>>
>> >>>> However, I don't have an r.terraflow.short command
>> >>>> I'm running Grass 7.0.0beta4 from OsGeo4W
>> >>>>
>> >>>> Cheers
>> >>>> Daniel
>> >>>> -------------- next part --------------
>> >>>> An HTML attachment was scrubbed...
>> >>>> URL:
>> >>>>
>> >>>> <http://lists.osgeo.org/pipermail/grass-user/attachments/20150112/a27c139f/attachment-0001.html>
>> >>>>
>> >>>> ------------------------------
>> >>>>
>> >>>> _______________________________________________
>> >>>> grass-user mailing list
>> >>>> grass-user at lists.osgeo.org
>> >>>> http://lists.osgeo.org/mailman/listinfo/grass-user
>> >>>>
>> >>>> End of grass-user Digest, Vol 105, Issue 16
>> >>>> *******************************************
>> >>>>
>> >>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> grass-user mailing list
>> >>>> grass-user at lists.osgeo.org
>> >>>> http://lists.osgeo.org/mailman/listinfo/grass-user
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> grass-user mailing list
>> >>> grass-user at lists.osgeo.org
>> >>> http://lists.osgeo.org/mailman/listinfo/grass-user
>> >>
>> >>
>> >
>> >
>> > _______________________________________________
>> > grass-user mailing list
>> > grass-user at lists.osgeo.org
>> > http://lists.osgeo.org/mailman/listinfo/grass-user
>
>