[GRASS-user] Very high resolution topographic map of Europe: need help and advices

[Markus Neteler]

Hi Felix,

On Mon, Aug 10, 2009 at 8:28 PM, Felix Schalck<felix.schalck at gmail.com> wrote:
> Hello,
>
> For some time now, I'm following sort of a personal objective to
> create the most precise (=high resolution) topographic map of Europe
> allowed by my comp (Xubuntu 9.04, AMD64, 3Gb RAM, 300+ HD space).
> Starting from the very scratch, I had to learn about DEMs and
> GIS/maptools - and I'm still not confortable with all the technicy
> behind. Fact is that the best data set available to serve my purpose
> seems to be the cgiar interpolated srtm3 geodata (no license problem
> here, since it is aimed for pure personal use) which has to somehow be
> translated into a big jpeg or png file.

Here you can use gdalwarp to merge all files into a big one. Enjoy
wildcards to do that in one line:

gdalwarp srtm_*.tif cgiar_srtm3_LL.tif

A big file is created. Then don't miss to add image pyramids:

gdaladdo srtm3_LL.tif 2 4 8 16 32 64

and you can open a file of several GB in no time with QGIS for
example.

I posted some GDAL tricks here:
http://gfoss.blogspot.com/2008/06/gdal-raster-data-tips-and-tricks.html

(if you want OGR for vectors, check:
http://gfoss.blogspot.com/2008/06/ogr-vector-data-tips-and-tricks.html
)

...
> The big problem happened to be the river
> data, since GSHHS provides only limited amount and precision of
> side-rivers, which resulted in chains of straight lines scattered all
> over a giant map, even after vectorization: it was a no-go.

No idea, I didn't try GMT so much.

> And then, a few days ago, I discovered nasa SWBD data and WMAP0, which
> seem to be of much higher resolution, linked to a nice GRASS GIS
> tutorial on the french wikipedia. I immedialely dug into this new
> software, quite complicated I must admit, but very powerful.

We hope you took the GRASS 6.4 version - even if still called "release
candidate" it is used by many in production. Myself, I am doing heavy
computations with it and it is so far the only convincing software
for GIS number crunching :-)

> I figured out how to import GeoTiff data into GRASS Raster files,

Hint: from GRASS 6.4 onwards you can use r.external to avoid
import but to just link an external file into GRASS. Nice when
registering 30GB of orthophotos in a few minutes...

For newcomers QGIS is a nice interface to GRASS, too, since
it comes with a GRASS toolbox.

> and how to
> display/export each one of them, but soon had to face new problems,
> especially when tried to reproject the data into LCC projection. So I
> decided to ask for help on this mailing list.
>
> My Current plan is to:
> 1. reproject the cgiar raster tiles OR one big merged raster into LCC
> projection (native srtm data seems to be a strange Mercator)

They are in LatLong but they (those I downloaded) don't/didn't have
the projection info in the metadata any more. Sad story. Maybe they
fixed it later...

Note that recent GDAL now provides the gdalhillshade tool for
easy hillshading of GeoTIFF/other format files.

Here my script (I did the same):

---------- snip ----------
#!/bin/sh
# mosaic European CGIAR SRTM to 250m LAEA DEM map, bilinear

gdalwarp -s_srs epsg:4326 -t_srs epsg:3035 -tr 250 250 -r bilinear \
                srtm_*.tif europe_all_srtmV4_cgiar_250m_LAEA_EU.tif

gdaladdo europe_all_srtmV4_cgiar_250m_LAEA_EU.tif 2 4 8 16 32 64

# generate shaded relief
gdalhillshade europe_all_srtmV4_cgiar_250m_LAEA_EU.tif \
                      europe_all_srtmV4_cgiar_250m_LAEA_EU_shaded.tif
gdaladdo europe_all_srtmV4_cgiar_250m_LAEA_EU_shaded.tif 2 4 8 16 32 64

---------- snap ----------

> 3. export a nice shaded topographic png

I would still use GDAL here and gdal_translate to a PNG (do
you really need this?)

> 4. extract rivers/coast vectors from SWBD files

At this point use r.external on the huge SRTM DEM GeoTIFF.
Then use r.terraflow/r.mapcalc/... (several options).

> 5. workout in Inkscape

Oh, at this point you'll like r.out.png, you can set previously
the resolution to something less (g.region) unless you don't
plan to print a huge poster.

> 6. paste the whole thing together in GIMP.

Why not ...

> The main problem right now seems to be the "tiling" of the topographic
> data. Each cgiar-cell (5°x5°) can be shown into a separate layer, but
> I'm unable to work them together.

See above: gdalwarp is your friend.

> And it looks like most of the tools
> provided by GRASS assume the raster map is the size of the location
> (r.proj for an example).

No, that's no the case. It's flexible.

> I tried r.patch but it produce wired results
> on top of giant files (I stopped when I hit the 2Gb limit),

Ah, so you need to compile with Large File Support (LFS):
./configure ... --enable-largefile

Then you can have huge files > 2GB (if it fails somewhere please
tell us but most common commands should be fine)

...
> I took the time to explain the whole thing with the hope to not only
> get help about my immediate raster division problem, but also about
> the "big-picture", eg: is GRASS the right tools to do this

My take is: GRASS together with GDAL tools are just perfect.

> (I installed GRASS because of the SWBD data tutorial, but it seems to me
> that swbd could also be plotted via psxy in GMT )? How would you
> GIS-gurus proceed to create a high resolution topographic map ?

Please take a look at ps.map in GRASS.

> Is one big png a good idea, or would it be smarter to divide the continent
> into 4 or more parts, and render each one into a separate png ?

It will depends on the memory management of the programs you use.
In GRASS, some users are using multi-GB files, so that works. Likewise
for GDAL. I don't know for GMT nor for Inkscape/Gimp.

Cheers
Markus

-- 
Markus Neteler
Foundation Edmund Mach (FEM) - Research and Innovation Centre
Environment and Natural Resources Area
Head of GIS and Remote Sensing Unit, Trento, Italy
Web:  http://gis.fem-environment.eu/