[GRASS-user] Very high resolution topographic map of Europe: need help and advices

Wed Aug 12 10:06:12 EDT 2009

Dear Markus,

Thank you soooo much for your response; it provided me with valuables
hints - but also with new questions.

First, following your avice, I finally decided to compile the new
GRASS6.4rc5. Having an amd64 cpu, there war no binary available; so I
had to do the job myself. Let's face it: manual configuration is a
pain for a newcommer like me, but once the ./configure script shows
the long awaited final recap, the make command - although quite long -
runs withouth a hitch. I reminds me of good ol' freebsd days, although
we had a lot more troubles finishing compilation without errors: great
job guys ! But the resulat was worth it; I don't know if it is the
work you've done for the last few years, or the gcc optiomization
flags (or perhaps both of it), but the resulting grass64 runs like a
fireball compared to my old 6.2 bin provided by the Ubuntu repos.
Again: great job ! Only now I fully understand what you meant by "it
is so far the only convincing software for GIS number crunching". But
let's go back to the topic: my high-resolution topographic map of
Europe.

Thanks to your advices, the production outline has changed to this:

1. Merge the cgiar TIFs (AND NOT THE GRASS RASTERS IF I GOT THIS
CORRECTLY) thanks to gdalwrap command. In what projection does this
command work ? Is it possible to wrap the TIF directlly in my lcc
projection ?

2. Add image pyramids with gdaladdo (< I frankly do not underdand this
one at all; what does it mean ?)

3. Shaded relief: I don't know your gdalhillshade command at all. Does
it produce the same results as the r.shaded.relief I was planning to
use ? Can you set the illumination angle with gdalhilshade ?

4. Re-gdaladdo for the shaded tif.

5. Import in GRASS and checkout results. If I'm right, I will have two
layers at this point, one with the relief colors, one with the
shadings.

6. Coastlines and Rivers. I was planning to use SWBD(coastlines and
main rivers) and VLMAP0 Data (for the secondary rivers). Here again,
you provide me with a complete new set of tools:
r.external/r.terraflow/r.mapcalc/. What is the general idea behind ?I
checked the man pages, but I don't really understand how to use them
for my purpose. My plan was to import the shapefile into the right
projection with rvin.ogr, and than export svg files for rework BEFORE
joining the river layer and the topographic layers; but perhaps your
way, once I understand it, is more efficient.

7. Export the shaded topography with r.out.png in one big png. Do I
need two files (one containing the shadings, one with the relief
colors) ?

8. Merge topography and hydrography layers in GIMP.

Could you (or anyone interested) please have a look a this ?

Again: thanks you very much for your time,

regards,
Felix

>> For some time now, I'm following sort of a personal objective to
>> create the most precise (=high resolution) topographic map of Europe
>> allowed by my comp (Xubuntu 9.04, AMD64, 3Gb RAM, 300+ HD space).
>> Starting from the very scratch, I had to learn about DEMs and
>> GIS/maptools - and I'm still not confortable with all the technicy
>> behind. Fact is that the best data set available to serve my purpose
>> seems to be the cgiar interpolated srtm3 geodata (no license problem
>> here, since it is aimed for pure personal use) which has to somehow be
>> translated into a big jpeg or png file.
>
> Here you can use gdalwarp to merge all files into a big one. Enjoy
> wildcards to do that in one line:
>
> gdalwarp srtm_*.tif cgiar_srtm3_LL.tif
>
> A big file is created. Then don't miss to add image pyramids:
>
> gdaladdo srtm3_LL.tif 2 4 8 16 32 64
>
> and you can open a file of several GB in no time with QGIS for
> example.
>
> I posted some GDAL tricks here:
> http://gfoss.blogspot.com/2008/06/gdal-raster-data-tips-and-tricks.html
>
> (if you want OGR for vectors, check:
> http://gfoss.blogspot.com/2008/06/ogr-vector-data-tips-and-tricks.html
> )
>
> ...
>> The big problem happened to be the river
>> data, since GSHHS provides only limited amount and precision of
>> side-rivers, which resulted in chains of straight lines scattered all
>> over a giant map, even after vectorization: it was a no-go.
>
> No idea, I didn't try GMT so much.
>
>> And then, a few days ago, I discovered nasa SWBD data and WMAP0, which
>> seem to be of much higher resolution, linked to a nice GRASS GIS
>> tutorial on the french wikipedia. I immedialely dug into this new
>> software, quite complicated I must admit, but very powerful.
>
> We hope you took the GRASS 6.4 version - even if still called "release
> candidate" it is used by many in production. Myself, I am doing heavy
> computations with it and it is so far the only convincing software
> for GIS number crunching :-)
>
>> I figured out how to import GeoTiff data into GRASS Raster files,
>
> Hint: from GRASS 6.4 onwards you can use r.external to avoid
> import but to just link an external file into GRASS. Nice when
> registering 30GB of orthophotos in a few minutes...
>
> For newcomers QGIS is a nice interface to GRASS, too, since
> it comes with a GRASS toolbox.
>
>> and how to
>> display/export each one of them, but soon had to face new problems,
>> especially when tried to reproject the data into LCC projection. So I
>> decided to ask for help on this mailing list.
>>
>> My Current plan is to:
>> 1. reproject the cgiar raster tiles OR one big merged raster into LCC
>> projection (native srtm data seems to be a strange Mercator)
>
> They are in LatLong but they (those I downloaded) don't/didn't have
> the projection info in the metadata any more. Sad story. Maybe they
> fixed it later...
>
> Note that recent GDAL now provides the gdalhillshade tool for
> easy hillshading of GeoTIFF/other format files.
>
> Here my script (I did the same):
>
> ---------- snip ----------
> #!/bin/sh
> # mosaic European CGIAR SRTM to 250m LAEA DEM map, bilinear
>
> gdalwarp -s_srs epsg:4326 -t_srs epsg:3035 -tr 250 250 -r bilinear \
>                srtm_*.tif europe_all_srtmV4_cgiar_250m_LAEA_EU.tif
>
> gdaladdo europe_all_srtmV4_cgiar_250m_LAEA_EU.tif 2 4 8 16 32 64
>
> # generate shaded relief
> gdalhillshade europe_all_srtmV4_cgiar_250m_LAEA_EU.tif \
>                      europe_all_srtmV4_cgiar_250m_LAEA_EU_shaded.tif
> gdaladdo europe_all_srtmV4_cgiar_250m_LAEA_EU_shaded.tif 2 4 8 16 32 64
>
> ---------- snap ----------
>
>> 3. export a nice shaded topographic png
>
> I would still use GDAL here and gdal_translate to a PNG (do
> you really need this?)
>
>> 4. extract rivers/coast vectors from SWBD files
>
> At this point use r.external on the huge SRTM DEM GeoTIFF.
> Then use r.terraflow/r.mapcalc/... (several options).
>
>> 5. workout in Inkscape
>
> Oh, at this point you'll like r.out.png, you can set previously
> the resolution to something less (g.region) unless you don't
> plan to print a huge poster.
>
>> 6. paste the whole thing together in GIMP.
>
> Why not ...
>
>> The main problem right now seems to be the "tiling" of the topographic
>> data. Each cgiar-cell (5°x5°) can be shown into a separate layer, but
>> I'm unable to work them together.
>
> See above: gdalwarp is your friend.
>
>> And it looks like most of the tools
>> provided by GRASS assume the raster map is the size of the location
>> (r.proj for an example).
>
> No, that's no the case. It's flexible.
>
>> I tried r.patch but it produce wired results
>> on top of giant files (I stopped when I hit the 2Gb limit),
>
> Ah, so you need to compile with Large File Support (LFS):
> ./configure ... --enable-largefile
>
> Then you can have huge files > 2GB (if it fails somewhere please
> tell us but most common commands should be fine)
>
> ...
>> I took the time to explain the whole thing with the hope to not only
>> get help about my immediate raster division problem, but also about
>> the "big-picture", eg: is GRASS the right tools to do this
>
> My take is: GRASS together with GDAL tools are just perfect.
>
>> (I installed GRASS because of the SWBD data tutorial, but it seems to me
>> that swbd could also be plotted via psxy in GMT )? How would you
>> GIS-gurus proceed to create a high resolution topographic map ?
>
> Please take a look at ps.map in GRASS.
>
>> Is one big png a good idea, or would it be smarter to divide the continent
>> into 4 or more parts, and render each one into a separate png ?
>
> It will depends on the memory management of the programs you use.
> In GRASS, some users are using multi-GB files, so that works. Likewise
> for GDAL. I don't know for GMT nor for Inkscape/Gimp.
>
> Cheers
> Markus
>
> --
> Markus Neteler
> Foundation Edmund Mach (FEM) - Research and Innovation Centre
> Environment and Natural Resources Area
> Head of GIS and Remote Sensing Unit, Trento, Italy
> Web:  http://gis.fem-environment.eu/
>