[gdal-dev] [GRASS-user] Slow import of GHSL

[Markus Neteler]

On Sat, Mar 11, 2017 at 7:01 PM, Markus Metz
<markus.metz.giswork at gmail.com> wrote:
> On Sat, Mar 11, 2017 at 8:53 AM, Nikos Alexandris <nik at nikosalexandris.net>
> wrote:
>>
>> Nikos Alexandris
>>
>>>> Why does (attempting to) import a 38m pixel resolution GHSL [0] GeoTIFF
>>>> layer, ie GHS_BUILT_LDS1990_GLOBE_R2016A_3857_38_v1_0_p1.tif, in GRASS'
>>>> db progress slow?
>
> because it is a very large raster map: Size is 507904, 647168

Nikos, for an even bigger map try

Global Surface Water (2000-2012, 30 m, Data coverage is from 80° north
to 60° south): http://landcover.usgs.gov/glc/WaterDescriptionAndDownloads.php
by USGS. 1.6GB in size.

Using gdalbuildvrt I created a VRT from the 504 GeoTIFF files.

After import into GRASS GIS, here the timings:

# final map size:
g.region -p
...
rows:       493200
cols:       1296001
cells:      639187693200

(handling only works in GRASS GIS 7.3.svn since Markus Metz's recent
improvements on global data import are needed).

Benchmarks:
- Import took 2h while reading the data from a CIFS mounted storage
box (slow) and writing on SSD.
- Displaying the entire map (639 giga-pixel) in GRASS GIS' display
(d.mon) took ~15 sec over a ssh tunnel from my laptop to the server,
since I am at a conference.

Fair deal I would say :-)

cheers,
Markus

-- 
https://www.mundialis.de/