[GRASS-dev] MBTiles format export tool (halfway)

[Hamish]

Hi,

If it is useful to anyone I've just written an addon module which will
let you export a grass raster map as a TMS tileset and uDig mapurl
file ready to be copied into Geopaparazzi, which will then convert it
to the MBTiles sqlite web tile database format. MBTiles is used for
Tilemill and other web tile softwares to avoid 2^n number of tiny tiles
clogging up your filesystem, handle deduplication, and other nice
things. GDAL support (read-only) since 1.10.

Relying on Geopaparazzi on Android to do the final conversion then
copying back the results is a bit awkward/slow, so I'd enjoy to see the
addon module create the MBTiles sqlite db directly. It looks pretty
straight forward, but sadly my SQL foo is lacking.

( https://github.com/mapbox/mbtiles-spec/blob/master/1.2/spec.md )

once that is working the idea is to buy the co-author a beer and rename
the module to r.out.mbtiles. The tile tree output would be preserved as
an optional output since you can copy it to an apache dir and use with
OpenLayers or do other post-processing directly.


notes, screentshot, and code:

http://grasswiki.osgeo.org/wiki/Android#Device_used_as_a_recording_slate_in_the_field
http://grasswiki.osgeo.org/wiki/AddOns/GRASS_6#r.out.mbtiles_prep
http://trac.osgeo.org/grass/browser/grass-addons/grass6/raster/r.out.mbtiles_prep


After some initial tests right now I am chugging through a 50000x50000
GDAL .vrt virtual mosaic of a directory of geo-ref'd JPEG orthophotos,
connected into a virtual raster layer with the r.external module.
180k+ tiles take a long time!, but I think it will (just) make it under
the spatialite 2GB filesize limit on Android by the time it's done.

(yay gdalbuildvrt + r.external + r.reclass)

I'm not sure if I got all the uDig mapurl request_type= settings
correct, I went for a throw everything at it and see what sticks
approach.

Other room-for-improvement areas are r,g,b band parallelizing in
r.out.gdal (even if only 3x speedup it would be a good win and have
very low throughput overheads), and also the parallelization patch for
gdal2tiles currently in gdal trac. The tile tree scanning to remove
unused metadata .aux.xml & converting png -> jpeg would probably be
much faster in a python script, since unlink() and PIL could be used
instead of all the expense of creating and destroying a zillion
rm and NetPBM processes for all the tiles.. many CPUs don't help if
you've already saturated the kernel %sys.


It's one day old and lightly tested; feedback is welcome.


regards,

-- 
Hamish <hamish.webmail at gmail.com>