[gdal-dev] GSoC GDAL2Tiles: MapTiler.org, Google Maps Projection Web, Source Code

Tue Aug 19 05:06:41 EDT 2008

Hi,

> I'm talking about 10 to 100 GigaBytes datasets, with which it doesn't
> seem reasonnable to generate a single merged file !

In case you use -of VRT it should not be problem. VRT is just an XML
file telling where is each file placed and what processing is
necessary - so in fact there is not big logical difference between
gdal_vrtmerge's VRT and gdaltindex result (one is using XML another
shape file for storing the same kind of link to raster data).

The stable version of GDAL2Tiles is able to handle such huge raster
files without problems.

The new version (in my sandbox) is not tested properly. As I've
written before >it is not yet ready for production<. Before I merge it
with gdal-trunk I will do some test on a raster dataset with size over
30 GB as well...

In case you anyway have WMS server for raster data, you can use
TileCache.org or GeoWebCache.org to cache the tiles.
Advantage of GDAL2Tiles over TileCache is practically zero
configuration of any server software. The result from the user point
of view is almost equal.

BTW It would be also possible to write an extension of GDAL2Tiles for
saving of generated tiles directly into a database (SQL / Amazon S3 /
Google BigTable - App Engine) instead of having small PNG files on a
disk. TileCache and GDAL2Tiles are close to each other...

Klokan

On Tue, Aug 19, 2008 at 10:31 AM, Guillaume Sueur
<guillaume.sueur at neogeo-online.net> wrote:
> Klokan Petr Přidal a écrit :
>> Hi Guillaume,
>>
>> Thank you for positive feedback ;-)
>>
>>> Just a question after a quick look at the doc : can the input file be a
>>> raster catalog (the gdaltindex thing) ? Raster data needing to be tiled
>>> fur web publication are sometimes already tiled in several pieces
>>> because of their size.
>>
>> I don't think you can use gdaltindex with other gdal tools (like
>> gdal_transform) either.
>> gdaltindex is used by UNM MapServer but not by other GDAL tools, as I know.
>>
>> You would need to use VRT for combining several rasters into one...
>> Use gdal_vrtmerge.py, it has similar syntax like gdaltindex. It is in
>> the source under /swig/python/samples/.
>> Result can be passed then to gdal2tiles and it should run well I hope.
>>
>> Is there any plan to merge gdal_vrtmerge into "gdal_merge -of VRT"?
>>
>> Best regards
>>
>> Klokan
>>
>>> Best regards,
>>>
>>> Guillaume
>>>
>>> Klokan Petr Přidal a écrit :
>>>> Hello everybody,
>>>>
>>>> what was done recently on GDAL2Tiles project:
>>>>
>>>> New cool example of the map generated by GDAL2Tiles:
>>>> ----------------------------------------------------------------------------------
>>>> I processed freely available USGS DRG map of Grand Canyon by
>>>> GDAL2Tiles. It is a nice example of what GDAL2Tiles can generate:
>>>>
>>>> http://www.maptiler.org/example-usgs-drg-grand-canyon-gtiff/
>>>> http://www.maptiler.org/example-usgs-drg-grand-canyon-gtiff/usgs-drg-grand-canyon.kml
>>>>
>>>> Source Code:
>>>> --------------------
>>>> - Fully functional source code for generating Google Maps tiles is in
>>>> my GDAL SVN sandbox (http://svn.osgeo.org/gdal/sandbox/klokan/).
>>>> The code was refactored a lot and it is almost prepared for merging to
>>>> stable svn trunk gdal2tiles.py, but some coding is still needed. I am
>>>> going to work on it during the next week.
>>>>
>>>> - I split the code into three files: 'gdal2tiles.py' (the main command
>>>> line utility), then 'globalmaptiles.py' (classes for global
>>>> mercator/geodetic tile bounds calculation, runs as command line
>>>> utility too), and supporting module 'templates.py' with sample HTML
>>>> viewers and base for generated XML files.
>>>>
>>>> - Implementation of several resampling algorithms for controlling the
>>>> quality of tile rendering. Default is 'average' used in the GDAL's
>>>> overview functionality. Complete list is:
>>>> 'average','near','bilinear','cubic','cubicspline','lanczos','antialias'.
>>>> The last one is high quality Lanczos based antialiasing algorithm, but
>>>> you will need Python Image Library for that.
>>>> Quality of the tile rendering is excellent I think. If you need it,
>>>> the simple and fast 'near' it is still available.
>>>>
>>>> - GDAL2Tiles generates the sample Google Maps Based viewer, OpenLayers
>>>> based Overlay viewer and KML files for Google Earth.
>>>>
>>>> - The utility automatically warp the raster to Google Maps projection
>>>> (EPSG:900913) by AutoCreateWarpedVRT().
>>>>
>>>> - For raster files with palette (USGS DRG, NOAA BSB, ...) a separate
>>>> VRT with "rasterize rgba" (GDAL 1.6dev) is recommended (later I would
>>>> like to automate this step).
>>>>
>>>> The code is NOT YET READY for production use (now supports only RGBA -
>>>> 4 bands - source files for example), so I recommend you to wait for
>>>> merged version which will appear in the GDAL trunk. I will send a note
>>>> when I will finish the merging.
>>>>
>>>> Documetation & MapTiler.org
>>>> ------------------------------------------
>>>>
>>>> - I decided to create a special website for distributing binary
>>>> installer and user-friendly GUI for GDAL2Tiles. I plan to develop
>>>> wxWidget based GUI a la step by step wizard. The user-friendly project
>>>> has name MapTiler, http://www.MapTiler.org/. Right now this web is
>>>> used only for documentation on for raster files for templates, which I
>>>> did not want to distribute with utility.
>>>>
>>>> I received great feedback from several users and realized that
>>>> GDAL2Tiles utility is expected by community. That pushed me to crete
>>>> this new project and change the name to MapTiler, which is more
>>>> straightforward and fits to the utility probably better, especially
>>>> for users who are not familiar with GDAL tools yet.
>>>> GDAL2Tiles will be maintained as GDAL command line utility. MapTiler
>>>> will implement just GUI and binary distribution.
>>>>
>>>> It would be great to have blessing of OSGEO for this project, I would
>>>> prefer to not make separate SVN repository for such small project,
>>>> better to keep it as part of GDAL. Would it be possible? Otherwise I
>>>> will create Google Code project.
>>>>
>>>> - I studied the OpenGIS proposal for WMTS standard, which is related
>>>> to TMS and I plan to send some comments to that standard. Is there
>>>> still free OSGEO slot for OpenGIS membership?
>>>>
>>>> - I created a website about Spherical Mercator and Google Maps
>>>> Projection and Coordinate system, for anybody who is interested to see
>>>> the math behind tiles:
>>>> http://www.maptiler.org/google-maps-coordinate-system-projection-epsg-900913-3785/
>>>>
>>>> Question to community:
>>>> ----------------------------------
>>>>
>>>> - Would it be possible to keep more files separately even in the main
>>>> GDAL tree or the utility really have to be merged to one file?
>>>>
>>>> - Is there anybody in the OpenLayers community who whould like to have
>>>> a look at the Opacity Control for OL? I prepared the bitmaps. Is there
>>>> any need on the OpenLayers templates? Now I am using /dev/ tree of
>>>> OpenLayers as Virtual Earth support is broken in the stable tree
>>>> (Safari for example).
>>>>
>>>> - Does anybody know how to correctly set the KML SuperOverlay
>>>> maxlodpixels and fadeextents so transparency and opacity runs well and
>>>> tiles are not disappearing in Google Earth?
>>>>
>>>> - Is anybody interested to help me with developing MapTiler (for
>>>> example compilation of latest GDAL with all modules, preparing of
>>>> Windows installer, help with wxPython...)
>>>> KyngChaos (thank you!) already had a look at binaries for MacOSX, but
>>>> we did not moved far yet as I need some functionality from 1.6dev.
>>>>
>>>> Thanks belong to everybody who gave me excellent advices on #gdal IRC,
>>>> especially to 'FrankW' and 'even'.
>>>>
>>>> BTW I am looking for an interesting job or paid projects... any offers? ;-)
>>>>
>>>> Is there anybody who would like to support further development of
>>>> MapTiler/GDAL2Tiles? There is plenty of features on todo list (merging
>>>> of produced tiles with existing tile tree, support for MapCruncher
>>>> tiles, Zoomify tiles, speed improvement - rewrite to C++ to allow
>>>> better usage of threads on multi-core processors, driver for remote
>>>> online TMS tiles for GDAL/MapServer to allow WMS from statically
>>>> published tiles, etc..).
>>>>
>>>> Best regards...
>>>>
>>>> Klokan Petr Pridal
>>>> _______________________________________________
>>>> gdal-dev mailing list
>>>> gdal-dev at lists.osgeo.org
>>>> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>>>>
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> gdal-dev mailing list
>>> gdal-dev at lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
>
>