[MapProxy] Generate complete cache from one scale

dirk.thalheim at bkg.bund.de dirk.thalheim at bkg.bund.de
Tue Aug 25 23:27:03 PDT 2015


thanks for your replies. I'm using a WMS as source and finally got this working. But it was a bit tricky and awkward to implement, as I wanted to use sqlite as cache storage.

First I defined for each zoom level an individual cache which uses as source the WMS at the highest zoom and the previous cache at lower zooms. Each cache uses an individual grid which defines the resolution for the zoom level. The tricky part was to define some pseudo resolutions for each level grid. So the zoom level numbers for each level cache matches the global zoom level number (this seems to be stored in the sqlite-DB). Furthermore when the lower cache needs to request its source, the cached data of the higher level (higher resolution) has to be returned.

With that configuration the seeding is done level by level starting at the highest one. Finally via Symlinks I create a master cache from the individual zoom levels, which will be used in the service.

Here comes the example configuration:
  # == zoom level caches for seeding ==
    sources: [my_wms] 
    grids: [MY_GRID_L15]
      type: sqlite
      directory: /cache
    sources: [cache_l15] 
    grids: [MY_GRID_L14]
      type: sqlite
      directory: /cache 
    sources: [cache_l14] 
    grids: [MY_GRID_L13]
      type: sqlite
      directory: /cache
  # and so on...
  # == master cache for the service ==
    sources: [dummy] 
    grids: [MY_GRID]
      type: sqlite
      directory: /cache
    srs: 'EPSG:25832'
    tile_size: [512, 512]
    res: [4891.96981025128, 2445.98490512564,  1222.99245256282,  611.49622628141,  305.748113140705,  152.874056570353,  76.4370282851763,  38.2185141425881,  19.1092570712941,  9.55462853564703,  4.77731426782352,  2.38865713391176,  1.19432856695588,  0.597164283477939,  0.29858214173897,  0.2]
    bbox: [-5120900.0, 4988722.914, 1140821.357, 9998100.0]
    bbox_srs: 'EPSG:25832'
    origin: 'ul'
    base: DE_EPSG_25832_DOP
    # here the first levels are pseudo resolutions, which won't be needed for seeding
    res: [4891, 4890, 4889, 4888, 4887, 4886, 4885, 4884, 4883, 4882, 4881, 4880, 4879, 4878, 4877, 0.2]
    base: DE_EPSG_25832_DOP
    res: [4891, 4890, 4889, 4888, 4887, 4886, 4885, 4884, 4883, 4882, 4881, 4880, 4879, 4878, 0.29858214173897]
    base: DE_EPSG_25832_DOP
    res: [4891, 4890, 4889, 4888, 4887, 4886, 4885, 4884, 4883, 4882, 4881, 4880, 4879, 0.597164283477939]
  # and so on ..

And for the seeding:
    caches: [cache_l15]
    levels: [15]
    caches: [cache_l14]
    levels: [14]
    caches: [cache_l13]
    levels: [13]
  # and so on...

Finally the linking of the master cache is done via following shell script:
for i in `seq 0 15`; do
    ln -s /cache/MY_GRID_$i/$i.mbtile /cache/MY_GRID/$i.mbtile 

Kind regards,

Dirk Thalheim

-----Ursprüngliche Nachricht-----
Von: Märt Reose [mailto:Mart.Reose at smit.ee] 
Gesendet: Mittwoch, 26. August 2015 07:39
An: Thalheim, Dirk <dirk.thalheim at bkg.bund.de>
Cc: mapproxy at lists.osgeo.org
Betreff: RE: [MapProxy] Generate complete cache from one scale


I had a similar task at hand. Which format are your aerials? If its wms, then sorry I cant help you, but if you have access to geotiff files, then you have hope.

Mine were geotiff so I decided to use mapserver to deliver the tiffs and mapproxy to cache the result.

As there were many tiffs I used gdaltindex to create a tileindex. Like: gdaltindex aerials.shp aerials/*.tif

Then created a map file with a layer definition like this:
  NAME "aerials"
  TILEINDEX "/srv/gis/aerials/aerials.shp"
  TILEITEM "Location"

Longer documentation: http://mapserver.org/input/raster.html
And map file: http://www.mapserver.org/mapfile/

It can look like a difficult task if you dont have experience in the field, but actually its quite easy.

Märt Reose

-----Original Message-----
From: mapproxy-bounces at lists.osgeo.org [mailto:mapproxy-bounces at lists.osgeo.org] On Behalf Of Oliver Tonnhofer
Sent: Tuesday, August 25, 2015 2:48 PM
To: dirk.thalheim at bkg.bund.de
Cc: mapproxy at lists.osgeo.org
Subject: Re: [MapProxy] Generate complete cache from one scale


> On 21.08.2015, at 11:38, dirk.thalheim at bkg.bund.de wrote:
> I have a source which provides raster data (aerial images) only for a single scale. Is it possible to configure mapproxy in that way, that a complete cache could be generated from that source?
> In theory it would be enough to cache the source scale and after that recursively generate all lower scale caches by resizing the previous cache. My first approach would be to configure a recursive cache configuration. But this seems to me like a huge configuration and I’m not pretty sure how to combine the single caches in a single one for production. Or is there a simpler way?

No, there is no easy way to handle this right now. But it would be a great feature for MapProxy. It would also help for services that only provide maps for higher scale, but not for low scales (overview). 


Oliver Tonnhofer  | Omniscale GmbH & Co KG  | http://omniscale.com
OpenStreetMap WMS and tile services         | http://maps.omniscale.com

MapProxy mailing list
MapProxy at lists.osgeo.org

More information about the MapProxy mailing list