[MapProxy] Generate complete cache from one scale

Tue Aug 25 23:27:03 PDT 2015

Hello,

thanks for your replies. I'm using a WMS as source and finally got this working. But it was a bit tricky and awkward to implement, as I wanted to use sqlite as cache storage.

First I defined for each zoom level an individual cache which uses as source the WMS at the highest zoom and the previous cache at lower zooms. Each cache uses an individual grid which defines the resolution for the zoom level. The tricky part was to define some pseudo resolutions for each level grid. So the zoom level numbers for each level cache matches the global zoom level number (this seems to be stored in the sqlite-DB). Furthermore when the lower cache needs to request its source, the cached data of the higher level (higher resolution) has to be returned.

With that configuration the seeding is done level by level starting at the highest one. Finally via Symlinks I create a master cache from the individual zoom levels, which will be used in the service.

Here comes the example configuration:
caches:
  # == zoom level caches for seeding ==
  cache_l15:
    sources: [my_wms] 
    grids: [MY_GRID_L15]
    cache:
      type: sqlite
      directory: /cache
  cache_l14:
    sources: [cache_l15] 
    grids: [MY_GRID_L14]
    cache:
      type: sqlite
      directory: /cache 
  cache_l13:
    sources: [cache_l14] 
    grids: [MY_GRID_L13]
    cache:
      type: sqlite
      directory: /cache
  # and so on...
  # == master cache for the service ==
  cache_master:
    sources: [dummy] 
    grids: [MY_GRID]
    cache:
      type: sqlite
      directory: /cache
grids:
  MY_GRID:
    srs: 'EPSG:25832'
    tile_size: [512, 512]
    res: [4891.96981025128, 2445.98490512564,  1222.99245256282,  611.49622628141,  305.748113140705,  152.874056570353,  76.4370282851763,  38.2185141425881,  19.1092570712941,  9.55462853564703,  4.77731426782352,  2.38865713391176,  1.19432856695588,  0.597164283477939,  0.29858214173897,  0.2]
    bbox: [-5120900.0, 4988722.914, 1140821.357, 9998100.0]
    bbox_srs: 'EPSG:25832'
    origin: 'ul'
  MY_GRID_L15:
    base: DE_EPSG_25832_DOP
    # here the first levels are pseudo resolutions, which won't be needed for seeding
    res: [4891, 4890, 4889, 4888, 4887, 4886, 4885, 4884, 4883, 4882, 4881, 4880, 4879, 4878, 4877, 0.2]
  MY_GRID_L14:
    base: DE_EPSG_25832_DOP
    res: [4891, 4890, 4889, 4888, 4887, 4886, 4885, 4884, 4883, 4882, 4881, 4880, 4879, 4878, 0.29858214173897]
  MY_GRID_L13:
    base: DE_EPSG_25832_DOP
    res: [4891, 4890, 4889, 4888, 4887, 4886, 4885, 4884, 4883, 4882, 4881, 4880, 4879, 0.597164283477939]
  # and so on ..

And for the seeding:
seeds:
  l15:
    caches: [cache_l15]
    levels: [15]
  l14:
    caches: [cache_l14]
    levels: [14]
  l13:
    caches: [cache_l13]
    levels: [13]
  # and so on...

Finally the linking of the master cache is done via following shell script:
for i in `seq 0 15`; do
    ln -s /cache/MY_GRID_$i/$i.mbtile /cache/MY_GRID/$i.mbtile 
done

Kind regards,

Dirk Thalheim

-----Ursprüngliche Nachricht-----
Von: Märt Reose [mailto:Mart.Reose at smit.ee] 
Gesendet: Mittwoch, 26. August 2015 07:39
An: Thalheim, Dirk <dirk.thalheim at bkg.bund.de>
Cc: mapproxy at lists.osgeo.org
Betreff: RE: [MapProxy] Generate complete cache from one scale

Hey!

I had a similar task at hand. Which format are your aerials? If its wms, then sorry I cant help you, but if you have access to geotiff files, then you have hope.

Mine were geotiff so I decided to use mapserver to deliver the tiffs and mapproxy to cache the result.

As there were many tiffs I used gdaltindex to create a tileindex. Like: gdaltindex aerials.shp aerials/*.tif

Then created a map file with a layer definition like this:
LAYER
  NAME "aerials"
  STATUS ON
  TILEINDEX "/srv/gis/aerials/aerials.shp"
  TILEITEM "Location"
  TYPE RASTER
END

Longer documentation: http://mapserver.org/input/raster.html
And map file: http://www.mapserver.org/mapfile/

It can look like a difficult task if you dont have experience in the field, but actually its quite easy.

Märt Reose

-----Original Message-----
From: mapproxy-bounces at lists.osgeo.org [mailto:mapproxy-bounces at lists.osgeo.org] On Behalf Of Oliver Tonnhofer
Sent: Tuesday, August 25, 2015 2:48 PM
To: dirk.thalheim at bkg.bund.de
Cc: mapproxy at lists.osgeo.org
Subject: Re: [MapProxy] Generate complete cache from one scale

Hi,

> On 21.08.2015, at 11:38, dirk.thalheim at bkg.bund.de wrote:
> 
> I have a source which provides raster data (aerial images) only for a single scale. Is it possible to configure mapproxy in that way, that a complete cache could be generated from that source?
>  
> In theory it would be enough to cache the source scale and after that recursively generate all lower scale caches by resizing the previous cache. My first approach would be to configure a recursive cache configuration. But this seems to me like a huge configuration and I’m not pretty sure how to combine the single caches in a single one for production. Or is there a simpler way?

No, there is no easy way to handle this right now. But it would be a great feature for MapProxy. It would also help for services that only provide maps for higher scale, but not for low scales (overview). 

Regards,
Oliver

-- 
Oliver Tonnhofer  | Omniscale GmbH & Co KG  | http://omniscale.com
OpenStreetMap WMS and tile services         | http://maps.omniscale.com

_______________________________________________
MapProxy mailing list
MapProxy at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapproxy