[Tilecache] Patch for WMTS in TileCache

Jim Groffen jim.groffen at lisasoft.com
Fri Sep 11 03:00:12 EDT 2009 

Hello Chris,

I apologize for the size of this email, but I'm trying to be complete. I've responded to your questions below and further on talk about an idea for incorporating your feedback.

1 Currently, TileCache is designed to support accessing any tile set
  with any cache backend. Since all tiling schemes essentially work the
  same -- specifically, they all work on a fixed grid, with a set of
  scales and reslutions that allows a multilevel fixed grid, typically
  with the same corners at all levels of the grid -- it is easiest to
  transform this grid into a fixed set.

  It seems like the WMTS layer does not do this, and invents its own
  cache instead.

  Is the cache set up by WMTS different enough that this is *really*
  neccesary?

TileCache does work with WMTS layers using disk cache and should work with any other cache too (I didn't test all of them). It does this by turning a WMTS layer definition into many TileCache compatible layers. Each TileCache layer is a unique combination of the style, dimensions and tile matrix sets for a layer. Before WMTS came along if you wanted the same layer with two different styles in TileCache you would make two different layers. That's how the WMTS implementation deals with it. Around line 118 of Service.py is where, during configuration file reading, a WMTS layer is found and a call to TileCache.Layers.WMTS.load_wmts_layer is made.

Given the following WMTS layer in the config file:

[basic]
type=WMTS
...
...
title=Basic
tile_matrix_sets=WholeWorld_CRS_84,World84-90_CRS_84

If was ask for a TMS capabilities document we see two layers, one for each tile matrix set. The name for each layer is a string join("_") of layer name, style if present, dimensions if present and tile matrix set. Every possible combination is treated as its own layer because they will be cached separately on disk. An example TMS capabilities file is attached.

It is true that when using a cache of type WMTS, unless passed a WMTSTile instance it will raise a TileCacheException. WMTSTile instances are at the moment only coming from a WMTS layer, because they need to have a tile matrix set. If the layer associated to the tile has style or dimension information then that gets used too.

2 It appears that WMTS has been developed as a 'Layer'. Layers are ways
  of accessing backend services -- so the MapServer layer accesses a
  MapServer service, the WMS layer accesses a WMS service, etc. WMTS
  does not appear to be this (since you're pointing at my WMS for your
  demo config). Instead, it appears that WMTS should be a *service*
  (TileCache/Services/ directory); the configuration parameters that
  you are using to define tilesets should potentially be added to *any*
  layer, and these configuration parameters are used to return metadata
  and tile requests whenever a request comes in in the WMTS service
  format.

  As it is now, presumably WMTS doesn't work with Mapnik, MapServer,
  GDAL, etc. backends. This ireally counter to the design of TileCache,
  and I'm sorry that was unclear.

  Note that a WMTS layer -- that is, a layer class which requested
  tiles from a WMTS service -- is not out of the question, and could be
  implemented -- as could a TMS layer, for example, but I don't think
  that either of those is what has been built here.

In my current implementation when a WMTS layer is seen in the config, TileCache.Layers.WMTS interprets it into many WMS layers. In other words TileCache.Layers.WMTS provides support for WMTS layer definitions in the config. This is as you say supporting WMTS for only one backend service and not all backend services.

Also: when a WMTS request comes in it goes to the WMTS Service, which in turn will hand it on to the WMTS Layer if it needs to get the tile from the backend. It would be possible to have the WMTS Service go to any layer to handle the backend as other Service implementations do, but it would mean pushing WMTS specific implementation bits down to the backends. The reason
I didn't do this is because right now "Layers.WMS" knows how to handle tiles the TileCache way where Layers.WMTS knows how to handle tiles the WMTS way. They are different and I can't consolidate them easily. I'll get into that after I've explained TileMatrixSets

3 I really think that -- althogh your effort was admirable -- I feel like
  a lot of this work was simply done in the wrong place/way.

Having as little impact on the existing TileCache implementation as possible and isolating the WMTS code has lead me to not implement WMTS the way it should have been?. When put that way it does seem like I put that goal over a correct implementation but that's not the case. I'll be honest - I didn't know TileCache all that well when I started but I know it much better now and these emails are improving my understanding even more. This is why the implementation is not exactly how you would like it to be. I'm aware that most users and contributors to TileCache won't be interested in WMTS and didn't want to be the cause of added complexity for them but I wouldn't sacrifice the quality of the implementation for it. Looking back, I should have badgered you earlier and more often than I did to get your feedback on my implementation plans.

4 Describe tile_matrix more, and I can probably comment on it. What is a
  tile_matrix? What does it map to most closely in existing code?

A tile matrix set controls what tiles are available for a layer. In TileCache terms a Tile Matrix Set replaces the bounding box, resolutions list, image size and CRS. A tile matrix set is made up of multiple "TileMatrix", one for each zoom level. This means a Tile Matrix maps to one element of the resolution array, but because WMTS supports different bounds and image sizes per zoom level, it also maps to bounding box and image size. In TileCache configuration I've made it so that you can set things like top left point and image width / height at the Tile Matrix Set level and all tile matrices for that level will inherit the same settings. The WMTSGuide-excerpt.txt attachment has the documentation I wrote on WMTS configuration of Tile Matrix Sets in it. It's from docs/WMTSGuide.txt This is why Layers.WMTS handles asking for tiles from the backend - why WMTS is its own backend.



Based on your feedback, let's think of an alternative implementation:

First let's get rid of Layers.WMTS and move the config handling part somewhere else (say the Service class, that's where other layers are created from config). Now when config sees a layer of any type with WMTS properties (styles, dimensions, tilematrixsets) it will instead create many layer instances to handle each style/dimension/tilematrixset combination. When a WMTS request comes in, the WMTS service will find and use the right layer for the unique style/dimension/tilematrixset combination of that request.

Now we are left with the problem that the way TileCache addresses tiles is different to how WMTS does it. I'd say they are different, but not incompatible. The way TileCache does it with resolutions, bbox, image size and crs can be represented in a TileMatrixSet, we could support the existing config files and generate a TileMatrixSet instead. Then each backend would use the tile matrix set instead of resolution, image width / height, bbox, crs. This would work without the need for code duplication but it would impact every layer and service. Also, not all of the capabilities of WMTS are supported by every layer (such as dimensions and multiple styles) - Of course they don't have to be supported right away or even at all in some cases.

Is this headed in the right direction? Is it better than what we have at the moment where WMTS is only supported for the WMS backend? While much of the code needed (at least for WMS service and back end, and the WMTS service) is already done I am daunted by the amount of work involved, any suggestions?



These last two points are less contentious.

5 If the ordering is always 'x,y' (or lon,lat) and not some freaky changed
  thing with projections, then I'd preer to just call it left_top and be
  done with it.

It will pretty much always be x, y. It's meant to be in the projection that the tile matrix set is using. Based on your feedback though I'm going to switch it to left_top and tone down the warning I have in the WMTSGuide.txt about it (that's in the attached excerpt too).

6 There is a pending change request...

The OGC recently have a motion to "Approve Web Map Tiling Service (WMTS)" [07-057r7] as an OGC implementation standard". The vote doesn't close until 2009-11-01 but if the change request I'm talking about is in there then I'll sort it out.

Regards,

Jim Groffen.


-----Original Message-----
From: Christopher Schmidt [mailto:crschmidt at metacarta.com]
Sent: Thursday, 3 September 2009 9:41 PM
To: Jim Groffen
Cc: tilecache at openlayers.org; Stefan Hansen; John Hudson
Subject: Re: [Tilecache] Patch for WMTS in TileCache

On Thu, Jun 25, 2009 at 07:38:26PM +0930, Jim Groffen wrote:
> Hello,

Sorry this has taken so long.

> Attached is a patch file that, if I made it correctly, will update TileCache revision 391 with the WMTS capabilities. I've added a guide (WMTSGuide.txt) and sample configuration file (wmts.cfg) to the docs folder that describes the new features.

Having briefly reviewed this patch, I have the following questions:

 * Currently, TileCache is designed to support accessing any tile set
   with any cache backend. Since all tiling schemes essentially work the
   same -- specifically, they all work on a fixed grid, with a set of
   scales and reslutions that allows a multilevel fixed grid, typically
   with the same corners at all levels of the grid -- it is easiest to
   transform this grid into a fixed set.

   It seems like the WMTS layer does not do this, and invents its own
   cache instead.

   Is the cache set up by WMTS different enough that this is *really*
   neccesary?

 * It appears that WMTS has been developed as a 'Layer'. Layers are ways
   of accessing backend services -- so the MapServer layer accesses a
   MapServer service, the WMS layer accesses a WMS service, etc. WMTS
   does not appear to be this (since you're pointing at my WMS for your
   demo config). Instead, it appears that WMTS should be a *service*
   (TileCache/Services/ directory); the configuration parameters that
   you are using to define tilesets should potentially be added to *any*
   layer, and these configuration parameters are used to return metadata
   and tile requests whenever a request comes in in the WMTS service
   format.

   As it is now, presumably WMTS doesn't work with Mapnik, MapServer,
   GDAL, etc. backends. This ireally counter to the design of TileCache,
   and I'm sorry that was unclear.

   Note that a WMTS layer -- that is, a layer class which requested
   tiles fro ma WMTS service -- is not out of the question, and could be
   implemented -- as could a TMS layer, for example, but I don't think
   that either of those is what has been built here.


 * top_left=-180,90
   Generally, this can simply be extracted from the bbox, though I see
   here that different levels can have different 'top left' expectations
   (hooray for setting up a standard that is completely different from
   everything else). I guess that this means that WMTS really isn't the
   same, though I still feel that it's probably better to not go bother
   writing another cache, and simply document that you can't use WMS and
   WMTS on the same cache. Porting each of the Layer and Cache classes
   to WMTS would mean duplicating tons of code.

> The implementation got a fair bit of testing in the OWS-6 testbed, but
> since then I've implemented support for MetaTiling, which only I've
> tested. A couple of things I came up with while merging with the
> latest revision and writing the documentation were:
>
>
> -          I avoided changes to existing TileCache files as much as
> possible. The goal was to keep the WMTS functionality contained to its
> Service, Layer and Disk module files as much as possible. When it
> comes to the changes to support reading TileMatrixSet configurations
> though I had a hard time separating this out from Service.py. I wasn't
> keen to subclass Service.py, but I don't like how much complexity has
> been added to the configuration reading. If anyone can suggest
> something that'd be great.

I really think that -- althogh your effort was admirable -- I feel like
a lot of this work was simply done in the wrong place/way.

>
> -          MetaTile was subclassed with WMTSTile so it could keep
> track of tile_matrix (used instead of resolution to go from tile
> co-ords to bounds). After implementing meta tiling for WMTS I think It
> may have been better to add tile_matrix to the Tile class instead, and
> WMTS specific code could look for it in the class if they wanted to
> use it.

Describe tile_matrix more, and I can probably comment on it. What is a
tile_matrix? What does it map to most closely in existing code?

>
> -          top_left configuration setting - I don't like this name
> because it causes a bit of confusion. The setting wants a coordinate
> which is usually ordered left,top and not top,left. The top_left
> matches the wording as it comes out in the WMTS capabilities document,
> but perhaps this should be something like 'origin'? The problem with
> origin is it doesn't state what corner of the bounds is used as the
> origin. I don't like left_top either - it may be more accurate than
> top_left most of the time but it still implies a specific order and
> now doesn't match WMTS (which is TopLeftCorner), so after all that I'm
> not sure what it should be called. Maybe top_left_corner?

If the ordering is always 'x,y' (or lon,lat) and not some freaky changed
thing with projections, then I'd preer to just call it left_top and be
done with it.

>
> -          There is a pending change request that was popular to make
> style always be specified in the path. When this change occurs, If no
> style is set in configuration then a default style name should be set
> and used in the paths which won't affect the WMS getMap requests. This
> would be easy enough to do but isn't *quite* yet standard WMTS
> behavior . Should it get done now?

Not enough information to understand the question.

Regards,
--
Christopher Schmidt
MetaCarta

The contents of this email are confidential and may be subject to legal or professional privilege and copyright. No representation is made that this email is free of viruses or other defects. If you have received this communication in error, you may not copy or distribute any part of it or otherwise disclose its contents to anyone. Please advise the sender of your incorrect receipt of this correspondence.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tms_capabilities.xml
Type: application/xml
Size: 734 bytes
Desc: tms_capabilities.xml
Url : http://lists.osgeo.org/pipermail/tilecache/attachments/20090911/ebe66803/tms_capabilities.rdf
-------------- next part --------------
Tile Matrix Sets
================
WMTS requires the ability to define Tile Matrix Sets. A tile 
matrix set controls what tiles are available for a layer. 
In TileCache terms a Tile Matrix Set replaces the layer bounding 
box, resolutions list, image size and CRS.

In the TileCache configuration file, each section represents a 
TileCache layer except for two special sections - the [cache] and 
[tilecache_options] sections. To define a Tile Matrix Set, add 
the following option to the [tilecache_options] section:

  tile_matrix_sets=WholeWorld_CRS_84,World84-90_CRS_84

This is a comma separated list of configuration file section names 
to treat as TileMatrixSets instead of a TileCache layer. 

A Tile Matrix Set consists of a series of Tile Matrices and some 
common parameters. As per TileCache layers, the section name becomes 
the name of the tile matrix set.

TileMatrixSet Configuration
=========================
crs
	CRS of the tile matrix set. Default is EPSG:4326. The tile matrix 
	set CRS and the layer CRS don't need to match.
identifiers
	Comma separated list of tile matrix identifiers.
top_left
	Upper left point of the matrix set in the CRS specified. 
	Applied to every tile matrix. EG: -180,90
	A warning: Being named top_left causes confusion as co-ordinates  
	must be in the order specified by the CRS, which for the 
	default EPSG:4326 is LEFT,TOP.
tile_size
	Width and height of each tile in the tile matrix set. EG: 256,256
wellknown_scale_set
	If set, the WMTS capabilities document will verify that the  
	tile matrix set conforms to (and can be overlayed with) tiles from 
	a tile matrix set that uses the same well known scale set. Does not 
	enforce conformity. Example: GlobalCRS84Pixel
<identifier>__matrix_size
	Width and height of the tile matrix in tiles.  
<identifier>__scale_denominator
	The scale denominator for the tile matrix identified by <identifier>  
<identifier>__tile_size
	You can set the tile size per tile matrix if you need to.  
<identifier>__top_left
	You can set the top left coordinate per tile matrix if you need to.
	This overrides the top_left setting for the tile matrix set if 
	there is one. Remember that this is a point using the CRS of this 
	tile matrix set, so not necessarily top,left (in fact by default 
	it's left,top).

More information about the Tilecache mailing list