[Tilecache] Patch for WMTS in TileCache

[Jim Groffen]

Hello again Chris,

> So, what The WMTS layer object actually does, as I understand it, is
> create a *set* of TileCache layers; each of those TileCache layers is
> (currently) a WMS layer. Is that correct? This makes me wonder if
> something that could (Thought not neccesarily 'should') be done is to
> change the code to instead be a utility which takes a set of WMTS
> configuration and turns it into a TileCache config.

That's right, it turns one WMTS definition in the config into many layers. It does do WMTS layers though as opposed to WMS layers because WMTS layers knows about WMTS specific stuff like per-zoom level TileMatrixSet stuff and supports getFeatureInfo. The alternative would be to make this generate any TileCache layer.

> It appears there is additional code in the form of the Tile and Layer,
> as well as the TileMatrix Layer object. I am less certain how these are
> meant to work, and concerned that people might not want to use them the
> way that they are currently designed. (For example, wanting to use WMTS
> against a Mapnik backend.) If, instead of a class which 'magically'
> creates multiple layers, there was a pre-processing step which generated
> the TileCache configs, it seems like it's possible that if someone
> wanted to (as I suggest) use Mapnik instead of WMS, they could tweak the
> output config.

TileMatrixSet contains the definition of a tile matrix set and holds a list of TileMatrix instances. The WMTS layers point to a particular tile matrix set and use that for tile addressing instead of crs, bbox and resolution.

Much of the complexity that caused the need for TileMatrix, TileMatrixSet, and a WMTS layer classes is due to the TileMatrixSet concept not being fully compatible with TileCache tile addressing. If we instead restrict the tile addressing capabilities to what TileCache already supports then much of this complexity will go away, and all layers will work without changes.

> In order to accomodate for the WMTS specific layer properties, the base
> layer could be extended; for example, you could add a "wmts_layerset"
> key/value param. Then all layers which had the same wmts_layerset would
> get put together in the output capabilities.

Cool! wmts_layerset is an exact example of a property that automagically happens now but would need to be pushed down into the regular layers (it's called wmts_layer at the moment, calling it wmts_layerset is better though). Other properties currently in the WMTS layer class that need to go somewhere else are:

- title (used in capabilities, common to all layers that share the same 'wmts_layerset')
- keywords (same as title)
- style (particular to this layer)
- dimensions (particular to this layer)
- units (used to go from a scale to a resolution)
- tile_matrix_set (particular to this layer)
- query_layers (to support getFeatureInfo)
- feature_count (to support getFeatureInfo)
- info_formats (to support getFeatureInfo)
- pixel_coord_params (to support getFeatureInfo, so you can override the i and j param names with something else if your WMS isn't using i and j).

Title and keyword could be stored for every layer, though they are unique to the wmts_layerset.

The tile_matrix_set would be needed for the layer to support WMTS cache format. When generating capabilities the WMTS Service uses this information in the same way dimension and style are used.

> Layers.WMS takes a tile (layername, x, y, z) and fetches it from a
> server. This is the part of the Layers.WMTS in 'renderTile'. This method
> looks pretty much the same as the WMS layer to me. The additional
> configuration properties on the layer look like something that could
> potentially be pushed as options into the base Layer.

The only difference is WMTS renderTile deals with style and dimension. Support for style and dimension for layers would need to be honored or ignored by each Layer, maybe throwing an exception stating style or dimension info isn't supported for that layer type. A while ago I saw a patch was added to support additional parameters to be passed through to the backend. Style and Dimension could be done with that instead. That way we'd inherit support for extra params implementation in other layers (if there is any, maybe that was WMS only).

> This leaves a couple things in the Layer.WMTS:
>  * GetFeatureInfo. In my opinion, this should be removed. TileCache
>    is really not designed to be fetching and caching feature data,
>    in my opinion; or if it is, then it should be something we work on
>    more thoroughly than just pushing it into this one layer type.

getFeatureInfo is an optional WMTS requirement, not a mandatory. The supporter in OWS-6 was keen for it to be supported.

> * Dimensions? Not entirely sure about this, but they just look like
>   extra KV Pairs, right?

Yep they are. WMS supports TIME and ELEVATION as named dimensions, plus any other named dimensions as well, but they get prefixed with "dim_". I was dealing with the "dim_" thing automagically. One layer can for example support a time dimension with any number of (say three) different values, an elevation dimension with say two different values and a wavelength with four different values. This does mean that we'd generate 3x2x4=24 unique TileCache layers for every possible combination of dimension value that results in a unique tile. It's rare for a layer to have more than one dimension if it has any, but it's easy to support as they are just extra key value pairs. Styles and tile matrix sets also multiply the number of possible visualizations and hence layers. It's like asking the question "Show me a map of this area at this point in time with this wavelength setting at this elevation in this style".

> * The "WMTSTile" stuff, which could be a Layer-level option -- "When
>   creating Tiles, use this subclass."

I think this problem is going to go away based on the solution.

>> Based on your feedback, let's think of an alternative implementation:
>>
>> First let's get rid of Layers.WMTS and move the config handling part
>> somewhere else (say the Service class, that's where other layers are
>> created from config). Now when config sees a layer of any type with
>> WMTS properties (styles, dimensions, tilematrixsets) it will instead
>> create many layer instances to handle each
>> style/dimension/tilematrixset combination. When a WMTS request comes
>> in, the WMTS service will find and use the right layer for the unique
>> style/dimension/tilematrixset combination of that request.
>
> Yes, I think this is how I would have it done. The only question in my
> mind, given that, is where the first part lives. My suggestion earlier
> of a config file 'preprocessor' which takes these config options and
> writes out a 'regular' tilecache config was designed to address this.

I like the automagic stuff myself. The existing implementation is pretty much done and it keeps TileCache configuration as a one step thing. I can see benefits to a config file generator though - gets rid of the config file reading complexity I added to Service.py by keeping it in a separate utility script - a script that could be run when TileCache is about to load
the config if the tilecache config file is not there. Which is going to be better for users of TileCache? there are benefits and drawbacks to both. I'll think about this one a bit more.

>> ...... Also, not all of the capabilities of WMTS are supported by
>> every layer (such as dimensions and multiple styles) - Of course they
>> don't have to be supported right away or even at all in some cases.
>
> It seems to me that 'dimensions' and 'multiple styles' are likely to be
> a WMS-only option. If we went with the 'preprocessing' method I
> suggested above, and someone said "Okay, here is my WMTS config, and I
> want you to write out a bunch of Mapnik layers, with these
> 'dimensions'", the script might throw an error saying "only WMS supports
> dimensions."

Agreed.

> Next, let's think about what the right implementation is. *My* desire is
> to continue to use bbox/resolution/etc. for all layers, and if we need
> to give the set of parameters that is a name, we can add additional data
> to do that. So for example, we might extend the TileCache config to
> support any sections which start with [tilematrix_blah] to describe a
> tilematrix; then, you have two options when setting up a layer:
>
> bbox=
> resolution=
> etc.=
>
> *or*
>
> tilematrix=blah
>
> The Service init can then take those and 'look up' the tile matrix and
> match it to the bbox/resolution.
>
> The difficulties I see with this are that I'm not sure that a layer in
> TileCache actually has all the config info you need: you mentioned that
> each 'zoom level' can basically use a different tileset, and TileCache
> doesn't really have a good way of doing that, and I"m not sure that I
> want to extend it to do so. So I guess then the question is: "how
> important is that?" Is it something that the spec absolutely requires?
> is it something that we can avoid? (And who the hell is ever going to
> use it?) Honestly, when there's a 'feature' in a spec that even ESRI
> hasn't seen fit to implement, I Get a little bit concerned at the idea that
> it is the Most Important Feature Ever; it seems an unlikely-to-be-used
> feature to me.
>
> If it really is important, then I guess we'll have to think about the
> right way to do it.

The importance of implementing as much of the spec as possible is
difficult for me to answer. My motivation is what's best for both TileCache
AND what's best for the WMTS spec. I agree that it's better to have the WMTS
service supported by all layers and I'm hoping when the dust settles a
nice way to support additional capabilities in the TileMatrixSet will
become apparent.

> So, here's what I think I would like, and you can tell me if this is
> going to be useful:
>
>  1. We keep the WMTS service. It looks good.

Great

>  2. We move the properties we need for WMTS "layers" into the base
>     layer.

Yep

>  3. We figure out whether we should be:
>     * Creating Layers on the fly when the config is loaded
>   or
>     * Creating layers via a seperate tool that uses a WMTS-like config
>       using the library functions you've already built. (The latter
>       allows users to tweak the resulting configs more easily; the
>       former allows the 'write and forget', but at the cost of the
>       config now being 'magical' in that it creates additional layers.)

Yep, I'm not sure which is best yet either.

>  4. We extend TileCache to support named descriptions of layer extents
>     and resolutions (and whatever else). (These are 'tilematrixes', I believe.)

Yep, that's exactly what a TileMatrixSet is. I wonder what the level of effort would be to move Layer properties like resolutions, crs, bbox etc into a LayerExtents class? Then we could have different LayerExtent implementations for different layer addressing schemes.

>  5. For now, we punt on the "WMTSTile" and "WMTSCache" until we get the
>      basics working. At that point, we can establish how to get the
>      tiles written on disk in a wya that you would access them through
>      the WMTS 'REST' service.

Agreed. My feeling is we won't need a WMTSTile anymore to support a WMTSCache. Instead as Tile includes the Layer it should get things like LayerExtent name (tile matrix set name), dimensions, styles from that now.

> Does that sound reasonable?

Coming along nicely Chris. Thanks!

I'm off to QLD for a week of fun in the sun but I'll still be listening to emails.

Regards,

Jim Groffen

The contents of this email are confidential and may be subject to legal or professional privilege and copyright. No representation is made that this email is free of viruses or other defects. If you have received this communication in error, you may not copy or distribute any part of it or otherwise disclose its contents to anyone. Please advise the sender of your incorrect receipt of this correspondence.