[mapserver-dev] MS RFC 68: Support for combining features from multiple layers (Call for discussion)

[Stephen Woodbridge]

On 2/11/2011 6:28 AM, Tamas Szekeres wrote:
>
> 2011/2/11 Stephen Woodbridge <woodbri at swoodbridge.com
> <mailto:woodbri at swoodbridge.com>>
>
>     This is more an abstract use case where separating the combining
>     from the clustering might be useful.
>
>     Today, you can combine multiple files using a tileindex, but only if
>     they have the same attributes, and description. In theory, it might
>     be nice to be able to have multiple source layers that have similar
>     attributes and to be able to combine them into a single layer using
>     some compatible subset of the source column attributes so that the
>     combined layer could be treated as a single layer with the benefits
>     of LABEL MINDISTANCE nnn, or querying it to get a single closest
>     object, etc.
>
>
> Hi Steve,
>
> I didn't understand exactly what Frank meant by combining and clustering
> should be different. Assuming you are talking about the same thing it
> looks like it is caused by a terminology confusion as we interpret the
> term 'combining' differently. In my variant 'combining' and 'clustering'
> are just the same: creating a single aggregated feature from multiple
> neighbouring features.

Ahhh! yes, now I understand what your we stay in this context. And below 
you are using the term union (which is a better term I think) for what I 
was calling combining above.

> Getting back to the use case you described it could also be implemented,
> but it may require to do some adjustments in the proposed implementation:
>
> 1. We would expect that the features are served from the union layer and
> not from the source layers (the source layers should be hidden in effect).
> 2. The ITEMS processing option should be set (probably at the union
> layer) to specify the set of the attributes we intend to use.
> 3. In the union layer we should maintain some state in layerinfo to
> track which layer is actually used to provide the features (in NextShape)
> 4. We should also provide the identification of the features (probably
> the tileindex would be sufficient to store the layer index?)
>
> There may be some problems if we would like to use the same symbology
> for the features on the union layer and the source layers. We might
> probably support STYLEITEM AUTO on the  union layer which could be
> impemented by retieving the the corresponding class from the source layer.
>
>
>     So I think the idea of building modular filters and being about to
>     chain them together might have some value. On the other hand, there
>     maybe be some significant value in performance to combining and
>     clustering in a single step, but you probably have a better idea
>     based on how things are structured.
>
>
> Isolating these options would not necessarily impact the performance and
> we iterate the features in a similar way for both cases. In this case we
> would probably set at least 3 layers chaining together to carry out the
> functionality we are targeting. But the option of chaining modular
> filters together reminds me to the discussions around RFC-22 which could
> not get noticeable support since it doesn't fit well in the concept of
> 'MapServer is not a full-featured GIS system, nor does it aspire to be.'
> The implementation should probably be 'simple' from the aspect of the
> user as well as the developer. (I'm on the developer's side in most cases.)

Well, I'm not implementing it so I will leave the details up to you. I 
do not think we need a general purpose GIS system, but at the same time 
I think it is a good idea to think about other useful features that fall 
out of some simple generalization of the code. Over the past 2-3 years 
we have done a lot of work to refactor and generalize internal mapserver 
code. I know you have been involved in parts of that. All I was trying 
to get at is that there is value in components that can be reused in 
other useful ways and tried to present a reasonable case. You clearly 
have thought alot about this problem and I will trust your judgment.

>     Regarding the whole RFC, I find it impressive and probably a little
>     intimidating also. My first concern would be that I hope this does
>     not impact performance in any significant way for people that are
>     not using it. Beyond that, it seems well thought out, but there is a
>     lot there and I think I need to read it again when my eyes are not
>     glazing over :) I'll try to reread it in the morning.
>
>
> I'm not sure what you guys mean by 'intimidating' here? Is this related
> to the functionality itself? Or the proposed implementation? Most of the

For my part, the scope of this project, adding a two pass rendering 
chain, adding clustering tools, is a lot of stuff. Even for me to review 
and understand and to think of the ramifications of it. There is a lot 
of meat in the RFC and clearly you have put a lot of effort into 
designing implementation and in writing the RFC. I was pleased that you 
seemed to have isolated much of the code from mapserver core and have 
clear well defined integration points, as opposed to spaghetti threading 
itself all through mapserver.

> functionality would be encapsulated in a separate layer data provider
> which could infact be a plugin not requiring to incorporate that in the
> MapServer codebase at all. The only noticeable thing we would add is the
> changes in the vtable creation but it could eventually be ommitted if we
> implement this 'union' layer separately. I'll do some investigation in
> this regard.

I think this would be a good addition if it does not overly complicate 
you implementation and would add some additional value.

Best regards,
   -Steve W

> Best regards,
>
> Tamas
>