MS RFC 22: Data adapter support (void of interest?)

Tamas Szekeres szekerest at GMAIL.COM
Thu Oct 26 17:17:26 EDT 2006


Steve,

> I was sort of sitting back and watching where the discussion with Frank
> was going
> to go since you both are amongst the most technically skilled and then
> comment a bit.
> I confess to being a bit lost in the technical details of RFC 22.
>

That's very kind of you. I consider the conversation between me and Frank is
closed. We have made a further illuminating talk on the IRC however we have
made our standpoints clear to each other.
At this point me - as the proposer - would like to see how the foundation of
this RFC was succeeded. With the absence of notes I cannot see how can
I make the things more clear and see whether the subject of the document is
difficult to follow or not. (I think this was not an issue for Frank)

> One thing I did notice is that for queries to work "as is" the adapter,
> specifically the
> cache adapter would need the ability to retrieve features by unique ID
> or better
> yet member shapes would need to be cached by hand from the msQueryBy...
> functions
> so we could pull them sequentially for presentation.
>

Hmmm... I guess I couldn't understand this conclusion exactly. The
most compelling behaviour of the cache adapter is to retrieve the
features using the mapservers 'bulk' mode by
utilizing WhichShapes and NextShape to the provider. All of the
subsequent renderings
and the QueryBy.. operations could be handled from the cache. Recall
that currently
serving QueryByShape requires at least n+1 provider accesses and 2 layeropens
where n is the number of the features found. And I am guessing even more
improvements for the further renderings by using msDrawQueryMap for example.

I'm a bit disappointed about the inevitable copyShape when serving
from the cache
since the caller has ownership of the features retrieved but I can be
reconciled to
it for now.

> My thoughts on caching are that it is very much necessary. I was
> thinking mostly
> from a query perspective but do set the value in a long running
> process. When I
> had thought about implementation it I was thinking that caching would
> be independent
> of the underlying datasource. That is, a filter that would sit between
> MapServer
> drawing and query functions.

Wow, that's exactly what I have addressed.

> That way we wouldn't have to change in the
> existing
> code base at all, nor would virtual providers have to worry about
> anything. Layer
> configuration for caching would be simple, STATUS On/Off/Query,
> SIZE/TUNING
> parameters and that's about it.
>

Well, one of the most important conclusion of the conversation with Frank was
that even the most good-looking solution requires modification when we are
thinking of how to implement them.

For me hooking into the vtable was the most trivial, but I also had to make some
changes as you can see.

Furthermore I'm not keen to propose a solution restricted solely to the
caching problem. Instead would want to give a framework where other
sorts of extensions to the the data provider could easily be done.

Support for cascading the adapters is the other point where one can
feel the solution a bit complex.

According to the ID-s I consider the providers not to keep the feature
indexes between the subsequent WhichShapes so I would restrict the
contets of the cache
to one retrieval at a given time. However I would allow to control how the
cached area is determined (not always the same as the rendering area)

> Your proposed solution goes well beyond the simple notions I had been
> thinking
> about. Also I was mulling over how to put GEOS functions in the
> pipeline (e.g.
> buffer a feature before drawing or querying with), and could only come
> up with
> using the PROCESSING tag so this would be better.
>

I would leave the original provider alone with the PROCESSING parameter so
with regard to this the provider would behave similarly. And I would also
give a support for getting the shape by reference and changing them
programmatically by the mapscript users for the subsequent renderings.

> I think it would be helpful to write RFC 23 on the caching adapter so
> they can
> be considered in tandem, that is, a concrete example of the benefits of
> RFC 22.
>

Agreed. But I wouldprevent the developers from a flood of the RFC-s
as long as this one seems to be definitely unsupported.
But it would be interesting to see some performance measures so I will
go ahead with it

> Consider this more than mild support.
>

Promising. Thanks.

Best Regards,

Tamas



More information about the mapserver-dev mailing list