MS RFC 22a: Feature cache for long running processes and query processing (update)

[Frank Warmerdam]

Tamas Szekeres wrote:
> 2007/6/28, Steve Lime <Steve.Lime at dnr.state.mn.us>:
>> I think the easiest way to achieve one-pass queries would be to simply
>> re-tool the result set structure to store a pile of features (linked 
>> list or other) in
>> memory. The id cache could be maintained for memory constrained 
>> situations.
>>
> 
> Steve,
> 
> I'm not totally sure about that. Then you should probably reimplement
> all of the functions involved in the layerObj->results member. At
> least the following files might be affected:
> 
> mapdraw.c
> mapquery.c
> mapgml.c
> mapogcfilter.c
> mapogcsos.c
> maprasterquery.c
> maptemplate.c
> mapwms.c
> 
> And it's quite difficult even to count the existing functions to be
> modified. I guess it's
> a large amount of work to implement this as well as fixing the bugs it 
> involves.

Steve / Tamas,

In my opinion, if we wanted a minimum complexity solution we would
add a mechanism to store cached features in the layer structure,
and just modify the query code to push them into this cache, and
the msGetShape() function to fetch them - avoiding going into any
provider specific logic at all.  The resultcache would stay unaltered,
but msGetShape() would first check the feature cache.

I'm not sure if there would be auto-styling issues with this approach
though since it would mostly just be used for query results I think
it would be ok.

I'd be prepared to write an RFC for this approach if we decide
against RFC 22a, though we are past our MapServer 5.0 RFC deadline.

In fact, I'm quite concerned about implementing something as big and
complex as RFC 22a in our last few weeks before release.  It doesn't
really leave very long for things to settle down in the development
tree.

Best regards,

-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | President OSGeo, http://osgeo.org