MS RFC 22a: Feature cache for long running processes and query processing (update)

[Steve Lime]

The one gotcha with one-pass queries regardless of approach is that presently the items
retrieved for the first pass don't match those on the second pass. In the first pass we grab just
enough to classify (if necessary) features via msLayerWhichItems. While the second pass gets
all items for presentation. We really should ditch msLayerWhichItems and get all items all the 
time.

The main impact of that would be needing to derive item (e.g. classitem, labelitem, etc...) index 
values at runtime. Basically:

   if (classitem && classitemindex == -1) /* lookup */
     classitemindex = msLayerGetItemIndexByName(...);
     if(classitemindex == -1) { /* item not found, throw an error */
       ...
     } 
   }

This would affect a few places in the code, principally mapdraw.c and maputil.c.

The initItemInfo functions for each of the vector drivers might need changes, although I hope
minor ones since we already support getting all items.

In reality this is a simplification of things which is a good thing. I would be willing to take this modification
on in consultation with driver owners.

Regarding RFC 22 what do folks think about implementing a simple one-pass cache using
the resultCacheMemberObj for 5.0? Then considering RFC 22 independently of queries independently
since there are plenty of good ideas there.

Steve


>>> On 6/28/2007 at 12:55 PM, in message <4683F62D.7070401 at mapgears.com>, Daniel
Morissette <dmorissette at MAPGEARS.COM> wrote:
> Steve Lime wrote:
>> That could be as simple as the shape just sitting there along with the 
> indexes, that is, the
>> resultCacheMemberObj gets a new member, a shapeObj.
>> 
>> The result cache is nothing more than an array of those members. (e.g. 
> layer->results) That
>> could be made more high performance and hidden behind accessor methods in 
> mapscript. Users
>> use getResult(int i) already. 
>> 
> 
> I like that. And you enable this query cache by default for all data 
> providers that suffer from the double-pass query problem (postgis, 
> oracle, etc, but *NOT* shapefiles) and provide a PROCESSING directive 
> that can be used to turn off the query cache if someone wants to disable 
> it to perform very large queries or for whatever reason, e.g.
> 
>    PROCESSING "QUERY_RESULT_CACHE" "OFF"
> 
> Daniel