MS RFC 22a: Feature cache for long running processes and query processing (update)

[Stephen Woodbridge]

Frank, Daniel,

Is this a case where we need to separate the machinery from the application?

What I mean by this is that having a good abstract machinery that will 
be flexible and to solve a variety of problems is a good thing. A second 
issue is solve specific problems with the machinery.

We do not have to leave that up to the user to solve, we can build that 
into the basic internal structures that implement the mapfile. If we 
know a layer will be used for both rendering and querying, then we build 
the internal representation using this machinery.

To the extent that we also expose it to users is a separate question and 
  one that should probably protect the novice from constructing poorly 
or invalidly nested layers, but should not prevent power users from 
taking advantage of it.

Simplicity is nice, but the only way to get that is to start pruning 
code. The more developers, the more code, the more solutions we tackle 
all add complexity to the product and it takes a lot of effort for 
everyone to stay on top of things.

I'm throwing this out as food for thought and not advocating a direction 
as the discussion sounds like it adds some interesting capabilities, but 
I have not had time to more than skim over the rfc.

-Steve W

Frank Warmerdam wrote:
> Daniel Morissette wrote:
>> If I understood the RFC correctly, the proposed solution to this 
>> problem will be to embed all layers inside a CACHE wrapper layer, 
>> correct? So by default all postgis/oracle/sde/etc connections will 
>> still suffer from the double-pass query problem unless we add the 
>> CACHE layer around them, right? This adds an additional thing that 
>> users needs to be aware of in order to tune their app.
> 
> Daniel,
> 
> I share this concern.  Essentially that we have put the onus
> on the user to setup a cache layer to get queries working
> at a reasonable speed instead of treating this internally
> as I had originally proposed.
> 
>> - What does nesting do if the outer layer is not of connection type 
>> cache, layerfilter or geomtrans? For instance what happens if I have a 
>> layer of type point (shapefile) which contains two nested layers of 
>> type point?
> 
> I believe nested layers that are not addressed by their parent or some
> other layer would never be used for anything, wouldn't show up in
> WMS layer lists, etc.
> 
>> - With respect to STYLEITEM AUTO the plan is to cache the class 
>> objects for each shape? Won't that be expensive for large layers?
> 
> I had this concern too, though it isn't clear to me what else could
> be done.  I trust we will only pay this class object penalty when
> styleitem auto is in effect.
> 
>> I feel bad for raising all those issues since you seem to have put 
>> quite a bit of hard work on this, and would need to put even more work 
>> to implement it. It also seems that everybody else likes the proposal, 
>> so hopefully it's just me who's not "getting it"... and in this case 
>> please show me the light.  :)
> 
> My conservative side feels very much the same.  The complexity frightens
> me, and yet I don't want to be too negative after all the effort Tamas
> has put in.
> 
> Best regards,