[mapserver-users] ONE PASS QUERY (RFC 52) - FEATURE OR BUG?

[Paul Ramsey]

Is it better to back out to the old behaviour or would defining a
filter object that allows complex query logic meet the need in a more
direct way? (Ie, is running multiple queries a feature or a workaround
for an even older limitation?)

P.

On Mon, Mar 22, 2010 at 9:21 PM, Lime, Steve D (DNR)
<Steve.Lime at state.mn.us> wrote:
> I think we're in need of a RFC 52a. Clearly the compound query handling the old approach afforded is of value to a group of users and that wasn't accounted for in the initial RFC. The work around Assefa had to do with WFS and a certain subset of OGC filters at the sprint is evidence that the approach was even used in the core code (I wasn't aware of that at the time). We (in 5.6.3) developed a work around that retains two sets of indexes one suitable for random access and one for a specific result set (e.g. cursor). It uses the already present tileindex property of a shapeObj to store the latter. I think we can have the best of both here by storing the two indexes and potentially we can revert to a single getShape() function in MapScript and revive the old queryfile format as a option as well. Just needs to be planned for now that the full impacts are better understood.
>
> Steve
> ________________________________________
> From: mapserver-users-bounces at lists.osgeo.org [mapserver-users-bounces at lists.osgeo.org] On Behalf Of Frank Warmerdam [warmerdam at pobox.com]
> Sent: Monday, March 22, 2010 10:16 PM
> To: Tamas Szekeres
> Cc: mapserver-users at lists.osgeo.org; BrainDrain
> Subject: Re: [mapserver-users] ONE PASS QUERY (RFC 52) - FEATURE OR BUG?
>
> Tamas Szekeres wrote:
>> In my understanding with the original approach the driver should:
>>
>> 1. Retain the result set of the queries at the layer (ie. in the
>> layerinfo structure) until the layer is open and no subsequent
>> whichShapes is called to 'invalidate' the query.
>
> Tamas,
>
> Your point here is that the query result should live until
> invalidated by another whichShapes, right?  I would agree with
> that, but draw on a layer does do a whichShapes, right?  So a
> draw is expected to invalidate a query, right?
>
>> 2. Provide such index in shapeObj which would allow to retieve in a
>> subsequent resultsGetShape within the result set.
>
> ok
>
>> 3. Retain the random access behaviour (getShape) for backward
>> compatibility in parallel to resultsGetShape.
>
> ok
>
>> Since the RFC doesn't contain explicit note about the opposite, the
>> drivers should also:
>>
>> 4. Preserve the behaviour of keeping separate set of results for
>> separate layer instances. In this regard a query on one layer should not
>> invalidate the results for a different layer instance of the same driver.
>
> This seems to be a lot to expect.  We go to significant effort with
> the connection pooling to allow reuse of a connection for different
> layers, and in effect in many drivers this connection also carries
> a bunch of context with it.  Certainly in the case of OGR an
> OGRLayer retains a concept of current query result, but it can
> be invalidated by lots of operations other than ResetReading() and
> GetNextFeature().  I would imagine this is true to a greater or
> lesser to other drivers that pool connections.
>
>> 5. Creating a clone of a layer should provide to use a separate query
>> (by keeping the results intact on the original layer). This would be
>> essential for msDrawQueryLayer to work when drawing the background
>> before the highlighted features.
>
> This is also quite impractical for some implementations - certainly
> for OGR.
>
>> 6. Using a drawQuery should not invalidate the results of a previous query.
>
> I don't know much about drawQuery but it does seem plausible to ask
> that drawQuery should not invalidate the query it is drawing.
>
>> 7. Drawing the map should not invalidate the results of a previous query.
>
> But drawing maps uses the feature access machinery like whichShapes
> doesn't it?  How can we expect map drawing not to invalidate a query?
>
> In retrospect, I'm not all that confident that we really considered
> the impact of RFC 52 on use cases such as those you raise.  I
> certainly didn't understand these impacts.  What is less clear to
> me is where to go from here.  RFC 52 was put in place because the
> old approach was giving terrible performance in some cases.
> But if we put the expectations you list into place there is no
> way it can be made fast on OGR short of maintaining distinct
> OGRDataset instances for each query in addition to the one used to
> draw the layer.  This could cause various performance and
> resource problems.
>
> Best regards,
> --
> ---------------------------------------+--------------------------------------
> I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
> light and sound - activate the windows | http://pobox.com/~warmerdam
> and watch the world go round - Rush    | Geospatial Programmer for Rent
>
> _______________________________________________
> mapserver-users mailing list
> mapserver-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-users
> _______________________________________________
> mapserver-users mailing list
> mapserver-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-users
>