[mapserver-users] ONE PASS QUERY (RFC 52) - FEATURE OR BUG?

[Lime, Steve D (DNR)]

I think we're in need of a RFC 52a. Clearly the compound query handling the old approach afforded is of value to a group of users and that wasn't accounted for in the initial RFC. The work around Assefa had to do with WFS and a certain subset of OGC filters at the sprint is evidence that the approach was even used in the core code (I wasn't aware of that at the time). We (in 5.6.3) developed a work around that retains two sets of indexes one suitable for random access and one for a specific result set (e.g. cursor). It uses the already present tileindex property of a shapeObj to store the latter. I think we can have the best of both here by storing the two indexes and potentially we can revert to a single getShape() function in MapScript and revive the old queryfile format as a option as well. Just needs to be planned for now that the full impacts are better understood.

Steve
________________________________________
From: mapserver-users-bounces at lists.osgeo.org [mapserver-users-bounces at lists.osgeo.org] On Behalf Of Frank Warmerdam [warmerdam at pobox.com]
Sent: Monday, March 22, 2010 10:16 PM
To: Tamas Szekeres
Cc: mapserver-users at lists.osgeo.org; BrainDrain
Subject: Re: [mapserver-users] ONE PASS QUERY (RFC 52) - FEATURE OR BUG?

Tamas Szekeres wrote:
> In my understanding with the original approach the driver should:
>
> 1. Retain the result set of the queries at the layer (ie. in the
> layerinfo structure) until the layer is open and no subsequent
> whichShapes is called to 'invalidate' the query.

Tamas,

Your point here is that the query result should live until
invalidated by another whichShapes, right?  I would agree with
that, but draw on a layer does do a whichShapes, right?  So a
draw is expected to invalidate a query, right?

> 2. Provide such index in shapeObj which would allow to retieve in a
> subsequent resultsGetShape within the result set.

ok

> 3. Retain the random access behaviour (getShape) for backward
> compatibility in parallel to resultsGetShape.

ok

> Since the RFC doesn't contain explicit note about the opposite, the
> drivers should also:
>
> 4. Preserve the behaviour of keeping separate set of results for
> separate layer instances. In this regard a query on one layer should not
> invalidate the results for a different layer instance of the same driver.

This seems to be a lot to expect.  We go to significant effort with
the connection pooling to allow reuse of a connection for different
layers, and in effect in many drivers this connection also carries
a bunch of context with it.  Certainly in the case of OGR an
OGRLayer retains a concept of current query result, but it can
be invalidated by lots of operations other than ResetReading() and
GetNextFeature().  I would imagine this is true to a greater or
lesser to other drivers that pool connections.

> 5. Creating a clone of a layer should provide to use a separate query
> (by keeping the results intact on the original layer). This would be
> essential for msDrawQueryLayer to work when drawing the background
> before the highlighted features.

This is also quite impractical for some implementations - certainly
for OGR.

> 6. Using a drawQuery should not invalidate the results of a previous query.

I don't know much about drawQuery but it does seem plausible to ask
that drawQuery should not invalidate the query it is drawing.

> 7. Drawing the map should not invalidate the results of a previous query.

But drawing maps uses the feature access machinery like whichShapes
doesn't it?  How can we expect map drawing not to invalidate a query?

In retrospect, I'm not all that confident that we really considered
the impact of RFC 52 on use cases such as those you raise.  I
certainly didn't understand these impacts.  What is less clear to
me is where to go from here.  RFC 52 was put in place because the
old approach was giving terrible performance in some cases.
But if we put the expectations you list into place there is no
way it can be made fast on OGR short of maintaining distinct
OGRDataset instances for each query in addition to the one used to
draw the layer.  This could cause various performance and
resource problems.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent

_______________________________________________
mapserver-users mailing list
mapserver-users at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users