[mapserver-users] ONE PASS QUERY (RFC 52) - FEATURE OR BUG?

[Tamas Szekeres]

2010/3/23 Frank Warmerdam <warmerdam at pobox.com>

>
> 4. Preserve the behaviour of keeping separate set of results for separate
>> layer instances. In this regard a query on one layer should not invalidate
>> the results for a different layer instance of the same driver.
>>
>
> This seems to be a lot to expect.  We go to significant effort with
> the connection pooling to allow reuse of a connection for different
> layers, and in effect in many drivers this connection also carries
> a bunch of context with it.  Certainly in the case of OGR an
> OGRLayer retains a concept of current query result, but it can
> be invalidated by lots of operations other than ResetReading() and
> GetNextFeature().  I would imagine this is true to a greater or
> lesser to other drivers that pool connections.
>
>
Frank,

It may be driver dependent, but I'm tending to think we should open up a new
connection/session/dataset whatever, for those queries which would retain
the results at the driver, corresponding to a particular result set stored
at layer level.  This is not the case for those queries where the results
are not retained (like drawing the layers / background) and the connection
pool approach could continue be used in these cases. That's why I suggested
an additional parameter in whichShapes to define the purpose of the query.



>
>  5. Creating a clone of a layer should provide to use a separate query (by
>> keeping the results intact on the original layer). This would be essential
>> for msDrawQueryLayer to work when drawing the background before the
>> highlighted features.
>>
>
> This is also quite impractical for some implementations - certainly
> for OGR.
>
>
I'm quite unsure how msDrawQueryLayer would ever work with OGR then. In this
case, MS_HILITE would require to draw the entire layer first and then the
highlighted shapes from the result set. Drawing the entire layer (regardless
of whether it's happening on a clone) would reset the spatialfilter on the
driver with the same connection, and a subsequent resultsGetShape would fail
to retrieve the same features.



>
> In retrospect, I'm not all that confident that we really considered
> the impact of RFC 52 on use cases such as those you raise.  I
> certainly didn't understand these impacts.  What is less clear to
> me is where to go from here.  RFC 52 was put in place because the
> old approach was giving terrible performance in some cases.
>

This is really a good question ;-)  RFC 52 is out in the stable branch and
prevents from a number of users to upgrade (we don't know the exact number
though). Leaving this version as it stands would bring in more people
involved in this version of the API, while we foresee some API change
shortly.
I think our best chance would be to provide the original version at the
drivers in parallel to the current that means: both getShape and
layerGetShape should work properly. This should probably be controlled by a
layer processing option at driver level. It would be reasonable to switch
the defaults to the 2 pass approach in the stable branch, while the users
could eventually override this in their mapfiles.

But if we put the expectations you list into place there is no
> way it can be made fast on OGR short of maintaining distinct
> OGRDataset instances for each query in addition to the one used to
> draw the layer.  This could cause various performance and
> resource problems.
>
>
While I don't exactly see the performance impacts, it may require a bit more
memory for sure. However since we intend to reuse the results of a query it
would definitely imply to store the corresponding reference of the
OGRDataset during the time interval when a subsequent access to the query
may happen.

I'm also hesitant to think that the 1pass option is better for all OGR data
sources in all cases. Having a couple of test scripts to see the performance
difference of the same query with these 2 methods would be helpful.


Best regards,

Tamas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/mapserver-users/attachments/20100323/1d6d9438/attachment-0001.html