[mapserver-users] ONE PASS QUERY (RFC 52) - FEATURE OR BUG?

[Tamas Szekeres]

Hi,

I second to these concerns, absolutely. With regards to RFC 52 there have
been a couple of breaking changes in MapServer 5.6 which prevents me from
upgrading to this version in my existing projects. I've already tried to
ring the bell in this topic with a couple of posts (see below), but it seems
the use case described here (as keeping long term mapObj references) is not
widely used and falls ouside of the general area of interest;

http://n2.nabble.com/Ready-for-5-6-2-td4743344.html#a4746772
http://n2.nabble.com/OGR-single-pass-query-issues-was-Ready-for-5-6-2-td4753764.html

By raising up your issues below I've studied RFC
52<http://mapserver.org/development/rfc/ms-rfc-52.html>again to see
the objectives, and it seems we are getting out of the sync
with the current implementation at the drivers.

In my understanding with the original approach the driver should:

1. Retain the result set of the queries at the layer (ie. in the layerinfo
structure) until the layer is open and no subsequent whichShapes is called
to 'invalidate' the query.
2. Provide such index in shapeObj which would allow to retieve in a
subsequent resultsGetShape within the result set.
3. Retain the random access behaviour (getShape) for backward compatibility
in parallel to resultsGetShape.


Since the RFC doesn't contain explicit note about the opposite, the drivers
should also:

4. Preserve the behaviour of keeping separate set of results for separate
layer instances. In this regard a query on one layer should not invalidate
the results for a different layer instance of the same driver.
5. Creating a clone of a layer should provide to use a separate query (by
keeping the results intact on the original layer). This would be essential
for msDrawQueryLayer to work when drawing the background before the
highlighted features.
6. Using a drawQuery should not invalidate the results of a previous query.
7. Drawing the map should not invalidate the results of a previous query.


Further notes: To provide correct implementations at the drivers it seems we
should provide a bit more information for the driver to distinguish between
the purpose of a query. For example whichShapes should take an additional
parameter (querymode) with the following predefined values:

MS_QUERY_SEQUENTIAL:  The returned shapes will be retrieved by nextShape (no
subsequent (result)getshape will happen). This would be used by the normal
drawing operations.
MS_QUERY_RANDOM: The results would be retrieved by nextShape. The driver
would provide oid-s as feature indexes to support to retrieve the features
by the original (2 pass) behaviour (getShape). No features are retained at
the driver for further access.
MS_QUERY_PRESERVE: The results would be retrieved by nextShape and
resultsGetShape. The drivers should store the result set at the layer for
further retrieval (1 pass).

The features retrieved by MS_QUERY_PRESERVE should be kept in a separate
location either in layerinfo or in a separate structure (a queryinfo for
example). The latter would provide the isolate the results from the layer
opened state.
MS_QUERY_SEQUENTIAL or MS_QUERY_RANDOM should not invalidate the results in
queryinfo.


We should also provide the user with an option to select between the default
of MS_QUERY_RANDOM/MS_QUERY_PRESERVE at layer level. Probably a layer
processing option would be sufficient.



Best regards,


Tamas








2010/3/21 BrainDrain <paulborodaev at gmail.com>

>
> Please read carefully.
> 'old style' (two pass) query
> advantages:
> - in mapscript (c#) layer's query methods are CUMULATIVE (relative to other
> layer's queries). Query result (success/failure) has no effect on other
> layers when I call map.savequery.
> - In this case (query file contains just oid's) - I CAN create COMPLEX
> queries by applying different parameters and mixing query types
> - Query result is oid (or row index in shapefile) - it's cool for creating
> advanced attr. postqueries
> and disadvantages (insignificant):
> - query binary (closed) format? no output to string (only to file)
> - query file sensitive to layer indexes (map file cleanups/refinements/some
> normalization can cause query file incompatibility)
> - if server data changed refreshing map image by old url to cgi with
> queryfile parameter doesn't perform requery (need custom http
> handler/module)
> AND ONE PASS QUERY (RFC 52)
> advantages:
> - open query file format
> - speedup (no random access)
> disadvantages (huge):
> - layer's query methods are NOT CUMULATIVE (relative to other layer's
> queries). I CAN'T create COMPLEX queries (by using different
> attribute/spatial queries (metadata driven) for different layers)!
> - query result - some (shape)indexes and when I'm querying many layers in
> sequence I NEED TO PRESERVE QUERY FILE FOR EVERY LAYER (on success result)
> (!) and than on feature attributes (for some layer) demand (delayed request
> - its a normal behavior, for. ex I'm requesting only shape names for some
> layer which has results - to build results tree) I NEED to REQUERY this
> layer by using preserver file! DAMN!
> - there is no way to completely reproduce old mapscript behavior:
>  1. I need to determine primary key ("id") field name (for ex. search for
> "USING UNIQUE") - no need before!
>  2. I need to retrieve all attributes (put to some cache) by
> resultsGetShape to get right oid - it's not time saving operation - no need
> before!
>  3. I need to rerequest (!) feature attributes (or extract from custom
> cache) by layer.getFeature BUT for shapefiles this function implementation
> uses internal index (row position) - no id attribute (!) - and there is no
> function to get it!
>  4. NO WAY TO REPRODUCE COMPLEX QUERYING BY USING QUERYFILE CGI PARAMETER!
> (Am I right?)
>
> _________________________________________________________________________________________
> To SPEED UP random access in old style querying I was able to use "IN
> (...)"
> operator (I suppose that most time critical web apps uses postgis/ms sql
> spatial/oracle spatial which supports it)
> As for pt. 4:
> at this time I decided to output every layer query results to postgis
> service table (with session_id, map_id, layer_id, class_id, feature_id,
> geometry of any type (should I preserve attributes in a BLOB?)) and to
> handle data from this layer with 3 service map file layers
> (point/line/polygon) which populates dynamically (copying class styles with
> replaced colors for highlighting + using layer filter expression + using
> class expressions). I can shrink service table once in a month to support
> old session map files/url's.
> But I've created web UI to update data from browser (drawing, attribute
> editing) - so I need to update service table entries if corresponding
> object
> was updated (if I wand to see actual selection objects)
>
> I'm Disappointed....
> --
> View this message in context:
> http://n2.nabble.com/ONE-PASS-QUERY-RFC-52-FEATURE-OR-BUG-tp4775048p4775048.html
> Sent from the Mapserver - User mailing list archive at Nabble.com.
> _______________________________________________
> mapserver-users mailing list
> mapserver-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20100323/8480d64f/attachment.htm>