WFS performance issue

[Arnulf Christl]

Frank Warmerdam wrote:
[...]
> My understanding is that there are two issues.  One is the unnecessary or
> confusing spatial query imposed by the .map bbox even when the wfs
> query has no spatial restriction.  The other is the fact that
> attribute filtering
> is done as a post-processing step by mapserver instead of being passed
> into the RDBMS as a WHERE style filter.
>
> I propose to dig into making MapServer queries able to handle NULL
> query rects in cases where no spatial filtering makes sense.  This can
> reduce query optimization confusion in some spatial databases (and OGR!)
> as well as help avoid problems like Tom K. ran into recently with the
> unexpected effect of his .map EXTENT on WFS queries.
>
> Assefa is offering to rewrite WFS attribute queries in a backend specific
> format (likely just adding support for SQL WHERE for Oracle, PostGIS
> and OGR) so that the attribute filtering can go down to the database
> instead of being done in MapServer as a postprocessing step.
>
> So I believe it is Assefa's change that will make a several orders
> of magnitude improvement in performance for your case.  My change
> might also be helpful in some cases.
>
>
>>The fun thing is that we *can* do it by using MapServer as a warped WMS.
>>We simply activate the internal (not OGC) FILTER expression which we add
>>to the WMS getMap as a vendor parameter. Setting DUMP=true you get a
>>valid GML - which is all we need to gazetteer, highlight, download and
>>edit the land parcel.
>
>
> Wow, thats pretty neat though I don't follow it completely.  But hopefully
> we will reach a point where things can be a little easier to optimize
> "out of the box".
>
> Best regards,

To get an impression of what we are talking about you can have a look
here, it went online yesterday.
http://www.flo.rlp.de/
Its all in German though, ...

Start with <Demo Zugang> and enter <demo> as user and pwd. The showcase
boasts a reduced dataset, contents are faked, the aerial photography res
is a meager 1m/pixel. It is a closed system for around 20T farmers in
Rhineland Palatinate. Within the closed system aerials have a res of
25cm/pixel and the database hosts the full set of 6.5 million land parcels.

Besides rendering the aerials (Mr.Sid), land parcels (Oracle), etc. as
WMS we use MapServer as WFS for finding land parcels. From the left
registers select <Flurstuecke> (land parcels) and enter a 4 into the
field <Stueck> (as it is a demo thats about all you can do). The request
returns the land parcel as GML, Mapbender transforms it into JavaScript
onmouseover and click events, so that clicking on it centers the map and
onmouseover highlights the geometry. The highlighting is done by sending
the transformed GML to a server side PHP which renders the land parcel
with a simple red outline and returns the image as overlay. So maybe
this might be a little slow and impractical across the atlantic. I am
very interested in performance comments from your side of the pond...

We do this for all searches. It is fantastic how easy things get when
you just stick to using standards (we just had a discussion on why is
Open Source using/pushing Open Standards). Dump any geometry into the
WFS (or slightly clobbered MapServer) and go search for it. The little
details take the time, if you search for the place name <Koblenz>
(register Ort) it works if you search for <koblenz> it wont find
anything. Tribute to a rather dumb shape/dbf behind as data.

Inside we go one step further, the farmer can digitize new objects and
store them in a central PostgreSQL/PostGIS, currently directly as WKT as
MapServer doesn't want the T inside WFS. We talked about this one and we
will probably stick with GeoServer WFS-T - so Barts suggestion might be
a good idea.

In the lower left corner you find a button named <Schlag> (field). It
returns all fields farmed by the current user as GML - to be downloaded
and edited/whatever with udig, jump... This is a simple MapServer WFS
outlet.

The productive system goes through https and is wrapped by a J2EE
webstart application that manages user authentication (using an Open
LDAP) and authorization by attributing a role in Mapbender. Besides it
load balances the system. The customer got us to duplicate everything(!)
on one GNU/Linux (quad Xeon) and two Windows servers (double Xeons).
Imagine all this Mr.Sid and Oracle connectivity crap and compiling not
on one operating system only but on two different ones... Anyway it
works fine so that we can attest MapServer to work excellent with both
OS (if we had a QM section on the MapServer homepage this could be
published there).

We are really grateful for you guys (all of us) doing all this good
work. We just fumble and tie the pieces together and the outcome
invariably is a great and stable application. Sorry for my undamped
enthusiasm, but I cannot stress this fact often enough!

Best Arnulf.

--
------------------------
Arnulf B. Christl
------------------------
http://www.ccgis.org
http://www.mapbender.org
------------------------