WFS performance issue

[Yewondwossen Assefa]

Arnulf,

  I have added the initial support to improve the performance (see
details in bug 1292 :
http://mapserver.gis.umn.edu/bugs/show_bug.cgi?id=1292).

  If you have a chance to do some testing using Postgis or Oracle please
let me know if it adresses some of the performance issues.

Later,

Arnulf Christl wrote:
> Frank Warmerdam wrote:
> [...]
>
>> My understanding is that there are two issues.  One is the unnecessary or
>> confusing spatial query imposed by the .map bbox even when the wfs
>> query has no spatial restriction.  The other is the fact that
>> attribute filtering
>> is done as a post-processing step by mapserver instead of being passed
>> into the RDBMS as a WHERE style filter.
>>
>> I propose to dig into making MapServer queries able to handle NULL
>> query rects in cases where no spatial filtering makes sense.  This can
>> reduce query optimization confusion in some spatial databases (and OGR!)
>> as well as help avoid problems like Tom K. ran into recently with the
>> unexpected effect of his .map EXTENT on WFS queries.
>>
>> Assefa is offering to rewrite WFS attribute queries in a backend specific
>> format (likely just adding support for SQL WHERE for Oracle, PostGIS
>> and OGR) so that the attribute filtering can go down to the database
>> instead of being done in MapServer as a postprocessing step.
>>
>> So I believe it is Assefa's change that will make a several orders
>> of magnitude improvement in performance for your case.  My change
>> might also be helpful in some cases.
>>
>>
>>> The fun thing is that we *can* do it by using MapServer as a warped WMS.
>>> We simply activate the internal (not OGC) FILTER expression which we add
>>> to the WMS getMap as a vendor parameter. Setting DUMP=true you get a
>>> valid GML - which is all we need to gazetteer, highlight, download and
>>> edit the land parcel.
>>
>>
>>
>> Wow, thats pretty neat though I don't follow it completely.  But
>> hopefully
>> we will reach a point where things can be a little easier to optimize
>> "out of the box".
>>
>> Best regards,
>
>
> To get an impression of what we are talking about you can have a look
> here, it went online yesterday.
> http://www.flo.rlp.de/
> Its all in German though, ...
>
> Start with <Demo Zugang> and enter <demo> as user and pwd. The showcase
> boasts a reduced dataset, contents are faked, the aerial photography res
> is a meager 1m/pixel. It is a closed system for around 20T farmers in
> Rhineland Palatinate. Within the closed system aerials have a res of
> 25cm/pixel and the database hosts the full set of 6.5 million land parcels.
>
> Besides rendering the aerials (Mr.Sid), land parcels (Oracle), etc. as
> WMS we use MapServer as WFS for finding land parcels. From the left
> registers select <Flurstuecke> (land parcels) and enter a 4 into the
> field <Stueck> (as it is a demo thats about all you can do). The request
> returns the land parcel as GML, Mapbender transforms it into JavaScript
> onmouseover and click events, so that clicking on it centers the map and
> onmouseover highlights the geometry. The highlighting is done by sending
> the transformed GML to a server side PHP which renders the land parcel
> with a simple red outline and returns the image as overlay. So maybe
> this might be a little slow and impractical across the atlantic. I am
> very interested in performance comments from your side of the pond...
>
> We do this for all searches. It is fantastic how easy things get when
> you just stick to using standards (we just had a discussion on why is
> Open Source using/pushing Open Standards). Dump any geometry into the
> WFS (or slightly clobbered MapServer) and go search for it. The little
> details take the time, if you search for the place name <Koblenz>
> (register Ort) it works if you search for <koblenz> it wont find
> anything. Tribute to a rather dumb shape/dbf behind as data.
>
> Inside we go one step further, the farmer can digitize new objects and
> store them in a central PostgreSQL/PostGIS, currently directly as WKT as
> MapServer doesn't want the T inside WFS. We talked about this one and we
> will probably stick with GeoServer WFS-T - so Barts suggestion might be
> a good idea.
>
> In the lower left corner you find a button named <Schlag> (field). It
> returns all fields farmed by the current user as GML - to be downloaded
> and edited/whatever with udig, jump... This is a simple MapServer WFS
> outlet.
>
> The productive system goes through https and is wrapped by a J2EE
> webstart application that manages user authentication (using an Open
> LDAP) and authorization by attributing a role in Mapbender. Besides it
> load balances the system. The customer got us to duplicate everything(!)
> on one GNU/Linux (quad Xeon) and two Windows servers (double Xeons).
> Imagine all this Mr.Sid and Oracle connectivity crap and compiling not
> on one operating system only but on two different ones... Anyway it
> works fine so that we can attest MapServer to work excellent with both
> OS (if we had a QM section on the MapServer homepage this could be
> published there).
>
> We are really grateful for you guys (all of us) doing all this good
> work. We just fumble and tie the pieces together and the outcome
> invariably is a great and stable application. Sorry for my undamped
> enthusiasm, but I cannot stress this fact often enough!
>
> Best Arnulf.
>
> --
> ------------------------
> Arnulf B. Christl
> ------------------------
> http://www.ccgis.org
> http://www.mapbender.org
> ------------------------
>

--
----------------------------------------------------------------
Assefa Yewondwossen
Software Analyst

Email: assefa at dmsolutions.ca
http://www.dmsolutions.ca/

Phone: (613) 565-5056 (ext 14)
Fax:   (613) 565-0925
----------------------------------------------------------------