FILTER slow

[Fawcett, David]

Stephan, 

I think that any time that you make that many comparisons in a filter or expression, it is going to be relatively expensive.  

PostGIS, with indexes may optimize it a some.  

I am curious how dynamic your data is.  

A common optimization concept is to pre-process and pre-classify your data as much as possible, so you need to do as little processing at the time of the request.  

Since you are hard-coding the address ID's into the filter for your map file, I am guessing that your data isn't too dynamic.  Would it be possible for you to add a column to your shapefile to store a 1 or a 0 to indicate whether the feature should be included into your layer.  Then, pre-classify your data to populate this column based on the address ID's.  

Then, your filter could be a simple regex /1/ or /0/.  Should run very quickly.  

David.

-----Original Message-----
From: UMN MapServer Users List [mailto:MAPSERVER-USERS at LISTS.UMN.EDU] On Behalf Of Stephan Holl
Sent: Thursday, September 13, 2007 6:16 AM
To: MAPSERVER-USERS at LISTS.UMN.EDU
Subject: Re: [UMN_MAPSERVER-USERS] FILTER slow


Hello Andreas,

Andreas Albarello <andreas.albarello at territoriumonline.com>, [20070913
- 13:00:29]

> Stephan Holl wrote:
> >>
> >> FILTER ([adr_id] IN '6446,6366,6483,...')
> > 
> > Thanks for the tip, but unfortunately it does not solve my problem. 
> > The filter does not get applied, it does not show up at all with 
> > your syntax above.
> 
> That's weird, this should indeed work and was confirmed the other day 
> by Lars-Göran Edholm on this mailing list. What Mapserver version are 
> working on? This is a shapefile layer, right? Are you sure you've got 
> it right? Notice the quotes around the comma-separated values (no 
> space between the values)! You could also try to do a string 
> comparison which might be necessary if adr_id is not a purely 
> numerical field:
> 
> FILTER ('[adr_in]' IN '6446,6366,6483')

Thanks for that, I had spaces between the entries... now it works, but I cannot see any speed improvements... Perhaps I should try postgis and greate Querys on-the-fly.

Perhaps there are other suggestions which may lead to more performance? Is it worth to index the shapefile in such cases (about 3000 entries)?

Best
	Stephan

-- 
Stephan Holl <stephan.holl at intevation.de>, http://intevation.de/~stephan
Tel: +49 (0)541-33 50 8 32 | Intevation GmbH | AG Osnabrück - HR B 18998
Geschäftsführer:  Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner