Expression Performance was: [ka-Map-users] KaMap performance tuning]

[Jan Hartmann]

Actually, it is in the documentation of the "expression" keyword in the 
"class" reference of the Mapfile reference 
(http://mapserver.gis.umn.edu/docs/reference/mapfile/class):

"As you might expect this level of complexity is slower to process" 
(i.e. logical expressions versus simple strings).

Jan

Stephen Woodbridge wrote:
> This showed up on the kaMap list but it seems more appropriate to the 
> mapserver list so I am forwarding it. Some times its the little things 
> that can kill performance, and they are not always obvious!
> 
> Thanks you Percy.
> 
> -Steve W
> 
> -------- Original Message --------
> Subject: [ka-Map-users] KaMap performance tuning
> Date: Thu, 24 May 2007 12:53:35 -0700
> From: percy <percyd at pdx.edu>
> To: ka-map-users at lists.maptools.org
> References: <200705231601.l4NG1GnU008459 at duke.maptools.org>
> 
> I recently was able to get an extremely complex geologic data shapefile 
> set to render much faster (after doing all of the other optimizations 
> mentioned by everyone else) by tuning the filter expressions.
> 
> I changed all class expressions from the following statement type
> CLASS
>  NAME 'Dominant:Clastic sediment;'
>  EXPRESSION ('[LithComp]' eq 'Dominant:Clastic sediment;')
>  STYLE
>   COLOR 255 255 0
>  END #STYLE
> END #CLASS
> 
> to
> 
> CLASS
>  NAME 'Dominant:Clastic sediment;'
>  EXPRESSION 'Dominant:Clastic sediment;'
> 
>  STYLE
>   COLOR 255 255 0
> 
>  END #STYLE
> END #CLASS
> 
> I was stunned at the performance increase this gave me, though it's 
> obvious why it helps so much (in hindsight). The only reason I was using 
> the former EXPRESSION statements was because I had used QGIS or AMEIN to 
> auto-generate the initial mapfile...
> 
> Hope this helps,
> Percy
> 
> 
> 
> Date: Tue, 22 May 2007 19:13:03 -0700
> From: Paul Ramsey <pramsey at refractions.net>
> Subject: Re: [ka-Map-users] KaMap performance tuning
> To: Jeff Dege <jdege at korterra.com>
> Cc: ka-map-users at lists.maptools.org
> Message-ID: <6207F842-AAF6-44EE-8A58-B08C960D4812 at refractions.net>
> Content-Type: text/plain; charset=ISO-8859-1; delsp=yes; format=flowed
> 
> 10:1 speed advantage for PostGIS is nice and all, but seems unlikely,
> even to this PostGIS zealot...
> 
> Does your layer have a FILTER on it that uses the attributes? PostGIS
> does that great, shapefiles do it terribly.
> 
> I would expect on a very large spatial file (your 500MB example) the
> PostGIS r-tree will be balanced better than the quadtree .qix file,
> but that should provide incremental advantage, not massive
> advantage.  One possibility is that you're running out of depth in
> your quadtree, maybe tree bumping up the depth in your shptree call.
> 
> P
> 
> 
>>  
>>> We'd never had trouble with the performance of shapefiles, until we  
>>> had to deal with one that contained the California streets.    500MB 
>>> .shp file, 600MB .dbf file, and performance with it was dog  slow.
>>>
>>> We pulled out interstates, highways, and main roads, into separate  
>>> shapefiles, so that we didn't have to touch the streets data until  
>>> we were zoomed in tight, and performance was still dog slow when we  
>>> did.
>>>
>>> We split the streets shapefile into hundreds of separate tiles,  
>>> shptreed each, built a tileindex on the bunch, then shptreed the  
>>> tileindex, and performance was a good bit better, but still not  
>>> acceptable.
>>>
>>> The people I've been talking to have told me that when rendering  
>>> everything, shapefiles are faster, but when rending small subsets  of 
>>> the data, PostGIS can be considerably faster.  Particularly if  you 
>>> use it's simplify() function to tune the resolution of the data  
>>> returned to the scale of the map.  I've only begun experimenting  
>>> with this, but nothing in my experience so far contradicts this.
>>>
>>> Shp2img, rendering a map with an extent of .04 decimal degrees,  took 
>>> 17.198 seconds to render the streets layer using (tiled)  shapefiles, 
>>> it took 0.219 seconds to render the streets layer using  PostGIS.
>>>
>>>