Expression Performance was: [ka-Map-users] KaMap performance tuning]

[Stephen Woodbridge]

This showed up on the kaMap list but it seems more appropriate to the 
mapserver list so I am forwarding it. Some times its the little things 
that can kill performance, and they are not always obvious!

Thanks you Percy.

-Steve W

-------- Original Message --------
Subject: [ka-Map-users] KaMap performance tuning
Date: Thu, 24 May 2007 12:53:35 -0700
From: percy <percyd at pdx.edu>
To: ka-map-users at lists.maptools.org
References: <200705231601.l4NG1GnU008459 at duke.maptools.org>

I recently was able to get an extremely complex geologic data shapefile 
set to render much faster (after doing all of the other optimizations 
mentioned by everyone else) by tuning the filter expressions.

I changed all class expressions from the following statement type
CLASS
  NAME 'Dominant:Clastic sediment;'
  EXPRESSION ('[LithComp]' eq 'Dominant:Clastic sediment;')
  STYLE
   COLOR 255 255 0
  END #STYLE
END #CLASS

to

CLASS
  NAME 'Dominant:Clastic sediment;'
  EXPRESSION 'Dominant:Clastic sediment;'

  STYLE
   COLOR 255 255 0

  END #STYLE
END #CLASS

I was stunned at the performance increase this gave me, though it's 
obvious why it helps so much (in hindsight). The only reason I was using 
the former EXPRESSION statements was because I had used QGIS or AMEIN to 
auto-generate the initial mapfile...

Hope this helps,
Percy



Date: Tue, 22 May 2007 19:13:03 -0700
From: Paul Ramsey <pramsey at refractions.net>
Subject: Re: [ka-Map-users] KaMap performance tuning
To: Jeff Dege <jdege at korterra.com>
Cc: ka-map-users at lists.maptools.org
Message-ID: <6207F842-AAF6-44EE-8A58-B08C960D4812 at refractions.net>
Content-Type: text/plain; charset=ISO-8859-1; delsp=yes; format=flowed

10:1 speed advantage for PostGIS is nice and all, but seems unlikely,
even to this PostGIS zealot...

Does your layer have a FILTER on it that uses the attributes? PostGIS
does that great, shapefiles do it terribly.

I would expect on a very large spatial file (your 500MB example) the
PostGIS r-tree will be balanced better than the quadtree .qix file,
but that should provide incremental advantage, not massive
advantage.  One possibility is that you're running out of depth in
your quadtree, maybe tree bumping up the depth in your shptree call.

P


>   
>> We'd never had trouble with the performance of shapefiles, until we  
>> had to deal with one that contained the California streets.    
>> 500MB .shp file, 600MB .dbf file, and performance with it was dog  
>> slow.
>>
>> We pulled out interstates, highways, and main roads, into separate  
>> shapefiles, so that we didn't have to touch the streets data until  
>> we were zoomed in tight, and performance was still dog slow when we  
>> did.
>>
>> We split the streets shapefile into hundreds of separate tiles,  
>> shptreed each, built a tileindex on the bunch, then shptreed the  
>> tileindex, and performance was a good bit better, but still not  
>> acceptable.
>>
>> The people I've been talking to have told me that when rendering  
>> everything, shapefiles are faster, but when rending small subsets  
>> of the data, PostGIS can be considerably faster.  Particularly if  
>> you use it's simplify() function to tune the resolution of the data  
>> returned to the scale of the map.  I've only begun experimenting  
>> with this, but nothing in my experience so far contradicts this.
>>
>> Shp2img, rendering a map with an extent of .04 decimal degrees,  
>> took 17.198 seconds to render the streets layer using (tiled)  
>> shapefiles, it took 0.219 seconds to render the streets layer using  
>> PostGIS.
>>
>>     
-- 
David Percy
Geospatial Data Manager
Geology Department
Portland State University
http://gisgeek.pdx.edu
503-725-3373


_______________________________________________
ka-Map-users mailing list
ka-Map-users at lists.maptools.org
http://lists.maptools.org/mailman/listinfo/ka-map-users