[mapserver-users] Raster data performance

[Stephen Woodbridge]

Fawcett, David (MPCA) wrote:
> I don’t know anything about your work flow, but if possible, you could 
> add a pre-processing step where you use a script to read your text file 
> and pre-classify the data.  You would end up with integers representing 
> each class as ‘cell’ values. 
> 
>  
> 
> Instead of :
> 
> EXPRESSION([pixel] >= -4 AND [pixel] < -3)
> 
>  
> 
> You could have: EXPRESSION “2”

This is the fastest one.

> Or
> 
> EXPRESSION /2/

Be careful with this because the // mean you are doing a regular 
expression evaluation which is slower. and you need to be more precise 
in the expression:

/2/ match a "2" anywhere in the string so would match any of the follow 
where "?" is a wildcard:
"2"
"2?"
"?2"
"2??"
"?2?"
"??2"

If you want to match exactly "2" with a regex then you would need /^2$/

Complex expressions are compiled into structures that have to be 
evaluated for each [pixel] so the more pixels you have the more 
computation you have and the more complex the expression the more 
computation that is involved.

-Steve W

>  
> 
> Matching a single string should be the fastest expression to evaluate.
> 
>  
> 
> David.
> 
>  
> 
> *From:* mapserver-users-bounces at lists.osgeo.org 
> [mailto:mapserver-users-bounces at lists.osgeo.org] *On Behalf Of *Tobias Weiß
> *Sent:* Wednesday, July 28, 2010 8:25 AM
> *To:* mapserver-users at lists.osgeo.org
> *Subject:* Re: [mapserver-users] Raster data performance
> 
>  
> 
> Hi Alex,
> 
> I tried some of the performance tips you mentioned, e.g. converted the 
> .txt to a tiled TIFF to optimize performance.  I can not see significant 
> changes in rendering speed.
> 
> gdal_translate -co TILED=YES input.txt output.tif
> 
> The major brake still is the expression stuff. I really wonder what 
> makes this task so time-consuming, it should not be more than a simple 
> comparison.
> 
> Do you have experiences with this issue? Or do I have to live with this 
> constraint?
> 
> Best, Tobi
> 
> ===
> Am 28.07.10 14:27, schrieb Alexandre Dube:
> 
> Hi Tobi,
> 
>   The following documentation topics about raster might help :
> 
> http://mapserver.org/input/raster.html#rasters-and-tile-indexing
> http://mapserver.org/input/raster.html#raster-display-performance-tips
> 
>   and about expressions :
> 
> http://mapserver.org/mapfile/expressions.html
> 
>   The string comparison type is the fastest, so you could try define 
> more classes but using string comparison to see if it gets faster.  
> Also, a good performance tip is to put your most frequent classes types 
> on top first.  For example, if 80% of your data has "pixel = -3" then 
> put it on top.
> 
> Kind regards,
> 
> Alexandre
> 
> 
> On 10-07-28 04:54 AM, Tobias Weiß wrote:
> 
> Hello list,
> 
> this is my first email to this list. I am working with MapServer, 
> OpenLayers & TileCache for almost 1 year now and the tools are really 
> working great together, when everything is configured correctly ;-)
> 
> At the moment I´m facing a performance problem with raster data that 
> serves as input data to render my map tiles. The spatial resolution of 
> this raster data is 720 (rows) x 360 (cols) and it is stored in a simple 
> .txt file (The size of the file is around 2 MB). I use TileCache to 
> prerender maptiles for different zoom levels that are later displayed in 
> a web application using OpenLayers.
> 
> I discovered, that the rendering speed of MapServer seems to mainly 
> depend on how many different classes I define within my mapfile, meaning 
> how many expressions MapServer has to evaluate during the rendering 
> process. Without any classes MapServer the rendering speed is really 
> fast, the more classes I use the slower it gets. With e.g. 10 different 
> classes defining 10 different styles / pixel colors the rendering time 
> for one tile can increase up to 60 sec.
> 
> 
> CLASS
> 
>   EXPRESSION ([pixel] >= -4 AND [pixel] < -3)
> 
>     STYLE
> 
>         COLOR 69 117 180
> 
>    END
> 
> END
> 
>   
> 
> 
> While working with shape files I haven´t had this problems and 
> prerendering map tiles was really fast. With raster data the rendering 
> time for one map and 5 different zoom levels can take up to several 
> hours and almost overburdens my CPU.
> 
> Can anybody give me a hint how I could improve this?
> 
> Best, Tobi
> 
>  
> 
> _______________________________________________
> 
> mapserver-users mailing list
> 
> mapserver-users at lists.osgeo.org <mailto:mapserver-users at lists.osgeo.org>
> 
> http://lists.osgeo.org/mailman/listinfo/mapserver-users
> 
>   
> 
> 
> 
> 
> -- 
> 
> Alexandre Dubé
> 
> Mapgears
> 
> www.mapgears.com <http://www.mapgears.com>
> 
>   
> 
>  
> 
>  
> 
> _______________________________________________
> 
> mapserver-users mailing list
> 
> mapserver-users at lists.osgeo.org <mailto:mapserver-users at lists.osgeo.org>
> 
> http://lists.osgeo.org/mailman/listinfo/mapserver-users
> 
>   
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> mapserver-users mailing list
> mapserver-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-users