<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="GENERATOR" content="MSHTML 8.00.6001.19154">
</head>
<body>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011">Hi,</span></font></div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"></span></font> </div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011">Absolutely do
</span></font><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011">not increase RAM, CPU or anything on that line. Instead, install all your data on a small slow laptop and make the system fast on that.</span></font></div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"></span></font> </div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"></span></font><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011">- Analyze your situaltion and think what you want to give for
your customers</span></font></div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"></span></font><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011">- Optimise the data for that purpose</span></font></div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011">- Final step is to acquire as much computing power as needed</span></font></div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"></span></font> </div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011">Let's assume that you are going to publish your data through WMS. You said you have 4 000 000 features and it takes a long time to load them. I believe
it is slow but fortunately you do not need to do it. The computer screen with 1152x864 resolution has only 955 328 pixels so your full screen view is trying to illustrate on average four features per pixel. If someone orders you to draw a map by hand with
a pen and paper in that way you would say it is lunatic and that's correct. The key to draw faster is to draw less.</span></font></div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"></span></font> </div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011">This link shows a course example of speeding vector renderign by making Mapserver to draw less.</span></font></div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"><a href="http://www.paikkatietoikkuna.fi/web/en/map-window?zoomLevel=6&coord=291620_6798042&mapLayers=base_35+100+!default!,34+100+&showMarker=false&forceCache=true">http://www.paikkatietoikkuna.fi/web/en/map-window?zoomLevel=6&coord=291620_6798042&mapLayers=base_35+100+!default!,34+100+&showMarker=false&forceCache=true</a></span></font></div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"></span></font> </div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011">There is a scale dependent group layer "Lan Parcel Register 2010". The group consists or the original data layer and two reduced sets. Original data is shown at
scale 1:100000 or bigger (big scale - small number in scale denominator) and it comes from a shapefile with 1.2 million polygons. In the application the switch to original data happens between named zoom levels "Town" and "Part of town". When zoomed out the
data comes from teh first reduced data set (10% sample) and finally from the second reduced data set (1% sample).
</span></font></div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"></span></font> </div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011">In this case I made the ten percent sample and one percent sample by selecting parcel IDs ending to one zero or two zeros, respectively. This way I can
broadcast the selected parcels uniformly on the whole map area for giving a hint about the places having parcels. There is also a maximum limit for the number of parcels which are rendered (MAXFEATURES 10000). I considered that ten thousand parcels per screen
is always enough.</span></font></div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"></span></font> </div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011">For other data some other basis could be better for reducing the number of features to render. For example it is usually enough to render at maximum a
few hundred biggest lakes an a map and that can be done by sorting data to descending order according to area and then applying MAXFEATURES.</span></font></div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"></span></font> </div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"><span class="737214108-12122011">Another good way for making Mapserver to draw less and thus draw faster is to generalize the geometries to have less vertises.
If you have a hundred pixels on a screen for showing a polygon then it will look just the same for the user with 20 or 20 000 vertises.</span></span></font></div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"><span class="737214108-12122011"></span></span></font> </div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"><span class="737214108-12122011">The mapfile used in the linked example is attached. It is a little bit modified and does not necesarily work straight ahead
but it should show the idea.</span></span></font></div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"><span class="737214108-12122011"></span></span></font> </div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"><span class="737214108-12122011">Here is my ordered list about making vector rendering fast. Other people can suggest re-ordering or adding new items.
I suppose there is something to think about when using classified layers and selecting classes with classitems and expressions.</span></span></font></div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"><span class="737214108-12122011"></span></span></font> </div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"><span class="737214108-12122011">1. Check the spatial indexes for shapefiles and in the database</span></span></font></div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"><span class="737214108-12122011">2. Check other database indexes. In 98 percent of cases when user says that database is slow it just does not have a proper
index for the query.</span></span></font></div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"><span class="737214108-12122011">3. Draw less - less features, less vertices</span></span></font></div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"><span class="737214108-12122011">==============================</span></span></font></div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"><span class="737214108-12122011">4. Draw simple - multiple styles and advanced labels have some cost</span></span></font></div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"><span class="737214108-12122011">5. Re-project data beforehand instead of on-the-fly</span></span></font></div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"><span class="737214108-12122011"></span></span></font> </div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"><span class="737214108-12122011"></span></span></font> </div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"><span class="737214108-12122011">-Jukka Rahkonen-</span></span></font></div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"><span class="737214108-12122011"></span></span></font> </div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"></span></font> </div>
<div dir="ltr" align="left"><font color="#0000ff" size="2" face="Arial"><span class="737214108-12122011"></span></font> </div>
<font color="#0000ff" size="2" face="Arial"></font><font color="#0000ff" size="2" face="Arial"></font><font color="#0000ff" size="2" face="Arial"></font><br>
<blockquote style="BORDER-LEFT: #0000ff 2px solid; PADDING-LEFT: 5px; MARGIN-LEFT: 5px; MARGIN-RIGHT: 0px" dir="ltr">
<div dir="ltr" lang="fi" class="OutlookMessageHeader" align="left">
<hr tabindex="-1">
<font size="2" face="Tahoma">Peter Maes<span class="737214108-12122011"><font color="#0000ff" face="Arial"> wrote:</font></span></font><br>
</div>
<div></div>
Hi Steve,
<div><font color="#0000ff" size="2" face="Arial"></font><font color="#0000ff" size="2" face="Arial"></font><br>
</div>
<div>Thanks for your reply. However, most of my datasources are vector data. I've created a spatial index on the shapefiles, but with 4 000 000 features it still takes a long time to load everything.</div>
<div><br>
</div>
<div>Should I increase ram, or cpu? or Both? the machines have 8 GB ram atm. I'm not sure what mapserver needs in heavy configurations.<br>
<br>
Besides the page on <a href="http://mapserver.org">mapserver.org</a> i dont find much on improving performance for load intensive systems.</div>
<div><br>
<br>
<div class="gmail_quote">2011/12/9 Lime, Steve D (DNR) <span dir="ltr"><<a href="mailto:Steve.Lime@state.mn.us">Steve.Lime@state.mn.us</a>></span><br>
<blockquote style="BORDER-LEFT: #ccc 1px solid; MARGIN: 0px 0px 0px 0.8ex; PADDING-LEFT: 1ex" class="gmail_quote">
<div lang="EN-US" vlink="purple" link="blue">
<div>
<p class="MsoNormal"><span style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">Apache should be plenty quick. Start with using fastcgi and optimizing your data: e.g. create spatial indexes, use optimized raster formats and then optimizing
your configuration (scale dependent layers w/things like reduced resolution layers where possible). -Steve<u></u><u></u></span></p>
<p class="MsoNormal"><span style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt"><u></u><u></u></span> </p>
<div style="BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<p class="MsoNormal"><b><span style="FONT-FAMILY: 'Tahoma','sans-serif'; FONT-SIZE: 10pt">From:</span></b><span style="FONT-FAMILY: 'Tahoma','sans-serif'; FONT-SIZE: 10pt">
<a href="mailto:mapserver-users-bounces@lists.osgeo.org" target="_blank">mapserver-users-bounces@lists.osgeo.org</a> [mailto:<a href="mailto:mapserver-users-bounces@lists.osgeo.org" target="_blank">mapserver-users-bounces@lists.osgeo.org</a>]
<b>On Behalf Of </b>Peter Maes<br>
<b>Sent:</b> Friday, December 09, 2011 6:31 AM<br>
<b>To:</b> <a href="mailto:mapserver-users@lists.osgeo.org" target="_blank">mapserver-users@lists.osgeo.org</a><br>
<b>Subject:</b> [mapserver-users] Mapserver performance on apache - heavy load systems<u></u><u></u></span></p>
</div>
<div>
<div></div>
<div class="h5">
<p class="MsoNormal"><u></u><u></u> </p>
<p class="MsoNormal"><span><span style="FONT-FAMILY: 'Arial','sans-serif'; BACKGROUND: white; FONT-SIZE: 10pt">Hi all,</span></span><u></u><u></u></p>
<div>
<p style="BACKGROUND: white" class="MsoNormal"><span style="FONT-FAMILY: 'Arial','sans-serif'; FONT-SIZE: 10pt"><u></u><u></u></span> </p>
</div>
<div>
<p style="BACKGROUND: white" class="MsoNormal"><span style="FONT-FAMILY: 'Arial','sans-serif'; FONT-SIZE: 10pt">We are planning to launch a major mapserver production site where i'm required to launch many mapserver instances in some form of cluster.<u></u><u></u></span></p>
</div>
<div>
<p style="BACKGROUND: white" class="MsoNormal"><span style="FONT-FAMILY: 'Arial','sans-serif'; FONT-SIZE: 10pt"><u></u><u></u></span> </p>
</div>
<div>
<p style="BACKGROUND: white" class="MsoNormal"><span style="FONT-FAMILY: 'Arial','sans-serif'; FONT-SIZE: 10pt">So I basically have some questions related to this:<u></u><u></u></span></p>
</div>
<div>
<p style="BACKGROUND: white" class="MsoNormal"><span style="FONT-FAMILY: 'Arial','sans-serif'; FONT-SIZE: 10pt"><u></u><u></u></span> </p>
</div>
<div>
<p style="BACKGROUND: white" class="MsoNormal"><span style="FONT-FAMILY: 'Arial','sans-serif'; FONT-SIZE: 10pt">- Does anyone have good tips on performance for running mapserver on apache? <u></u><u></u></span></p>
</div>
<div>
<p style="BACKGROUND: white" class="MsoNormal"><span style="FONT-FAMILY: 'Arial','sans-serif'; FONT-SIZE: 10pt">- Any specific build parameters I should pay attention to? <u></u><u></u></span></p>
</div>
<div>
<p style="BACKGROUND: white" class="MsoNormal"><span style="FONT-FAMILY: 'Arial','sans-serif'; FONT-SIZE: 10pt">- Does it run as a native apache module, or is it required to work through cgi? In which case: will i get better performance by using fastcgi? (we
are not using databases but shapefiles). <u></u><u></u></span></p>
</div>
<div>
<p style="BACKGROUND: white" class="MsoNormal"><span style="FONT-FAMILY: 'Arial','sans-serif'; FONT-SIZE: 10pt">- Should I consider lighttpd or nginx instead?<u></u><u></u></span></p>
</div>
<div>
<p style="BACKGROUND: white" class="MsoNormal"><span style="FONT-FAMILY: 'Arial','sans-serif'; FONT-SIZE: 10pt"><u></u><u></u></span> </p>
</div>
<div>
<p style="BACKGROUND: white" class="MsoNormal"><span style="FONT-FAMILY: 'Arial','sans-serif'; FONT-SIZE: 10pt">- Any other tips related to clustering mapserver instances? :).<u></u><u></u></span></p>
</div>
<div>
<p style="BACKGROUND: white" class="MsoNormal"><span style="FONT-FAMILY: 'Arial','sans-serif'; FONT-SIZE: 10pt"><u></u><u></u></span> </p>
</div>
<div>
<p style="BACKGROUND: white" class="MsoNormal"><span style="FONT-FAMILY: 'Arial','sans-serif'; FONT-SIZE: 10pt">Note that we are using live data, so we cannot cache them, except for a minute or so.<u></u><u></u></span></p>
</div>
<div>
<p style="BACKGROUND: white" class="MsoNormal"><span style="FONT-FAMILY: 'Arial','sans-serif'; FONT-SIZE: 10pt"><u></u><u></u></span> </p>
</div>
<div>
<p style="BACKGROUND: white" class="MsoNormal"><span style="FONT-FAMILY: 'Arial','sans-serif'; FONT-SIZE: 10pt">Thanks for your time.<u></u><u></u></span></p>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
</body>
</html>