<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<!--[if !mso]>
<style>
v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style>
<![endif]-->
<title>RE: [UMN_MAPSERVER-USERS] System Configuration</title>
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"Dutch801 Rm BT";
panose-1:0 0 0 0 0 0 0 0 0 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.EmailStyle19
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle22
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.Section1
{page:Section1;}
/* List Definitions */
@list l0
{mso-list-id:244844146;
mso-list-type:hybrid;
mso-list-template-ids:-247557240 -623608790 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:-;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:.75in;
text-indent:-.25in;
font-family:"Calibri","sans-serif";
mso-fareast-font-family:Calibri;
mso-bidi-font-family:"Times New Roman";}
@list l1
{mso-list-id:951596379;
mso-list-type:hybrid;
mso-list-template-ids:1273527814 121961620 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l1:level1
{mso-level-number-format:bullet;
mso-level-text:-;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:.75in;
text-indent:-.25in;
font-family:"Calibri","sans-serif";
mso-fareast-font-family:Calibri;
mso-bidi-font-family:"Times New Roman";}
@list l2
{mso-list-id:2050177321;
mso-list-template-ids:1103010026;}
@list l2:level1
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Bruce –<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>I am not a MapScript user, so we can hope someone else will help
you with that step. A RAID 10 array is about your best choice for this kind of
situation (the more disks the better), but you’re fundamentally limited in that
only so many simultaneous reads/writes can be handled by one disk. And since
disk seeks are the slowest thing you can do, that’s going to hang you up. If
you have a disk with a 10ms random seek time, and you’re jumping around doing
random, small, reads and writes to satisfy many users, then even if the reads
and writes themselves are free you’re not going to be able to do more than 100
10ms seeks a second.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Switching from mode=browse to mode=map (I’m using the CGI
querystring parameter syntax) will make a huge difference – although it may
well expose something else entirely as your new bottleneck!<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoListParagraph style='margin-left:.75in;text-indent:-.25in;
mso-list:l0 level1 lfo3'><![if !supportLists]><span style='font-size:11.0pt;
font-family:"Calibri","sans-serif";color:#1F497D'><span style='mso-list:Ignore'>-<span
style='font:7.0pt "Times New Roman"'>
</span></span></span><![endif]><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Ed<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Ed McNierney<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Chief Mapmaker<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Demand Media / TopoZone.com<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>73 Princeton Street, Suite 305<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>North Chelmsford, MA 01863<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><a href="mailto:ed@topozone.com">ed@topozone.com</a><o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Phone: +1 (978) 251-4242<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Fax: +1 (978) 251-1396<o:p></o:p></span></p>
<div>
<div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'>
<p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> UMN MapServer Users
List [mailto:MAPSERVER-USERS@LISTS.UMN.EDU] <b>On Behalf Of </b>Bruce Cheney<br>
<b>Sent:</b> Thursday, November 08, 2007 6:31 PM<br>
<b>To:</b> MAPSERVER-USERS@LISTS.UMN.EDU<br>
<b>Subject:</b> Re: [UMN_MAPSERVER-USERS] System Configuration - [SPAM] Email
found in subject<o:p></o:p></span></p>
</div>
</div>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif";
color:blue'>Ed,</span><o:p></o:p></p>
<p class=MsoNormal> <o:p></o:p></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif";
color:blue'>I very much appreciate your help. Very insightful. Yes
expectations are 50,000 users at least at the same time.</span><o:p></o:p></p>
<p class=MsoNormal> <o:p></o:p></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif";
color:blue'>Disk subsystem RAID 10 and NTFS.</span><o:p></o:p></p>
<p class=MsoNormal> <o:p></o:p></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif";
color:blue'>Changing the mode sounds like a very good idea and I am guilty as
charged - I am writing to temporary files on the server. I have a
gut feel that this is the big slowdown in our application. I could not
find where to change the mode that you talk about. Somehow slipping through the
fingers . . . </span><o:p></o:p></p>
<p class=MsoNormal> <o:p></o:p></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif";
color:blue'>I must have to alter this process that I am using: </span><o:p></o:p></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif";
color:blue'>1. Image generated using PHP with calls $map->draw() and
$image->saveWebImage() </span><o:p></o:p></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif";
color:blue'>2. Definition of storage location in the mapfile by
properties ImagePath and ImageURL</span><o:p></o:p></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif";
color:blue'>3. The image name and path are substituted in my HTML
code generation by referencing the name generated in 1 above.</span><o:p></o:p></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif";
color:blue'>What part am I missing? Can I somehow have the image generated
and saved on the users machine directly and reference that instead?
Now I am showing some lack of understanding for sure.</span><o:p></o:p></p>
<p class=MsoNormal> <o:p></o:p></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif";
color:blue'>All your suggestions are great. Makes perfect sense,
anything that can use direct access instead of having to filter and process
improves the speed. </span><o:p></o:p></p>
<p class=MsoNormal> <o:p></o:p></p>
<div>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif";
color:blue'>Thanks again.</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal> <o:p></o:p></p>
</div>
<div>
<p class=MsoNormal> <o:p></o:p></p>
</div>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Dutch801 Rm BT","serif"'>Bruce
Cheney</span><o:p></o:p></p>
<p class=MsoNormal><strong><span style='font-size:10.0pt;font-family:"Arial","sans-serif";
color:navy'>Gateway Mapping, Inc</span></strong><o:p></o:p></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif";
color:#0080FF'><a href="http://www.gatewaymapping.com/">www.gatewaymapping.com</a></span><o:p></o:p></p>
<p class=MsoNormal><em><span style='font-size:10.0pt;font-family:"Arial","sans-serif";
color:gray'>801.221.7656</span></em><o:p></o:p></p>
<div>
<p class=MsoNormal> <o:p></o:p></p>
</div>
<p class=MsoNormal><o:p> </o:p></p>
<div class=MsoNormal align=center style='text-align:center'>
<hr size=2 width="100%" align=center>
</div>
<p class=MsoNormal style='margin-bottom:12.0pt'><b><span style='font-size:10.0pt;
font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;
font-family:"Tahoma","sans-serif"'> Ed McNierney [mailto:ed@topozone.com] <br>
<b>Sent:</b> Thursday, November 08, 2007 2:59 PM<br>
<b>To:</b> Bruce Cheney; MAPSERVER-USERS@LISTS.UMN.EDU<br>
<b>Subject:</b> RE: [UMN_MAPSERVER-USERS] System Configuration - [SPAM] Email
found in subject</span><o:p></o:p></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Bruce –<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Thanks; that’s a good start. 5,000 requests at a time is a
lot. If your estimate the average user will look at a map for 10 seconds
before requesting another one, you’re talking about 50,000 simultaneous human
users.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Thanks for posting the map file – that helps. There is one
huge thing you should be aware of and think about right away – the difference
between mode=browse and mode=map. If you are using browse mode with an
HTML template (as it appears you are) you should reconsider that decision.
When you use MapServer in browse mode, it needs to generate the map image,
legend image, etc. and then <i>write them all to a local disk</i> before
returning an HTML template to the client with the URLs of those images embedded
in it. Writing to disk is the slowest thing you can do on your
server. Having 5,000 simultaneous disk writes, with simultaneous reads to
the same disk, is very nearly pure evil. That is a huge drag on
performance. You really can’t sustain nearly as many users on a given
machine in browse mode as you could in map mode. What kind of disk
subsystem is being used for the temp files?<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Map mode (mode=map) is a simple HTTP request for an image, and
the image, after being built, is sent directly to the client with no disk I/O
involved. <i>Much</i> nicer. <i>Much</i> faster.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>If you must use browse mode, then I’d suggest you create a RAM
disk to hold the temp images, with appropriate monitoring to clean out old
files promptly. But can you pre-build a set of legends, etc. so you can
just embed those and not crank them out every time? You can also get some
benefit from storing our input shapefiles on a RAM disk, but don’t devote any
RAM to that purpose if you’re still writing temp files to hard disk.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Let me first toss off a few generic suggestions that probably
won’t help much but are worth doing:<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoListParagraph style='text-indent:-.25in'><span style='font-size:
11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>1.</span><span
style='font-size:7.0pt;color:#1F497D'> </span><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Get
rid of unused fonts in your font file. MapServer will reach out and touch
each one for each map request.<o:p></o:p></span></p>
<p class=MsoListParagraph style='text-indent:-.25in'><span style='font-size:
11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>2.</span><span
style='font-size:7.0pt;color:#1F497D'> </span><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>If
your shapefiles don’t change often, preprocess them to create a separate shapefile
for each CLASS, so you don’t have to filter as much.<o:p></o:p></span></p>
<p class=MsoListParagraph style='text-indent:-.25in'><span style='font-size:
11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>3.</span><span
style='font-size:7.0pt;color:#1F497D'> </span><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>If
you must use CLASSes, organize them so the most commonly-used class comes first
in each LAYER.<o:p></o:p></span></p>
<p class=MsoListParagraph style='text-indent:-.25in'><span style='font-size:
11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>4.</span><span
style='font-size:7.0pt;color:#1F497D'> </span><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>If
you have a certain number of combinations of shapefiles used in a given request,
create multiple map files with only those layers. For example, if your 12
layers are one set of 4 base layers that are always used, and then 8 more
layers only one of which is displayed at a time, create 8 map files, each with
only 5 layers instead of 12 – the four base layers and one specific overlay
layer. Then use your application code to figure out which map file to
select.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>And make sure you’re using shptree to generate spatial indexes
for all your shapefiles!<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>This looks like it should be an application capable of running
pretty darn quickly.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoListParagraph style='margin-left:.75in;text-indent:-.25in'><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>-</span><span
style='font-size:7.0pt;color:#1F497D'>
</span><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Ed<o:p></o:p></span></p>
<p class=MsoListParagraph style='margin-left:.75in'><span style='font-size:
11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Ed McNierney<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Chief Mapmaker<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Demand Media / TopoZone.com<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>73 Princeton Street, Suite 305<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>North Chelmsford, MA 01863<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><a href="mailto:ed@topozone.com">ed@topozone.com</a><o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Phone: +1 (978) 251-4242<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Fax: +1 (978) 251-1396<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<div>
<div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'>
<p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> UMN MapServer Users
List [mailto:MAPSERVER-USERS@LISTS.UMN.EDU] <b>On Behalf Of </b>Bruce Cheney<br>
<b>Sent:</b> Thursday, November 08, 2007 12:17 PM<br>
<b>To:</b> MAPSERVER-USERS@LISTS.UMN.EDU<br>
<b>Subject:</b> Re: [UMN_MAPSERVER-USERS] System Configuration<o:p></o:p></span></p>
</div>
</div>
<p class=MsoNormal><o:p> </o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Many good
questions. I will see if I can catch them all. A bit on the nature of the
application.</span> <o:p></o:p></p>
<ul type=disc>
<li class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;
mso-list:l2 level1 lfo1'><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>We
are only using vector data (we assumed that the raster would slow it
down).</span> <o:p></o:p></li>
<li class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;
mso-list:l2 level1 lfo1'><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>We
are serving many different maps stored separately with the same
composition of layers. Each map has 6 layers (4 polygon and 2 point layers
for labeling). </span><o:p></o:p></li>
<li class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;
mso-list:l2 level1 lfo1'><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Each
of the different map sets has differing quantities of features. For
the most significant layers they average around 10,000 features but may be
as high as 100,000. </span><o:p></o:p></li>
<li class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;
mso-list:l2 level1 lfo1'><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>A
majority of the requests are to display a small area of one of the maps so
the rendering focuses in to a few features. The user will query a
database which will allow for viewing the map that is zoomed to the area
of interest.</span> <o:p></o:p></li>
<li class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;
mso-list:l2 level1 lfo1'><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>No
layer reprojecting (we assumed this would also slow it down).</span> <o:p></o:p></li>
<li class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;
mso-list:l2 level1 lfo1'><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>The
output map is PNG with dimensions 419 X 403.</span> <o:p></o:p></li>
<li class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;
mso-list:l2 level1 lfo1'><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>We
are using PHP_mapscript to generate the requests. The parameters for
the map generation come from a database and the user requested
location. So there are a few lines of code to find the location on
the map and generate the images.</span> <o:p></o:p></li>
<li class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;
mso-list:l2 level1 lfo1'><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>The
mapfile contains about 12 layers. Several layers to display the
primary polygon layer thematically and a couple extra to show the polygons
with outlines. </span><o:p></o:p></li>
<li class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;
mso-list:l2 level1 lfo1'><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Data
is stored in Shapefiles</span> <o:p></o:p></li>
</ul>
<p class=MsoNormal style='margin-left:.5in'><o:p> </o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>I made
attempts to stream-line the use of extra features to ensure the speed. I
certainly may be using items that hurt instead of help. Here is the
mapfile. Now that I look at the mapfile there may be a couple items that
I originally had intended to use but are now just relics and time wasters.</span><o:p></o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif";color:black'><<postforwebforum.map>>
</span><br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Now as for the
users. We are assuming 5000 simultaneous - all at the same instant. This
would assume a substantially larger group of users accessing the site at the
same time. We assume this to be the peak stress for Stage 1 of the
app. </span><o:p></o:p></p>
<p class=MsoNormal style='margin-bottom:12.0pt'><br>
<br>
<o:p></o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Bruce -</span>
<o:p></o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>My
channeling sensors went off when Frank rang <g>.</span> <o:p></o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>It is
certainly true that a bit of experience and contemplation can help you discover
optimization opportunities that aren't immediately self-evident. Can you
describe the nature of your map application? Are you using raster data,
vector data, or both? What size is your data, in numbers of features
and/or files? What kind of disk subsystem is being used? Is there
layer reprojection going on?</span><o:p></o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Generalizations
are rarely helpful (except for this one). It's like being told the
average man is 5' 7" tall - it tells you nothing about how tall I
am. MapServer performance depends on a number of factors, but the best
place to start is a detailed understanding of what exactly you're trying to do
with MapServer.</span><o:p></o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>It would be
most helpful to us if you could post your map file and a sample URL request,
preferably one that is externally (publicly) visible. And can you define
what you mean by "simultaneous" users? Do you mean 5,000 map
requests all being generated at exactly the same time? Or do you mean
5,000 human users asking for a new map every X seconds or so? And if the
latter, what value are you using for X?</span><o:p></o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>
- Ed</span> <o:p></o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Ed McNierney</span>
<br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Chief Mapmaker</span>
<br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Demand Media /
TopoZone.com</span> <br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>73 Princeton
Street, Suite 305</span> <br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>North
Chelmsford, MA 01863</span> <br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Phone:
978-251-4242, Fax: 978-251-1396</span> <br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>ed@topozone.com</span>
<o:p></o:p></p>
<p class=MsoNormal style='margin-bottom:12.0pt'><o:p> </o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>-----Original
Message-----</span> <br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>From: UMN
MapServer Users List [</span><a href="mailto:MAPSERVER-USERS@LISTS.UMN.EDU"><span
style='font-size:10.0pt;font-family:"Arial","sans-serif"'>mailto:MAPSERVER-USERS@LISTS.UMN.EDU</span></a><span
style='font-size:10.0pt;font-family:"Arial","sans-serif"'>] On Behalf Of Frank
Warmerdam</span> <br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Sent:
Wednesday, November 07, 2007 8:26 PM</span> <br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>To:
MAPSERVER-USERS@LISTS.UMN.EDU</span> <br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Subject: Re:
[UMN_MAPSERVER-USERS] System Configuration</span> <o:p></o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Bruce Cheney
wrote:</span> <br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>> We have
been given a requirement to support 5000 simultaneous users. </span><br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>> What we
are finding is that MapServer bogs down around 400 </span><br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>>
simultaneous users on a test machine. It looks like it is likely </span><br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>> slowing
because of the threading issue. We haven't tested on a </span><br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>> production
machine but are estimating that it should support double </span><br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>> what are
test machine could handle (double the processor and RAM). So </span><br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>> at least
800 simultaneous users. Divide that out with the 5000 and we </span><br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>> need a
minimum of 6-7 web servers supporting MapServer. We will </span><br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>> certainly
scale this as is needed but I do need some idea going in as to what is going to
be required.</span> <o:p></o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Bruce,</span>
<o:p></o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>I'm curious
how many map requests per minute you expect 800 simultaneous users to generate.</span>
<o:p></o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>> Does
this sound like results that others expect or is this quantity </span><br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>> above what
others have tested? Also Does anyone know of a solution in </span><br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>> the works
to run make mapserver thread safe and/or up the overall </span><br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>>
speed? I am not complaining about the speed just wondering what is in </span><br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>> the works.</span>
<o:p></o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>In various
aspects MapServer is already thread safe though there are also known
"unsafe" components, and some components are wrapped by big locks
that significantly reduce the value of multiple threads.</span><o:p></o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Progress
occurs by fits and starts, largely based on support from user organizations
depending on multi-threading. For instance, in 5.0 I implement locking
around OGR for a client of mine in Australia.</span><o:p></o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>(This is a
subtle way of suggesting you hire someone to make this happen if it is what you
want!)</span> <o:p></o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>All this
aside, by default MapServer is *massively multi-threaded*.</span> <br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>I say this
since the default operation is to start a new cgi instance for each request -
each is essentially an independent thread.</span><o:p></o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Of course,
the downside of whole-process cgi style multithreading is that very little
context is preserved from request to request. Map files, data file
headers, etc all need to be reparsed for each request.</span><o:p></o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>My point
here is that you need to think carefully about the application flow to take
much advantage of multiple threading within a single process.</span><o:p></o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Also, if I
may channel Ed, if you wanted to squeeze more performance out of mapserver, you
really need to start by figuring out what it is spending it's time doing.
Where is it spending it's time?</span><o:p></o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'> o
waiting for disk? (perhaps you are reading more data than you need?)</span>
<br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'> o
rendering (perhaps your data is overdense, or you are using expensive</span> <br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>
rendering options?)</span> <br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'> o
parsing mapfiles (perhaps you mapfile has too many unused layers?) etc.</span> <o:p></o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Best
regards,</span> <br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>-- </span><br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>---------------------------------------+--------------------------------</span>
<br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>---------------------------------------+------</span>
<br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>I set the
clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com</span>
<br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>light and sound
- activate the windows | </span><a href="http://pobox.com/~warmerdam"><span
style='font-size:10.0pt;font-family:"Arial","sans-serif"'>http://pobox.com/~warmerdam</span></a>
<br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>and watch the world
go round - Rush | President OSGeo, </span><a
href="http://osgeo.org"><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>http://osgeo.org</span></a>
<o:p></o:p></p>
</div>
</body>
</html>