<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Hi Bruce,<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'> On
the “scale relatively quickly” front, you should look at Amazon’s
EC2/S3 services. I’ve recently worked with it and find it an attractive
platform for scaling <a href="http://www.cadmaps.com/gisblog">http://www.cadmaps.com/gisblog</a><o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>The stack I like is Ubuntu+Java+ Postgresql/PostGIS + Apache2
mod_jk Tomcat + Geoserver + custom SVG or XAML clients run out of Tomcat <o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'> If
you use the larger instances the cost is higher but it sounds like you plan on
some heavy raster services (WMS,WCS) and lots of memory will help.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Small EC2 instance provides $0.10/hr:<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";
color:black'>1.7 GB of memory, 1 EC2 Compute Unit (1 virtual core with 1 EC2
Compute Unit), 160 GB of instance storage, 32-bit platform<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Large EC2 instances provide $0.40/hr:<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";
color:black'>7.5 GB of memory, 4 EC2 Compute Units (2 virtual cores with 2 EC2
Compute Units each), 850 GB of instance storage, 64-bit platform</span><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Extra large EC2 instances $0.80/hr:<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";
color:black'>15 GB of memory, 8 EC2 Compute Units (4 virtual cores with 2 EC2
Compute Units each), 1690 GB of instance storage, 64-bit platform<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";
color:black'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Note: that the instances do not need to be permanent. Some people
(WeoGeo) have been using a couple of failover small instances and then starting
new large instances for specific requirements. The idea is to start and stop
instances as required rather than having ongoing infrastructure costs. It only takes
a minute or so to start an ec2 instance. If you are running a corporate service
there may be parts of the day with very little use so you just schedule your
heavy duty instances for peak times. If you can connect your raster to S3
buckets rather than instance storage you have built in replicated backup.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>I know that Java JAI can easily eat up memory and is core to
Geoserver WMS/WCS so you probably want to look at large memory footprint for
any platform with lots of raster service. I’m partial to Geoserver
because of its Java foundation. I think I would try to keep the Apache2
mod_jk Tomcat Geoserver on a separate server instance from PostGIS. This might avoid
problems for instance startup since your database would need to be loaded
separately. The instance ami resides in a 10G partition the balance of data
will probably reside on a /mnt partition separate from ec2-run-instances. You
may be able to avoid datadir problems by adding something like Elastra to the
mix. Elastra beta is a wrapper for PostgreSql that puts the datadir on S3
rather than local to an instance. I suppose they still keep indices(GIST et al)
on the local instance. <o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>(I still think it an interesting exercise to see what could be
done connecting PostGIS to AWS SimpleDB services.)<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>So thinking out loud here is a possible architecture– <o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'> Basic permanent setup<o:p></o:p></span></p>
<p class=MsoNormal style='text-indent:.5in'><span style='font-size:11.0pt;
font-family:"Calibri","sans-serif";color:#1F497D'>put raster in S3 – this
may require some customization of Geoserver, <o:p></o:p></span></p>
<p class=MsoNormal style='text-indent:.5in'><span style='font-size:11.0pt;
font-family:"Calibri","sans-serif";color:#1F497D'>build a datadir in a PostGIS
and backup to S3<o:p></o:p></span></p>
<p class=MsoNormal style='text-indent:.5in'><span style='font-size:11.0pt;
font-family:"Calibri","sans-serif";color:#1F497D'>create a private ami for Postgresql/PostGIS<o:p></o:p></span></p>
<p class=MsoNormal style='text-indent:.5in'><span style='font-size:11.0pt;
font-family:"Calibri","sans-serif";color:#1F497D'>create a private ami for the
load balancer instance<o:p></o:p></span></p>
<p class=MsoNormal style='text-indent:.5in'><span style='font-size:11.0pt;
font-family:"Calibri","sans-serif";color:#1F497D'>create a private ami with
your service stack for both a small and large instance for flexibility, <o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'> Startup services<o:p></o:p></span></p>
<p class=MsoNormal style='text-indent:.5in'><span style='font-size:11.0pt;
font-family:"Calibri","sans-serif";color:#1F497D'>start a balancer instance<o:p></o:p></span></p>
<p class=MsoNormal style='text-indent:.5in'><span style='font-size:11.0pt;
font-family:"Calibri","sans-serif";color:#1F497D'>point your DNS CNAME to this
balancer instance<o:p></o:p></span></p>
<p class=MsoNormal style='text-indent:.5in'><span style='font-size:11.0pt;
font-family:"Calibri","sans-serif";color:#1F497D'>start a PostGis instance (you
could have more than one if necessary but it would be easier to just scale to a
larger instance type if the load demands it)<o:p></o:p></span></p>
<p class=MsoNormal style='text-indent:.5in'><span style='font-size:11.0pt;
font-family:"Calibri","sans-serif";color:#1F497D'>have a scripted download from
an S3 BU to your PostGIS datadir (I’m assuming a relatively static data
resource)<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'> Variable services<o:p></o:p></span></p>
<p class=MsoNormal style='text-indent:.5in'><span style='font-size:11.0pt;
font-family:"Calibri","sans-serif";color:#1F497D'>start service stack instance
and connect to PostGIS<o:p></o:p></span></p>
<p class=MsoNormal style='text-indent:.5in'><span style='font-size:11.0pt;
font-family:"Calibri","sans-serif";color:#1F497D'>update balancer to see new
instance – this could be tricky<o:p></o:p></span></p>
<p class=MsoNormal style='text-indent:.5in'><span style='font-size:11.0pt;
font-family:"Calibri","sans-serif";color:#1F497D'>repeat previous two steps
as needed <o:p></o:p></span></p>
<p class=MsoNormal style='text-indent:.5in'><span style='font-size:11.0pt;
font-family:"Calibri","sans-serif";color:#1F497D'>at night scale back – cron
scaling for a known cycle or use a controller like weoceo to detect and respond
to load fluctuation<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>By the way the public AWS ami with the best resources that I
have found is Ubuntu 7.10 Gutsy. The debian dependency tools are much easier to
use and the resources are plentiful.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>I’ve been toying with using an AWS stack adapted for
serving some larger Postgis vector sets such as fully connected census demographic
data and block polygons here in US. The idea would be to populate the data
directly from the census SF* and TIGER with a background Java bot. There are
some potentially novel 3D viewing approaches possible with xaml. Anyway lots of
fun to have access to virtual systems like this. <o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>As you can see I’m excited anyway.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>randy<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'>
<p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> discuss-bounces@lists.osgeo.org
[mailto:discuss-bounces@lists.osgeo.org] <b>On Behalf Of </b>Bruce.Bannerman@dpi.vic.gov.au<br>
<b>Sent:</b> Monday, February 18, 2008 6:35 PM<br>
<b>To:</b> OSGeo Discussions<br>
<b>Subject:</b> [OSGeo-Discuss] OS Spatial environment 'sizing'<o:p></o:p></span></p>
</div>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>IMO:</span> <br>
<br>
<br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Hello everyone,</span>
<br>
<br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>I'm trying to
get a feel for server 'sizing' for a **hypothetical** Corporate environment to
support OS Spatial apps.</span> <br>
<br>
<br>
<br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Assume that:</span>
<br>
<br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>- this is a
dedicated environment to allow the use of OS Spatial applications to serve
Corporate OGC Services.</span> <br>
<br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>- the
applications of interest are GeoServer, Deegree, GeoNetwork, MapServer,
MapGuide and Postgres/PostGIS. </span><br>
<br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>- the
environment may need to scale relatively quickly. </span><br>
<br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>- it will be
required to serve in the vicinty of 5 to 10 TB of data initially (WMS, WFS,
WCS).</span> <br>
<br>
<br>
<br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Can anyone shed
some light on the following questions please?</span> <br>
<br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>- I'm assuming
a Linux installation (SLES, Redhat or Debian) or possibly Intel Solaris. Has
anyone experienced any issues in these (or other) environments that they'd like
to share?</span> <br>
<br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>- Are there any
recommendations as to dedicated network bandwidth that should be allocated?</span>
<br>
<br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>- Has anyone
done any work with load balancing and would like to share their experiences?</span>
<br>
<br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>- Of the above
OS Spatial products, which ones could co-exist on the same server (excluding Postgres/PostGIS)?</span>
<br>
<br>
<br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Any thoughts
are appreciated.</span> <br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'><br>
<br>
Bruce Bannerman</span> <br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Australia</span><o:p></o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Notice:</span><span
style='background:red'><br>
</span><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>This
email and any attachments may contain information that is personal,
confidential,<br>
legally privileged and/or copyright.</span><span style='font-size:10.0pt'> </span><span
style='font-size:10.0pt;font-family:"Arial","sans-serif"'>No part of it should
be reproduced, adapted or communicated without the prior written consent of the
copyright owner. </span><o:p></o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>It is the
responsibility of the recipient to check for and remove viruses.</span><o:p></o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>If you have
received this email in error, please notify the sender by return email, delete
it from your system and destroy any copies. You are not authorised to use,
communicate or rely on the information contained in this email.</span><o:p></o:p></p>
<p><span style='font-size:10.0pt;font-family:"Arial","sans-serif";color:green'>Please
consider the environment before printing this email.</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p> <o:p></o:p></p>
<p> <o:p></o:p></p>
</div>
</body>
</html>