hardware

[Ed McNierney]

Brent -

But I think that's what I said - I'm sure you saw the results you saw in
one specific situation and one specific configuration comparing one pair
of different machines.  I don't think that justifies the generic
recommendation that step 1 should be to move to Linux.  The choice of
operating system, when placed in the universe of possible performance
optimizations, is probably pretty far down the list after hard disk,
RAM, network, caching, and CPU selection.  And I list those hardware
choices in a very rough order of priority, with the first two being by
far the most important.

I have no experience in comparing AMD vs. Intel CPUs, although I do run
both.

I also have found, subjectively, that in most cases raster and vector
reprojection (tasks TopoZone does all the time) are far more efficient
than expected.

It is entirely possible that Kim's application issue is primarily one of
optimizing hardware for PostgreSQL/PostGIS performance - MapServer may
be the easy part if the application is dominated by database query time.


Part of the problem of asking about hardware recommendations on this
list is that a MapServer/PostgreSQL/PostGIS application is fairly
complex, and there are many such beasts.  It is exceptionally hard to
extrapolate one user's experience to provide useful advice for another
user with a different application.

Except for recommending fast disks and more RAM!

	- Ed

-----Original Message-----
From: Brent Wood [mailto:pcreso at pcreso.com] 
Sent: Tuesday, January 08, 2008 1:02 PM
To: Ed McNierney; MAPSERVER-USERS at LISTS.UMN.EDU
Subject: Re: [UMN_MAPSERVER-USERS] hardware


--- Ed McNierney <ed at TOPOZONE.COM> wrote:

> Brent -
> 
> I have to disagree with such a sweeping generalization concerning
Windows and
> Linux servers.  Having run both Windows and Linux servers for
MapServer,
> MySQL, PostGIS and other applications for several years, I find it
hard to
> claim that on identical hardware one is noticeably faster or slower
than the
> other.  Your mileage can (and obviously does) vary, but since you're
running
> on two different machines your "inferior" system may well be better
suited to
> the load you're giving it than the Windows system.  I'm sure there are
> exceptions, and applications where there IS a noticeable difference
between
> otherwise identical systems, but my experience does not show that a
broad
> statement based on your observations on one pair of servers is
warranted.


Hi Ed, 

I agree in general, but with this particular Mapserver/PostGIS/Java
application, this is exactly the result we had. The tech support was
more
Windows capable than Linux, the result was totally unexpected, and so
far
unexplained. Hence the abstracting of the PostGIS/Mapserver using WMS to
support the application. We found no need to further separate PostGIS &
Mapserver to meet performance needs, but this is obviously feasible.

The performance of a repeat query under Windows & Linux I have tried
several
times, & Linux has been invariably faster than Windows the second time
around,
but as I said, the initial un-cached query times were similar. In
situations
where a substantial amount of of data is in memory vs being read from
disk, I
believe Linux (especially 64bit) will outperform Windows (generally
still
32bit).

Another example, I recently helped put a 16 core mini-cluster for
fisheries
modelling together. (Note: very different loads to a mapserver/PostGIS
application- mostly diskless). We benchmarked AMD vs Intel & Linux vs
Windows,
fairly thoroughly, with surprising (to me, at least) results.

Supposedly similar AMD A64 dual core & Intel Core Duo cpus (according to
published benchmarks) ran very differently, with AMD winning by 15-20%
runtimes
on several hour long iterations. On the same (AMD & Intel) hardware,
Linux beat
Windows by 20-30%. The Windows binary running under wine ran 10-15%
faster than

natively under Windows.

This may differ from your experiences, but that's why a forum like this
usefully allows a variety of input from many people. It's also why
running your
own benchmarks for your own application is a very good idea.

I recommend where possible people do benchmark their own applications,
because
I have found it not uncommon for particular apps to behave unexpectedly,
and
identifying specific hardware bottlenecks can obviously be very useful.

> First, the load Kim is talking about is quite light.  If there are 200
hits
> per hour, and a 3x increase is expected, that's 600 hits per hour or
10 hits
> per minute; one hit every six seconds.

Perhaps, 1.5m points is a rough fixture, but how many are to be rendered
in any
one map? Introducing polygons makes it very hard to know what PostGIS
response
times will be, without knowing exactly how complex the polygons are.
High
resolution coastlines with millions of points per feature vs square
cells. The
former may well require more than 6 seconds to return a query result. Is
PostGIS or Mapserver doing any vector or raster reprojection? Again,
this can
substantially impact performance.

I agree, the simple numbers suggest that the load will not be huge, but
the
main thrust (or at least intent) of my response was that optimising data
structures,
pre-reprojecting data if required, etc, will often yield better
performance
improvements than adding another spindle to an array or a couple of cpu
cores. 


> 
> Second, I think it's very important to separate the MapServer and
PostGIS
> portions of the application, at least for discussion purposes.  The
optimal
> arrangement for each application may not be the same, and there may
need to
> be some compromise if they're to run on one system.

I agree here totally. Note my illustration of abstracting
mapserver/postgis
from the application server. The suggested loads, as you detail them,
don't
really imply that further separation via hardware is really needed, at
least
not without more information as above. And asking for hardware advice
without
an indicative budget is always difficult to answer except in
generalities.

More fast disk & memory :-) 


> Third, I do agree that fast RAID 5 disks and lots of RAM are always a
good
> idea!
> 
> Finally, I think the most important think to remember is to work with
what
> you know.  If your staff are accustomed to and trained in using
Windows
> systems, moving to Linux is hard and expensive - in terms of training,
> consultants, time to resolve issues, etc.  The reverse is equally
true. 
> There is a high cost of moving to an operating system environment you
don't
> know, and there needs to be a very good reason why you're willing to
incur
> that cost.  The suggestion that Kim "go to Linux ASAP" is a very
expensive
> suggestion if the needed Linux skills aren't available.  IMHO, the
first step
> is to ensure optimal hardware design using the operating system Kim's
most
> familiar with.

Except that Kim did say they were looking to do this anyway. My
experience
suggests Linux _may_ offer substantial performance benefits. Quite a
different
situation to where they are a strictly Windows house with no intent to
migrate
at all. In which case I would agree with you 100% :-)


If they have the time & resources, a benchmarking exercise of a
prototype
application to give some real world numbers could be very useful. But
generally
this is unrealistically expensive & people ask lists like this for
advice
instead :-)
 


Cheers,

  Brent