Benchmarks

[Arnulf Christl]

Hi,
the questions of benchmarking comes up every now an then. There has been
quite a lot of additional communication going on, some of which the
posters decided to keep off the list because of all the implications Ed
implicated...
:-)
Maybe we should gather some postings covering aspects why it is
pracitcally impossible to produce meaningful and verifiable benchmarks.
I remember that somebody wanted to do that but lost track what came out
of it. Does anybody know or is interested?

Basically it burns down to the problem that geodata and requirements are
too heterogeneous to be comparable so that you can always compare only
one dataset with one specific question in mind (the complexity issue).

Arnulf.

Ed McNierney wrote:

>Ken -
>
>I think the complexity is one issue.  I think the other issue is that
>these are expensive products, and it's hard to see an independent lab
>doing the tests unless the vendors donate their software.  And if ESRI
>is unwilling to publish the results of the tests they control, I'd doubt
>very much that they'd be willing to give free software so someone else
>could publish tests they DON'T control!
>
>        - Ed
>
>-----Original Message-----
>From: UMN MapServer Users List [mailto:MAPSERVER-USERS at LISTS.UMN.EDU] On
>Behalf Of Ken Waters
>Sent: Monday, August 16, 2004 2:33 PM
>To: MAPSERVER-USERS at LISTS.UMN.EDU
>Subject: Re: [UMN_MAPSERVER-USERS] Benchmarks
>
>All,
>
>Last weekend I was at an ESRI seminar, in conjunction with their annual
>conference.  Somehow this question came up.  The ESRI "unofficial"
>answer was that they have such statistics but have elected to not share
>them.  Fair enough.
>
>I think it would be great for someone who is not associated with any
>company or other interests to do an objective test of the different
>solutions.
>Maybe this hasn't happened due to the complexity that Ed refers to here.
>
>Ken
>
>Ed McNierney wrote:
>
>
>
>>Rich -
>>
>>Are you aware of any benchmarks that compare ArcIMS to anything?
>>Benchmarking Internet map servers is a extremely complicated subject,
>>and I have not seen anyone make a serious attempt at doing it for ANY
>>products.
>>
>>        - Ed
>>
>>Ed McNierney
>>President and Chief Mapmaker
>>TopoZone.com / Maps a la carte, Inc.
>>73 Princeton Street, Suite 305
>>North Chelmsford, MA  01863
>>ed at topozone.com
>>(978) 251-4242
>>
>>-----Original Message-----
>>From: UMN MapServer Users List [mailto:MAPSERVER-USERS at LISTS.UMN.EDU]
>>On Behalf Of Rich Binns
>>Sent: Monday, August 16, 2004 1:52 PM
>>To: MAPSERVER-USERS at LISTS.UMN.EDU
>>Subject: [UMN_MAPSERVER-USERS] Benchmarks
>>
>>Was wondering if anyone could direct me to where I could find
>>benchmark info comparing Mapserver to proprietary solutions,
>>
>>
>specifically ArcIMS.
>
>
>


--
----------------------------
CCGIS Christl & Stamm GbR
----------------------------
http://www.ccgis.de
http://www.mapbender.org
----------------------------
Service: +49 (0)228 90826 0
Direct:  +49 (0)228 90826 23
Fax:     +49 (0)228 90826 11
----------------------------
CCGIS Christl & Stamm GbR
Siemensstraße 8
53121 Bonn
GERMANY