[Benchmarking] Test System Specs

[Paul Ramsey]

Does anyone have any brilliant ideas about how to get around the
apples/oranges issues we will face working in a de-centralized
fashion?

The cleverest idea to date is using EC2 instances, but I fear it may
be too clever by half: first that we have no idea how "real" the
performance characteristics of the instances are; and, more important,
that we certainly can't guarantee that at different times of day we
are getting a truly "equivalent" machine in all characteristics of I/O
and CPU performance. If someone is willing to run a longitudinal study
of EC2 performance, that would allay by fears: basically gather
daisy++ i/o stats every hour for a few days, and also run a CPU
intensive process that requires a fixed number of cycles at a regular
interval and see how much real time it consumes on each run.

A less clever but effective route is to find someone who will host a
benchmarking cluster for us for a few months. A 2-4 core server
machine with a 2-4G memory, and another machine of whatever
specifications for generating load.

P.