[Benchmarking] Test matrix (aka putting tests on a diet)

[Paul Spencer]

On 2009-09-27, at 10:40 AM, Andrea Aime wrote:

> yestdays with Jeff we were talking about the size of the images that
> the tests will use to load the servers, whether it's better to
> hit the servers with common tile size requests (256x256) or common
> screen size request (anything between 640x480 to 1024x768, which also
> includes the common 768x768 base metatile request that tile caches
> love to issue).

personally I'd like to see the tests based on an assumption other than  
serving tile requests.  Tile requests are typically cached for  
performance and only run once to generate the cache, either ahead of  
time in a seeding environment specially optimized for this or  
dynamically with some meta-tiling parameters.  Both of these are  
somewhat special cases that are fine tuned by hand to achieve optimal  
performance, they don't suit more general purpose applications of GIS  
(turning layers on an off, etc).  In the interest of keeping the task  
at hand manageable within the timeframe available, I think that  
picking a size that would suit most non-tiled applications like  
600x400 would be a reasonable compromise.

Cheers

Paul

__________________________________________

    Paul Spencer
    Chief Technology Officer
    DM Solutions Group Inc
    http://research.dmsolutions.ca/