[Benchmarking] Test matrix (aka putting tests on a diet)

[Shanmugam Ganeshkumar]

Paul +1
A typical WMS request is not tile based. The real world experience varies
anything between 600x400 (512x512, if you want to consider tile) to 1024x768
(4x3 tiles). As Paul puts it, tiles based measurement is good only for tile
based service (GWC, TileCache, ArcGIS TMS). If the benchmarking is on the
GS, MS and ArcGIS and WMS/WFS as protocols then you should consider one
agreed tile size and do more fine grain tests rather than diluting by
testing few tile sizes. Make sure that content is NOT cached  to measure the
real throughput of the server.

imo, 600x400 is still on the lower side of the size, 800x600 is practical
(slightly more than 3 tiles x 2tiles) and making it meaningful for vector
rendering also. If one wants to play safe on system resources, then 600x400
is an ideal size.

When you do the benchmarking on the cached content, then one should use
256x256 since thats the tile size it stores. Then you can measure
the throughput as, how many tiles per second.

my 2 cents.


Shanmugam

Geospatial Consultant
Singapore


2009/9/27 Paul Spencer <pspencer at dmsolutions.ca>

>
> On 2009-09-27, at 10:40 AM, Andrea Aime wrote:
>
>  yestdays with Jeff we were talking about the size of the images that
>> the tests will use to load the servers, whether it's better to
>> hit the servers with common tile size requests (256x256) or common
>> screen size request (anything between 640x480 to 1024x768, which also
>> includes the common 768x768 base metatile request that tile caches
>> love to issue).
>>
>
> personally I'd like to see the tests based on an assumption other than
> serving tile requests.  Tile requests are typically cached for performance
> and only run once to generate the cache, either ahead of time in a seeding
> environment specially optimized for this or dynamically with some
> meta-tiling parameters.  Both of these are somewhat special cases that are
> fine tuned by hand to achieve optimal performance, they don't suit more
> general purpose applications of GIS (turning layers on an off, etc).  In the
> interest of keeping the task at hand manageable within the timeframe
> available, I think that picking a size that would suit most non-tiled
> applications like 600x400 would be a reasonable compromise.
>
> Cheers
>
> Paul
>
> __________________________________________
>
>   Paul Spencer
>   Chief Technology Officer
>   DM Solutions Group Inc
>   http://research.dmsolutions.ca/
>
>
> _______________________________________________
> Benchmarking mailing list
> Benchmarking at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/benchmarking
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/benchmarking/attachments/20090927/ada24d44/attachment.html