[Benchmarking] Feedback and some activity, Please!

[Arnulf Christl]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi All,
thanks for picking this up an pointing out the issues. I think this
will make up a big part of the benchmark report (if folks allow me to
suggest this). One result of this benchmark exercise might be that you
cannot compare apples with penguins - or some such.

So please keep it coming, I will record and register and propose ideas
of how to bring things together eventually.

I am still missing some commitment from the Desktop GIS folks. The
promise is that we will be honest in every comparison and that it will
be fun.

Cheers,
Arnulf

On 06/25/2013 01:30 PM, Marco Hugentobler wrote:
> Hi Andrea
> 
> Here are csv and jmx files:
> 
> https://dl.dropboxusercontent.com/u/45385184/21781.csv 
> https://dl.dropboxusercontent.com/u/45385184/qgis_av_21781.jmx
> 
>> Seeing what kind of requests you made (zoom level and image size)
>> is
> going to be interesting.
> 
> Image 1000 x 600 - 1500 x 900, Zoom level 1:800 - 1:5'000. Zoom
> level has been selected because most of the cadasdral layers are
> visible on that scale. And yes, image encoding is a very important
> factor in this benchmark. However, I think it is a realistic
> scenario and image encoding as well as palette generation is indeed
> a very imortant thing for a WMS server.
> 
>> however that really turns the benchmark into a png encoding
>> contest
> 
> In UMN as well as QGIS server, the palette generation for png8 is 
> implemented as part of the server code (it does not come from a
> 3rd party lib). So I think comparing that in a benchmark makes
> sense. If a test would be driven by rendering speed, you could also
> say that the test measures the performance of the underlying
> graphic lib.
> 
> In any case, I agree that benchmark results are very specific and
> a different test may show a totally different picture.
> 
> Regards, Marco
> 
> On 25.06.2013 12:01, Andrea Aime wrote:
>> On Tue, Jun 25, 2013 at 11:13 AM, Marco Hugentobler 
>> <marco.hugentobler at sourcepole.ch 
>> <mailto:marco.hugentobler at sourcepole.ch>> wrote:
>> 
>>> So is Stefan shares the data and qgis project, can you share
>>> the
>> load test itself? I imagine it's some sort of jmeter thing?
>> 
>> Yes, it is basically a copy of the foss4g benchmark jmeter file, 
>> only with layers / styles replaced (and outputformat is png8). 
>> Sampling extents were generated with python script from foss4g 
>> benchmark.
>> 
>> 
>> Can you share it? Or, at least, can you share the command line
>> used to generate it? Seeing what kind of requests you made (zoom
>> level and image size) is going to be interesting.
>> 
>> I understand the usage of png8 to some extent, however that
>> really turns the benchmark into a png encoding contest, unless
>> rendering is very slow I'd estimate 50-70% of the time is spent 
>> building the palette and encoding the output PNG (I took the
>> FOSS4G 2010 benchmarks recently and made some optimization to
>> GeoServer, now 75% of the time is spent encoding the PNGs (plain
>> ones, no png8) if I use the fastest PNG encoder available, and it
>> gets way worse if I leave it with the standard Java PNG encoder.
>> Now... imagine adding on the fly palette computation on top of
>> that :-)
>> 
>> Cheers Andrea
>> 
>> -- == Our support, Your Success! Visit
>> http://opensdi.geo-solutions.it for more information. ==
>> 
>> Ing. Andrea Aime @geowolf Technical Lead
>> 
>> GeoSolutions S.A.S. Via Poggio alle Viti 1187 55054  Massarosa
>> (LU) Italy phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39
>> 339 8844549
>> 
>> http://www.geo-solutions.it http://twitter.com/geosolutions_it
>> 
>> -------------------------------------------------------
>> 
>> 
>> _______________________________________________ Benchmarking
>> mailing list Benchmarking at lists.osgeo.org 
>> http://lists.osgeo.org/mailman/listinfo/benchmarking
> 
> 
> -- Dr. Marco Hugentobler Sourcepole -  Linux & Open Source
> Solutions Weberstrasse 5, CH-8004 Zürich, Switzerland 
> marco.hugentobler at sourcepole.ch http://www.sourcepole.ch Technical
> Advisor QGIS Project Steering Committee
> 
> 
> 
> _______________________________________________ Benchmarking
> mailing list Benchmarking at lists.osgeo.org 
> http://lists.osgeo.org/mailman/listinfo/benchmarking
> 


- -- 
Arnulf Christl
http://metaspatial.net
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iEYEARECAAYFAlHJht4ACgkQXmFKW+BJ1b3UqACfR4F6EkdTQtBmvkFLTJzVwBiW
iZgAniMnVpP7NXomZaF+QeGR9PL3n4FJ
=vkaa
-----END PGP SIGNATURE-----