[Benchmarking] Feedback and some activity, Please!

[Pirmin Kalberer]

Am Dienstag, 25. Juni 2013, 14.02:39 schrieb Arnulf Christl:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hi All,
> thanks for picking this up an pointing out the issues. I think this
> will make up a big part of the benchmark report (if folks allow me to
> suggest this). One result of this benchmark exercise might be that you
> cannot compare apples with penguins - or some such.
> 
> So please keep it coming, I will record and register and propose ideas
> of how to bring things together eventually.
> 
> I am still missing some commitment from the Desktop GIS folks. The
> promise is that we will be honest in every comparison and that it will
> be fun.

In case you mean the QGIS Server team (or maybe ESRI?) with Desktop GIS folks, 
I'm sorry, that we can't commit for this years exercise. We don't have enough 
resources for defining and implementing a new benchmark. My proposal for 
repeating an existing benchmark was in light of your deadline with only the 
Mapserver team in, since a major part of the work would have already been done 
for MapServer and QGIS Server.

Regards
Pirmin

> 
> On 06/25/2013 01:30 PM, Marco Hugentobler wrote:
> > Hi Andrea
> > 
> > Here are csv and jmx files:
> > 
> > https://dl.dropboxusercontent.com/u/45385184/21781.csv
> > https://dl.dropboxusercontent.com/u/45385184/qgis_av_21781.jmx
> > 
> >> Seeing what kind of requests you made (zoom level and image size)
> >> is
> > 
> > going to be interesting.
> > 
> > Image 1000 x 600 - 1500 x 900, Zoom level 1:800 - 1:5'000. Zoom
> > level has been selected because most of the cadasdral layers are
> > visible on that scale. And yes, image encoding is a very important
> > factor in this benchmark. However, I think it is a realistic
> > scenario and image encoding as well as palette generation is indeed
> > a very imortant thing for a WMS server.
> > 
> >> however that really turns the benchmark into a png encoding
> >> contest
> > 
> > In UMN as well as QGIS server, the palette generation for png8 is
> > implemented as part of the server code (it does not come from a
> > 3rd party lib). So I think comparing that in a benchmark makes
> > sense. If a test would be driven by rendering speed, you could also
> > say that the test measures the performance of the underlying
> > graphic lib.
> > 
> > In any case, I agree that benchmark results are very specific and
> > a different test may show a totally different picture.
> > 
> > Regards, Marco
> > 
> > On 25.06.2013 12:01, Andrea Aime wrote:
> >> On Tue, Jun 25, 2013 at 11:13 AM, Marco Hugentobler
> >> <marco.hugentobler at sourcepole.ch
> >> 
> >> <mailto:marco.hugentobler at sourcepole.ch>> wrote:
> >>> So is Stefan shares the data and qgis project, can you share
> >>> the
> >> 
> >> load test itself? I imagine it's some sort of jmeter thing?
> >> 
> >> Yes, it is basically a copy of the foss4g benchmark jmeter file,
> >> only with layers / styles replaced (and outputformat is png8).
> >> Sampling extents were generated with python script from foss4g
> >> benchmark.
> >> 
> >> 
> >> Can you share it? Or, at least, can you share the command line
> >> used to generate it? Seeing what kind of requests you made (zoom
> >> level and image size) is going to be interesting.
> >> 
> >> I understand the usage of png8 to some extent, however that
> >> really turns the benchmark into a png encoding contest, unless
> >> rendering is very slow I'd estimate 50-70% of the time is spent
> >> building the palette and encoding the output PNG (I took the
> >> FOSS4G 2010 benchmarks recently and made some optimization to
> >> GeoServer, now 75% of the time is spent encoding the PNGs (plain
> >> ones, no png8) if I use the fastest PNG encoder available, and it
> >> gets way worse if I leave it with the standard Java PNG encoder.
> >> Now... imagine adding on the fly palette computation on top of
> >> that :-)
> >> 
> >> Cheers Andrea
> >> 
> >> -- == Our support, Your Success! Visit
> >> http://opensdi.geo-solutions.it for more information. ==
> >> 
> >> Ing. Andrea Aime @geowolf Technical Lead
> >> 
> >> GeoSolutions S.A.S. Via Poggio alle Viti 1187 55054  Massarosa
> >> (LU) Italy phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39
> >> 339 8844549
> >> 
> >> http://www.geo-solutions.it http://twitter.com/geosolutions_it
> >> 
> >> -------------------------------------------------------
> >> 
> >> 
> >> _______________________________________________ Benchmarking
> >> mailing list Benchmarking at lists.osgeo.org
> >> http://lists.osgeo.org/mailman/listinfo/benchmarking
> > 
> > -- Dr. Marco Hugentobler Sourcepole -  Linux & Open Source
> > Solutions Weberstrasse 5, CH-8004 Zürich, Switzerland
> > marco.hugentobler at sourcepole.ch http://www.sourcepole.ch Technical
> > Advisor QGIS Project Steering Committee
> > 
> > 
> > 
> > _______________________________________________ Benchmarking
> > mailing list Benchmarking at lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/benchmarking
> 
> - --
> Arnulf Christl
> http://metaspatial.net
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with undefined - http://www.enigmail.net/
> 
> iEYEARECAAYFAlHJht4ACgkQXmFKW+BJ1b3UqACfR4F6EkdTQtBmvkFLTJzVwBiW
> iZgAniMnVpP7NXomZaF+QeGR9PL3n4FJ
> =vkaa
> -----END PGP SIGNATURE-----
> _______________________________________________
> Benchmarking mailing list
> Benchmarking at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/benchmarking
-- 
Pirmin Kalberer
Sourcepole  -  Linux & Open Source Solutions
http://www.sourcepole.com