[OSGeo-Discuss] How to assess rendering quality?

Michael P. Gerlek mpg at lizardtech.com
Wed Oct 10 09:34:57 PDT 2007 

All I could add to what Ed and Traian have said is:

- try to keep the tests neutral in all other respects: if you can, you
  want to minimize any indications of what tool the image came from, remove
  extraneous toolbars and scrollbars, etc -- anything subconsciously biasing
  or visually distracting

- the quality (and settings) of the display monitor are very important.  If
  you can, do the tests on the same monitor with the same lighting in the
  room

-mpg

 

> -----Original Message-----
> From: discuss-bounces at lists.osgeo.org 
> [mailto:discuss-bounces at lists.osgeo.org] On Behalf Of Traian Stanev
> Sent: Wednesday, October 10, 2007 7:26 AM
> To: OSGeo Discussions
> Subject: RE: [OSGeo-Discuss] How to assess rendering quality?
> 
> 
> Antialiasing, when done wrong is just "making the edges 
> intermediate colors".
> 
> When done right it also involves subpixel positioning which 
> improves not only the visual appearance but also the relative 
> accuracy of lines -- i.e. visual weight is most correctly 
> distributed over exact position of the line given the 
> discrete sampling frequency (pixels). There is actually solid 
> scientific basis in that (Niquist-Shannon theorem). 
> "Sharpness of detail" in non-AA lines is fake accuracy.
> 
> Unfortunately antialiasing is also hard to get right, because 
> it depends on gamma correction. For example compare the 
> antialiased output here 
> (http://www.realtimerendering.com/gamma10.png with gamma = 
> 1.0) and here with gamma 2.2 
> (http://www.realtimerendering.com/gamma22.png). They are both 
> antialiased using the same algorithm, yet one looks better 
> than the other.
> 
> Asking people to compare smooth versus sharp will be 
> sensitive not only to the subjective appeal of the picture, 
> but also by the ambient lighting conditions, the gamma used 
> by the AA algorithm versus the gamma of the monitor used for 
> the test, and by whether or not the AA algorithm used 
> subpixel positioning. Also, it depends on whether the image 
> reading application honors the gamma value stored in the 
> image. In addition, you will get "fanboy bias" where people 
> will prefer the output they have seen before. This comes up 
> with things like font glyph rasterization surveys, where 
> people who are more familiar with Macs prefer blurry but 
> correct glyphs while people who use predominantly Windows 
> prefer sharp but deformed glyphs, only because that's what 
> they are used to seeing. So you will need to have lots of 
> questions that ask for the same thing using screenshots that 
> do not say which product they are from. You should also 
> include output from a third party app like Adobe Acrobat. I 
> would also include test questions where the two images being 
> compared are identical.
> 
> 
> 
> Traian
> 
> 
> 
> 
> 
> 
> > -----Original Message-----
> > From: discuss-bounces at lists.osgeo.org [mailto:discuss-
> > bounces at lists.osgeo.org] On Behalf Of Ed McNierney
> > Sent: Wednesday, October 10, 2007 9:02 AM
> > To: OSGeo Discussions
> > Subject: RE: [OSGeo-Discuss] How to assess rendering quality?
> > 
> > Gilles -
> > 
> > Just keep in mind that subjective metrics are, after all, 
> subjective,
> > and some of your metrics are mutually exclusive.  
> "Smoothness of lines"
> > is normally accomplished by antialiasing those lines, 
> making the edges
> > intermediate colors and a little "soft".  This is a good 
> thing, but is
> > incompatible with "sharpness of details", which is best accomplished
> > without antialiasing but with more jagged artifacts on curves and
> > diagonal lines.
> > 
> > 	- Ed
> > 
> > Ed McNierney
> > Chief Mapmaker
> > Demand Media / TopoZone.com
> > 73 Princeton Street, Suite 305
> > North Chelmsford, MA  01863
> > Phone: 978-251-4242, Fax: 978-251-1396
> > ed at topozone.com
> > 
> > 
> > 
> > 
> > -----Original Message-----
> > From: discuss-bounces at lists.osgeo.org [mailto:discuss-
> > bounces at lists.osgeo.org] On Behalf Of Gilles Bassière
> > Sent: Wednesday, October 10, 2007 8:38 AM
> > To: OSGeo Discussions
> > Subject: Re: [OSGeo-Discuss] How to assess rendering quality?
> > 
> > Hi Michael,
> > 
> > I'm more concerned with subjective metrics, I actually plan 
> to survey
> > some users. The questionnaire will include some map samples 
> and gather
> > user preference for each criteria. My problem is to identify what
> > questions/criteria should I ask to a user.
> > 
> > Gilles
> > 
> > 
> > Michael P. Gerlek wrote:
> > > Gilles-
> > >
> > > Is your idea to measure the quality by having a human 
> look at outputs
> > (subjective metrics) or automatically via some analysis routine
> > (objective metrics)?
> > >
> > > -mpg
> > >
> > >
> > >
> > >
> > >> -----Original Message-----
> > >> From: discuss-bounces at lists.osgeo.org
> > >> [mailto:discuss-bounces at lists.osgeo.org] On Behalf Of Gilles
> > Bassière
> > >> Sent: Wednesday, October 10, 2007 3:01 AM
> > >> To: discuss at lists.osgeo.org
> > >> Subject: [OSGeo-Discuss] How to assess rendering quality?
> > >>
> > >> Hi list,
> > >>
> > >> I'm doing a comparative study of OpenSource cartographic servers
> > >> (Mapserver, Geoserver and Mapnik). Beside raw performance and
> > >> features,
> > >> I'd like to assess the rendering quality, say how pretty
> > >> produced maps
> > >> are. Precisely, I'm interested in the quality of the 
> drawing work,
> > my
> > >> point is not about symbology, nor styling of maps.
> > >>
> > >> I have some problems to find a set of objective criteria I could
> > >> benchmark my servers against. So far, I have already 
> identified the
> > >> following:
> > >> - sharpness of details
> > >> - smoothness of lines
> > >> - uniformity of colors
> > >>
> > >> I'm open to any comments. Do you think these criteria 
> are consistent
> > >> regarding the purpose of my study? Does anyone have 
> other criteria
> > to
> > >> suggest?
> > >>
> > >> Regards
> > >>
> > >> --
> > >> Gilles Bassiere
> > >> MAKINA CORPUS
> > >> 30 rue des Jeuneurs
> > >> FR-75011 PARIS
> > >> http://www.makina-corpus.com
> > >>
> > >> _______________________________________________
> > >> Discuss mailing list
> > >> Discuss at lists.osgeo.org
> > >> http://lists.osgeo.org/mailman/listinfo/discuss
> > >>
> > >>
> > > _______________________________________________
> > > Discuss mailing list
> > > Discuss at lists.osgeo.org
> > > http://lists.osgeo.org/mailman/listinfo/discuss
> > >
> > 
> > 
> > --
> > Gilles Bassiere
> > MAKINA CORPUS
> > 30 rue des Jeuneurs
> > FR-75011 PARIS
> > http://www.makina-corpus.com
> > 
> > _______________________________________________
> > Discuss mailing list
> > Discuss at lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/discuss
> > _______________________________________________
> > Discuss mailing list
> > Discuss at lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/discuss
> 
> 
> _______________________________________________
> Discuss mailing list
> Discuss at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/discuss
>

More information about the Discuss mailing list