[OSGeo-Discuss] 5 Star OSGeo project maturity rating

[P Kishor]

On Mon, Jun 7, 2010 at 9:21 AM, Daniel Morissette
<dmorissette at mapgears.com> wrote:
> Cameron,
>
> AFAIK the objective of incubation is *NOT* to rate the maturity of projects:
> it is to verify that they have an open and active users and developers
> community, open and documented decision and development processes, that the
> source code is free from IP issues, and that as such the project seems
> viable and OSGeo is ready to stand behind it. That's the way incubation
> works today and I don't think we can go beyond that without hurting some
> people and eventually fragmenting our community.
>
> Of course it is easier for more mature projects to pass all those tests, but
> graduating incubation is not a software (source code) maturity indicator.
>
> Nowhere in the incubation process do we evaluate the quality, robustness,
> performance, user-friendlyness, usability, etc. of the software... so I
> repeat it: a star rating based solely on incubation status would mislead the
> users and could have some ill side-effects.
>

I agree with Daniel. Star ratings are misguided. Stars convey a
value-judgment that is neither intentional, nor calculated nor meant
to be conveyed. Nevertheless, a browser looking at a project that is
rated 3 stars versus a project that is rated 5 stars is bound to take
away an "opinion" that was never meant to be given.

Just state clearly what "graduated from incubation" means, indicate
whether a project has graduated or not, and then let the browser/user
decide.


> Daniel
>
> Cameron Shorter wrote:
>>
>> Jason,
>> I agree that it is important that any rating system has had a lot of
>> thought put behind it, which is why I've suggested using the existing OSGeo
>> graduation rating system - which has had input from many of us in the OSGeo
>> community.
>>
>> I do think that Andrea has highlighted a couple of additional points which
>> should be rolled into the OSGeo incubation criteria - but until that
>> happens, we should use what we have, which is guidelines for projects going
>> into incubation (assigned 3 stars), and criteria for projects completing
>> graduation (assigned 4 stars).
>>
>>
>> Bruce Bannerman wrote:
>>>
>>> Jason / Cameron,
>>>
>>> >From the potential utiliser / implementer viewpoint:
>>>
>>> I’d like to think that any project that has graduated OSGeo Incubation
>>> could be considered a quality project with all of the vectors described by
>>> Andrea.
>>>
>>> This proposed rating system implies that this may not be the case.
>>>
>>> Comments?
>>>
>>>
>>> Bruce
>>
>>
>> Daniel Morissette wrote:
>>>
>>> I'm also not too keen on a star ranking system, especially if it is
>>> mostly based on having passed incubation or not.
>>>
>>> To me, passing incubation is more an indication of good process
>>> management and long term viability than an indication of software
>>> quality/robustness and ability to really solve the user's needs. However, a
>>> star ranking system makes me think of hotel/restaurant rating and would
>>> mislead the user to think that a software with 4 stars (because it passed
>>> incubation) does a better job than others with 2 or 3 which is not
>>> necessarily the case.
>>>
>>> If the goal is to denote whether a project has passed incubation or not
>>> then let's call the rating that way (which is what we currently do when we
>>> differentiate between graduated and in-incubation projects on
>>> www.osgeo.org). If we want to create a "project maturity rating" then it
>>> will have to take into account several variables as Andrea wrote earlier...
>>> and then defining those variables and evaluating each piece of software
>>> against them will be quite a task.
>>>
>>> In the end, I just wanted to register the fact that I too am worried
>>> about the possible side-effects of a poorly handled rating system on our
>>> communities.
>>>
>>> Daniel
>>>
>>>
>>> Cameron Shorter wrote:
>>>>
>>>> On 06/06/10 10:14, Jason Birch wrote:
>>>>>
>>>>> IMHO getting into rating projects is just asking for trouble,
>>>>> infighting, bitterness, and people/projects walking away from OSGeo.
>>>>>
>>>>
>>>> Jason, this is a valid concern with decent founding. However I think the
>>>> potential for conflict is not as bad as you may think, and there is a very
>>>> strong user community desire for, and value to be gained from such ratings.
>>>>
>>>> 1. We already have a rating system, based upon:
>>>> * Project has completed incubation
>>>> * Project is in incubation
>>>> * Project is not in incubation
>>>> What I'm suggesting is that we apply a star system to these stages.
>>>>
>>>> 2. We already have a criteria for defining this rating, (which may be
>>>> refined), which reduces the subjectiveness and hence the potential for
>>>> conflict.
>>>>
>>>
>>>
>>
>>
>
>
> --
> Daniel Morissette
> http://www.mapgears.com/
> _______________________________________________
> Discuss mailing list
> Discuss at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/discuss
>



-- 
Puneet Kishor http://www.punkish.org
Carbon Model http://carbonmodel.org
Charter Member, Open Source Geospatial Foundation http://www.osgeo.org
Science Commons Fellow, http://sciencecommons.org/about/whoweare/kishor
Nelson Institute, UW-Madison http://www.nelson.wisc.edu
-----------------------------------------------------------------------
Assertions are politics; backing up assertions with evidence is science
=======================================================================