[OSGeo-Discuss] 5 Star OSGeo project maturity rating

[Daniel Morissette]

Cameron,

AFAIK the objective of incubation is *NOT* to rate the maturity of 
projects: it is to verify that they have an open and active users and 
developers community, open and documented decision and development 
processes, that the source code is free from IP issues, and that as such 
the project seems viable and OSGeo is ready to stand behind it. That's 
the way incubation works today and I don't think we can go beyond that 
without hurting some people and eventually fragmenting our community.

Of course it is easier for more mature projects to pass all those tests, 
but graduating incubation is not a software (source code) maturity 
indicator.

Nowhere in the incubation process do we evaluate the quality, 
robustness, performance, user-friendlyness, usability, etc. of the 
software... so I repeat it: a star rating based solely on incubation 
status would mislead the users and could have some ill side-effects.

Daniel

Cameron Shorter wrote:
> Jason,
> I agree that it is important that any rating system has had a lot of 
> thought put behind it, which is why I've suggested using the existing 
> OSGeo graduation rating system - which has had input from many of us in 
> the OSGeo community.
> 
> I do think that Andrea has highlighted a couple of additional points 
> which should be rolled into the OSGeo incubation criteria - but until 
> that happens, we should use what we have, which is guidelines for 
> projects going into incubation (assigned 3 stars), and criteria for 
> projects completing graduation (assigned 4 stars).
> 
> 
> Bruce Bannerman wrote:
>> Jason / Cameron,
>>
>> >From the potential utiliser / implementer viewpoint:
>>
>> I’d like to think that any project that has graduated OSGeo Incubation 
>> could be considered a quality project with all of the vectors 
>> described by Andrea.
>>
>> This proposed rating system implies that this may not be the case.
>>
>> Comments?
>>
>>
>> Bruce
> 
> 
> Daniel Morissette wrote:
>> I'm also not too keen on a star ranking system, especially if it is 
>> mostly based on having passed incubation or not.
>>
>> To me, passing incubation is more an indication of good process 
>> management and long term viability than an indication of software 
>> quality/robustness and ability to really solve the user's needs. 
>> However, a star ranking system makes me think of hotel/restaurant 
>> rating and would mislead the user to think that a software with 4 
>> stars (because it passed incubation) does a better job than others 
>> with 2 or 3 which is not necessarily the case.
>>
>> If the goal is to denote whether a project has passed incubation or 
>> not then let's call the rating that way (which is what we currently do 
>> when we differentiate between graduated and in-incubation projects on 
>> www.osgeo.org). If we want to create a "project maturity rating" then 
>> it will have to take into account several variables as Andrea wrote 
>> earlier... and then defining those variables and evaluating each piece 
>> of software against them will be quite a task.
>>
>> In the end, I just wanted to register the fact that I too am worried 
>> about the possible side-effects of a poorly handled rating system on 
>> our communities.
>>
>> Daniel
>>
>>
>> Cameron Shorter wrote:
>>> On 06/06/10 10:14, Jason Birch wrote:
>>>> IMHO getting into rating projects is just asking for trouble, 
>>>> infighting, bitterness, and people/projects walking away from OSGeo.
>>>>
>>>
>>> Jason, this is a valid concern with decent founding. However I think 
>>> the potential for conflict is not as bad as you may think, and there 
>>> is a very strong user community desire for, and value to be gained 
>>> from such ratings.
>>>
>>> 1. We already have a rating system, based upon:
>>> * Project has completed incubation
>>> * Project is in incubation
>>> * Project is not in incubation
>>> What I'm suggesting is that we apply a star system to these stages.
>>>
>>> 2. We already have a criteria for defining this rating, (which may be 
>>> refined), which reduces the subjectiveness and hence the potential 
>>> for conflict.
>>>
>>
>>
> 
> 


-- 
Daniel Morissette
http://www.mapgears.com/