[Foss4g2009] Workshops and tutorials

[Paul Ramsey]

Sayeth Yoda: Read carefully the notes Dave Patton left on workshop
evaluation on wiki.

Once you rank things, particularly if you take the time and effort to
do multiple criteria and scoring, diverging from the ranking becomes
hard.  Allow me to suggest my own game theoretic variant:

- Each committee member picks the 12 workshops he/she thinks should be
included and provides his/her reasoning.
- After everyone has presented, there is an opportunity for people to
change their selections (perhaps they like someones reasonings).
- Then you tot up the number of committee members in favour of each
workshop in order to achieve a "ranking". Then you discuss the 2/4
items clustered around the dividing line and see if there is a
consensus decision on moving them out/in.
- Then you're done.

The hard part about breaking rankings is that for everything you might
want to move *in*, you have to move something *out*, and that doesn't
work often. My scheme has the following properties:

- There is no "scoring" on the part of committee members, so
"plumping" doesn't affect results (only voting for 4 favorites, or
voting with a zero-or-max strategy).
- The "choose 12" scheme forces people to avoid scoring everything in
a narrow wishywashy range from 2.5 to 2.71, a common problem that
washes out difference. Yes, there are hard choices, everyone should
make them together.
- There is a chance to talk before casting results into stone. And
reasoning is presented organically, rather than in fixed bins.
- There is an option to move items around at the end, but it requires consensus.

P.

On Tue, Feb 3, 2009 at 11:29 AM, Cameron Shorter
<cameron.shorter at gmail.com> wrote:
> Mark, I think that process is a good idea.
>
> I'd add that there should be a weighting given to each criteria.
>
> Also, who is going to do the ranking? Are you going to open up voting to all
> OSGeo? We do want equal representation from all attendees, so we might need
> to give certain groups extra weighting in their votes (Eg, the SSI,
> representing local attendees maybe should get higher votes)
>
> Mark Leslie wrote:
>>
>> With the call for workshops and tutorials going out yesterday (presumably)
>> we need to sort out how we should go about ranking and selecting them.  I
>> seem to remember putting together a suggestion but for the life of me I have
>> no idea where I put it.
>>
>> My suggestion is that we basically have some simple guidelines and let the
>> organising committee run with them.  I feel it's more important to exert our
>> will over the workshops since we have fewer of them than previous years and
>> we need to ensure we have a high ratio of beginner workshops, our presenters
>> are as experienced as possible, etc.
>>
>> Tutorials I expect will be a different balance.  24 slots gives us more
>> flexibility, and I would like to see the developers and advanced users
>> catered to as much as beginners.  I  also feel that we should be able to
>> define four clear tracks once we can see what submissions we have to work
>> with.
>>
>> The criteria that I've been discussing to date are (in no particular
>> order):
>>
>> * Presenter experience with the material and FOSS4Gs (more important for
>> workshops)
>> * Beginner oriented (only for workshops)
>> * Fit with the "User Driven" theme
>> * Fit with other tutorials and available tracks (only for tutorials)
>> * Hands-on content (mandatory for workshops, preferred for tutorials)
>> * Balance of user level within the program (only for tutorials)
>> * Quality of the content of the response
>> * Quality of the written description (if they can't write a decent
>> description how will they write a decent tutorial?)
>>
>> Volker was working on an app to allow us to vote on these, which would
>> make life easier, but I think we need to be aware that simply taking the top
>> N in the ranking won't necessarily provide us with the balance of content we
>> need.  I haven't heard if he has finished or not.
>>
>> Any thoughts on our process?
>>
>
>
> --
> Cameron Shorter
> Geospatial Systems Architect
> Tel: +61 (0)2 8570 5050
> Mob: +61 (0)419 142 254
>
> Think Globally, Fix Locally
> Geospatial Solutions enhanced with Open Standards and Open Source
> http://www.lisasoft.com
>
> _______________________________________________
> Foss4g2009 mailing list
> Foss4g2009 at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/foss4g2009
>