[OSGeo-Discuss] FOSS4G presentation review process

[Volker Mische]

Hi all,

On 10/01/2012 06:10 PM, Barry Rowlingson wrote:
> In our bid for FOSS4G 2013 Nottingham, we didn't precisely say how we
> intended to select presentations for the main track of the conference.
> Some discussion amongst the committee has been going on, and we think
> it necessary to informally poll the community to get a feel for what
> method is preferred.
> 
> Previous FOSS4Gs have not used anonymous reviews (note: the Academic
> Track will be a double-blind review process, we are discussing the
> main conference presentations here), and have used a blend of
> committee reviews and community reviews. Note that even with a
> numerical ranking system its normally still necessary to do a manual
> step to get a balanced conference.
> 
> The big change we could do would be to have anonymous community
> reviews. Proposals would be rated based on title and abstract only.
> The arguments for this include:
> 
>  * selection is on quality of proposal rather than bigness of name
>  * rating procedure can prevent up-votes from whoever has the most
> followers on twitter
>  * promotes inclusivity:
> http://2012.jsconf.eu/2012/09/17/beating-the-odds-how-we-got-25-percent-women-speakers.html
> 
> and against arguments include:
> 
>  * some names are big draws, and it would be disappointing to not have
> someone because their abstract wasn't that exciting.
>  * previous FOSS4Gs have used non-anonymous reviewing and that worked
> fine. Why change it?
>  * it may be hard to distil an exciting talk into an abstract without
> losing the excitement.
> 
> So, as this would be quite a change for FOSS4G, what do you - the
> OSGeo community at large - think? I do have a google poll nearly ready
> on this, but lets have a bit of a debate here and maybe it won't even
> be necessary.

I think an anonymous selection process makes a lot of sense. I
personally always hoped that people don't do a "please up-vote me"
campaigns on blogs or Twitter, but it happened. It will still be
possible as people could publish the titles of the abstract, but I hope
this won't happen and everyone will play along nicely.

One thing we have to keep in mind, that this conference is different
from the JSConf.eu. The JSConf.eu is about the bleeding edge an what's
hot in the fast changing JavaScript world. The audience are definitely
non-beginners. At the FOSS4G the audience is way more wide-spread. It
ranges from beginners to absolute pros. Hence there are also talks that
are kind of the same every year. Things that come to my mind are my own
talks, which are always about GeoCouch, or the "State of ..." talks.
They have a place, but you'd know upfront the the "State of GeoServer"
e.g. is done by one of the big names of GeoServer and respectively a
talk mentioning GeoCouch is probably me. What I want to say is, you
can't fully prevent that people up-vote well known names.

Of course there still needs to be the review process by the programm
committee that makes the final call, so that we e.g. don't have 5 talks
from the same person.

To conclude: I'm in favour of trying it and seeing how it works.

Cheers,
  Volker