[OSGeo-Discuss] FOSS4G presentation review process

[Schlagel, Joel D IWR]

I believe anonymous reviews has a place as a component of paper selection - as a compliment to editorial review and professional judgement.    FOSS4G conference is the number one marketing opportunity for the OSGEO community.  We should make a deliberate effort to have a balance between inward focused technical / developer oriented presentations and outward focused policy / success / benefit type good news presentations.   

-joel


________________________________________
From: discuss-bounces at lists.osgeo.org [discuss-bounces at lists.osgeo.org] on behalf of Paul Ramsey [pramsey at opengeo.org]
Sent: Monday, October 01, 2012 2:43 PM
To: Volker Mische
Cc: osgeo-discuss
Subject: Re: [OSGeo-Discuss] FOSS4G presentation review process

I'm in favour too. It has potential, let's see how an anonymous
community process works in practice.

P.

On Mon, Oct 1, 2012 at 11:31 AM, Volker Mische <volker.mische at gmail.com> wrote:
> Hi all,
>
> On 10/01/2012 06:10 PM, Barry Rowlingson wrote:
>> In our bid for FOSS4G 2013 Nottingham, we didn't precisely say how we
>> intended to select presentations for the main track of the conference.
>> Some discussion amongst the committee has been going on, and we think
>> it necessary to informally poll the community to get a feel for what
>> method is preferred.
>>
>> Previous FOSS4Gs have not used anonymous reviews (note: the Academic
>> Track will be a double-blind review process, we are discussing the
>> main conference presentations here), and have used a blend of
>> committee reviews and community reviews. Note that even with a
>> numerical ranking system its normally still necessary to do a manual
>> step to get a balanced conference.
>>
>> The big change we could do would be to have anonymous community
>> reviews. Proposals would be rated based on title and abstract only.
>> The arguments for this include:
>>
>>  * selection is on quality of proposal rather than bigness of name
>>  * rating procedure can prevent up-votes from whoever has the most
>> followers on twitter
>>  * promotes inclusivity:
>> http://2012.jsconf.eu/2012/09/17/beating-the-odds-how-we-got-25-percent-women-speakers.html
>>
>> and against arguments include:
>>
>>  * some names are big draws, and it would be disappointing to not have
>> someone because their abstract wasn't that exciting.
>>  * previous FOSS4Gs have used non-anonymous reviewing and that worked
>> fine. Why change it?
>>  * it may be hard to distil an exciting talk into an abstract without
>> losing the excitement.
>>
>> So, as this would be quite a change for FOSS4G, what do you - the
>> OSGeo community at large - think? I do have a google poll nearly ready
>> on this, but lets have a bit of a debate here and maybe it won't even
>> be necessary.
>
> I think an anonymous selection process makes a lot of sense. I
> personally always hoped that people don't do a "please up-vote me"
> campaigns on blogs or Twitter, but it happened. It will still be
> possible as people could publish the titles of the abstract, but I hope
> this won't happen and everyone will play along nicely.
>
> One thing we have to keep in mind, that this conference is different
> from the JSConf.eu. The JSConf.eu is about the bleeding edge an what's
> hot in the fast changing JavaScript world. The audience are definitely
> non-beginners. At the FOSS4G the audience is way more wide-spread. It
> ranges from beginners to absolute pros. Hence there are also talks that
> are kind of the same every year. Things that come to my mind are my own
> talks, which are always about GeoCouch, or the "State of ..." talks.
> They have a place, but you'd know upfront the the "State of GeoServer"
> e.g. is done by one of the big names of GeoServer and respectively a
> talk mentioning GeoCouch is probably me. What I want to say is, you
> can't fully prevent that people up-vote well known names.
>
> Of course there still needs to be the review process by the programm
> committee that makes the final call, so that we e.g. don't have 5 talks
> from the same person.
>
> To conclude: I'm in favour of trying it and seeing how it works.
>
> Cheers,
>   Volker
> _______________________________________________
> Discuss mailing list
> Discuss at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/discuss
_______________________________________________
Discuss mailing list
Discuss at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/discuss