[FOSS4G2016] [Program] Abstract review process draft

[Steven Feldman]

My thanks also to Volker and the team selecting the programme. I know how difficult it is

Cheers
______
Steven

NB - this mail is not influenced by having 2 talks accepted :) Can’t wait for August

> On 28 Apr 2016, at 20:49, Marc Vloemans <marcvloemans1 at gmail.com> wrote:
> 
> Hi
> Although I have only voted as a community member, I really applaud the care you guys have applied in approaching this delicate matter!!!
> 
> We present a very well balanced program.
> 
> Some typos aside, send it out as-is. It is the thought(fullness) that counts. Kudos to Volker et al.
> 
> Vriendelijke groet,
> Marc Vloemans
> 
> 
>> Op 28 apr. 2016 om 21:37 heeft Volker Mische <volker.mische at gmail.com> het volgende geschreven:
>> 
>> Hi LOC and Program-Committee,
>> 
>> here's the draft of a blog post I'd like to publish tomorrow morning, so
>> there isn't much time for a review :) It's about our review process.
>> This blog post will be mentioned in the acceptance/decline emails I'll
>> also send out tomorrow.
>> 
>> Cheers,
>> Volker
>> 
>> 
>> Abstract review process for FOSS4G 2016
>> =======================================
>> 
>> The final selection of the presentations for the FOSS4G used to be a
>> controversy. Reviewing so many abstracts takes a lot of effort, so it's
>> not possible to tell the rejected presenters in detail why their
>> abstract wasn't accepted. Though it's possible to be transparent about
>> the process that was applied. That's what this blog post is about.
>> 
>> Let's start with some numbers. There were 281 abstracts submitted, 181
>> were accepted. There was a community review and a separate review by the
>> program committee. Both reviews were blind, without knowing the author's
>> name.
>> 
>> Next the program committee met several times using a Google Docs
>> spreadsheet which had the community rank, reviewer's rank and the
>> submitted details in it. This time it was including the author's name.
>> 
>> 
>> The selection process
>> ---------------------
>> 
>> ### Step 1: Get rid of the lowest ranked ones
>> 
>> We were picking a number to eliminate the abstracts that had *both* a
>> low rank (less thank 180) by the community and by the reviewers. This
>> way we got the numbers down by 47 to 233.
>> 
>> 
>> ### Step 2: Go through highest ranked presentations and limit those by
>> author and company
>> 
>> Only the abstracts where both the community as well as the program
>> committee were ranking them above 180 were taken into account, that were
>> 142 talks. We went trough them to limit the number of presentations to a
>> maximum of two per person (with exceptions: some people are so active in
>> the FOSS4G community, that they represent so many different projects, so
>> that it make sense to have them give more than two talks. We also
>> limited the number of talks by company. This wasn't a hard-number, but
>> depending on how many different topics were covered. This reduced the
>> total talks by 11 to around 222.
>> 
>> 
>> ### Step 3: Go through the "undecided" ones
>> 
>> Those presentations where the community and the program committee didn't
>> agree on got special care. Those were about 90 talks. We sorted them by
>> the community voting. Then everyone from the committee was able to argue
>> for talks being in our out. This also included talks being excluded
>> because of the limitations outlined in step 2. This was the most time
>> consuming step which lead to a lot of good discussions. In the end it
>> brought the total count down by 41 to the 181 we accepted.
>> 
>> 
>> ### Step 4: Group talks by topic
>> 
>> As we were meeting already twice, we decided to take the grouping
>> offline. All members were working on the spreadsheet of the final
>> selection and adding tags of what the talks are about. The program
>> committee chair then took the time to do the final grouping for the website.
>> 
>> 
>> ### Impact of the community vote
>> 
>> If you compare the top 181 talks from the community voting with the
>> final program the overlap is 77% (when taking the talks removed in step
>> 2 into account it's even 80%).
>> 
>> When the committee made decisions that were contrary to the community
>> voting, it had numerous reasons. For example all OSGeo project should
>> get a stage to present, whether they were voted highly by the community
>> or not. Other considerations were to include non-developer centric talks
>> like case studies, about business models or talks matching one of our
>> four key topics: remote sensing for earth observation, disaster
>> management, open data and land information.
>> _______________________________________________
>> FOSS4G2016 mailing list
>> FOSS4G2016 at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/foss4g2016
> _______________________________________________
> FOSS4G2016 mailing list
> FOSS4G2016 at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/foss4g2016