[OSGeo-Discuss] FOSS4G presentation review process [SEC=UNCLASSIFIED]

[Barry Rowlingson]

On Tue, Oct 2, 2012 at 1:01 AM, Bruce Bannerman <B.Bannerman at bom.gov.au> wrote:
> Agreed.
>
> Well said Cameron, with the aside that there may be an interesting talk from
> a previously little known person.
>
> I suggest leaving this to the discretion of the LOC and interested parties
> who subscribe to that year’s FOSS4G mailing list.
>
> A popularity campaign is not required or wanted.

 With my statistician hat on, and not speaking as a member of the
committee, it seems that we have two processes going on - what sounds
like a good talk, and who sounds like a good speaker. Maybe we should
run two review systems - one with *just* names and not abstracts or
titles, and the other with just abstracts and no names. That would
give us a measure of who the community wanted to see at the
conference, and what the community thought were great talks unbiased
by the name. The committee would then take both these reports into
consideration for the final selection.

 My extreme statistician hat gave me another idea. For each review,
present a random speaker with a random talk abstract, and ask for a
rating on the whole package. With enough randomized reviews, it would
be possible to get a ranking for speakers and talks as well as a
correlation between speakers and talks. Perhaps we could even suggest
that if speaker A did talk C instead of talk B, more people would be
interested!

There may be ways to stop popularity-contest ballot stuffing -
reviewers could get a random subset of the presentations for review,
with no guarantee that their friend's proposal is going to be there -
and prevent them reloading the page until it appears. Or you could
present multiple random pairs of proposals and ask which of the two
you'd attend.

 Committee hat back on, I'm glad we're having this discussion.

Barry