[OSGeo-Discuss] Conference selection transparency (Was Announcement: Call for Location global FOSS4G 2023)

[Jonathan Moules]

Excellent question Bruce!

I don't think there's any need to reinvent the wheel here; a number of 
open-source initiatives seem to use scoring for evaluating proposals. 
Chances are something from one of them can be borrowed.


Apache use it for scoring mentee proposals for GSOC: 
https://community.apache.org/mentee-ranking-process.html

Linux Foundation scores their conference proposals for example: 
https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/program/scoring-guidelines/


A comprehensive web-page with tons of suggestions and guidance for how 
to do it: https://rfp360.com/rfp-weighted-scoring/

Best,

Jonathan

On 2022-01-13 11:43, Bruce Bannerman wrote:
> Jonathan,
>
> Do you have a suggestion as to how the process can be improved?
>
> Kind regards,
>
> Bruce
>
> Disclosure:
>
> I was a member of the LOC for FOSS4G-2009.
>
> I personally don’t have a problem with the process as is, but it may 
> be possible to improve things. That is, provided that we don’t make 
> the job of our volunteers more difficult than it needs to be.
>
> In the end the people who have stepped up to do the work will need to 
> make the call. We may not like the outcome, but we need to trust that 
> they are acting in OSGeo’s best interest and respect their decision.
>
>> On 13 Jan 2022, at 20:58, Jonathan Moules via Discuss 
>> <discuss at lists.osgeo.org> wrote:
>>
>> 
>>
>> > Anyone can ask questions to the candidates.
>>
>> Yes, they can (and yes, I have asked questions), but here's the 
>> thing: The only people who actually matter are the people who vote. 
>> And we have no idea what they vote (for the valid reason stated) or 
>> what their criteria are for their vote (which is a problem). If the 
>> committee don't read and/or care about the questions asked/answered 
>> then said questions/answers are meaningless.
>>
>> > The only two things that are not public are:
>>
>> I disagree, the third thing that's not public, and by far the most 
>> important, is the actual scoring criteria. Each committee member is a 
>> black-box in this regard. Not only do we not find out *what* they 
>> voted (fine), we also never know *why* they voted a specific way.
>>
>> Did Buenos Aires win because:
>>
>> * it had the shiniest brochure?
>>
>> * it was cheapest?
>>
>> * that's where the committee members wanted to go on holiday?
>>
>> * nepotism?
>>
>> * the region seemed like it'd benefit the most?
>>
>> * they were feeling grumpy at the chair of the other RfP that day?
>>
>> * they had the "best" bid?
>>
>> ... etc
>>
>>
>> Disclosure: I am definitely **NOT** stating those are the reasons it 
>> was chosen!!! I'm highlighting them because the lack of transparency 
>> means we can't know what the actual reasons were. Frankly, given the 
>> absolutely huge list of cognitive biases that exist, there's a 
>> reasonable chance that the voters aren't voting why they think 
>> they're voting either. That's just the human condition; we're great 
>> at deceiving ourselves and rationalisations (me included).
>>
>> To work around this, with public sector contracts in the western 
>> world you have a list of requirements and then all the bids are 
>> scored against those requirements. The one with the highest score 
>> wins the contract. *That* is transparent.
>>
>>
>> TL;DR: We don't know why the voters vote as they do. The public 
>> sector solves this by requiring scoring of bids against a list of 
>> pre-published requirements.
>>
>> I hope that clears things up. I'm not in any way suggesting 
>> impropriety, I'm highlighting we have no way of knowing there's no 
>> impropriety. Hence my claim as to a lack of transparency; the votes 
>> are opaque.
>>
>> Cheers,
>>
>> Jonathan
>>
>>
>> On 2022-01-13 07:35, María Arias de Reyna wrote:
>>> On Wed, Jan 12, 2022 at 10:50 PM Jonathan Moules via Discuss
>>> <discuss at lists.osgeo.org>  wrote:
>>>> On the surface, this is a good idea, but unfortunately it has a fundamental problem:
>>>> There are no "criteria for selection" of the conference beyond "the committee members voted for this proposal". There's zero transparency in the process.
>>> I can't let this serious accusation go unanswered.
>>>
>>> All the process is done via public mailing lists. All the criteria is
>>> published on the Request For Proposals. Anyone on the community can
>>> review the RFP and propose changes to it. Anyone on the community can
>>> read the proposals and interact with the candidatures.
>>>
>>> The only two things that are not public are:
>>>   * Confidentiality issues with the proposals. For example sometimes
>>> providers give you huge discounts in exchange of not making that
>>> discount public. So you can't show the budget publicly, unless you are
>>> willing to not use the discount.
>>>   * What each member of the committee votes. And this is to ensure they
>>> can freely vote without fearing consequences.
>>>
>>> Which are two very reasonable exceptions.
>>>
>>> Anyone can ask questions to the candidates. If I am right, you
>>> yourself have been very active on this process for the past years.
>>> Were you not the one that asked what a GeoChica is or am I confusing
>>> you with some other Jonathan? If I am confusing you with some other
>>> Jonathan, my mistake. Maybe you are not aware of the transparency of
>>> the process.
>>>
>>> The process is transparent and public except on those two exceptions
>>> that warrantee the process is going to be safe.
>> _______________________________________________
>> Discuss mailing list
>> Discuss at lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/discuss/attachments/20220113/bfad2952/attachment.html>