[OSGeo-Discuss] Conference selection transparency (Was Announcement: Call for Location global FOSS4G 2023)

Thu Jan 13 06:25:40 PST 2022

Quoting " To work around this, with public sector contracts in the western world you have a list of requirements and then all the bids are scored against those requirements. The one with the highest score wins the contract. *That* is transparent. "

Really...? And cognitive bias suddenly does not play a role anymore when you score a good friend vs a hated enemy against a "list of requirements"....? It might look transparent but is not the tiniest bit more fair.

--
Barend Köbben

From: Discuss <discuss-bounces at lists.osgeo.org> on behalf of Jonathan Moules via Discuss <discuss at lists.osgeo.org>
Organisation: LightPear
Reply to: "jonathan-lists at lightpear.com" <jonathan-lists at lightpear.com>
Date: Thursday, 13 January 2022 at 13:13
To: Bruce Bannerman <bruce.bannerman.osgeo at gmail.com>
Cc: "discuss at lists.osgeo.org" <discuss at lists.osgeo.org>
Subject: Re: [OSGeo-Discuss] Conference selection transparency (Was Announcement: Call for Location global FOSS4G 2023)

Excellent question Bruce!

I don't think there's any need to reinvent the wheel here; a number of open-source initiatives seem to use scoring for evaluating proposals. Chances are something from one of them can be borrowed.

Apache use it for scoring mentee proposals for GSOC: https://community.apache.org/mentee-ranking-process.html

Linux Foundation scores their conference proposals for example: https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/program/scoring-guidelines/

A comprehensive web-page with tons of suggestions and guidance for how to do it: https://rfp360.com/rfp-weighted-scoring/

Best,

Jonathan
On 2022-01-13 11:43, Bruce Bannerman wrote:
Jonathan,

Do you have a suggestion as to how the process can be improved?

Kind regards,

Bruce

Disclosure:

I was a member of the LOC for FOSS4G-2009.

I personally don’t have a problem with the process as is, but it may be possible to improve things. That is, provided that we don’t make the job of our volunteers more difficult than it needs to be.

In the end the people who have stepped up to do the work will need to make the call. We may not like the outcome, but we need to trust that they are acting in OSGeo’s best interest and respect their decision.

On 13 Jan 2022, at 20:58, Jonathan Moules via Discuss <discuss at lists.osgeo.org><mailto:discuss at lists.osgeo.org> wrote:

> Anyone can ask questions to the candidates.

Yes, they can (and yes, I have asked questions), but here's the thing: The only people who actually matter are the people who vote. And we have no idea what they vote (for the valid reason stated) or what their criteria are for their vote (which is a problem). If the committee don't read and/or care about the questions asked/answered then said questions/answers are meaningless.

> The only two things that are not public are:

I disagree, the third thing that's not public, and by far the most important, is the actual scoring criteria. Each committee member is a black-box in this regard. Not only do we not find out *what* they voted (fine), we also never know *why* they voted a specific way.

Did Buenos Aires win because:

* it had the shiniest brochure?

* it was cheapest?

* that's where the committee members wanted to go on holiday?

* nepotism?

* the region seemed like it'd benefit the most?

* they were feeling grumpy at the chair of the other RfP that day?

* they had the "best" bid?

... etc

Disclosure: I am definitely *NOT* stating those are the reasons it was chosen!!! I'm highlighting them because the lack of transparency means we can't know what the actual reasons were. Frankly, given the absolutely huge list of cognitive biases that exist, there's a reasonable chance that the voters aren't voting why they think they're voting either. That's just the human condition; we're great at deceiving ourselves and rationalisations (me included).

To work around this, with public sector contracts in the western world you have a list of requirements and then all the bids are scored against those requirements. The one with the highest score wins the contract. *That* is transparent.

TL;DR: We don't know why the voters vote as they do. The public sector solves this by requiring scoring of bids against a list of pre-published requirements.

I hope that clears things up. I'm not in any way suggesting impropriety, I'm highlighting we have no way of knowing there's no impropriety. Hence my claim as to a lack of transparency; the votes are opaque.

Cheers,

Jonathan

On 2022-01-13 07:35, María Arias de Reyna wrote:

On Wed, Jan 12, 2022 at 10:50 PM Jonathan Moules via Discuss

<discuss at lists.osgeo.org><mailto:discuss at lists.osgeo.org> wrote:

On the surface, this is a good idea, but unfortunately it has a fundamental problem:

There are no "criteria for selection" of the conference beyond "the committee members voted for this proposal". There's zero transparency in the process.

I can't let this serious accusation go unanswered.

All the process is done via public mailing lists. All the criteria is

published on the Request For Proposals. Anyone on the community can

review the RFP and propose changes to it. Anyone on the community can

read the proposals and interact with the candidatures.

The only two things that are not public are:

 * Confidentiality issues with the proposals. For example sometimes

providers give you huge discounts in exchange of not making that

discount public. So you can't show the budget publicly, unless you are

willing to not use the discount.

 * What each member of the committee votes. And this is to ensure they

can freely vote without fearing consequences.

Which are two very reasonable exceptions.

Anyone can ask questions to the candidates. If I am right, you

yourself have been very active on this process for the past years.

Were you not the one that asked what a GeoChica is or am I confusing

you with some other Jonathan? If I am confusing you with some other

Jonathan, my mistake. Maybe you are not aware of the transparency of

the process.

The process is transparent and public except on those two exceptions

that warrantee the process is going to be safe.
_______________________________________________
Discuss mailing list
Discuss at lists.osgeo.org<mailto:Discuss at lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/discuss/attachments/20220113/ba627919/attachment.html>