[OSGeo-Discuss] Conference selection transparency (Was Announcement: Call for Location global FOSS4G 2023)
Jonathan Moules
jonathan-lists at lightpear.com
Thu Jan 13 06:32:04 PST 2022
> And cognitive bias suddenly does not play a role anymore when you
score a good friend vs a hated enemy against a "list of
requirements"....? It might look transparent but is not the tiniest bit
more fair.
Sure the biases will still be there, but the justification for the score
is written down for all to see. Hence: Transparent. It'll be available
for the entire community to then read; if it's a rationalisation it'll
be there for all to see (and call out).
Suggestions for even more fairness are welcome.
On 2022-01-13 14:25, Kobben, Barend (UT-ITC) wrote:
>
> Quoting "To work around this, with public sector contracts in the
> western world you have a list of requirements and then all the bids
> are scored against those requirements. The one with the highest score
> wins the contract. *That* is transparent. "
>
> Really...? And cognitive bias suddenly does not play a role anymore
> when you score a good friend vs a hated enemy against a "list of
> requirements"....? It might look transparent but is not the tiniest
> bit more fair.
>
> /-- /
>
> /Barend Köbben/
>
> *From: *Discuss <discuss-bounces at lists.osgeo.org> on behalf of
> Jonathan Moules via Discuss <discuss at lists.osgeo.org>
> *Organisation: *LightPear
> *Reply to: *"jonathan-lists at lightpear.com" <jonathan-lists at lightpear.com>
> *Date: *Thursday, 13 January 2022 at 13:13
> *To: *Bruce Bannerman <bruce.bannerman.osgeo at gmail.com>
> *Cc: *"discuss at lists.osgeo.org" <discuss at lists.osgeo.org>
> *Subject: *Re: [OSGeo-Discuss] Conference selection transparency (Was
> Announcement: Call for Location global FOSS4G 2023)
>
> Excellent question Bruce!
>
> I don't think there's any need to reinvent the wheel here; a number of
> open-source initiatives seem to use scoring for evaluating proposals.
> Chances are something from one of them can be borrowed.
>
> Apache use it for scoring mentee proposals for GSOC:
> https://community.apache.org/mentee-ranking-process.html
>
> Linux Foundation scores their conference proposals for example:
> https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/program/scoring-guidelines/
>
> A comprehensive web-page with tons of suggestions and guidance for how
> to do it: https://rfp360.com/rfp-weighted-scoring/
>
> Best,
>
> Jonathan
>
> On 2022-01-13 11:43, Bruce Bannerman wrote:
>
> Jonathan,
>
> Do you have a suggestion as to how the process can be improved?
>
> Kind regards,
>
> Bruce
>
> Disclosure:
>
> I was a member of the LOC for FOSS4G-2009.
>
> I personally don’t have a problem with the process as is, but it
> may be possible to improve things. That is, provided that we don’t
> make the job of our volunteers more difficult than it needs to be.
>
> In the end the people who have stepped up to do the work will need
> to make the call. We may not like the outcome, but we need to
> trust that they are acting in OSGeo’s best interest and respect
> their decision.
>
>
>
> On 13 Jan 2022, at 20:58, Jonathan Moules via Discuss
> <discuss at lists.osgeo.org> <mailto:discuss at lists.osgeo.org> wrote:
>
> > Anyone can ask questions to the candidates.
>
> Yes, they can (and yes, I have asked questions), but here's
> the thing: The only people who actually matter are the people
> who vote. And we have no idea what they vote (for the valid
> reason stated) or what their criteria are for their vote
> (which is a problem). If the committee don't read and/or care
> about the questions asked/answered then said questions/answers
> are meaningless.
>
> > The only two things that are not public are:
>
> I disagree, the third thing that's not public, and by far the
> most important, is the actual scoring criteria. Each committee
> member is a black-box in this regard. Not only do we not find
> out *what* they voted (fine), we also never know *why* they
> voted a specific way.
>
> Did Buenos Aires win because:
>
> * it had the shiniest brochure?
>
> * it was cheapest?
>
> * that's where the committee members wanted to go on holiday?
>
> * nepotism?
>
> * the region seemed like it'd benefit the most?
>
> * they were feeling grumpy at the chair of the other RfP that day?
>
> * they had the "best" bid?
>
> ... etc
>
> Disclosure: I am definitely **NOT** stating those are the
> reasons it was chosen!!! I'm highlighting them because the
> lack of transparency means we can't know what the actual
> reasons were. Frankly, given the absolutely huge list of
> cognitive biases that exist, there's a reasonable chance that
> the voters aren't voting why they think they're voting either.
> That's just the human condition; we're great at deceiving
> ourselves and rationalisations (me included).
>
> To work around this, with public sector contracts in the
> western world you have a list of requirements and then all the
> bids are scored against those requirements. The one with the
> highest score wins the contract. *That* is transparent.
>
> TL;DR: We don't know why the voters vote as they do. The
> public sector solves this by requiring scoring of bids against
> a list of pre-published requirements.
>
> I hope that clears things up. I'm not in any way suggesting
> impropriety, I'm highlighting we have no way of knowing
> there's no impropriety. Hence my claim as to a lack of
> transparency; the votes are opaque.
>
> Cheers,
>
> Jonathan
>
> On 2022-01-13 07:35, María Arias de Reyna wrote:
>
> On Wed, Jan 12, 2022 at 10:50 PM Jonathan Moules via Discuss
>
> <discuss at lists.osgeo.org> <mailto:discuss at lists.osgeo.org> wrote:
>
> On the surface, this is a good idea, but unfortunately it has a fundamental problem:
>
> There are no "criteria for selection" of the conference beyond "the committee members voted for this proposal". There's zero transparency in the process.
>
> I can't let this serious accusation go unanswered.
>
> All the process is done via public mailing lists. All the criteria is
>
> published on the Request For Proposals. Anyone on the community can
>
> review the RFP and propose changes to it. Anyone on the community can
>
> read the proposals and interact with the candidatures.
>
> The only two things that are not public are:
>
> * Confidentiality issues with the proposals. For example sometimes
>
> providers give you huge discounts in exchange of not making that
>
> discount public. So you can't show the budget publicly, unless you are
>
> willing to not use the discount.
>
> * What each member of the committee votes. And this is to ensure they
>
> can freely vote without fearing consequences.
>
> Which are two very reasonable exceptions.
>
> Anyone can ask questions to the candidates. If I am right, you
>
> yourself have been very active on this process for the past years.
>
> Were you not the one that asked what a GeoChica is or am I confusing
>
> you with some other Jonathan? If I am confusing you with some other
>
> Jonathan, my mistake. Maybe you are not aware of the transparency of
>
> the process.
>
> The process is transparent and public except on those two exceptions
>
> that warrantee the process is going to be safe.
>
> _______________________________________________
> Discuss mailing list
> Discuss at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/discuss/attachments/20220113/f2fbec6f/attachment.html>
More information about the Discuss
mailing list