<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>> <span
style="font-size:12.0pt;font-family:"Georgia",serif;mso-fareast-language:EN-US"
lang="EN-GB">And cognitive bias suddenly does not play a role
anymore when you score a good friend vs a hated enemy against a
"list of requirements"....? It might look transparent but is not
the tiniest bit more fair.</span></p>
<p><span
style="font-size:12.0pt;font-family:"Georgia",serif;mso-fareast-language:EN-US"
lang="EN-GB">Sure the biases will still be there, but the
justification for the score is written down for all to see.
Hence: Transparent. It'll be available for the entire community
to then read; if it's a rationalisation it'll be there for all
to see (and call out). <br>
</span></p>
<p><span
style="font-size:12.0pt;font-family:"Georgia",serif;mso-fareast-language:EN-US"
lang="EN-GB">Suggestions for even more fairness are welcome.<br>
</span></p>
<p><span
style="font-size:12.0pt;font-family:"Georgia",serif;mso-fareast-language:EN-US"
lang="EN-GB"><br>
</span></p>
<div class="moz-cite-prefix">On 2022-01-13 14:25, Kobben, Barend
(UT-ITC) wrote:<br>
</div>
<blockquote type="cite"
cite="mid:BF6C9886-F535-4350-88C4-74A3CAF99CA6@utwente.nl">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}@font-face
{font-family:Georgia;
panose-1:2 4 5 2 5 4 5 2 3 3;}@font-face
{font-family:"Times New Roman \(Body CS\)";
panose-1:2 11 6 4 2 2 2 2 2 4;}@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0cm;
font-size:10.0pt;
font-family:"Courier New";}span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;}span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Georgia",serif;
color:windowtext;
font-weight:normal;
font-style:normal;}.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}div.WordSection1
{page:WordSection1;}</style>
<div class="WordSection1">
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Georgia",serif;mso-fareast-language:EN-US"
lang="EN-GB">Quoting "</span><span lang="EN-GB">
</span><span
style="font-size:12.0pt;font-family:"Georgia",serif;mso-fareast-language:EN-US"
lang="EN-GB">To work around this, with public sector
contracts in the western world you have a list of
requirements and then all the bids are scored against those
requirements. The one with the highest score wins the
contract. *That* is transparent. "<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Georgia",serif;mso-fareast-language:EN-US"
lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Georgia",serif;mso-fareast-language:EN-US"
lang="EN-GB">Really...? And cognitive bias suddenly does not
play a role anymore when you score a good friend vs a hated
enemy against a "list of requirements"....? It might look
transparent but is not the tiniest bit more fair.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Georgia",serif;mso-fareast-language:EN-US"
lang="EN-GB"><o:p> </o:p></span></p>
<div>
<div>
<p class="MsoNormal"><i><span
style="font-family:"Georgia",serif;color:black">-- <o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span
style="font-family:"Georgia",serif;color:black">Barend
Köbben<o:p></o:p></span></i></p>
</div>
</div>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Georgia",serif;mso-fareast-language:EN-US"
lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Georgia",serif;mso-fareast-language:EN-US"
lang="EN-GB"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF
1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span
style="font-size:12.0pt;color:black">From: </span></b><span
style="font-size:12.0pt;color:black">Discuss
<a class="moz-txt-link-rfc2396E" href="mailto:discuss-bounces@lists.osgeo.org"><discuss-bounces@lists.osgeo.org></a> on behalf of
Jonathan Moules via Discuss
<a class="moz-txt-link-rfc2396E" href="mailto:discuss@lists.osgeo.org"><discuss@lists.osgeo.org></a><br>
<b>Organisation: </b>LightPear<br>
<b>Reply to: </b><a class="moz-txt-link-rfc2396E" href="mailto:jonathan-lists@lightpear.com">"jonathan-lists@lightpear.com"</a>
<a class="moz-txt-link-rfc2396E" href="mailto:jonathan-lists@lightpear.com"><jonathan-lists@lightpear.com></a><br>
<b>Date: </b>Thursday, 13 January 2022 at 13:13<br>
<b>To: </b>Bruce Bannerman
<a class="moz-txt-link-rfc2396E" href="mailto:bruce.bannerman.osgeo@gmail.com"><bruce.bannerman.osgeo@gmail.com></a><br>
<b>Cc: </b><a class="moz-txt-link-rfc2396E" href="mailto:discuss@lists.osgeo.org">"discuss@lists.osgeo.org"</a>
<a class="moz-txt-link-rfc2396E" href="mailto:discuss@lists.osgeo.org"><discuss@lists.osgeo.org></a><br>
<b>Subject: </b>Re: [OSGeo-Discuss] Conference selection
transparency (Was Announcement: Call for Location global
FOSS4G 2023)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p>Excellent question Bruce!<o:p></o:p></p>
<p>I don't think there's any need to reinvent the wheel here; a
number of open-source initiatives seem to use scoring for
evaluating proposals. Chances are something from one of them
can be borrowed.<o:p></o:p></p>
<p><o:p> </o:p></p>
<p>Apache use it for scoring mentee proposals for GSOC: <a
href="https://community.apache.org/mentee-ranking-process.html"
moz-do-not-send="true" class="moz-txt-link-freetext">
https://community.apache.org/mentee-ranking-process.html</a><o:p></o:p></p>
<p>Linux Foundation scores their conference proposals for
example: <a
href="https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/program/scoring-guidelines/"
moz-do-not-send="true" class="moz-txt-link-freetext">
https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/program/scoring-guidelines/</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p>A comprehensive web-page with tons of suggestions and
guidance for how to do it:
<a href="https://rfp360.com/rfp-weighted-scoring/"
moz-do-not-send="true" class="moz-txt-link-freetext">https://rfp360.com/rfp-weighted-scoring/</a><o:p></o:p></p>
<p>Best,<o:p></o:p></p>
<p>Jonathan<o:p></o:p></p>
<div>
<p class="MsoNormal">On 2022-01-13 11:43, Bruce Bannerman
wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">Jonathan,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Do you have a suggestion as to how the
process can be improved?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Kind regards,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Bruce<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Disclosure:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I was a member of the LOC for
FOSS4G-2009.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I personally don’t have a problem with
the process as is, but it may be possible to improve
things. That is, provided that we don’t make the job of
our volunteers more difficult than it needs to be.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">In the end the people who have stepped
up to do the work will need to make the call. We may not
like the outcome, but we need to trust that they are
acting in OSGeo’s best interest and respect their
decision.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal" style="margin-bottom:12.0pt">On 13
Jan 2022, at 20:58, Jonathan Moules via Discuss
<a href="mailto:discuss@lists.osgeo.org"
moz-do-not-send="true"><discuss@lists.osgeo.org></a>
wrote:<o:p></o:p></p>
</blockquote>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p>> Anyone can ask questions to the candidates.<o:p></o:p></p>
<p>Yes, they can (and yes, I have asked questions), but
here's the thing: The only people who actually matter
are the people who vote. And we have no idea what they
vote (for the valid reason stated) or what their
criteria are for their vote (which is a problem). If the
committee don't read and/or care about the questions
asked/answered then said questions/answers are
meaningless.<o:p></o:p></p>
<p>> The only two things that are not public are:<o:p></o:p></p>
<p>I disagree, the third thing that's not public, and by
far the most important, is the actual scoring criteria.
Each committee member is a black-box in this regard. Not
only do we not find out *what* they voted (fine), we
also never know *why* they voted a specific way.<o:p></o:p></p>
<p>Did Buenos Aires win because:<o:p></o:p></p>
<p>* it had the shiniest brochure?<o:p></o:p></p>
<p>* it was cheapest?<o:p></o:p></p>
<p>* that's where the committee members wanted to go on
holiday?<o:p></o:p></p>
<p>* nepotism?<o:p></o:p></p>
<p>* the region seemed like it'd benefit the most?<o:p></o:p></p>
<p>* they were feeling grumpy at the chair of the other
RfP that day?<o:p></o:p></p>
<p>* they had the "best" bid?<o:p></o:p></p>
<p>... etc<o:p></o:p></p>
<p><o:p> </o:p></p>
<p>Disclosure: I am definitely *<b>NOT</b>* stating those
are the reasons it was chosen!!! I'm highlighting them
because the lack of transparency means we can't know
what the actual reasons were. Frankly, given the
absolutely huge list of cognitive biases that exist,
there's a reasonable chance that the voters aren't
voting why they think they're voting either. That's just
the human condition; we're great at deceiving ourselves
and rationalisations (me included).<o:p></o:p></p>
<p>To work around this, with public sector contracts in
the western world you have a list of requirements and
then all the bids are scored against those requirements.
The one with the highest score wins the contract. *That*
is transparent.<o:p></o:p></p>
<p><o:p> </o:p></p>
<p>TL;DR: We don't know why the voters vote as they do.
The public sector solves this by requiring scoring of
bids against a list of pre-published requirements.<o:p></o:p></p>
<p>I hope that clears things up. I'm not in any way
suggesting impropriety, I'm highlighting we have no way
of knowing there's no impropriety. Hence my claim as to
a lack of transparency; the votes are opaque.<o:p></o:p></p>
<p>Cheers,<o:p></o:p></p>
<p>Jonathan<o:p></o:p></p>
<p><o:p> </o:p></p>
<div>
<p class="MsoNormal">On 2022-01-13 07:35, María Arias de
Reyna wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<pre>On Wed, Jan 12, 2022 at 10:50 PM Jonathan Moules via Discuss<o:p></o:p></pre>
<pre><a href="mailto:discuss@lists.osgeo.org" moz-do-not-send="true"><discuss@lists.osgeo.org></a> wrote:<o:p></o:p></pre>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<pre>On the surface, this is a good idea, but unfortunately it has a fundamental problem:<o:p></o:p></pre>
<pre>There are no "criteria for selection" of the conference beyond "the committee members voted for this proposal". There's zero transparency in the process.<o:p></o:p></pre>
</blockquote>
<pre>I can't let this serious accusation go unanswered.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>All the process is done via public mailing lists. All the criteria is<o:p></o:p></pre>
<pre>published on the Request For Proposals. Anyone on the community can<o:p></o:p></pre>
<pre>review the RFP and propose changes to it. Anyone on the community can<o:p></o:p></pre>
<pre>read the proposals and interact with the candidatures.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>The only two things that are not public are:<o:p></o:p></pre>
<pre> * Confidentiality issues with the proposals. For example sometimes<o:p></o:p></pre>
<pre>providers give you huge discounts in exchange of not making that<o:p></o:p></pre>
<pre>discount public. So you can't show the budget publicly, unless you are<o:p></o:p></pre>
<pre>willing to not use the discount.<o:p></o:p></pre>
<pre> * What each member of the committee votes. And this is to ensure they<o:p></o:p></pre>
<pre>can freely vote without fearing consequences.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Which are two very reasonable exceptions.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Anyone can ask questions to the candidates. If I am right, you<o:p></o:p></pre>
<pre>yourself have been very active on this process for the past years.<o:p></o:p></pre>
<pre>Were you not the one that asked what a GeoChica is or am I confusing<o:p></o:p></pre>
<pre>you with some other Jonathan? If I am confusing you with some other<o:p></o:p></pre>
<pre>Jonathan, my mistake. Maybe you are not aware of the transparency of<o:p></o:p></pre>
<pre>the process.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>The process is transparent and public except on those two exceptions<o:p></o:p></pre>
<pre>that warrantee the process is going to be safe.<o:p></o:p></pre>
</blockquote>
<p class="MsoNormal">_______________________________________________<br>
Discuss mailing list<br>
<a href="mailto:Discuss@lists.osgeo.org"
moz-do-not-send="true" class="moz-txt-link-freetext">Discuss@lists.osgeo.org</a><br>
<a
href="https://lists.osgeo.org/mailman/listinfo/discuss"
moz-do-not-send="true" class="moz-txt-link-freetext">https://lists.osgeo.org/mailman/listinfo/discuss</a><o:p></o:p></p>
</div>
</blockquote>
</blockquote>
</div>
</blockquote>
</body>
</html>