<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Excellent question Bruce!</p>
<p> I don't think there's any need to reinvent the wheel here; a
number of open-source initiatives seem to use scoring for
evaluating proposals. Chances are something from one of them can
be borrowed.</p>
<p><br>
</p>
<p>Apache use it for scoring mentee proposals for GSOC:
<a class="moz-txt-link-freetext" href="https://community.apache.org/mentee-ranking-process.html">https://community.apache.org/mentee-ranking-process.html</a></p>
<p>Linux Foundation scores their conference proposals for example:
<a class="moz-txt-link-freetext" href="https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/program/scoring-guidelines/">https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/program/scoring-guidelines/</a></p>
<br>
<p>A comprehensive web-page with tons of suggestions and guidance
for how to do it: <a class="moz-txt-link-freetext" href="https://rfp360.com/rfp-weighted-scoring/">https://rfp360.com/rfp-weighted-scoring/</a></p>
<p>Best,</p>
<p>Jonathan<br>
</p>
<div class="moz-cite-prefix">On 2022-01-13 11:43, Bruce Bannerman
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:18CFED39-A5E7-46FE-9E3B-E51487724338@gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">Jonathan,</div>
<div dir="ltr"><br>
</div>
<div dir="ltr">Do you have a suggestion as to how the process can
be improved?</div>
<div dir="ltr"><br>
</div>
<div dir="ltr">Kind regards,</div>
<div dir="ltr"><br>
</div>
<div dir="ltr">Bruce</div>
<div dir="ltr"><br>
</div>
<div dir="ltr">Disclosure:</div>
<div dir="ltr"><br>
</div>
<div dir="ltr">I was a member of the LOC for FOSS4G-2009.</div>
<div dir="ltr"><br>
</div>
<div dir="ltr">I personally don’t have a problem with the process
as is, but it may be possible to improve things. That is,
provided that we don’t make the job of our volunteers more
difficult than it needs to be.</div>
<div dir="ltr"><br>
</div>
<div dir="ltr">In the end the people who have stepped up to do the
work will need to make the call. We may not like the outcome,
but we need to trust that they are acting in OSGeo’s best
interest and respect their decision.</div>
<div dir="ltr"><br>
<blockquote type="cite">On 13 Jan 2022, at 20:58, Jonathan
Moules via Discuss <a class="moz-txt-link-rfc2396E" href="mailto:discuss@lists.osgeo.org"><discuss@lists.osgeo.org></a> wrote:<br>
<br>
</blockquote>
</div>
<blockquote type="cite">
<div dir="ltr">
<meta http-equiv="Content-Type" content="text/html;
charset=UTF-8">
<p>> Anyone can ask questions to the candidates.</p>
<p>Yes, they can (and yes, I have asked questions), but here's
the thing: The only people who actually matter are the
people who vote. And we have no idea what they vote (for the
valid reason stated) or what their criteria are for their
vote (which is a problem). If the committee don't read
and/or care about the questions asked/answered then said
questions/answers are meaningless.<br>
</p>
<p>> The only two things that are not public are:</p>
<p> I disagree, the third thing that's not public, and by far
the most important, is the actual scoring criteria. Each
committee member is a black-box in this regard. Not only do
we not find out *what* they voted (fine), we also never know
*why* they voted a specific way.</p>
<p>Did Buenos Aires win because:</p>
<p>* it had the shiniest brochure?</p>
<p>* it was cheapest?</p>
<p>* that's where the committee members wanted to go on
holiday?</p>
<p>* nepotism?</p>
<p>* the region seemed like it'd benefit the most?</p>
<p>* they were feeling grumpy at the chair of the other RfP
that day?</p>
<p>* they had the "best" bid?<br>
</p>
<p>... etc<br>
</p>
<p><br>
</p>
<p>Disclosure: I am definitely *<b>NOT</b>* stating those are
the reasons it was chosen!!! I'm highlighting them because
the lack of transparency means we can't know what the actual
reasons were. Frankly, given the absolutely huge list of
cognitive biases that exist, there's a reasonable chance
that the voters aren't voting why they think they're voting
either. That's just the human condition; we're great at
deceiving ourselves and rationalisations (me included).<br>
</p>
<p>To work around this, with public sector contracts in the
western world you have a list of requirements and then all
the bids are scored against those requirements. The one with
the highest score wins the contract. *That* is transparent.</p>
<p><br>
</p>
<p>TL;DR: We don't know why the voters vote as they do. The
public sector solves this by requiring scoring of bids
against a list of pre-published requirements.</p>
<p>I hope that clears things up. I'm not in any way suggesting
impropriety, I'm highlighting we have no way of knowing
there's no impropriety. Hence my claim as to a lack of
transparency; the votes are opaque.</p>
<p>Cheers,</p>
<p>Jonathan<br>
</p>
<p><br>
</p>
<div class="moz-cite-prefix">On 2022-01-13 07:35, María Arias
de Reyna wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAERgKeA2K-izYev8XtxN_-W4GE_aYnPVNE4_4jCJKn3PGd8tZA@mail.gmail.com">
<pre class="moz-quote-pre" wrap="">On Wed, Jan 12, 2022 at 10:50 PM Jonathan Moules via Discuss
<a class="moz-txt-link-rfc2396E" href="mailto:discuss@lists.osgeo.org" moz-do-not-send="true"><discuss@lists.osgeo.org></a> wrote:
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">On the surface, this is a good idea, but unfortunately it has a fundamental problem:
There are no "criteria for selection" of the conference beyond "the committee members voted for this proposal". There's zero transparency in the process.
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">I can't let this serious accusation go unanswered.
All the process is done via public mailing lists. All the criteria is
published on the Request For Proposals. Anyone on the community can
review the RFP and propose changes to it. Anyone on the community can
read the proposals and interact with the candidatures.
The only two things that are not public are:
* Confidentiality issues with the proposals. For example sometimes
providers give you huge discounts in exchange of not making that
discount public. So you can't show the budget publicly, unless you are
willing to not use the discount.
* What each member of the committee votes. And this is to ensure they
can freely vote without fearing consequences.
Which are two very reasonable exceptions.
Anyone can ask questions to the candidates. If I am right, you
yourself have been very active on this process for the past years.
Were you not the one that asked what a GeoChica is or am I confusing
you with some other Jonathan? If I am confusing you with some other
Jonathan, my mistake. Maybe you are not aware of the transparency of
the process.
The process is transparent and public except on those two exceptions
that warrantee the process is going to be safe.
</pre>
</blockquote>
<span>_______________________________________________</span><br>
<span>Discuss mailing list</span><br>
<span><a class="moz-txt-link-abbreviated" href="mailto:Discuss@lists.osgeo.org">Discuss@lists.osgeo.org</a></span><br>
<span><a class="moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/discuss">https://lists.osgeo.org/mailman/listinfo/discuss</a></span><br>
</div>
</blockquote>
</blockquote>
</body>
</html>