[QGIS-Developer] Tightening AI submission policy?

[Greg Troxel]

Julien Cabieces via QGIS-Developer <qgis-developer at lists.osgeo.org>
writes:

[I'm in favor of the stricter policy.]

>> - LLMs may only be used as an improved auto-completion mechanism, or
>> for repeated tasks (mechanical refactoring) that could potentially be
>> completed with a deterministic algorithm.
>
> Shall we not also allow AI when it's used as a chatbot to answer
> speific/technical questions.

I see this as crossing a line that we don't today need to cross.  One
can separate "problems from LLMs arising from PRs" into:

  1) effects on the project: review burden, low-quality code, code without
  adequately clear licensing, asking humans to read LLM-generated text

  2) more general concerns: the ethics of using tools built with abusive
  scraping, proprietary/centralization concerns, environmental concerns

Of course, different people see all of this differently.  (I have run
into people who think LLMs are beings worthy of being protected by a
Code of Conduct, implying that those who think LLM contributions should
be banned are discriminatory people who should be shunned by other
humans.)

So far QGIS (and gdal as insiration) are addressing category 1 only.
This means

  human engages with an LLM to figure out a problem
  human comes to understand the code and larger issues
  human writes a patch and commit message
  human submits that to qgis

is not a problem (not a category 1 problem).

> I also seen contributors using it to identify where and how to fix an
> issue. I've never tried it myself and don't know if it's something that
> actually work. Would it be allowed ?

Under the policy text I see, it would not be prohibited, if the human
creates the fix after gaining understanding.   If the patch comes from
the LLM, it's not allowed.

My own view on this got stricter after I and several others spent an
hour reviewing a vibe-coded patch to an unrelated project.  There was a
lot of vague tutorial/generalities in the commit message and comments,
unrelated code rototilling, and in the end we decided it was wrong.




There's a further issue not addressed by the revised policy, which is
LLM-generated text being sent to mailing lists, matrix room, etc.  I
have recently seen an instance (in an unrelated project) where someone
asked a question (about a fairly difficult subject), and got a few
generally-helpful accurate answers from humans.  Then, a very long
answer appeared, superficially appearing authoritative, but it got
things wrong.   Pretty obviously this was LLM-generated text.   I see
this as being in the same category as PRs and would suggest amending the
policy to be

  - All contributions including code, ticket comments, commit messages, mailing list messages, and chat messages and  should be fully understood by the author(s) submitting them to the project.

except that I find the gdal text to be not strict enough.   While the
"mild LLM assistance is not prohibited if you end up with a contribution
that is understood, accurate enough, and not overly verbose", my
perception is that people who think it's ok to use LLMs think these
problems are not serious and they're going to go ahead anyway.

A big point is that it's not ok to ask humans to read text generated by
an LLM.  Perhaps that could be emphasized in the modified policy.

Also, while prohibiting LLM usage, I think it's important to avoid
prohibiting straight machine translation or grammar editing of text.
The point is that if the translated text has the same semantics as the
original, that's fine, but if there new semantics, it's LLM-generated
text, not a translation.  We should not exclude people who are not
fluent in English as part of stopping vibe-coded PRs.