[QGIS-Developer] Floating an idea: ban AI based contributed from non-core developers?

[Even Rouault]

> One key point here. It is really **hard** to decrypt the quality of 
> LLM generated code, because it is trained to be plausible.
>
I triple on that. I've always found that (in depth) reviewing was a task 
consuming a lot of mental energy, often more than coding, but in pre-LLM 
times, you knew that the contributor had already invested a lot of 
energy into creating their submission, so that made your own energy 
investment acceptable. Now, the reviewer is mostly the only one that 
sweats.  When you make comments and get a reply, you never know if the 
reply comes from the human, the machine or a mix of them. This is really 
really really unpleasant.  In our highly decentralized community, the 
lack of direct interaction pre-LLM could also be a challenge (mechanisms 
behind the working of human psychology have been forged by natural 
selection over ~ 3 million years of face-to-face interactions, add tens 
of extra millions years inherited from our closest cousins in the animal 
reign, compared to hardly 50 years of digital communication), but at 
least you knew that at @someone was a peer.  Now you don't know anymore. 
You're becoming effectively the slave of an agent.

Among the things I dislike is the wall'o'text LLM generated PR 
descriptions. I believe we should ban them and require contributors to 
come with their own succinct and to-the-point descriptions. Or if they 
can't come up with one, just none. That makes at least less stuff to 
review!  But then the "human in the loop" policy no longer works.

Maybe we should charge a symbolic fee to new contributors for each PR 
they submit, until one is accepted? I bet this would make them think 
twice. Unfortunately that's not a github feature.

>  Knowing this, a friendly contributor, willing to getting involved in 
> the long term, and who will use AI as a helper for its first 
> contributions, should know that it will put a very high charge on 
> reviewers. So this is like a poisoned gift.
>
One difficulty is that it is likely really hard for someone not having 
already experienced themselves reviewing LLM generated PRs to know how 
it feels to deal with them.

Writing all the above with https://github.com/qgis/QGIS/pull/65778 in 
mind. Took me maybe one hour of digging (yes not fast but CMake is full 
of subtelties) to realize that there is (likely) a much simpler fix than 
the one suggested (I may be wrong, let's see what the bot answers). I 
believe that 6 months ago I would have approved the exact same PR 
content without much thinking. So one could argue that the extra 
skepticism when confronting to LLM generated PR is a good thing, but 
frankly it is exhausting. I believe there's a new job lurking: "mental 
coach/psychologist for developers dealing with LLM contributions"

-- 
http://www.spatialys.com
My software is free, but my time generally not.