LLM policy

[Regina Obe]

Greg and Even,

I think we are mostly in agreement and especially the important points.

> "Regina Obe" <lr at pcorp.us> writes:
> 
> > Again I say we can come to the same conclusion by not debating what is
> > everyone.
> 
> The problem is that this statement accepts that the CoC, which applies to
> "everyone", might apply to LLMs as pseudo-people, both as
>   - creators of content later submitted
>   - autonomous agent submitting content disjoint from a human
> 
> The problem is not just disagreement about "are LLMs people too" but the
> very concept that a code of conduct can protect probabalistic machine
models
> from a human choice to prohibit them.  Saying we don't need to debate it
--
> without abandoning the assertion -- is a claim that the opinions of those
that
> find "CoC protects LLMs" to be ludicrous are not worthy of discussion.
> 

I'm not saying an LLM policy should not be considered so sorry if it sounded
like that was what I was saying.
Yes if the code of conduct (and also our Contributing doc) is not clear
enough, then we need an additional policy specifically for the usage of LLMs
and should probably reference it in those files to be clear.

What I am arguing is the idea that the "everyone" in our code of conduct
means only humans.

I tend to think very anthropomorphically about things and that is
intentional to prevent me from suffering what I consider clouds of
judgement.

For example if we think that everyone refers to people, does that mean I
can't bring my video camera to an event.

I may not be able to bring those things not because they are not human but
because they inconvenience people/things more valued in the community
than they help me and I should be allowed to bring them unless expressly
stated otherwise.  That is a judgement call by the people who are managing
the community or the venue - e.g. someone highly valued in the community
being inconvenienced by a video camera.

Just like other things allowed in, that camera deserves to not be mutilated
by someone who happens to not like cameras. So yes the Coc applies to it, if
I welcome it, it has the same rights as everything else I welcome.

So in that sense the camera has rights.  Our environment has the right not
to be wasted by humans and it isn't human.

Ultimately the purpose of a code of conduct is to help increase the value of
our work for the world in the most efficient way and to welcome valuable
resources into our community.  I don't really see it as being something
specific to humans.

> LLMs are only being talked about specifically because essentially
everything
> else you are calling about as bad is near-universally considered not ok.
For
> example, if we had people who were interacting wtih postgis in order to
gain
> trust and plant a vulnerability (xz style), then we'd say that wasn't ok,
and we
> would almost certainly not be having a debate.
> 

I fail to see the difference here.  Whether it's an LLM trying to gain favor
or a person so they can plant a vulnerability
the conclusion is clear to everyone, it's bad.  I don't need to know it was
an LLM who did it to conclude it is bad
and I'm still going to kick out that one from my community whether it be LLM
or a human because it's bad for my community.


> > My pace-maker could be co-opted, should I be deeply concerned, Yes,
> > should I ban the pace-maker from my body? No.
> 
> I think it's right to be concerned.  There's a real risk from proprietary
software
> -- these days I wouldn't really be surprised for a new pacemaker to be
> BLE/wifi connected and have the facebook SDK in it for social login to
some
> portal, with who knows what along for the ride.
> There's a real risk that future code updates will embed "AI" which may or
may
> not make sense (vs neural nets trained on consent-obtained high-quality
data,
> might already be there).  That's how far I think people building things
have lost
> their way.  A rant from the best-known ranter about this:
>   https://softwarefreedom.org/resources/2010/transparent-medical-
> devices.pdf
> 

Exactly.  We don't need to argue the merits of respecting an AI.
We need to stop bad actors end of story.  If we need to say in our code of
conduct
(we welcome everyone except "You LLM blah blah with link to reason why or
caveats", that's fine.
 
You can say it's easier for an AI to be a bad actor
so you want to deliberately mention and block them or at least a class of AI
you consider hopeless.

> Do I think you should ban it?  No, of course not, you have to make a
> risk/benefit tradeoff, mitigate risks that can be mitigated, and hope for
the
> best.
> 
Agree

> 
> But that isn't the question on the table for projects.  It's whether code
> produced by LLMs is acceptable, and the reasons it isn't are as you
mentioned
> upthread: unclear copyright status due to being a derived work of stolen
> training data, a track record of being wrong, and transferring effort from
> submitters to reviewers/maintainers.
> 
> There's a further question, which is if full-on open-loop agent
interaction is
> acceptable.
> 
Agree, except we don't allow any stolen work period whether it be LLM or
someone who stole code from another project that doesn't allow code copy.

> (That's before one considers the ethical issues of using LLMs trained on
stolen
> code, for many, the environment costs of LLMs, and for some, the coming
> financial fraud of IPO to index conversion before the bubble bursts as
> companies that can't possibly make money fail.  

Agree


I consider it unethical to use
> LLMs that are trained on stolen data or data from aggressive scraping.
That's
> a personal judgement, and I realize opinions differ.)

Agree