LLM policy

[Greg Troxel]

"Regina Obe" <lr at pcorp.us> writes:

> Again I say we can come to the same conclusion by not debating what is
> everyone.

The problem is that this statement accepts that the CoC, which applies
to "everyone", might apply to LLMs as pseudo-people, both as
  - creators of content later submitted
  - autonomous agent submitting content disjoint from a human

The problem is not just disagreement about "are LLMs people too" but the
very concept that a code of conduct can protect probabalistic machine
models from a human choice to prohibit them.  Saying we don't need to
debate it -- without abandoning the assertion -- is a claim that the
opinions of those that find "CoC protects LLMs" to be ludicrous are not
worthy of discussion.

> If a human kills you do you consider them more welcome than AI?  You
> don't need AI to be unethical so the argument seems silly -- just dump
> anthrax in peoples water.  Just throw an A-bomb on a continent. Put
> guards in to prevent the action not the thing or everyone doing it.

This feels like a pile of non-sequiturs.

> I still stand by "you judge the actions not the thing doing it, if you
> have concluded all things of that class are bad, then fine, block all
> those".

Many people have concluded that all extant LLMs are bad.

> If something causes people to be addicts, it's bad period - that could
> be a drug dealer or AI.  I see no point in isolating AI out as if it
> were the only thing capable of evil in this world and simply because
> we deem it not human and therefore not worthy of respect.

LLMs are only being talked about specifically because essentially
everything else you are calling about as bad is near-universally
considered not ok.  For example, if we had people who were interacting
wtih postgis in order to gain trust and plant a vulnerability (xz
style), then we'd say that wasn't ok, and we would almost certainly not
be having a debate.

> My pace-maker could be co-opted, should I be deeply concerned, Yes,
> should I ban the pace-maker from my body? No.

I think it's right to be concerned.  There's a real risk from
proprietary software -- these days I wouldn't really be surprised for a
new pacemaker to be BLE/wifi connected and have the facebook SDK in it
for social login to some portal, with who knows what along for the ride.
There's a real risk that future code updates will embed "AI" which may
or may not make sense (vs neural nets trained on consent-obtained
high-quality data, might already be there).  That's how far I think
people building things have lost their way.  A rant from the best-known
ranter about this:
  https://softwarefreedom.org/resources/2010/transparent-medical-devices.pdf

Do I think you should ban it?  No, of course not, you have to make a
risk/benefit tradeoff, mitigate risks that can be mitigated, and hope
for the best.


But that isn't the question on the table for projects.  It's whether
code produced by LLMs is acceptable, and the reasons it isn't are as you
mentioned upthread: unclear copyright status due to being a derived work
of stolen training data, a track record of being wrong, and transferring
effort from submitters to reviewers/maintainers.

There's a further question, which is if full-on open-loop agent
interaction is acceptable.

(That's before one considers the ethical issues of using LLMs trained on
stolen code, for many, the environment costs of LLMs, and for some, the
coming financial fraud of IPO to index conversion before the bubble
bursts as companies that can't possibly make money fail.  I consider it
unethical to use LLMs that are trained on stolen data or data from
aggressive scraping.  That's a personal judgement, and I realize
opinions differ.)