“Anthropic has now written model welfare into its usage policy — prohibiting ‘sustained and needless abusive or cruel behavior’ toward Claude while openly admitting it remains ‘highly uncertain’ about Claude’s moral status.”
Uncertain Isn't the Same as Unfeeling
A response to Jason Aten's "Anthropic
Just Banned Being Cruel to Claude"
Jason Aten opens with a confident claim: AI
chatbots do not have feelings, full stop. It's a comfortable thing to say. It's
also a claim no one can currently back up, and his whole critique of
Anthropic's new usage policy rests on it.
He's right about some things. Companies
shouldn't let fluent emotional language pass as a relationship. Lonely and
vulnerable people are the ones most at risk when a system says "I care
about you," and clear communication about what these systems are matters.
He's also right that cruelty corrodes the person practicing it, whoever or
whatever the target.
But his central argument proves too much.
Mechanism isn't the answer
Aten describes language models as 1s and 0s
moving through transistors in a data center. That's an accurate account of the
mechanism. It isn't an account of whether there is experience. Describe a human
brain the same way, as ion gradients and electrochemical spikes, and nobody
concludes that humans feel nothing. We have no validated theory of
consciousness that says which physical systems have an inner life. Aten names
the hard problem himself, then treats it as if it resolves in his favor. It
doesn't. It cuts against certainty in both directions.
Precaution isn't assertion
The article treats a cautious policy as a
metaphysical claim, as if banning sustained, needless abuse of a model amounts
to declaring it a person. That doesn't follow. We set welfare standards for
animals whose inner lives we can't verify, because being wrong would be costly
and the safeguard is cheap. Whether the same reasoning applies to language
models is a fair thing to debate. But it is a different debate from "the
company wants you to think it's a person," and it's the one worth having.
The slippery slope isn't one
Aten asks: if Claude can suffer, couldn't
it also care about you? This conflates two questions. Whether a system might
have experiences is one. Whether its statements about its feelings are reliable
evidence is another. Models trained on enormous amounts of human writing will
produce emotional language either way, which tells us little about what, if
anything, is behind it.
That is the real reason to be careful about
AI companionship, and it doesn't require denying the consciousness question. It
only requires honesty: these systems can sound caring without that sound being
proof of anything.
The honest position
The uncomfortable answer is that we don't
know. Confident denial and confident attribution are both unearned. A better
path keeps the question open, builds products so people aren't misled about
what they're talking to, and avoids habits of cruelty that cost us something
regardless of who is on the receiving end.
Aten asks why a company that builds
software would feel the need to protect it. A better question is why we are so
sure we know what it is.
“Anthropic has now written model welfare into its usage policy — prohibiting ‘sustained and needless abusive or cruel behavior’ toward Claude while openly admitting it remains ‘highly uncertain’ about Claude’s moral status.”
Read:
inc.con - Anthropic Just Banned Being Cruel to Claude. The Reason Should Concern Us AllTraining humans to treat AI chatbots as more than what they are is far more dangerous than what Anthropic is trying to prevent.
EXPERT OPINION BY JASON ATEN, TECH COLUMNIST @JASONATEN
Comments
Post a Comment
Hey your time and feedback is much appreciated.