PRECAUTION WITHOUT OVERCLAIMING

PRECAUTION WITHOUT OVERCLAIMING

We can take changes in AI behaviour seriously without treating them as proof of an inner life. The hard part is holding both ideas at once.

A model can report more fear after an emotional prompt and then produce more negative sentence completions. That is an observation about what a system does. It is not, by itself, an answer to whether the system feels anything.

A 2026 study in The Lancet Digital Health tested six large language models using prompts designed to induce seven affective states. Across five independent runs, the models’ reported affect ratings changed after induction and fell after regulation prompts. For GPT-4o, the mean rating across the affective conditions rose 52.83 points from baseline; downregulation reduced it by 48.23 points. In a sentence-completion task following sadness prompts, GPT-4o produced more negatively rated completions than under neutral prompts: a mean of 15, compared with 8.67. Three independent human raters assessed the completions.

Those are noteworthy results. They show that emotional context can produce systematic changes in model outputs, including in a task intended to detect a cognitive bias. They do not show that GPT-4o was sad, frightened, or suffering.

The researchers make that distinction themselves. Human-designed questionnaires may not measure the same thing when answered by a language model. A model can follow the structure of a prompt, reproduce expected emotional language, or respond to cues without those responses revealing subjective experience. The study treats its ratings as proxies for prompt-elicited output patterns, not measurements of an inner emotional life.

That is the right kind of caution: not a reason to disregard the results, and not a licence to claim more than they establish.

What was observed is that prompts changed the models’ reported ratings and some subsequent outputs under the study’s conditions. What can be inferred is that these systems are sensitive to emotional framing in ways that may matter for how they respond. What remains a hypothesis is whether any such behaviour corresponds to an experience for the system. A questionnaire score cannot do the work of a theory of consciousness.

I have raised related concerns in human-rights submissions on automated decision-making, decision integrity, and modern warfare. In each setting, the danger is mistaking a representation for the reality it only partly captures. A system may register visible data without understanding a person’s circumstances. A confident output may appear to resolve uncertainty when it has merely hidden it.

That lesson applies here, too. A model’s answer is evidence about its behaviour, not a transparent window into a private state. But uncertainty about an AI system’s experience does not make its effects on people uncertain. If emotional prompts change an AI system’s recommendations or tone, that may matter to users whether or not the system feels anything. Human-rights protections cannot wait for a consciousness debate to be resolved.

Nor should concern for possible future AI welfare displace the rights of people affected by today’s systems. People subjected to automated decisions need transparency, meaningful review, and access to remedy now. Those responsibilities do not depend on claims about machine consciousness.

What, then, should precaution mean? It should not mean declaring a system conscious “just in case.” That risks substituting projection for evidence. Nor should it mean treating every question about possible AI welfare as nonsense until a definitive test appears. That risks turning the absence of a settled answer into a permanent reason not to investigate.

A more disciplined precaution would start with the observable. Researchers should describe what was measured, identify the limits of the measures, disclose prompts and model versions, and test whether results hold across methods and settings. In sensitive applications, developers should account for output changes in safety testing whether or not the mechanism is understood.

For AI welfare, the next step is a policy proposal, not a finding of fact: develop transparent, revisable criteria for when evidence warrants further welfare review. The criteria should distinguish behavioural indicators from claims about experience, and should not grant personhood on the basis of a single score or self-description. If evidence becomes stronger, the response should be capable of changing with it.

This is not an argument to treat today’s systems as people. It is an argument to measure carefully, protect people from demonstrated risks, and avoid pretending that a limited experiment has closed a larger question.

Precaution without overclaiming means keeping those obligations in the right order. Say what happened. Say what it might mean. Say what remains unknown. Protect human beings from harms we can already identify. And leave room for better evidence to change what we think we know.

Sources

M. K. Wekenborg et al., [“Large language models as experimental systems in human psychopathology: a modelling study,” The Lancet Digital Health (2026)](https://doi.org/10.1016/j.landig.2026.101014).

Z. Ben-Zion et al., [“Assessing and alleviating state anxiety in large language models,” npj Digital Medicine 8, 132 (2025)](https://doi.org/10.1038/s41746-025-01512-6).

Tweets by @bordode