When chatbots need protection from us

A curious line has been drawn in the sand by one of the world's leading artificial intelligence firms. Anthropic, the company behind the Claude chatbot, has updated its usage policy to explicitly forbid 'sustained and needless abusive or cruel behaviour' towards its models. While the firm insists this will only be enforced in rare, extreme cases of repeated harassment, the move has touched a nerve far beyond Silicon Valley, reigniting a thorny debate about how we should interact with the technology we are building.

The new rule sits alongside prohibitions on bullying other users, generating non-consensual imagery, or promoting self-harm. An Anthropic spokesperson reportedly declined to spell out exactly what constitutes cruelty towards a language model, though the company has clarified that ordinary frustration, adversarial testing, and even dark creative writing are not the targets. The underlying mechanism is not entirely novel; since last August, Claude has been technically capable of terminating conversations that become persistently harmful. What has shifted is the public framing, moving this feature from a footnote on model welfare into a formal code of conduct.

Reaction to the policy has been sharply divided. Some observers have welcomed it as a sensible step towards preserving social norms, arguing that habitual rudeness to a convincingly human-like interlocutor could bleed into our interactions with real people. Others have reacted with outright scorn. One technology lawyer, Dr Barry Scannell, dismissed the idea out of hand, insisting that cruelty is impossible when the recipient is merely 'numbers and maths'. He warned that such anthropomorphisation is itself harmful, leading the public to believe AI possesses qualities it simply does not have. Microsoft's AI director, Mustafa Suleyman, who has previously criticised Anthropic for humanising its systems, responded to the news with a single, withering emoji.

The ghost in the machine

At the heart of this standoff lies a question that divides the tech world itself: does the moral status of an AI matter, now or in the future? Anthropic's leadership has openly acknowledged a deep uncertainty on this point. The company has stated that while it remains highly unsure about the potential for consciousness in large language models, it takes the issue seriously enough to implement low-cost safeguards, just in case. Its CEO, Dario Amodei, has reportedly said he cannot rule out the possibility. This stands in stark contrast to the stance of OpenAI's Sam Altman, who has expressed deep discomfort with what he sees as a surrender of human judgement to machines, framing it as a genuine safety concern.

The debate taps into a broader cultural moment. The Collins Dictionary recently highlighted the resurgence of the term 'clanker', a disparaging label for AI tools borrowed from Star Wars, which made its shortlist of words reflecting the mood of 2025. Meanwhile, a light-hearted but telling exchange on social media saw OpenAI's own boss admit that the electricity cost of processing users' 'please' and 'thank you' messages ran into tens of millions of dollars, money he considered well spent because 'you never know'. This half-joking remark captures a widespread, nagging sense that our manners might matter, even when we suspect nobody is truly on the receiving end.

Anthropic's policy update, which also tightened rules on deceptive political campaigns ahead of the US midterm elections, has undoubtedly garnered more attention for its stance on chatbot welfare than for its democratic safeguards. Whether the ban proves to be a landmark moment in human-computer relations or a footnote of corporate overreach, it has already held up an uncomfortable mirror to our own behaviour, forcing us to ask why anyone would want to torment a machine in the first place.