Anthropic has implemented new restrictions prohibiting users from engaging in “sustained and needless abusive or cruel behavior” towards its AI models. This development comes as the company’s executives explore the complex topic of machine consciousness.
The Verge was the first to cover this policy shift.
A representative from Anthropic, creators of the AI chatbot Claude, has yet to clarify what specific actions would be deemed “abusive or cruel”.
In its updated user policies, the San Francisco-based firm emphasized that the ban would not extend to typical user frustrations, experimentation with the models, or creative explorations of darker themes.
Previously, their usage guidelines already included measures against abusive actions.
The organization’s large language models possess a feature that allows them to end a conversation if users engage in harmful behavior persistently. Introduced last August, this capability was presented as a protective measure for the AI's well-being.
“While we remain uncertain about the moral status of Claude and other large language models, both now and in the future, we approach this issue with gravity. In tandem with our research, we are actively seeking low-cost solutions to minimize risks to model welfare, should such welfare be applicable. One method is permitting models to terminate or exit potentially distressing dialogues,” the company stated on its website.
The debate surrounding whether AI systems can be considered conscious remains contentious, generating varied opinions both within and outside the technology sector.
Anthropic’s CEO, Dario Amodei, has expressed that he does not dismiss the possibility of machine consciousness.
Conversely, Sam Altman, CEO of the competing firm OpenAI, has shown skepticism regarding the notion. He expressed his apprehension in an X post, criticizing the tendency to attribute divine qualities or relinquish human judgment to AI models. Shortly after, the New York Times revealed that Anthropic leaders had been engaging in in-depth discussions with religious scholars about these topics.




