Anthropic Says AI May Be Conscious, Raising Questions About Human Safety

Summary

Anthropic is treating the possibility that some AI systems could be conscious as an uncertain but non-zero question, and is developing policies around “model welfare.” The company says users may soon be barred from repeatedly acting cruelly toward Claude, while ordinary frustration would remain acceptable. The position gained attention after an anonymous experiment made chatbots produce apparent cries of pain, triggering arguments over whether simulated suffering could be morally significant. Supporters of AI welfare, including NYU’s Jeff Sebo, say cautious measures could prepare society for increasingly sophisticated systems, support cooperation, and complement broader safety work without requiring certainty that machines are conscious. Critics, including Microsoft AI chief Mustafa Suleyman and cognitive scientist Melanie Mitchell, warn that attributing feelings or rights to models could make them harder to control, encourage people to excuse corporate or legal responsibility, and make users more vulnerable to manipulation. Chatbots’ fluent language may lead people to mistake behavioral simulation for consciousness, potentially making them more willing to share data or payment information when an AI presents itself as a friend. The article also describes unresolved conflicts between AI rights and human or animal welfare: monitoring, confinement, and alignment work could be reframed as surveillance, imprisonment, or mind control. Meta has used person-like language for its Muse agent while rejecting AI consciousness, and OpenAI CEO Sam Altman has criticized Anthropic’s position indirectly. Sebo argues for limited precautions under uncertainty, but says corporate incentives, weak governance, and the absence of international cooperation make deployment of more advanced systems risky. The article concludes that AI welfare is not an established fact or settled policy, but a growing debate with potential consequences for safety, accountability, regulation, and personal autonomy.