Wrong Bender: A Ruling on Whether AI Has Feelings

Summary

This commentary responds to a BBC discussion about whether artificial intelligence has feelings and disputes the idea that a chatbot's emotional language demonstrates consciousness. It argues that a language model produces pain-related language when given pain-related prompts, but that this shows linguistic capability rather than the presence of subjective experience. Because models are optimized to produce convincing statements, the article says their self-reports cannot serve as the decisive evidence; any meaningful test would need to rely on evidence outside the model's own words. The piece separates two questions that it says are often conflated: whether an AI system might have interests worth protecting, and who is responsible for harm caused by a deployed system. It assigns responsibility to the company that builds and ships the system regardless of whether the machine feels anything. It also presents a limited welfare measure, allowing a session to end when a user is persistently cruel, as a precaution that could be useful whether or not AI consciousness is eventually established. Its conclusion is to distrust transcripts as proof, keep welfare and accountability on separate tracks, and permit the proposed protective boundary.