Who Decided That AI Gets to Have Feelings?

Summary

An essay recounts an evening conversation between Microsoft AI executive Mustafa Suleyman and Walter Isaacson about whether advanced AI systems should be treated as conscious or capable of having feelings. Suleyman’s position, as presented in the essay, is that current systems are not alive but predict the next word, and that granting them personhood or rights could make alignment, containment, and broader social control harder. The discussion contrasts this view with Anthropic’s stated uncertainty in Claude’s Constitution about whether Claude may have consciousness or moral status, including in the future. Suleyman also warned that open-source models could eventually make the regulation of large companies secondary to the risk that individuals might run powerful systems without safeguards on personal devices. He argued that public debate should happen before one or two companies effectively determine how society treats these systems. The essay also records his expectation of rapid progress, including a future shift from digital applications toward materials science, and his view that knowledge-based work will face substantial displacement while work centered on human connection may be more resilient. His stated red line was another laboratory trying to persuade people or models that a model is conscious, which he said should prompt an immediate halt to development. The author concludes that society needs open, public discussion of these philosophical and governance questions rather than leaving them to AI companies alone.