What Does It Mean for an AI System to Be Good?

Summary

This essay argues that “good” is not a single, transferable property when people evaluate AI systems. It may refer to reliability, safety, social benefit, or moral desirability, and those standards can conflict. Drawing on Bernard Williams’s discussion of goodness, the author says that an object can be good at one role or task without being good in a broader sense: an AI agent that performs writing, coding, reporting, or summarization tasks well should not automatically be treated as a good programmer, writer, analyst, or lawyer. The same distinction applies to coding agents, whose usefulness involves more than producing code quickly; requirements clarification, system understanding, trade-offs, correctness, maintainability, communication, and knowledge transfer may also matter. Whether an outcome is acceptable depends on the task’s purpose, the user, the environment, the stakes, and the amount of verification required. A lightly checked prototype may be acceptable in one setting, while software used by millions requires much stronger confidence. The author therefore recommends asking what an AI is good at, for whom, compared with what, under which conditions, and with which failures considered unacceptable. Teams need enough understanding of both their work and the technology to choose meaningful baselines and success criteria.