Coding AI Without Deterministic Outcomes

Summary

An Aha! engineering essay argues that building features powered by large language models creates two frustrations that differ from conventional software development. Engineers cannot reliably trace an output back to a specific instruction or logical decision, so changing a prompt can produce a different result without revealing why the original failed. The work therefore resembles interacting with another person more than debugging deterministic code. The author also argues that LLM outputs rarely have one perfect answer, making it difficult to define when a feature is finished; “good enough” replaces a precise completion standard. Testing text outputs is difficult because language is flexible, and using another LLM as an evaluator can reproduce the same uncertainty rather than remove it. Prompting is described as a learnable skill, but practice does not eliminate the underlying lack of determinism. The essay suggests that AI feature work may suit engineers who enjoy persuasion, experimentation, and interaction, just as different engineering roles suit different preferences.