Why the AI Determinism Debate Misses the Real Problem

Summary

The author, a computer scientist who studies deterministic sequences and has worked with machine learning at a low level, argues that the debate over nondeterministic large language models often identifies the wrong problem. In a two-by-two framework, an LLM can be random and wrong, repeatable and wrong, random but correct, or repeatable and correct. The author says users generally want the last two outcomes: correctness matters, while variation among correct answers is acceptable unless zero variation is itself part of the required result. The article notes that deterministic training and inference are technically possible by fixing the seeds of pseudorandom number generators and preventing nondeterministic CPU or GPU execution, with details such as rounding modes and cross-platform behavior still requiring care. However, repeatability alone only places every answer in either the “same and wrong” or “same and correct” category; it does not select the correct one. The author also explains that sampling from a wider probability distribution can produce different correct answers, whereas a sharply concentrated distribution produces the same answer. The conclusion is that demands for determinism often mask a deeper demand for reliable correctness.