For as long as software has existed, we have always typed the code. You opened the file, followed the logic, wrote the code, and code-reviewed it (yourself, or by a peer), until you were satisfied it would do what it claimed. For ordinary code, that still holds. Deterministic code does exactly what its source says, every time.
But that is not the code we are writing anymore.
Today the code has become non-deterministic. More and more of what we build now runs on AI logic. A function no longer computes an answer; it asks a model for one. A branch is no longer an if-statement; it is a prompt. An agent decides, at runtime, what to do next.
So, the same input can produce different behavior on different runs, because the real logic no longer lives in the lines you wrote. It lives in a model’s weights, memory, retrieved data, orchestration, sampling, settings, and more. So it shifts with context, with temperature, with version.
Now look at that code. You see a function call and a prompt string. It looks orderly, and you come away feeling you know what it does. You do not. The behavior was never in the text. And worse, the text gave you confidence anyway.
That false confidence is dangerous, and new studies are starting to measure it. Researchers at Stanford ran a controlled study of forty-seven developers across multiple security tasks. The developers using an AI assistant wrote less secure code, yet believed it was more secure. They were applying an old-world habit to code that no longer works that way.
In fact, my own belief is that even conventional software is not always deterministic; concurrency, external services, clocks, randomness, and undefined behavior already make it non-deterministic. But that was tolerable. Nondeterminism itself is not the real danger. False confidence is.
That is why the craft has to change. Writing the code yourself, and reviewing it yourself or with a peer, was how you earned confidence that it worked. For non-deterministic code, that confidence is false. You cannot hand-verify behavior that shifts from run to run, and your unit tests cannot pin it either.
So you stop judging the code you wrote and start “evaluating” what it does — many runs, many cases, counting how often it is right, held to a bar. That is what evals are, and that is why they, not your own hands, become the thing you trust.
Check the output, not the code.