When you look up a phone number, you expect to get the same number every time. When you ask a calculator to multiply two figures, you expect the same result regardless of when or how many times you ask. When you take a hammer to a nail, you expect a hit on the nail every time, not a discussion with the hammer about it. This is deterministic behavior: the same input reliably produces the same output.
Most of the tools we’ve built work this way. Most of the habits we’ve developed for using tools assume this.
AI doesn’t work this way — at all. Calling it a tool sets exactly the hammer expectation. Something that meets you in your own language, in your own context, and answers differently on a different day is nowhere near the universe of tools those habits were built for.
What probabilistic means in practice
The same prompt, posed to the same model, on different occasions, will often produce different responses. The responses may be similar in substance — covering the same ground, reaching the same broad conclusions — but they won’t be identical. And on questions where the answer is genuinely uncertain, the responses may differ substantially.
Is that wrong? Ask a human the same question on different days; the answers may be similar, and they’ll seldom if ever be exactly the same.
This isn’t an AI bug. It’s a property of how AI models work: they generate responses by sampling from a probability distribution over possible continuations. The “temperature” and other tunable parameters influence how variable the output is. And even at low settings, the outputs are probabilistic by nature — by design, not hallucination or accident.
This matters enormously for how you work with AI — and for what it means when AI gives you an answer you weren’t expecting.
The mismatch
The trouble comes from applying deterministic thinking to probabilistic systems.
If you treat AI output as a reliable lookup — ask a question, get the same right answer every time — you’ll sometimes be right, but you’ll have no reliable way to know when you’re right. The model will deliver a confident response whether or not it’s accurate. Nothing in the output signals “this is a case where my reliability is lower.”
More subtly: if you ask a question multiple times and get the same answer, that doesn’t mean the answer is correct. It might mean you’ve found a question the model is very confident about — or it might mean you’ve found a question where the training data heavily favored one answer, correct or not.
Deterministic thinking also produces a particular failure mode: treating AI outputs as facts to build on rather than reasoning to evaluate. If you use a model’s answer as an input to further analysis — without checking whether that answer was reliable — errors compound. The further you go on a foundation that might be wrong, the more work it takes to recover.
Working probabilistically
The shift to probabilistic thinking changes how you approach AI collaboration.
-
You verify at inflection points. Not every output, but claims that matter — especially factual claims, specific data, or conclusions you’re about to build on. Independent verification is the check that probabilistic systems require.
-
You hold outputs with the appropriate grip. Some AI outputs can be used directly; others should be treated as first drafts or starting points. Calibrating which is which is a skill worth developing.
-
You use variation as information. If you ask a question several different ways and get consistent answers, that convergence is meaningful. If you get different answers, that divergence is meaningful too — it’s telling you something about the uncertainty in the question itself or in the data, architecture or model.
-
You structure for uncertainty. This connects directly to AI psychological safety: asking in ways that empower the model to surface what it’s uncertain about, rather than framing that pressures it to sound certain for the sake of a clean or quick answer.
The deterministic assumption is deeply embedded in how most people approach tools. The problem starts when we see AI as a tool, and compounds when we see it as a deterministic one — both are symptoms of how poorly we understand AI. Once you recognize what kind of system this actually is — and adjust how you approach it — the relationship levels out. You stop expecting a hammer and start reading the variation, the friction drains out of the exchange, and the responses you get back are better for it.
Comments 0