Using AI Without the Guesswork: What Recent Papers Show
Most people treat large language models as black boxes that sometimes give different answers to the same question. That variability is real, but it is not an unavoidable property of the models. Several recent papers show practical ways to get consistent, repeatable results – either by making the model itself behave deterministically, or by keeping the model out of the parts of a system that need to be reliable.
The Problem Is Mostly in the Serving Stack
Even with temperature set to zero, the same prompt can produce different outputs when it runs on a production server. The main reason is not sampling. Floating-point addition is not associative, so the order of operations matters. Dynamic batching changes that order from one request to the next.
Thinking Machines Lab showed that the kernels used in a normal forward pass are themselves deterministic. The nondeterminism appears because reduction strategies change with batch size. They built batch-invariant versions of the critical operations (normalization, matrix multiplies, and attention). With those kernels, a thousand identical prompts produced one identical completion instead of dozens of different ones. The performance cost was moderate, not catastrophic.
Microsoft Research took a different route with LLM-42. Instead of rewriting the kernels, they keep the fast path with normal dynamic batching and add a verify-and-rollback step. The system generates a window of tokens at full speed, then checks those tokens under a fixed reduction order. It keeps matching tokens, discards the rest, and regenerates them. Overhead appears only for the requests that actually need determinism. When only a fraction of traffic requires identical outputs, throughput stays close to the non-deterministic baseline.
Integer Arithmetic Removes the Root Cause
A smaller line of work goes further and eliminates floating-point arithmetic. One group ran the full forward pass of a small model using only integer operations. The same prompt produced bitwise-identical logits across different GPUs, CPUs, batch sizes, and even different vendors. Integer addition is associative, so reduction order no longer matters.
The approach is still limited to smaller models and careful quantization, but it demonstrates that the numerical source of variability can be removed rather than merely contained.
Hybrid Systems Keep the Model Out of the Decision Path
A different strategy accepts that the model will remain somewhat variable and refuses to let it control anything important. In a 2026 paper on hybrid symbolic–probabilistic pipelines, a deterministic engine handles every state change and every decision. The language model is used only after the fact, to explain or rephrase results that have already been computed.
Across a hundred repeated runs on structured documents, the hybrid system produced identical execution traces every time. A pure language-model pipeline did not.
The same pattern appears in other domains. Models propose code, intermediate representations, or candidate constructions; an external interpreter, constraint solver, or symbolic engine checks and executes them. When the model is asked to perform exact arithmetic or string algorithms on its own, accuracy collapses.
When it generates a short program that an ordinary interpreter runs, accuracy reaches 100 percent on the tested tasks. The model supplies the structure; the deterministic component supplies the correctness.
What This Means in Practice
None of these papers claim that every use of a language model should be forced into a deterministic mode. Sampling is useful when the goal is exploration or when multiple plausible answers are acceptable. The useful distinction is between parts of a system that can tolerate variation and parts that cannot.
For the parts that cannot, the options are now concrete: run inference with batch-invariant kernels or a verify-rollback loop; move critical calculations into ordinary code or a symbolic engine; or keep the model in a strictly advisory role.
All of these approaches have been demonstrated with open implementations or detailed descriptions. The remaining work is mostly engineering – deciding where in a given pipeline to draw the line between variable and fixed, and paying the modest cost of the deterministic path only where needed.