Eliminating Nondeterminism in LLM Inference with Integer-Only Arithmetic
This article presents DetLLM, a system that eliminates nondeterminism in LLM inference by performing all operations using integer-only arithmetic, achieving bit-identical outputs across different hardware platforms while maintaining comparable perplexity and dramatically improving decode speed through kernel optimizations.