{"ID":2877360,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2508.21186","arxiv_id":"2508.21186","title":"Manifold Trajectories in Next-Token Prediction: From Replicator Dynamics to Softmax Equilibrium","abstract":"Decoding in large language models is often described as scoring tokens and normalizing with softmax. We give a minimal, self-contained account of this step as a constrained variational principle on the probability simplex. The discrete, normalization-respecting ascent is the classical multiplicative-weights (entropic mirror) update; its continuous-time limit is the replicator flow. From these ingredients we prove that, for a fixed context and temperature, the next-token distribution follows a smooth trajectory inside the simplex and converges to the softmax equilibrium. This formalizes the common ``manifold traversal'' intuition at the output-distribution level. The analysis yields precise, practice-facing consequences: temperature acts as an exact rescaling of time along the same trajectory, while top-k and nucleus sampling restrict the flow to a face with identical guarantees. We also outline a controlled account of path-dependent score adjustments and their connection to loop-like, hallucination-style behavior. We make no claims about training dynamics or internal representations; those are deferred to future work.","short_abstract":"Decoding in large language models is often described as scoring tokens and normalizing with softmax. We give a minimal, self-contained account of this step as a constrained variational principle on the probability simplex. The discrete, normalization-respecting ascent is the classical multiplicative-weights (entropic m...","url_abs":"https://arxiv.org/abs/2508.21186","url_pdf":"https://arxiv.org/pdf/2508.21186v1","authors":"[\"Christopher R. Lee-Jenkins\"]","published":"2025-08-28T20:00:22Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.AI\",\"math.DS\"]","methods":"[\"Language Model\"]","has_code":false}