{"ID":2848151,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.26697","arxiv_id":"2510.26697","title":"The End of Manual Decoding: Towards Truly End-to-End Language Models","abstract":"The \"end-to-end\" label for LLMs is a misnomer. In practice, they depend on a non-differentiable decoding process that requires laborious, hand-tuning of hyperparameters like temperature and top-p. This paper introduces AutoDeco, a novel architecture that enables truly \"end-to-end\" generation by learning to control its own decoding strategy. We augment the standard transformer with lightweight heads that, at each step, dynamically predict context-specific temperature and top-p values alongside the next-token logits. This approach transforms decoding into a parametric, token-level process, allowing the model to self-regulate its sampling strategy within a single forward pass. Through extensive experiments on eight benchmarks, we demonstrate that AutoDeco not only significantly outperforms default decoding strategies but also achieves performance comparable to an oracle-tuned baseline derived from \"hacking the test set\"-a practical upper bound for any static method. Crucially, we uncover an emergent capability for instruction-based decoding control: the model learns to interpret natural language commands (e.g., \"generate with low randomness\") and adjusts its predicted temperature and top-p on a token-by-token basis, opening a new paradigm for steerable and interactive LLM decoding.","short_abstract":"The \"end-to-end\" label for LLMs is a misnomer. In practice, they depend on a non-differentiable decoding process that requires laborious, hand-tuning of hyperparameters like temperature and top-p. This paper introduces AutoDeco, a novel architecture that enables truly \"end-to-end\" generation by learning to control its...","url_abs":"https://arxiv.org/abs/2510.26697","url_pdf":"https://arxiv.org/pdf/2510.26697v2","authors":"[\"Zhichao Wang\",\"Dongyang Ma\",\"Xinting Huang\",\"Deng Cai\",\"Tian Lan\",\"Jiahao Xu\",\"Haitao Mi\",\"Xiaoying Tang\",\"Yan Wang\"]","published":"2025-10-30T17:01:43Z","proceeding":"cs.CL","tasks":"[\"cs.CL\",\"cs.AI\"]","methods":"[\"Transformer\",\"Large Language Model\",\"Language Model\"]","has_code":false}