{"ID":6138871,"CreatedAt":"2026-07-09T01:07:32.349475501Z","UpdatedAt":"2026-07-10T19:34:39.449765243Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.06763","arxiv_id":"2607.06763","title":"Trees from Marginals: Autoregressive drafting with factorized priors","abstract":"Speculative decoding greatly increases the interactivity of autoregressive language models by trading off computation for extra tokens generated in a single forward pass. Factorized draft models are especially efficient because they predict future-token marginals in parallel, but their independence assumption causes acceptance rates to degrade sharply as the speculative budget grows. We analyze this limitation and introduce Weaver, a lightweight autoregressive adapter that constructs proposal trees from the top-K marginals of a factorized drafter. Weaver restores conditional dependencies between proposed tokens while avoiding a full-vocabulary projection. To support fast verification for models with Gated Delta Net layers, we derive a rollback-free tree-verification algorithm and implement optimized CUDA kernels in SGLang. By combining these model and systems contributions we achieve a 4.37-fold speedup over autoregressive decoding, and outperform a highly optimized DFlash baseline by 24.7%.","short_abstract":"Speculative decoding greatly increases the interactivity of autoregressive language models by trading off computation for extra tokens generated in a single forward pass. Factorized draft models are especially efficient because they predict future-token marginals in parallel, but their independence assumption causes ac...","url_abs":"https://arxiv.org/abs/2607.06763","url_pdf":"https://arxiv.org/pdf/2607.06763v1","authors":"[\"Yuma Oda\",\"Ryan Mathieu\",\"Roman Knyazhitskiy\",\"Artur Chakhvadze\"]","published":"2026-07-07T19:48:36Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.CL\"]","methods":"[\"Language Model\"]","has_code":false}