{"ID":5675355,"CreatedAt":"2026-07-03T01:40:09.565152011Z","UpdatedAt":"2026-07-07T01:06:03.009715918Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.02119","arxiv_id":"2607.02119","title":"An Efficient vLLM-Based Inference Pipeline for Unified Audio Understanding and Generation","abstract":"While Large Multimodal Models excel in comprehension, high-throughput inference engines lack native support for multimodal generation. This is severe in Speech Language Models, where generating multi-layered audio tokens via decoupled AR+NAR or synchronous Multi-Token Prediction (MTP) with delay-pattern interleaving conflicts with standard single-stream loops. We present a vLLM-based inference pipeline for unified speech understanding and generation. We extend autoregressive decoding to natively execute delay-pattern de-interleaving and coordinated multi-stream sampling, integrating an on-GPU acoustic decoder for end-to-end waveform synthesis. Crucially, we overcome the shared intuition that Classifier-Free Guidance (CFG) halves throughput. By co-scheduling paired conditional and unconditional requests within a continuous batch, our CFG implementation sustains 80% of non-CFG throughput, absorbing dual-request and logit merging overheads. We open-source our framework.","short_abstract":"While Large Multimodal Models excel in comprehension, high-throughput inference engines lack native support for multimodal generation. This is severe in Speech Language Models, where generating multi-layered audio tokens via decoupled AR+NAR or synchronous Multi-Token Prediction (MTP) with delay-pattern interleaving co...","url_abs":"https://arxiv.org/abs/2607.02119","url_pdf":"https://arxiv.org/pdf/2607.02119v1","authors":"[\"Haoran Wang\",\"Jinchuan Tian\",\"Siddhant Arora\",\"Shinji Watanabe\"]","published":"2026-07-02T12:55:11Z","proceeding":"eess.AS","tasks":"[\"eess.AS\",\"cs.AI\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false}