{"ID":2884941,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2508.04946","arxiv_id":"2508.04946","title":"REINA: Regularized Entropy Information-Based Loss for Efficient Simultaneous Speech Translation","abstract":"Simultaneous Speech Translation (SimulST) systems stream in audio while simultaneously emitting translated text or speech. Such systems face the significant challenge of balancing translation quality and latency. We introduce a strategy to optimize this tradeoff: wait for more input only if you gain information by doing so. Based on this strategy, we present Regularized Entropy INformation Adaptation (REINA), a novel loss to train an adaptive policy using an existing non-streaming translation model. We derive REINA from information theory principles and show that REINA helps push the reported Pareto frontier of the latency/quality tradeoff over prior works. Utilizing REINA, we train a SimulST model on French, Spanish and German, both from and into English. Training on only open source or synthetically generated data, we achieve state-of-the-art (SOTA) streaming results for models of comparable size. We also introduce a metric for streaming efficiency, quantitatively showing REINA improves the latency/quality trade-off by as much as 21% compared to prior approaches, normalized against non-streaming baseline BLEU scores.","short_abstract":"Simultaneous Speech Translation (SimulST) systems stream in audio while simultaneously emitting translated text or speech. Such systems face the significant challenge of balancing translation quality and latency. We introduce a strategy to optimize this tradeoff: wait for more input only if you gain information by doin...","url_abs":"https://arxiv.org/abs/2508.04946","url_pdf":"https://arxiv.org/pdf/2508.04946v3","authors":"[\"Nameer Hirschkind\",\"Joseph Liu\",\"Xiao Yu\",\"Mahesh Kumar Nandwana\"]","published":"2025-08-07T00:25:58Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.CL\",\"eess.AS\"]","methods":"[]","has_code":false}