{"ID":3053344,"CreatedAt":"2026-06-04T04:41:36.695875263Z","UpdatedAt":"2026-06-06T02:42:15.249915999Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.04378","arxiv_id":"2606.04378","title":"DLLG: Dynamic Logit-Level Gating of LLM Experts","abstract":"Leveraging multiple specialized LLMs can combine complementary strengths, but existing approaches trade adaptability for stability: routing commits prematurely, heuristic ensembling depends on fragile proxies, and parameter merging introduces interference. We propose DLLG (Dynamic Logit-Level Gating), a dynamic logit-level ensembling framework that learns token-level expert fusion from sparse response-level supervision. A lightweight gating module predicts step-wise fusion weights, linking trajectory-level correctness to generation without token-level labels or expert retraining. Across diverse reasoning and code benchmarks, DLLG consistently outperforms strong routing, heuristic ensembling, and parameter-merging baselines across model scales, highlighting learned logit-level fusion as a robust and scalable paradigm for integrating specialized experts.","short_abstract":"Leveraging multiple specialized LLMs can combine complementary strengths, but existing approaches trade adaptability for stability: routing commits prematurely, heuristic ensembling depends on fragile proxies, and parameter merging introduces interference. We propose DLLG (Dynamic Logit-Level Gating), a dynamic logit-l...","url_abs":"https://arxiv.org/abs/2606.04378","url_pdf":"https://arxiv.org/pdf/2606.04378v1","authors":"[\"Bingnan Li\",\"Zhaoyang Zhang\",\"Xiaoze Liu\",\"Yantao Shen\",\"Shuli Jiang\",\"Shuo Yang\",\"Wei Xia\",\"Zhuowen Tu\",\"Stefano Soatto\"]","published":"2026-06-03T02:51:56Z","proceeding":"cs.CL","tasks":"[\"cs.CL\"]","methods":"[\"Large Language Model\"]","has_code":false}