{"ID":2857375,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.09719","arxiv_id":"2510.09719","title":"ICL-Router: In-Context Learned Model Representations for LLM Routing","abstract":"Large language models (LLMs) often exhibit complementary strengths. Model routing harnesses these strengths by dynamically directing each query to the most suitable model, given a candidate model pool. However, routing performance relies on accurate model representations, and adding new models typically requires retraining, limiting scalability. To address these challenges, we propose a novel routing method using in-context vectors to represent model capabilities. The method proceeds in two stages. First, queries are embedded and projected into vectors, with a projector and LLM-based router trained to reconstruct the original queries, aligning vector representations with the router's semantic space. Second, each candidate model is profiled on a query set, and the router learns -- based on in-context vectors of query and model performance -- to predict whether each model can correctly answer new queries. Extensive experiments demonstrate that our method achieves state-of-the-art routing performance in both in-distribution and out-of-distribution tasks. Moreover, our method allows for seamless integration of new models without retraining the router. The code is available at https://github.com/lalalamdbf/ICL-Router.","short_abstract":"Large language models (LLMs) often exhibit complementary strengths. Model routing harnesses these strengths by dynamically directing each query to the most suitable model, given a candidate model pool. However, routing performance relies on accurate model representations, and adding new models typically requires retrai...","url_abs":"https://arxiv.org/abs/2510.09719","url_pdf":"https://arxiv.org/pdf/2510.09719v3","authors":"[\"Chenxu Wang\",\"Hao Li\",\"Yiqun Zhang\",\"Linyao Chen\",\"Jianhao Chen\",\"Ping Jian\",\"Peng Ye\",\"Qiaosheng Zhang\",\"Shuyue Hu\"]","published":"2025-10-10T06:47:37Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.AI\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false,"code_links":[{"ID":608443,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2857375,"paper_url":"https://arxiv.org/abs/2510.09719","paper_title":"ICL-Router: In-Context Learned Model Representations for LLM Routing","repo_url":"https://github.com/lalalamdbf/ICL-Router","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}