{"ID":2897677,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2507.05346","arxiv_id":"2507.05346","title":"LoRA-Augmented Generation (LAG) for Knowledge-Intensive Language Tasks","abstract":"The proliferation of fine-tuned language model experts for specific tasks and domains signals the need for efficient selection and combination methods. We propose LoRA-Augmented Generation (LAG) for leveraging large libraries of knowledge and task-specific LoRA adapters. LAG requires no additional training or access to data, and efficiently filters, retrieves, and applies experts on a per-token and layer basis. We evaluate LAG on various knowledge-intensive tasks, achieving superior performance over existing data-free methods. We explore scenarios where additional data is available, demonstrating LAG's compatibility with alternative solutions such as retrieval-augmented generation (RAG).","short_abstract":"The proliferation of fine-tuned language model experts for specific tasks and domains signals the need for efficient selection and combination methods. We propose LoRA-Augmented Generation (LAG) for leveraging large libraries of knowledge and task-specific LoRA adapters. LAG requires no additional training or access to...","url_abs":"https://arxiv.org/abs/2507.05346","url_pdf":"https://arxiv.org/pdf/2507.05346v2","authors":"[\"William Fleshman\",\"Benjamin Van Durme\"]","published":"2025-07-07T18:00:01Z","proceeding":"cs.CL","tasks":"[\"cs.CL\",\"cs.AI\",\"cs.LG\"]","methods":"[\"RAG\",\"Language Model\",\"LoRA\"]","has_code":false}