{"ID":2843744,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2511.10674","arxiv_id":"2511.10674","title":"Continual Learning of Domain Knowledge from Human Feedback in Text-to-SQL","abstract":"Large Language Models (LLMs) can generate SQL queries from natural language questions but struggle with database-specific schemas and tacit domain knowledge. We introduce a framework for continual learning from human feedback in text-to-SQL, where a learning agent receives natural language feedback to refine queries and distills the revealed knowledge for reuse on future tasks. This distilled knowledge is stored in a structured memory, enabling the agent to improve execution accuracy over time. We design and evaluate multiple variations of a learning agent architecture that vary in how they capture and retrieve past experiences. Experiments on the BIRD benchmark Dev set show that memory-augmented agents, particularly the Procedural Agent, achieve significant accuracy gains and error reduction by leveraging human-in-the-loop feedback. Our results highlight the importance of transforming tacit human expertise into reusable knowledge, paving the way for more adaptive, domain-aware text-to-SQL systems that continually learn from a human-in-the-loop.","short_abstract":"Large Language Models (LLMs) can generate SQL queries from natural language questions but struggle with database-specific schemas and tacit domain knowledge. We introduce a framework for continual learning from human feedback in text-to-SQL, where a learning agent receives natural language feedback to refine queries an...","url_abs":"https://arxiv.org/abs/2511.10674","url_pdf":"https://arxiv.org/pdf/2511.10674v2","authors":"[\"Thomas Cook\",\"Kelly Patel\",\"Sivapriya Vellaichamy\",\"Udari Madhushani Sehwag\",\"Saba Rahimi\",\"Zhen Zeng\",\"Sumitra Ganesh\"]","published":"2025-11-10T05:29:10Z","proceeding":"cs.CL","tasks":"[\"cs.CL\",\"cs.AI\",\"cs.DB\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false}