{"ID":5443809,"CreatedAt":"2026-07-01T02:07:11.383974684Z","UpdatedAt":"2026-07-03T15:14:29.341850669Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.31808","arxiv_id":"2606.31808","title":"Large Databases Need Small, Open-Weight Language Models","abstract":"Language model systems built around proprietary APIs often operate on a token-based cost model. This becomes prohibitively expensive in the context of large databases, where LM-enhanced relational operators can incur costs exceeding $10,000 for a single set of experiments, hindering thorough research and practical deployment. In this paper, we demonstrate that quantized, open-weight models running locally on just 16GB of VRAM can match or exceed the accuracy of closed-source counterparts at lower latency and a fraction of the price, challenging the prevailing assumption that closed-source LM APIs are necessary for effective LM-database integration. We present and analyze the key system optimizations required to efficiently deploy these open-weight models within an LM-DB system. By integrating these local models into the BlendSQL v0.1.0 framework, we demonstrate a 390x reduction in overall costs and 3.8x reduction in latency compared to a proprietary LM API. We make our code available at https://github.com/CapitalOne-Research/play-by-the-type-rules/tree/main/sembench.","short_abstract":"Language model systems built around proprietary APIs often operate on a token-based cost model. This becomes prohibitively expensive in the context of large databases, where LM-enhanced relational operators can incur costs exceeding $10,000 for a single set of experiments, hindering thorough research and practical depl...","url_abs":"https://arxiv.org/abs/2606.31808","url_pdf":"https://arxiv.org/pdf/2606.31808v1","authors":"[\"Parker Glenn\",\"Alfy Samuel\"]","published":"2026-06-30T15:25:10Z","proceeding":"cs.AI","tasks":"[\"cs.AI\",\"cs.DB\"]","methods":"[\"Language Model\"]","has_code":false,"code_links":[{"ID":613810,"CreatedAt":"2026-07-01T02:07:11.383974684Z","UpdatedAt":"2026-07-01T02:07:11.383974684Z","DeletedAt":null,"paper_id":5443809,"paper_url":"https://arxiv.org/abs/2606.31808","paper_title":"Large Databases Need Small, Open-Weight Language Models","repo_url":"https://github.com/CapitalOne-Research/play-by-the-type-rules","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}