{"ID":2828554,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2512.14792","arxiv_id":"2512.14792","title":"IaC Generation with LLMs: An Error Taxonomy and A Study on Configuration Knowledge Injection","abstract":"Large Language Models (LLMs) currently exhibit low success rates in generating correct and intent-aligned Infrastructure as Code (IaC). This research investigated methods to improve LLM-based IaC generation, specifically for Terraform, by systematically injecting structured configuration knowledge. To facilitate this, an existing IaC-Eval benchmark was significantly enhanced with cloud emulation and automated error analysis. Additionally, a novel error taxonomy for LLM-assisted IaC code generation was developed. A series of knowledge injection techniques was implemented and evaluated, progressing from Naive Retrieval-Augmented Generation (RAG) to more sophisticated Graph RAG approaches. These included semantic enrichment of graph components and modeling inter-resource dependencies. Experimental results demonstrated that while baseline LLM performance was poor (27.1% overall success), injecting structured configuration knowledge increased technical validation success to 75.3% and overall success to 62.6%. Despite these gains in technical correctness, intent alignment plateaued, revealing a \"Correctness-Congruence Gap\" where LLMs can become proficient \"coders\" but remain limited \"architects\" in fulfilling nuanced user intent.","short_abstract":"Large Language Models (LLMs) currently exhibit low success rates in generating correct and intent-aligned Infrastructure as Code (IaC). This research investigated methods to improve LLM-based IaC generation, specifically for Terraform, by systematically injecting structured configuration knowledge. To facilitate this,...","url_abs":"https://arxiv.org/abs/2512.14792","url_pdf":"https://arxiv.org/pdf/2512.14792v1","authors":"[\"Roman Nekrasov\",\"Stefano Fossati\",\"Indika Kumara\",\"Damian Andrew Tamburri\",\"Willem-Jan van den Heuvel\"]","published":"2025-12-16T14:58:00Z","proceeding":"cs.AI","tasks":"[\"cs.AI\",\"cs.SE\"]","methods":"[\"RAG\",\"Large Language Model\",\"Language Model\"]","has_code":false}