{"ID":2846115,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2511.02352","arxiv_id":"2511.02352","title":"SWE-Sharp-Bench: A Reproducible Benchmark for C# Software Engineering Tasks","abstract":"AI coding agents have shown great progress on Python software engineering benchmarks like SWE-Bench, and for other languages like Java and C in benchmarks like Multi-SWE-Bench. However, C# -- a prominent enterprise language ranking #5 in the TIOBE index -- remains absent from such benchmarks. We introduce SWE-Sharp-Bench, a reproducible software engineering benchmark for C# featuring 150 instances from 17 repositories. Evaluating identical model-agent configurations across languages reveals a significant performance gap: while 70% of Python tasks in SWE-Bench Verified are solved, only 40% of our C# tasks are resolved. We open-source SWE-Sharp-Bench and our entire curation pipeline.","short_abstract":"AI coding agents have shown great progress on Python software engineering benchmarks like SWE-Bench, and for other languages like Java and C in benchmarks like Multi-SWE-Bench. However, C# -- a prominent enterprise language ranking #5 in the TIOBE index -- remains absent from such benchmarks. We introduce SWE-Sharp-Ben...","url_abs":"https://arxiv.org/abs/2511.02352","url_pdf":"https://arxiv.org/pdf/2511.02352v3","authors":"[\"Sanket Mhatre\",\"Yasharth Bajpai\",\"Sumit Gulwani\",\"Emerson Murphy-Hill\",\"Gustavo Soares\"]","published":"2025-11-04T08:16:23Z","proceeding":"cs.SE","tasks":"[\"cs.SE\"]","methods":"[]","has_code":false}