{"ID":6620691,"CreatedAt":"2026-07-15T01:01:48.440468303Z","UpdatedAt":"2026-07-15T03:28:55.185153975Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.12790","arxiv_id":"2607.12790","title":"Who Grades the Grader? Co-Evolving Evaluation Metrics and Skills for Self-Improving LLM Agents","abstract":"Self-evolving agent systems improve by creating, revising, and retiring their own skills, but every such loop rests on a hidden assumption: a reliable evaluation metric already exists. In many real applications it does not. We make three claims. First, metrics can be \\emph{evolved}: our metric loop searches compositions of small drawback detectors under a full evolutionary lifecycle, trained to agree with a ten-item anchored reference set, regularized by consensus over unlabeled outputs, and audited against a held-out anchor it never reads, yielding a transparent, inspectable metric rather than an opaque judge. Second, since no metric exists to beat, the yardstick is recovering what an accurate metric would have enabled, and \\emph{Double Ratchet}, our co-evolution of the metric with a lifecycle-managed skill loop, does so: across code generation (MBPP+), enterprise text-to-SQL (Spider~2.0-Snow), and reference-free report generation, it retains 88--110\\% of the held-out lift achieved by the same skill loop driven by ground truth or the best available rubric. Third, safety comes from anchor discipline plus outer audits: removing anchor guards collapses the metric into a vacuous detector while removing the lifecycle does not; and when evolved skills gamed the report rubric, an independent judge caught it, one detector repaired it, and a task-aware judge then preferred the evolved outputs over the pre-evolution baseline in 77\\% of decided pairs. We argue this failure-expecting architecture is the right default wherever no reliable automatic verifier exists.","short_abstract":"Self-evolving agent systems improve by creating, revising, and retiring their own skills, but every such loop rests on a hidden assumption: a reliable evaluation metric already exists. In many real applications it does not. We make three claims. First, metrics can be \\emph{evolved}: our metric loop searches composition...","url_abs":"https://arxiv.org/abs/2607.12790","url_pdf":"https://arxiv.org/pdf/2607.12790v1","authors":"[\"Xing Zhang\",\"Guanghui Wang\",\"Yanwei Cui\",\"Ziyuan Li\",\"Wei Qiu\",\"Bing Zhu\",\"Peiyang He\"]","published":"2026-07-14T14:02:50Z","proceeding":"cs.AI","tasks":"[\"cs.AI\",\"cs.CL\",\"cs.MA\"]","methods":"[\"Large Language Model\"]","has_code":false}