{"ID":2852652,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.17314","arxiv_id":"2510.17314","title":"Auto-Rubric: Learning From Implicit Weights to Explicit Rubrics for Reward Modeling","abstract":"Conventional reward modeling relies on gradient descent over neural weights, creating opaque, data-hungry \"black boxes.\" We propose a paradigm shift from implicit to explicit reward parameterization, recasting optimization from continuous weight spaces to the discrete space of natural language rubrics. We introduce a training-free framework based on iterative rubric learning: it locally induces discriminative criteria via verification-driven refinement, and globally compresses the candidate criteria pool into a compact core set by maximizing an information-theoretic coding rate objective. We organize the compressed core set into a hierarchical rubric structure -- high-level evaluation dimensions supported by concrete verification checks -- serving as an interpretable, portable reward function. Empirically, our approach challenges prevailing data scaling assumptions: using only 70 preference pairs, our rubric-guided judges outperform fully trained reward models on diverse benchmarks. For instance, Qwen3-8B equipped with our learned rubrics achieves 80.91% on RewardBench2, surpassing the specialized Skywork-Reward-V2-Qwen3-8B (78.20%). These results demonstrate that alignment signals are highly compressible and can be effectively captured through explicit symbolic search.","short_abstract":"Conventional reward modeling relies on gradient descent over neural weights, creating opaque, data-hungry \"black boxes.\" We propose a paradigm shift from implicit to explicit reward parameterization, recasting optimization from continuous weight spaces to the discrete space of natural language rubrics. We introduce a t...","url_abs":"https://arxiv.org/abs/2510.17314","url_pdf":"https://arxiv.org/pdf/2510.17314v2","authors":"[\"Lipeng Xie\",\"Sen Huang\",\"Zhuo Zhang\",\"Anni Zou\",\"Yunpeng Zhai\",\"Dingchao Ren\",\"Kezun Zhang\",\"Haoyuan Hu\",\"Boyin Liu\",\"Haoran Chen\",\"Zhaoyang Liu\",\"Bolin Ding\"]","published":"2025-10-20T09:01:37Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.AI\"]","methods":"[\"Generative Adversarial Network\"]","has_code":false}