{"ID":5675207,"CreatedAt":"2026-07-03T01:40:09.565152011Z","UpdatedAt":"2026-07-05T07:38:12.860786193Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.01830","arxiv_id":"2607.01830","title":"Many Voices, One Reward: Multi-Role Rubric Generation for LLM Judging and Reward Modeling","abstract":"Reliable reward and preference signals are critical for evaluating and optimizing large language models on open-ended tasks. Rubric-based judges offer a transparent way to decompose such judgments into explicit evaluation criteria, but existing annotation-free rubric generators typically rely on a single generic evaluator. As a result, they may overlook important dimensions of human preference, a failure mode we term dimensional blind spots. To address this limitation, we propose Multi-Role Rubric Generation (MRRG), a training-free and reference-free framework that elicits evaluation criteria from multiple complementary roles and consolidates them into an auditable rubric-based scorer. This scorer can be used both to validate pairwise preferences and to provide rewards for GRPO-style Reinforcement Learning with Verifiable Rewards (RLVR). Experiments on preference validation benchmarks show that MRRG consistently outperforms single-role rubric generation baselines across multiple backbone models. Further RLVR experiments demonstrate that MRRG yields a stronger reward signal for improving open-ended generation.","short_abstract":"Reliable reward and preference signals are critical for evaluating and optimizing large language models on open-ended tasks. Rubric-based judges offer a transparent way to decompose such judgments into explicit evaluation criteria, but existing annotation-free rubric generators typically rely on a single generic evalua...","url_abs":"https://arxiv.org/abs/2607.01830","url_pdf":"https://arxiv.org/pdf/2607.01830v1","authors":"[\"Dazhi Fu\",\"Jiuding Yang\",\"Yiwen Guo\",\"Jicong Fan\"]","published":"2026-07-02T07:50:38Z","proceeding":"cs.LG","tasks":"[\"cs.LG\"]","methods":"[\"Reinforcement Learning\",\"Large Language Model\",\"Language Model\"]","has_code":false}