{"ID":2855268,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.13651","arxiv_id":"2510.13651","title":"What is the objective of reasoning with reinforcement learning?","abstract":"We show that several popular algorithms for reinforcement learning in large language models with binary rewards can be viewed as stochastic gradient ascent on a monotone transform of the probability of a correct answer given a prompt. In particular, the transformation associated with rejection sampling algorithms is the logarithm and that associated with the GRPO algorithm is the arcsine of the square root.","short_abstract":"We show that several popular algorithms for reinforcement learning in large language models with binary rewards can be viewed as stochastic gradient ascent on a monotone transform of the probability of a correct answer given a prompt. In particular, the transformation associated with rejection sampling algorithms is th...","url_abs":"https://arxiv.org/abs/2510.13651","url_pdf":"https://arxiv.org/pdf/2510.13651v1","authors":"[\"Damek Davis\",\"Benjamin Recht\"]","published":"2025-10-15T15:13:38Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"math.OC\"]","methods":"[\"Reinforcement Learning\",\"Language Model\"]","has_code":false}