What is the objective of reasoning with reinforcement learning?

cs.LG arXiv:2510.13651
View PDF arXiv JSON

Abstract

We show that several popular algorithms for reinforcement learning in large language models with binary rewards can be viewed as stochastic gradient ascent on a monotone transform of the probability of a correct answer given a prompt. In particular, the transformation associated with rejection sampling algorithms is the logarithm and that associated with the GRPO algorithm is the arcsine of the square root.

PDF Viewer