{"ID":2843805,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2511.06812","arxiv_id":"2511.06812","title":"Convergence of Actor-Critic Learning for Mean Field Games and Mean Field Control in Continuous Spaces","abstract":"We establish the convergence of the deep actor-critic reinforcement learning algorithm presented in [Angiuli et al., 2023a] in the setting of continuous state and action spaces with an infinite discrete-time horizon. This algorithm provides solutions to Mean Field Game (MFG) or Mean Field Control (MFC) problems depending on the ratio between two learning rates: one for the value function and the other for the mean field term. In the MFC case, to rigorously identify the limit, we introduce a discretization of the state and action spaces, following the approach used in the finite-space case in [Angiuli et al., 2023b]. The convergence proofs rely on a generalization of the two-timescale framework introduced in [Borkar, 1997]. We further extend our convergence results to Mean Field Control Games, which involve locally cooperative and globally competitive populations. Finally, we present numerical experiments for linear-quadratic problems in one and two dimensions, for which explicit solutions are available.","short_abstract":"We establish the convergence of the deep actor-critic reinforcement learning algorithm presented in [Angiuli et al., 2023a] in the setting of continuous state and action spaces with an infinite discrete-time horizon. This algorithm provides solutions to Mean Field Game (MFG) or Mean Field Control (MFC) problems dependi...","url_abs":"https://arxiv.org/abs/2511.06812","url_pdf":"https://arxiv.org/pdf/2511.06812v1","authors":"[\"Jean-Pierre Fouque\",\"Mathieu Laurière\",\"Mengrui Zhang\"]","published":"2025-11-10T07:55:34Z","proceeding":"math.OC","tasks":"[\"math.OC\",\"cs.LG\",\"math.PR\"]","methods":"[\"Reinforcement Learning\"]","has_code":false}