{"ID":2874538,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2509.04027","arxiv_id":"2509.04027","title":"CoT-Space: A Theoretical Framework for Internal Slow-Thinking via Reinforcement Learning","abstract":"Reinforcement Learning (RL) has become a pivotal approach for enhancing the reasoning capabilities of Large Language Models (LLMs). However, a significant theoretical gap persists, as traditional token-level RL frameworks fail to align with the reasoning-level nature of complex, multi-step thought processes like Chain-of-Thought (CoT). To address this challenge, we introduce CoT-Space, a novel theoretical framework that recasts LLM reasoning from a discrete token-prediction task to an optimization process within a continuous, reasoning-level semantic space. This shift in perspective serves as a conceptual bridge, revitalizing foundational principles from classical learning theory to analyze the unique dynamics of LLMs. By analyzing this process from both a noise perspective and a risk perspective, we demonstrate that the convergence to an optimal CoT length is a natural consequence of the fundamental trade-off between underfitting and overfitting. Furthermore, extensive experiments provide strong empirical validation for our theoretical findings. Our framework not only provides a coherent explanation for empirical phenomena such as overthinking but also offers a solid theoretical foundation to guide the future development of more effective and generalizable reasoning agents. We open-source our code at https://github.com/ZyGan1999/CoT-Space.","short_abstract":"Reinforcement Learning (RL) has become a pivotal approach for enhancing the reasoning capabilities of Large Language Models (LLMs). However, a significant theoretical gap persists, as traditional token-level RL frameworks fail to align with the reasoning-level nature of complex, multi-step thought processes like Chain-...","url_abs":"https://arxiv.org/abs/2509.04027","url_pdf":"https://arxiv.org/pdf/2509.04027v2","authors":"[\"Zeyu Gan\",\"Hao Yi\",\"Yong Liu\"]","published":"2025-09-04T09:02:16Z","proceeding":"cs.AI","tasks":"[\"cs.AI\",\"cs.CL\"]","methods":"[\"Reinforcement Learning\",\"Large Language Model\",\"Language Model\",\"Generative Adversarial Network\"]","has_code":false,"code_links":[{"ID":610154,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2874538,"paper_url":"https://arxiv.org/abs/2509.04027","paper_title":"CoT-Space: A Theoretical Framework for Internal Slow-Thinking via Reinforcement Learning","repo_url":"https://github.com/ZyGan1999/CoT-Space","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}