{"ID":2828071,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2512.15489","arxiv_id":"2512.15489","title":"Nemotron-Math: Efficient Long-Context Distillation of Mathematical Reasoning from Multi-Mode Supervision","abstract":"High-quality mathematical reasoning supervision requires diverse reasoning styles, long-form traces, and effective tool integration, capabilities that existing datasets provide only in limited form. Leveraging the multi-mode generation ability of gpt-oss-120b, we introduce Nemotron-Math, a large-scale mathematical reasoning dataset containing 7.5M solution traces across high, medium, and low reasoning modes, each available both with and without Python tool-integrated reasoning (TIR). The dataset integrates 85K curated AoPS problems with 262K community-sourced StackExchange-Math problems, combining structured competition tasks with diverse real-world mathematical queries. We conduct controlled evaluations to assess the dataset quality. Nemotron-Math consistently outperforms the original OpenMathReasoning on matched AoPS problems. Incorporating StackExchange-Math substantially improves robustness and generalization, especially on HLE-Math, while preserving accuracy on math competition benchmarks. To support efficient long-context training, we develop a sequential bucketed strategy that accelerates 128K context-length fine-tuning by 2--3$\\times$ without significant accuracy loss. Overall, Nemotron-Math enables state-of-the-art performance, including 100\\% maj@16 accuracy on AIME 2024 and 2025 with Python TIR.","short_abstract":"High-quality mathematical reasoning supervision requires diverse reasoning styles, long-form traces, and effective tool integration, capabilities that existing datasets provide only in limited form. Leveraging the multi-mode generation ability of gpt-oss-120b, we introduce Nemotron-Math, a large-scale mathematical reas...","url_abs":"https://arxiv.org/abs/2512.15489","url_pdf":"https://arxiv.org/pdf/2512.15489v1","authors":"[\"Wei Du\",\"Shubham Toshniwal\",\"Branislav Kisacanin\",\"Sadegh Mahdavi\",\"Ivan Moshkov\",\"George Armstrong\",\"Stephen Ge\",\"Edgar Minasyan\",\"Feng Chen\",\"Igor Gitman\"]","published":"2025-12-17T14:37:41Z","proceeding":"cs.AI","tasks":"[\"cs.AI\"]","methods":"[]","has_code":false}