{"ID":2921976,"CreatedAt":"2026-06-02T02:42:49.606572591Z","UpdatedAt":"2026-06-02T05:10:42.929021005Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.00539","arxiv_id":"2606.00539","title":"GNMR: Runtime Stability Control for Low-Precision Large Language Model Training","abstract":"Training stability is a key bottleneck in low-precision language model training: efficient low-cost paths can still produce short-lived numerical risks at a small set of operators. We formulate this as runtime stability control and present Gradient Norm-to-Mean Ratio (GNMR), a lightweight controller that compares each recoverable unit's current gradient norm with its historical mean. Together with $Δ$-GNMR for abrupt short-window increases, GNMR maps local risk signals to bounded recovery actions under a hard $\\mathrm{maxO}$ budget and a short lock interval, without changing the numerical format, kernel, or backend recipe. Across activation-quantization stress, DeepSeek-style recipe-level training, and LLaMA-2 13B fine-tuning, GNMR preserves high-fidelity quality with sparse, budgeted recovery. These results support GNMR as a backend-agnostic controller to improve low-precision training stability while preserving low-cost execution.","short_abstract":"Training stability is a key bottleneck in low-precision language model training: efficient low-cost paths can still produce short-lived numerical risks at a small set of operators. We formulate this as runtime stability control and present Gradient Norm-to-Mean Ratio (GNMR), a lightweight controller that compares each...","url_abs":"https://arxiv.org/abs/2606.00539","url_pdf":"https://arxiv.org/pdf/2606.00539v1","authors":"[\"Boao Kong\",\"Weichen Jia\",\"Engao Zhang\",\"Guohong Li\",\"Yonghan Dong\",\"Yao Wang\",\"Yaoyuan Wang\",\"Yunke Peng\",\"Kun Yuan\"]","published":"2026-05-30T05:11:13Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"math.OC\",\"stat.ML\"]","methods":"[\"Language Model\"]","has_code":false}