{"ID":6536417,"CreatedAt":"2026-07-14T01:21:01.169441415Z","UpdatedAt":"2026-07-14T11:39:13.791169299Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.10263","arxiv_id":"2607.10263","title":"Sharper Analysis of Single-Loop Methods for Bilevel Optimization","abstract":"Bilevel optimization underpins many machine learning applications, including hyperparameter optimization, meta-learning, neural architecture search, and reinforcement learning. While hypergradient-based methods have advanced significantly, a gap persists between theoretical guarantees and practical single-loop implementations required for efficiency. We bridge this gap by establishing sharper convergence results for single-loop approximate implicit differentiation (AID) and iterative differentiation (ITD) methods, leveraging our proposed analytical framework, decoupled norm analysis (DNA). For AID, we improve the convergence rate from $\\mathcal{O}(κ^6/K)$ to $\\mathcal{O}(κ^5/K)$, where $κ$ is the condition number of the inner-level problem. For ITD, we prove that the asymptotic error is $\\mathcal{O}(κ^2)$, exactly matching the known lower bound and improving upon the previous $\\mathcal{O}(κ^3)$ guarantee. Numerical experiments on synthetic and real tasks corroborate our theoretical findings.","short_abstract":"Bilevel optimization underpins many machine learning applications, including hyperparameter optimization, meta-learning, neural architecture search, and reinforcement learning. While hypergradient-based methods have advanced significantly, a gap persists between theoretical guarantees and practical single-loop implemen...","url_abs":"https://arxiv.org/abs/2607.10263","url_pdf":"https://arxiv.org/pdf/2607.10263v1","authors":"[\"Yubo Zhou\",\"Jun Shu\",\"Luo Luo\",\"Junmin Liu\",\"Deyu Meng\",\"Guang Dai\",\"Haishan Ye\"]","published":"2026-07-11T11:41:42Z","proceeding":"cs.LG","tasks":"[\"cs.LG\"]","methods":"[\"Reinforcement Learning\"]","has_code":false}