{"ID":2833869,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2512.02494","arxiv_id":"2512.02494","title":"A Fully First-Order Layer for Differentiable Optimization","abstract":"Differentiable optimization layers enable learning systems to make decisions by solving embedded optimization problems. However, computing gradients via implicit differentiation requires solving a linear system with Hessian terms, which is both compute- and memory-intensive. To address this challenge, we propose a novel algorithm that computes the gradient using only first-order information. The key insight is to rewrite the differentiable optimization as a bilevel optimization problem and leverage recent advances in bilevel methods. Specifically, we introduce an active-set Lagrangian hypergradient oracle that avoids Hessian evaluations and provides finite-time, non-asymptotic approximation guarantees. We show that an approximate hypergradient can be computed using only first-order information in $\\tilde{\\oo}(1)$ time, leading to an overall complexity of $\\tilde{\\oo}(δ^{-1}ε^{-3})$ for constrained bilevel optimization, which matches the best known rate for non-smooth non-convex optimization. Furthermore, we release an open-source Python library that can be easily adapted from existing solvers. Our code is available here: https://github.com/guaguakai/FFOLayer.","short_abstract":"Differentiable optimization layers enable learning systems to make decisions by solving embedded optimization problems. However, computing gradients via implicit differentiation requires solving a linear system with Hessian terms, which is both compute- and memory-intensive. To address this challenge, we propose a nove...","url_abs":"https://arxiv.org/abs/2512.02494","url_pdf":"https://arxiv.org/pdf/2512.02494v1","authors":"[\"Zihao Zhao\",\"Kai-Chia Mo\",\"Shing-Hei Ho\",\"Brandon Amos\",\"Kai Wang\"]","published":"2025-12-02T07:36:03Z","proceeding":"cs.LG","tasks":"[\"cs.LG\"]","methods":"[]","has_code":false,"code_links":[{"ID":606359,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2833869,"paper_url":"https://arxiv.org/abs/2512.02494","paper_title":"A Fully First-Order Layer for Differentiable Optimization","repo_url":"https://github.com/guaguakai/FFOLayer","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}