{"ID":2854732,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.14844","arxiv_id":"2510.14844","title":"Provable Unlearning with Gradient Ascent on Two-Layer ReLU Neural Networks","abstract":"Machine Unlearning aims to remove specific data from trained models, addressing growing privacy and ethical concerns. We provide a theoretical analysis of a simple and widely used method - gradient ascent - used to reverse the influence of a specific data point without retraining from scratch. Leveraging the implicit bias of gradient descent towards solutions that satisfy the Karush-Kuhn-Tucker (KKT) conditions of a margin maximization problem, we quantify the quality of the unlearned model by evaluating how well it satisfies these conditions w.r.t. the retained data. To formalize this idea, we propose a new success criterion, termed \\textbf{$(ε, δ, τ)$-successful} unlearning, and show that, for both linear models and two-layer neural networks with high dimensional data, a properly scaled gradient-ascent step satisfies this criterion and yields a model that closely approximates the retrained solution on the retained data. We also show that gradient ascent performs successful unlearning while still preserving generalization in a synthetic Gaussian-mixture setting.","short_abstract":"Machine Unlearning aims to remove specific data from trained models, addressing growing privacy and ethical concerns. We provide a theoretical analysis of a simple and widely used method - gradient ascent - used to reverse the influence of a specific data point without retraining from scratch. Leveraging the implicit b...","url_abs":"https://arxiv.org/abs/2510.14844","url_pdf":"https://arxiv.org/pdf/2510.14844v1","authors":"[\"Odelia Melamed\",\"Gilad Yehudai\",\"Gal Vardi\"]","published":"2025-10-16T16:16:36Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.CR\",\"cs.NE\",\"stat.ML\"]","methods":"[]","has_code":false}