{"ID":2921133,"CreatedAt":"2026-06-02T02:42:49.606572591Z","UpdatedAt":"2026-06-04T06:21:04.369492701Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.01787","arxiv_id":"2606.01787","title":"Stochastic convergence of parallel asynchronous adaptive first-order methods","abstract":"A new class of asynchronous adaptive first-order optimization methods is introduced, comprising asynchronous variants of several popular algorithms. Versions of these methods using momentum and/or inexact normalization are also considered. The convergence of methods in the class on non-convex functions is analyzed in a fully stochastic setting, and is shown to be (up to logarithmic factors) of order O(1/sqrt{t}) under reasonable assumptions. Numerical experiments suggest that such asynchronous adaptive algorithms are very relevant in heterogeneous large-scale machine learning systems.","short_abstract":"A new class of asynchronous adaptive first-order optimization methods is introduced, comprising asynchronous variants of several popular algorithms. Versions of these methods using momentum and/or inexact normalization are also considered. The convergence of methods in the class on non-convex functions is analyzed in a...","url_abs":"https://arxiv.org/abs/2606.01787","url_pdf":"https://arxiv.org/pdf/2606.01787v1","authors":"[\"Serge Gratton\",\"Philippe L. Toint\"]","published":"2026-06-01T07:08:32Z","proceeding":"cs.AI","tasks":"[\"cs.AI\",\"math.OC\"]","methods":"[]","has_code":false}