{"ID":2896188,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2507.07853","arxiv_id":"2507.07853","title":"Optimization Guarantees for Square-Root Natural-Gradient Variational Inference","abstract":"Variational inference with natural-gradient descent often shows fast convergence in practice, but its theoretical convergence guarantees have been challenging to establish. This is true even for the simplest cases that involve concave log-likelihoods and use a Gaussian approximation. We show that the challenge can be circumvented for such cases using a square-root parameterization for the Gaussian covariance. This approach establishes novel convergence guarantees for natural-gradient variational-Gaussian inference and its continuous-time gradient flow. Our experiments demonstrate the effectiveness of natural gradient methods and highlight their advantages over algorithms that use Euclidean or Wasserstein geometries.","short_abstract":"Variational inference with natural-gradient descent often shows fast convergence in practice, but its theoretical convergence guarantees have been challenging to establish. This is true even for the simplest cases that involve concave log-likelihoods and use a Gaussian approximation. We show that the challenge can be c...","url_abs":"https://arxiv.org/abs/2507.07853","url_pdf":"https://arxiv.org/pdf/2507.07853v1","authors":"[\"Navish Kumar\",\"Thomas Möllenhoff\",\"Mohammad Emtiyaz Khan\",\"Aurelien Lucchi\"]","published":"2025-07-10T15:33:28Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.AI\",\"stat.ML\"]","methods":"[]","has_code":false}