{"ID":2895342,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2507.09103","arxiv_id":"2507.09103","title":"CoVAE: Consistency Training of Variational Autoencoders","abstract":"Current state-of-the-art generative approaches frequently rely on a two-stage training procedure, where an autoencoder (often a VAE) first performs dimensionality reduction, followed by training a generative model on the learned latent space. While effective, this introduces computational overhead and increased sampling times. We challenge this paradigm by proposing Consistency Training of Variational AutoEncoders (CoVAE), a novel single-stage generative autoencoding framework that adopts techniques from consistency models to train a VAE architecture. The CoVAE encoder learns a progressive series of latent representations with increasing encoding noise levels, mirroring the forward processes of diffusion and flow matching models. This sequence of representations is regulated by a time dependent $β$ parameter that scales the KL loss. The decoder is trained using a consistency loss with variational regularization, which reduces to a conventional VAE loss at the earliest latent time. We show that CoVAE can generate high-quality samples in one or few steps without the use of a learned prior, significantly outperforming equivalent VAEs and other single-stage VAEs methods. Our approach provides a unified framework for autoencoding and diffusion-style generative modeling and provides a viable route for one-step generative high-performance autoencoding. Our code is publicly available at https://github.com/gisilvs/covae.","short_abstract":"Current state-of-the-art generative approaches frequently rely on a two-stage training procedure, where an autoencoder (often a VAE) first performs dimensionality reduction, followed by training a generative model on the learned latent space. While effective, this introduces computational overhead and increased samplin...","url_abs":"https://arxiv.org/abs/2507.09103","url_pdf":"https://arxiv.org/pdf/2507.09103v1","authors":"[\"Gianluigi Silvestri\",\"Luca Ambrogioni\"]","published":"2025-07-12T01:32:08Z","proceeding":"stat.ML","tasks":"[\"stat.ML\",\"cs.LG\"]","methods":"[\"Diffusion Model\",\"Variational Autoencoder\"]","has_code":false,"code_links":[{"ID":612180,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2895342,"paper_url":"https://arxiv.org/abs/2507.09103","paper_title":"CoVAE: Consistency Training of Variational Autoencoders","repo_url":"https://github.com/gisilvs/covae","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}