{"ID":5937939,"CreatedAt":"2026-07-07T03:14:33.014478982Z","UpdatedAt":"2026-07-07T10:29:49.167883842Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.03809","arxiv_id":"2607.03809","title":"Stable Global Weighting of Flow Mixtures using Simplex Exponential Moving Average","abstract":"Normalising flows provide a powerful variational family for approximate inference, yet individual architectures often fail to generalise across heterogeneous posterior geometries. We revisit mixture-based flow formulations and introduce \\emph{AMF\\mbox{-}VI\\mbox{-}sEMA}, a two-stage framework featuring a \\emph{stable global weighting} mechanism based on a \\emph{Simplex Exponential Moving Average} (sEMA) update. In Stage~1, a heterogeneous set of experts (\\textsc{RealNVP}, \\textsc{MAF}, \\textsc{RBIG}) are trained independently to specialise in distinct structural regimes. In Stage~2, expert parameters are frozen and global mixture weights are learned through a temperature-controlled softmax of average log-likelihoods, followed by a smooth EMA update on the probability simplex. This design produces a tractable, data-agnostic gating mechanism (without per-sample gating or gradient backpropagation through weights) that adaptively reallocates capacity while avoiding component collapse. We evaluate the framework on ten posterior benchmarks: six canonical 2D synthetic families (Banana, X-Shaped, Bimodal, Multimodal, Two-moons, Rings) and four real/low-dimensional Bayesian targets (BLR, BPR, Weibull, Real-GMM2), with stronger baselines (\\textsc{NICE}, \\textsc{ResFlow}, and EM-Mixing). Comprehensive evaluation covers NLL, KL divergence, Wasserstein-2 distance, and MMD, together with diagnostics of mixture dynamics, hyperparameter sensitivity, and cross-seed robustness. Empirically, \\emph{AMF\\mbox{-}VI\\mbox{-}sEMA} achieves consistent NLL improvements over its predecessor \\emph{AMF\\mbox{-}VI} and avoids the catastrophic transport failures of single-flow baselines, while maintaining stable weight trajectories ($N_{\\mathrm{eff}}{\u003e}1.4$ on all datasets) with minimal computational overhead.","short_abstract":"Normalising flows provide a powerful variational family for approximate inference, yet individual architectures often fail to generalise across heterogeneous posterior geometries. We revisit mixture-based flow formulations and introduce \\emph{AMF\\mbox{-}VI\\mbox{-}sEMA}, a two-stage framework featuring a \\emph{stable gl...","url_abs":"https://arxiv.org/abs/2607.03809","url_pdf":"https://arxiv.org/pdf/2607.03809v1","authors":"[\"Benjamin Wiriyapong\",\"Oktay Karakus\",\"Can Eyupoglu\",\"Kirill Sidorov\"]","published":"2026-07-04T10:31:10Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"stat.ML\"]","methods":"[]","has_code":false}