{"ID":5937821,"CreatedAt":"2026-07-07T03:14:33.014478982Z","UpdatedAt":"2026-07-08T22:40:00.334127079Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.04535","arxiv_id":"2607.04535","title":"ManifoldFlow: SPD-Relaxed Stiefel Layers with Learnable Singular Spectrum","abstract":"Orthogonal and Stiefel layers give neural weights exact spectral control, but they also impose a strong modeling constraint: all represented singular values are fixed at one. Many settings that benefit from an orthonormal basis still need direction-dependent attenuation or amplification. We introduce ManifoldFlow, a minimal relaxation of a fixed-spectrum Stiefel layer that keeps the basis on the Stiefel manifold while learning a bounded positive spectrum through W = Q S^{1/2}, with Q^T Q = I and S positive definite. Since W^T W = S, the eigenvalues of S are exactly the squared singular values of the realized weight, making eigenvalue clipping a direct singular-value control mechanism. Across paired sequence, tabular, and image experiments, the learnable SPD spectrum improves the fixed-spectrum Stiefel counterpart in the reported settings where the Stiefel prior is useful, with the largest gains in recurrent language-model projections. Boundary cases in convolutional classifier heads clarify the intended scope: ManifoldFlow is not a universal dense-layer replacement, but a spectrum-learnable Stiefel relaxation for settings where an orthonormal basis is a useful prior. When the basis should be orthonormal, its spectrum need not be frozen. Code available at https://github.com/Hik289/manifold_flow","short_abstract":"Orthogonal and Stiefel layers give neural weights exact spectral control, but they also impose a strong modeling constraint: all represented singular values are fixed at one. Many settings that benefit from an orthonormal basis still need direction-dependent attenuation or amplification. We introduce ManifoldFlow, a mi...","url_abs":"https://arxiv.org/abs/2607.04535","url_pdf":"https://arxiv.org/pdf/2607.04535v1","authors":"[\"Haiwen Yi\",\"Xinyuan Song\"]","published":"2026-07-05T22:46:17Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"stat.ML\"]","methods":"[]","has_code":false,"code_links":[{"ID":613989,"CreatedAt":"2026-07-07T03:14:33.014478982Z","UpdatedAt":"2026-07-07T03:14:33.014478982Z","DeletedAt":null,"paper_id":5937821,"paper_url":"https://arxiv.org/abs/2607.04535","paper_title":"ManifoldFlow: SPD-Relaxed Stiefel Layers with Learnable Singular Spectrum","repo_url":"https://github.com/Hik289/manifold_flow","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}