{"ID":3006153,"CreatedAt":"2026-06-03T03:09:48.883664427Z","UpdatedAt":"2026-06-04T19:14:31.964469513Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.02993","arxiv_id":"2606.02993","title":"Neural Networks Provably Learn Spectral Representations for Group Composition","abstract":"Understanding how structured internal structure emerges during neural network training is central to the study of deep learning. We investigate this phenomenon through the group composition task, where a two-layer neural network is trained to predict $g_1 \\star g_2$ for elements of a finite group $G$. By lifting the projected gradient flow to the Fourier domain, we demonstrate that the training dynamics are governed by a Riemannian gradient ascent on a representation-theoretic energy functional. We prove that, under random initialization, this flow drives each neuron to converge almost surely toward a single irreducible representation, while the cross-layer Fourier coefficients achieve a rotational rank-one alignment. This framework provides a representation-theoretic account of feature learning and characterizes a novel low-rank compression phenomenon for matrix-valued group representations. Moreover, for Abelian groups, we provide a complete population-level description: random initialization promotes uniform diversification across nontrivial representations and induces Haar-uniform phases, jointly approximating the indicator via a majority-vote mechanism. We further prove that both phase alignment and representation competition emerge with exponential convergence rates.","short_abstract":"Understanding how structured internal structure emerges during neural network training is central to the study of deep learning. We investigate this phenomenon through the group composition task, where a two-layer neural network is trained to predict $g_1 \\star g_2$ for elements of a finite group $G$. By lifting the pr...","url_abs":"https://arxiv.org/abs/2606.02993","url_pdf":"https://arxiv.org/pdf/2606.02993v1","authors":"[\"Jianliang He\",\"Leda Wang\",\"Fengzhuo Zhang\",\"Siyu Chen\",\"Zhuoran Yang\"]","published":"2026-06-02T01:04:21Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"math.OC\",\"math.RT\",\"math.ST\",\"stat.ML\"]","methods":"[]","has_code":false}