{"ID":2823431,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2601.00276","arxiv_id":"2601.00276","title":"Task-Driven Kernel Flows: Label Rank Compression and Laplacian Spectral Filtering","abstract":"We present a theory of feature learning in wide L2-regularized networks showing that supervised learning is inherently compressive. We derive a kernel ODE that predicts a \"water-filling\" spectral evolution and prove that for any stable steady state, the kernel rank is bounded by the number of classes ($C$). We further demonstrate that SGD noise is similarly low-rank ($O(C)$), confining dynamics to the task-relevant subspace. This framework unifies the deterministic and stochastic views of alignment and contrasts the low-rank nature of supervised learning with the high-rank, expansive representations of self-supervision.","short_abstract":"We present a theory of feature learning in wide L2-regularized networks showing that supervised learning is inherently compressive. We derive a kernel ODE that predicts a \"water-filling\" spectral evolution and prove that for any stable steady state, the kernel rank is bounded by the number of classes ($C$). We further...","url_abs":"https://arxiv.org/abs/2601.00276","url_pdf":"https://arxiv.org/pdf/2601.00276v1","authors":"[\"Hongxi Li\",\"Chunlin Huang\"]","published":"2026-01-01T09:28:31Z","proceeding":"cs.LG","tasks":"[\"cs.LG\"]","methods":"[]","has_code":false}