{"ID":2877732,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2508.20036","arxiv_id":"2508.20036","title":"Eigenvalue distribution of the Neural Tangent Kernel in the quadratic scaling","abstract":"We compute the asymptotic eigenvalue distribution of the neural tangent kernel of a two-layer neural network under a specific scaling of dimension. Namely, if $X\\in\\mathbb{R}^{n\\times d}$ is an i.i.d random matrix, $W\\in\\mathbb{R}^{d\\times p}$ is an i.i.d $\\mathcal{N}(0,1)$ matrix and $D\\in\\mathbb{R}^{p\\times p}$ is a diagonal matrix with i.i.d bounded entries, we consider the matrix \\[ \\mathrm{NTK} = \\frac{1}{d}XX^\\top \\odot \\frac{1}{p} σ'\\left( \\frac{1}{\\sqrt{d}}XW \\right)D^2 σ'\\left( \\frac{1}{\\sqrt{d}}XW \\right)^\\top \\] where $σ'$ is a pseudo-Lipschitz function applied entrywise and under the scaling $\\frac{n}{dp}\\to γ_1$ and $\\frac{p}{d}\\to γ_2$. We describe the asymptotic distribution as the free multiplicative convolution of the Marchenko--Pastur distribution with a deterministic distribution depending on $σ$ and $D$.","short_abstract":"We compute the asymptotic eigenvalue distribution of the neural tangent kernel of a two-layer neural network under a specific scaling of dimension. Namely, if $X\\in\\mathbb{R}^{n\\times d}$ is an i.i.d random matrix, $W\\in\\mathbb{R}^{d\\times p}$ is an i.i.d $\\mathcal{N}(0,1)$ matrix and $D\\in\\mathbb{R}^{p\\times p}$ is a...","url_abs":"https://arxiv.org/abs/2508.20036","url_pdf":"https://arxiv.org/pdf/2508.20036v1","authors":"[\"Lucas Benigni\",\"Elliot Paquette\"]","published":"2025-08-27T16:41:01Z","proceeding":"math.PR","tasks":"[\"math.PR\",\"stat.ML\"]","methods":"[]","has_code":false}