{"ID":2890992,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2507.18555","arxiv_id":"2507.18555","title":"Neural Tangent Kernels and Fisher Information Matrices for Simple ReLU Networks with Random Hidden Weights","abstract":"Fisher information matrices and neural tangent kernels (NTK) for 2-layer ReLU networks with random hidden weight are argued. We discuss the relation between both notions as a linear transformation and show that spectral decomposition of NTK with concrete forms of eigenfunctions with major eigenvalues. We also obtain an approximation formula of the functions presented by the 2-layer neural networks.","short_abstract":"Fisher information matrices and neural tangent kernels (NTK) for 2-layer ReLU networks with random hidden weight are argued. We discuss the relation between both notions as a linear transformation and show that spectral decomposition of NTK with concrete forms of eigenfunctions with major eigenvalues. We also obtain an...","url_abs":"https://arxiv.org/abs/2507.18555","url_pdf":"https://arxiv.org/pdf/2507.18555v2","authors":"[\"Jun'ichi Takeuchi\",\"Yoshinari Takeishi\",\"Noboru Murata\",\"Kazushi Mimura\",\"Ka Long Keith Ho\",\"Hiroshi Nagaoka\"]","published":"2025-07-24T16:26:52Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"stat.ML\"]","methods":"[]","has_code":false}