{"ID":2831884,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2512.08091","arxiv_id":"2512.08091","title":"Complexity of One-Dimensional ReLU DNNs","abstract":"We study the expressivity of one-dimensional (1D) ReLU deep neural networks through the lens of their linear regions. For randomly initialized, fully connected 1D ReLU networks (He scaling with nonzero bias) in the infinite-width limit, we prove that the expected number of linear regions grows as $\\sum_{i = 1}^L n_i + \\mathop{o}\\left(\\sum_{i = 1}^L{n_i}\\right) + 1$, where $n_\\ell$ denotes the number of neurons in the $\\ell$-th hidden layer. We also propose a function-adaptive notion of sparsity that compares the expected regions used by the network to the minimal number needed to approximate a target within a fixed tolerance.","short_abstract":"We study the expressivity of one-dimensional (1D) ReLU deep neural networks through the lens of their linear regions. For randomly initialized, fully connected 1D ReLU networks (He scaling with nonzero bias) in the infinite-width limit, we prove that the expected number of linear regions grows as $\\sum_{i = 1}^L n_i +...","url_abs":"https://arxiv.org/abs/2512.08091","url_pdf":"https://arxiv.org/pdf/2512.08091v1","authors":"[\"Jonathan Kogan\",\"Hayden Jananthan\",\"Jeremy Kepner\"]","published":"2025-12-08T23:01:41Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"stat.ML\"]","methods":"[]","has_code":false}