{"ID":2858887,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.07202","arxiv_id":"2510.07202","title":"An in-depth look at approximation via deep and narrow neural networks","abstract":"In 2017, Hanin and Sellke showed that the class of arbitrarily deep, real-valued, feed-forward and ReLU-activated networks of width w forms a dense subset of the space of continuous functions on R^n, with respect to the topology of uniform convergence on compact sets, if and only if w\u003en holds. To show the necessity, a concrete counterexample function f:R^n-\u003eR was used. In this note we actually approximate this very f by neural networks in the two cases w=n and w=n+1 around the aforementioned threshold. We study how the approximation quality behaves if we vary the depth and what effect (spoiler alert: dying neurons) cause that behavior.","short_abstract":"In 2017, Hanin and Sellke showed that the class of arbitrarily deep, real-valued, feed-forward and ReLU-activated networks of width w forms a dense subset of the space of continuous functions on R^n, with respect to the topology of uniform convergence on compact sets, if and only if w\u003en holds. To show the necessity, a...","url_abs":"https://arxiv.org/abs/2510.07202","url_pdf":"https://arxiv.org/pdf/2510.07202v1","authors":"[\"Joris Dommel\",\"Sven A. Wegner\"]","published":"2025-10-08T16:34:45Z","proceeding":"cs.LG","tasks":"[\"cs.LG\"]","methods":"[]","has_code":false}