{"ID":2855401,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.14068","arxiv_id":"2510.14068","title":"On the expressivity of sparse maxout networks","abstract":"We study the expressivity of sparse maxout networks, where each neuron takes a fixed number of inputs from the previous layer and employs a, possibly multi-argument, maxout activation. This setting captures key characteristics of convolutional or graph neural networks. We establish a duality between functions computable by such networks and a class of virtual polytopes, linking their geometry to questions of network expressivity. In particular, we derive a tight bound on the dimension of the associated polytopes, which serves as the central tool for our analysis. Building on this, we construct a sequence of depth hierarchies. While sufficiently deep sparse maxout networks are universal, we prove that if the required depth is not reached, width alone cannot compensate for the sparsity of a fixed indegree constraint.","short_abstract":"We study the expressivity of sparse maxout networks, where each neuron takes a fixed number of inputs from the previous layer and employs a, possibly multi-argument, maxout activation. This setting captures key characteristics of convolutional or graph neural networks. We establish a duality between functions computabl...","url_abs":"https://arxiv.org/abs/2510.14068","url_pdf":"https://arxiv.org/pdf/2510.14068v1","authors":"[\"Moritz Grillo\",\"Tobias Hofmann\"]","published":"2025-10-15T20:18:18Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.AI\",\"math.CO\"]","methods":"[\"Graph Neural Network\"]","has_code":false}