{"ID":2862202,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.01105","arxiv_id":"2510.01105","title":"Geometric Analysis of Neural Regression Collapse via Intrinsic Dimension","abstract":"Neural multivariate regression underpins a wide range of domains, including control, robotics, and finance, yet the geometry of its learned representations remains poorly characterized. While neural collapse has been shown to benefit generalization in classification, we find that analogous collapse in regression consistently degrades performance. To explain this contrast, we analyze regression models through the lens of intrinsic dimension. Across control tasks and synthetic datasets, we estimate the intrinsic dimension of last-layer features (ID_H) and compare it with that of the regression targets (ID_Y). Collapsed models exhibit ID_H \u003c ID_Y, leading to over-compression and poor generalization, whereas non-collapsed models typically maintain ID_H \u003e ID_Y. For the non-collapsed models, performance with respect to ID_H depends on the data quantity and noise levels. From these observations, we identify two regimes (over-compressed and under-compressed) that determine when expanding or reducing feature dimensionality improves performance. Our results provide new geometric insights into neural regression collapse and suggest practical strategies for enhancing generalization.","short_abstract":"Neural multivariate regression underpins a wide range of domains, including control, robotics, and finance, yet the geometry of its learned representations remains poorly characterized. While neural collapse has been shown to benefit generalization in classification, we find that analogous collapse in regression consis...","url_abs":"https://arxiv.org/abs/2510.01105","url_pdf":"https://arxiv.org/pdf/2510.01105v2","authors":"[\"George Andriopoulos\",\"Zixuan Dong\",\"Bimarsha Adhikari\",\"Keith Ross\"]","published":"2025-10-01T16:50:57Z","proceeding":"cs.LG","tasks":"[\"cs.LG\"]","methods":"[]","has_code":false}