{"ID":2893688,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2507.13543","arxiv_id":"2507.13543","title":"Loss-Complexity Landscape and Model Structure Functions","abstract":"We develop a framework for dualizing the Kolmogorov structure function $h_x(α)$, which then allows using computable complexity proxies. We establish a mathematical analogy between information-theoretic constructs and statistical mechanics, introducing a suitable partition function and free energy functional. We explicitly prove the Legendre-Fenchel duality between the structure function and free energy, showing detailed balance of the Metropolis kernel, and interpret acceptance probabilities as information-theoretic scattering amplitudes. A susceptibility-like variance of model complexity is shown to peak precisely at loss-complexity trade-offs interpreted as phase transitions. Practical experiments with linear and tree-based regression models verify these theoretical predictions, explicitly demonstrating the interplay between the model complexity, generalization, and overfitting threshold.","short_abstract":"We develop a framework for dualizing the Kolmogorov structure function $h_x(α)$, which then allows using computable complexity proxies. We establish a mathematical analogy between information-theoretic constructs and statistical mechanics, introducing a suitable partition function and free energy functional. We explici...","url_abs":"https://arxiv.org/abs/2507.13543","url_pdf":"https://arxiv.org/pdf/2507.13543v4","authors":"[\"Alexander Kolpakov\"]","published":"2025-07-17T21:31:45Z","proceeding":"cs.IT","tasks":"[\"cs.IT\",\"cs.AI\",\"cs.LG\",\"math-ph\"]","methods":"[]","has_code":false}