{"ID":2871067,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2509.12185","arxiv_id":"2509.12185","title":"The Morgan-Pitman Test of Equality of Variances and its Application to Machine Learning Model Evaluation and Selection","abstract":"Model selection in non-linear models often prioritizes performance metrics over statistical tests, limiting the ability to account for sampling variability. We propose the use of a statistical test to assess the equality of variances in forecasting errors. The test builds upon the classic Morgan-Pitman approach, incorporating enhancements to ensure robustness against data with heavy-tailed distributions or outliers with high variance, plus a strategy to make residuals from machine learning models statistically independent. Through a series of simulations and real-world data applications, we demonstrate the test's effectiveness and practical utility, offering a reliable tool for model evaluation and selection in diverse contexts.","short_abstract":"Model selection in non-linear models often prioritizes performance metrics over statistical tests, limiting the ability to account for sampling variability. We propose the use of a statistical test to assess the equality of variances in forecasting errors. The test builds upon the classic Morgan-Pitman approach, incorp...","url_abs":"https://arxiv.org/abs/2509.12185","url_pdf":"https://arxiv.org/pdf/2509.12185v1","authors":"[\"Argimiro Arratia\",\"Alejandra Cabaña\",\"Ernesto Mordecki\",\"Gerard Rovira-Parra\"]","published":"2025-09-15T17:47:38Z","proceeding":"stat.ML","tasks":"[\"stat.ML\",\"cs.LG\",\"math.ST\"]","methods":"[\"Generative Adversarial Network\"]","has_code":false}