{"ID":2835533,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2511.23212","arxiv_id":"2511.23212","title":"Asymptotic Theory and Phase Transitions for Variable Importance in Quantile Regression Forests","abstract":"Quantile Regression Forests (QRF) are widely used for non-parametric conditional quantile estimation, yet statistical inference for variable importance measures remains challenging due to the non-smoothness of the loss function and the complex bias-variance trade-off. In this paper, we develop a asymptotic theory for variable importance defined as the difference in pinball loss risks. We first establish the asymptotic normality of the QRF estimator by handling the non-differentiable pinball loss via Knight's identity. Second, we uncover a \"phase transition\" phenomenon governed by the subsampling rate $β$ (where $s \\asymp n^β$). We prove that in the bias-dominated regime ($β\\ge 1/2$), which corresponds to large subsample sizes typically favored in practice to maximize predictive accuracy, standard inference breaks down as the estimator converges to a deterministic bias constant rather than a zero-mean normal distribution. Finally, we derive the explicit analytic form of this asymptotic bias and discuss the theoretical feasibility of restoring valid inference via analytic bias correction. Our results highlight a fundamental trade-off between predictive performance and inferential validity, providing a theoretical foundation for understanding the intrinsic limitations of random forest inference in high-dimensional settings.","short_abstract":"Quantile Regression Forests (QRF) are widely used for non-parametric conditional quantile estimation, yet statistical inference for variable importance measures remains challenging due to the non-smoothness of the loss function and the complex bias-variance trade-off. In this paper, we develop a asymptotic theory for v...","url_abs":"https://arxiv.org/abs/2511.23212","url_pdf":"https://arxiv.org/pdf/2511.23212v1","authors":"[\"Tomoshige Nakamura\",\"Hiroshi Shiraishi\"]","published":"2025-11-28T14:18:05Z","proceeding":"stat.ML","tasks":"[\"stat.ML\",\"cs.LG\",\"math.ST\"]","methods":"[]","has_code":false}