{"ID":5675966,"CreatedAt":"2026-07-03T01:40:09.565152011Z","UpdatedAt":"2026-07-04T19:15:18.090787218Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.01417","arxiv_id":"2607.01417","title":"Conditional Inference Trees and Forests for Feature Selection","abstract":"Conditional inference trees (CIT) and conditional inference forests (CIF) reduce split-selection bias by testing features before choosing split thresholds, but repeated permutation tests and threshold searches can make these methods computationally expensive. We study CIT and CIF as top-$k$ feature-ranking methods for downstream prediction using real-data benchmarks, runtime ablations, and synthetic feature-recovery experiments. At a fixed node, if the features and permutation budget do not depend on the node responses, Bonferroni-corrected $+1$ Monte Carlo permutation $p$-values control nodewise rejection under the complete permutation null. CIF ranks 4th among 17 classification methods on 22 datasets and 3rd among 18 regression methods on 8 datasets. With Bonferroni correction held fixed, the CIF runtime ablations indicate that adaptive stopping and the number of thresholds searched have the largest measured effect on runtime: turning off adaptive stopping and using exact threshold search increase fitting time by 4.0--8.4$\\times$ and 1.9--10.8$\\times$, respectively, while downstream score changes are at most 0.011. Sparse high-$p$ simulations indicate that forest feature sampling can leave informative features out of many split decisions. Overall, the results support CIF as a top-$k$ feature-ranking method in the evaluated downstream prediction benchmarks.","short_abstract":"Conditional inference trees (CIT) and conditional inference forests (CIF) reduce split-selection bias by testing features before choosing split thresholds, but repeated permutation tests and threshold searches can make these methods computationally expensive. We study CIT and CIF as top-$k$ feature-ranking methods for...","url_abs":"https://arxiv.org/abs/2607.01417","url_pdf":"https://arxiv.org/pdf/2607.01417v1","authors":"[\"Robert Milletich\",\"Justin Downes\",\"Steve Goley\",\"Newel Hirst\"]","published":"2026-07-01T19:23:07Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"stat.ML\"]","methods":"[]","has_code":false}