{"ID":2864277,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.03277","arxiv_id":"2510.03277","title":"Quantile-Scaled Bayesian Optimization Using Rank-Only Feedback","abstract":"Bayesian Optimization (BO) is widely used for optimizing expensive black-box functions, particularly in hyperparameter tuning. However, standard BO assumes access to precise objective values, which may be unavailable, noisy, or unreliable in real-world settings where only relative or rank-based feedback can be obtained. In this study, we propose Quantile-Scaled Bayesian Optimization (QS-BO), a principled rank-based optimization framework. QS-BO converts ranks into heteroscedastic Gaussian targets through a quantile-scaling pipeline, enabling the use of Gaussian process surrogates and standard acquisition functions without requiring explicit metric scores. We evaluate QS-BO on synthetic benchmark functions, including one- and two-dimensional nonlinear functions and the Branin function, and compare its performance against Random Search. Results demonstrate that QS-BO consistently achieves lower objective values and exhibits greater stability across runs. Statistical tests further confirm that QS-BO significantly outperforms Random Search at the 1\\% significance level. These findings establish QS-BO as a practical and effective extension of Bayesian Optimization for rank-only feedback, with promising applications in preference learning, recommendation, and human-in-the-loop optimization where absolute metric values are unavailable or unreliable.","short_abstract":"Bayesian Optimization (BO) is widely used for optimizing expensive black-box functions, particularly in hyperparameter tuning. However, standard BO assumes access to precise objective values, which may be unavailable, noisy, or unreliable in real-world settings where only relative or rank-based feedback can be obtained...","url_abs":"https://arxiv.org/abs/2510.03277","url_pdf":"https://arxiv.org/pdf/2510.03277v1","authors":"[\"Tunde Fahd Egunjobi\"]","published":"2025-09-28T11:03:18Z","proceeding":"stat.ML","tasks":"[\"stat.ML\",\"cs.LG\",\"math.ST\"]","methods":"[]","has_code":false}