{"ID":5438852,"CreatedAt":"2026-07-01T01:17:58.482524686Z","UpdatedAt":"2026-07-03T12:27:39.719939621Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.31577","arxiv_id":"2606.31577","title":"Localized Conformal Prediction for Image Classification with Vision-Language Models","abstract":"Conformal predictions have attracted significant attention in the field of uncertainty quantification, mainly because of their strong marginal coverage guarantees. Full conditional guarantee is not an attainable goal, a well known fact in conformal predictions literature. As a result, several approaches have tried to approximate this behavior by adapting the conformal sets of test-time samples according to their similarity to calibration examples. Although the latter has gained traction and shown impressive performances for regression problems, its application to image classification remains under-explored. We conduct an extensive benchmarking on natural image classification tasks with vision-language models (VLMs), using our open source implementation of a recent localized conformal prediction algorithm. We show that straightforward usage of the cosine similarity between test-time and calibration visual features, an intuitive choice for VLMs, is not sufficient to improve over the non-local baselines. In response, we propose a simple non-linear transformation of the cosine similarities, which conserves marginal coverage guarantees and achieves statistically significant mean set sizes reduction. Code is available at https://github.com/cfuchs2023/lcp-vlm/.","short_abstract":"Conformal predictions have attracted significant attention in the field of uncertainty quantification, mainly because of their strong marginal coverage guarantees. Full conditional guarantee is not an attainable goal, a well known fact in conformal predictions literature. As a result, several approaches have tried to a...","url_abs":"https://arxiv.org/abs/2606.31577","url_pdf":"https://arxiv.org/pdf/2606.31577v1","authors":"[\"Clément Fuchs\",\"Tim Bary\",\"Benoît Macq\"]","published":"2026-06-30T12:34:40Z","proceeding":"cs.CV","tasks":"[\"cs.CV\",\"cs.LG\"]","methods":"[\"Language Model\"]","has_code":false,"code_links":[{"ID":613787,"CreatedAt":"2026-07-01T01:17:58.482524686Z","UpdatedAt":"2026-07-01T01:17:58.482524686Z","DeletedAt":null,"paper_id":5438852,"paper_url":"https://arxiv.org/abs/2606.31577","paper_title":"Localized Conformal Prediction for Image Classification with Vision-Language Models","repo_url":"https://github.com/cfuchs2023/lcp-vlm","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}