{"ID":2883668,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2508.07835","arxiv_id":"2508.07835","title":"Effortless Vision-Language Model Specialization in Histopathology without Annotation","abstract":"Recent advances in Vision-Language Models (VLMs) in histopathology, such as CONCH and QuiltNet, have demonstrated impressive zero-shot classification capabilities across various tasks. However, their general-purpose design may lead to suboptimal performance in specific downstream applications. While supervised fine-tuning methods address this issue, they require manually labeled samples for adaptation. This paper investigates annotation-free adaptation of VLMs through continued pretraining on domain- and task-relevant image-caption pairs extracted from existing databases. Our experiments on two VLMs, CONCH and QuiltNet, across three downstream tasks reveal that these pairs substantially enhance both zero-shot and few-shot performance. Notably, with larger training sizes, continued pretraining matches the performance of few-shot methods while eliminating manual labeling. Its effectiveness, task-agnostic design, and annotation-free workflow make it a promising pathway for adapting VLMs to new histopathology tasks. Code is available at https://github.com/DeepMicroscopy/Annotation-free-VLM-specialization.","short_abstract":"Recent advances in Vision-Language Models (VLMs) in histopathology, such as CONCH and QuiltNet, have demonstrated impressive zero-shot classification capabilities across various tasks. However, their general-purpose design may lead to suboptimal performance in specific downstream applications. While supervised fine-tun...","url_abs":"https://arxiv.org/abs/2508.07835","url_pdf":"https://arxiv.org/pdf/2508.07835v1","authors":"[\"Jingna Qiu\",\"Nishanth Jain\",\"Jonas Ammeling\",\"Marc Aubreville\",\"Katharina Breininger\"]","published":"2025-08-11T10:39:27Z","proceeding":"cs.CV","tasks":"[\"cs.CV\"]","methods":"[\"Language Model\"]","has_code":false,"code_links":[{"ID":611005,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2883668,"paper_url":"https://arxiv.org/abs/2508.07835","paper_title":"Effortless Vision-Language Model Specialization in Histopathology without Annotation","repo_url":"https://github.com/DeepMicroscopy/Annotation-free-VLM-specialization","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}