{"ID":2851856,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.19802","arxiv_id":"2510.19802","title":"Class-Aware Prototype Learning with Negative Contrast for Test-Time Adaptation of Vision-Language Models","abstract":"Vision-Language Models (VLMs) demonstrate impressive zero-shot generalization through large-scale image-text pretraining, yet their performance can drop once the deployment distribution diverges from the training distribution. To address this, Test-Time Adaptation (TTA) methods update models using unlabeled target data. However, existing approaches often ignore two key challenges: prototype degradation in long-tailed distributions and confusion between semantically similar classes. To tackle these issues, we propose \\textbf{C}lass-Aware \\textbf{P}rototype \\textbf{L}earning with \\textbf{N}egative \\textbf{C}ontrast(\\textbf{CPL-NC}), a lightweight TTA framework designed specifically for VLMs to enhance generalization under distribution shifts. CPL-NC introduces a \\textit{Class-Aware Prototype Cache} Module that dynamically adjusts per-class capacity based on test-time frequency and activation history, with a rejuvenation mechanism for inactive classes to retain rare-category knowledge. Additionally, a \\textit{Negative Contrastive Learning} Mechanism identifies and constrains hard visual-textual negatives to improve class separability. The framework employs asymmetric optimization, refining only textual prototypes while anchoring on stable visual features. Experiments on 15 benchmarks show that CPL-NC consistently outperforms prior TTA methods across both ResNet-50 and ViT-B/16 backbones.","short_abstract":"Vision-Language Models (VLMs) demonstrate impressive zero-shot generalization through large-scale image-text pretraining, yet their performance can drop once the deployment distribution diverges from the training distribution. To address this, Test-Time Adaptation (TTA) methods update models using unlabeled target data...","url_abs":"https://arxiv.org/abs/2510.19802","url_pdf":"https://arxiv.org/pdf/2510.19802v1","authors":"[\"Xiaozhen Qiao\",\"Jingkai Zhao\",\"Yuqiu Jiang\",\"Xianda Guo\",\"Zhe Sun\",\"Hongyuan Zhang\",\"Xuelong Li\"]","published":"2025-10-22T17:38:35Z","proceeding":"cs.CV","tasks":"[\"cs.CV\"]","methods":"[\"Language Model\"]","has_code":false}