{"ID":2884798,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2508.06382","arxiv_id":"2508.06382","title":"Text as Any-Modality for Zero-Shot Classification by Consistent Prompt Tuning","abstract":"The integration of prompt tuning with multimodal learning has shown significant generalization abilities for various downstream tasks. Despite advancements, existing methods heavily depend on massive modality-specific labeled data (e.g., video, audio, and image), or are customized for a single modality. In this study, we present Text as Any-Modality by Consistent Prompt Tuning (TaAM-CPT), a scalable approach for constructing a general representation model toward unlimited modalities using solely text data. TaAM-CPT comprises modality prompt pools, text construction, and modality-aligned text encoders from pre-trained models, which allows for extending new modalities by simply adding prompt pools and modality-aligned text encoders. To harmonize the learning across different modalities, TaAM-CPT designs intra- and inter-modal learning objectives, which can capture category details within modalities while maintaining semantic consistency across different modalities. Benefiting from its scalable architecture and pre-trained models, TaAM-CPT can be seamlessly extended to accommodate unlimited modalities. Remarkably, without any modality-specific labeled data, TaAM-CPT achieves leading results on diverse datasets spanning various modalities, including video classification, image classification, and audio classification. The code is available at https://github.com/Jinx630/TaAM-CPT.","short_abstract":"The integration of prompt tuning with multimodal learning has shown significant generalization abilities for various downstream tasks. Despite advancements, existing methods heavily depend on massive modality-specific labeled data (e.g., video, audio, and image), or are customized for a single modality. In this study,...","url_abs":"https://arxiv.org/abs/2508.06382","url_pdf":"https://arxiv.org/pdf/2508.06382v1","authors":"[\"Xiangyu Wu\",\"Feng Yu\",\"Yang Yang\",\"Jianfeng Lu\"]","published":"2025-08-08T15:13:05Z","proceeding":"cs.CV","tasks":"[\"cs.CV\"]","methods":"[]","has_code":false,"code_links":[{"ID":611118,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2884798,"paper_url":"https://arxiv.org/abs/2508.06382","paper_title":"Text as Any-Modality for Zero-Shot Classification by Consistent Prompt Tuning","repo_url":"https://github.com/Jinx630/TaAM-CPT","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}