{"ID":2888890,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2507.22872","arxiv_id":"2507.22872","title":"TR-PTS: Task-Relevant Parameter and Token Selection for Efficient Tuning","abstract":"Large pre-trained models achieve remarkable performance in vision tasks but are impractical for fine-tuning due to high computational and storage costs. Parameter-Efficient Fine-Tuning (PEFT) methods mitigate this issue by updating only a subset of parameters; however, most existing approaches are task-agnostic, failing to fully exploit task-specific adaptations, which leads to suboptimal efficiency and performance. To address this limitation, we propose Task-Relevant Parameter and Token Selection (TR-PTS), a task-driven framework that enhances both computational efficiency and accuracy. Specifically, we introduce Task-Relevant Parameter Selection, which utilizes the Fisher Information Matrix (FIM) to identify and fine-tune only the most informative parameters in a layer-wise manner, while keeping the remaining parameters frozen. Simultaneously, Task-Relevant Token Selection dynamically preserves the most informative tokens and merges redundant ones, reducing computational overhead. By jointly optimizing parameters and tokens, TR-PTS enables the model to concentrate on task-discriminative information. We evaluate TR-PTS on benchmark, including FGVC and VTAB-1k, where it achieves state-of-the-art performance, surpassing full fine-tuning by 3.40% and 10.35%, respectively. The code are available at https://github.com/synbol/TR-PTS.","short_abstract":"Large pre-trained models achieve remarkable performance in vision tasks but are impractical for fine-tuning due to high computational and storage costs. Parameter-Efficient Fine-Tuning (PEFT) methods mitigate this issue by updating only a subset of parameters; however, most existing approaches are task-agnostic, failin...","url_abs":"https://arxiv.org/abs/2507.22872","url_pdf":"https://arxiv.org/pdf/2507.22872v1","authors":"[\"Siqi Luo\",\"Haoran Yang\",\"Yi Xin\",\"Mingyang Yi\",\"Guangyang Wu\",\"Guangtao Zhai\",\"Xiaohong Liu\"]","published":"2025-07-30T17:47:13Z","proceeding":"cs.CV","tasks":"[\"cs.CV\"]","methods":"[]","has_code":false,"code_links":[{"ID":611591,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2888890,"paper_url":"https://arxiv.org/abs/2507.22872","paper_title":"TR-PTS: Task-Relevant Parameter and Token Selection for Efficient Tuning","repo_url":"https://github.com/synbol/TR-PTS","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}