{"ID":2828681,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2512.14880","arxiv_id":"2512.14880","title":"Task Matrices: Linear Maps for Cross-Model Finetuning Transfer","abstract":"Results in interpretability suggest that large vision and language models learn implicit linear encodings when models are biased by in-context prompting. However, the existence of similar linear representations in more general adaptation regimes has not yet been demonstrated. In this work, we develop the concept of a task matrix, a linear transformation from a base to finetuned embedding state. We demonstrate that for vision and text models and ten different datasets, a base model augmented with a task matrix achieves results surpassing linear probes, sometimes approaching finetuned levels. Our results validate the existence of cross-layer linear encodings between pretrained and finetuned architectures. Moreover, we show that a data-based approximation for such encodings is both efficient and generalizable to multiple domains. We make our implementation publicly available.","short_abstract":"Results in interpretability suggest that large vision and language models learn implicit linear encodings when models are biased by in-context prompting. However, the existence of similar linear representations in more general adaptation regimes has not yet been demonstrated. In this work, we develop the concept of a t...","url_abs":"https://arxiv.org/abs/2512.14880","url_pdf":"https://arxiv.org/pdf/2512.14880v1","authors":"[\"Darrin O' Brien\",\"Dhikshith Gajulapalli\",\"Eric Xia\"]","published":"2025-12-16T19:51:29Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.CL\",\"cs.CV\"]","methods":"[\"Language Model\"]","has_code":false}