{"ID":3083536,"CreatedAt":"2026-06-05T06:46:15.197025399Z","UpdatedAt":"2026-06-07T06:54:00.442624098Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.06494","arxiv_id":"2606.06494","title":"TailLoR: Protecting Principal Components in Parameter-Efficient Continual Learning","abstract":"Parameter-efficient finetuning methods based on spectral decomposition have enabled progress in Continual Learning. In this paper we introduce TailLoR, which utilizes the singular bases U and V of the pre-trained weights as a fixed reference frame to learn a low-rank update applied to the singular value matrix. A soft spectral penalty discourages updates aligned with dominant singular directions, reducing interference while routing fine-grained adaptation into the highly flexible, long-tail spectral coordinates.","short_abstract":"Parameter-efficient finetuning methods based on spectral decomposition have enabled progress in Continual Learning. In this paper we introduce TailLoR, which utilizes the singular bases U and V of the pre-trained weights as a fixed reference frame to learn a low-rank update applied to the singular value matrix. A soft...","url_abs":"https://arxiv.org/abs/2606.06494","url_pdf":"https://arxiv.org/pdf/2606.06494v1","authors":"[\"Marius Dragoi\",\"Ioana Pintilie\",\"Alexandra Dragomir\",\"Antonio Barbalau\",\"Florin Brad\"]","published":"2026-06-04T17:59:55Z","proceeding":"cs.LG","tasks":"[\"cs.LG\"]","methods":"[]","has_code":false}