{"ID":3004908,"CreatedAt":"2026-06-03T03:09:48.883664427Z","UpdatedAt":"2026-06-05T10:38:01.117085634Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.03465","arxiv_id":"2606.03465","title":"Rethinking the Role of Tensor Decompositions in Post-Training LLM Compression","abstract":"Post-training compression is essential for deploying large language models (LLMs) under tight resource constraints. Tensor decompositions have emerged as a promising direction, offering compact parameterizations well suited to Transformer weight structures. However, existing studies evaluate these methods in narrow settings, leaving unclear whether tensorization is effective at large-scale deployment. We systematically evaluate tensor compression across dense and MoE architectures, establishing performance trade-offs grounded in both empirical analysis and theoretical analysis. We identify a fundamental mismatch between the shared subspaces assumed by tensor decompositions and the heterogeneous representations learned by modern LLMs, thereby delineating their practical limits and clarifying their viable role in large-scale deployment. The code is available at https://github.com/brain-lab-research/TT-LLM.","short_abstract":"Post-training compression is essential for deploying large language models (LLMs) under tight resource constraints. Tensor decompositions have emerged as a promising direction, offering compact parameterizations well suited to Transformer weight structures. However, existing studies evaluate these methods in narrow set...","url_abs":"https://arxiv.org/abs/2606.03465","url_pdf":"https://arxiv.org/pdf/2606.03465v1","authors":"[\"Artur Zagitov\",\"Alexander Miasnikov\",\"Maxim Krutikov\",\"Vladimir Aletov\",\"Gleb Molodtsov\",\"Nail Bashirov\",\"Artem Tsedenov\",\"Aleksandr Beznosikov\"]","published":"2026-06-02T10:45:21Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.AI\"]","methods":"[\"Transformer\",\"Large Language Model\",\"Language Model\"]","has_code":false,"code_links":[{"ID":612717,"CreatedAt":"2026-06-03T03:09:48.883664427Z","UpdatedAt":"2026-06-03T03:09:48.883664427Z","DeletedAt":null,"paper_id":3004908,"paper_url":"https://arxiv.org/abs/2606.03465","paper_title":"Rethinking the Role of Tensor Decompositions in Post-Training LLM Compression","repo_url":"https://github.com/brain-lab-research/TT-LLM","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}