{"ID":2892194,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2507.22920","arxiv_id":"2507.22920","title":"Discrete Tokenization for Multimodal LLMs: A Comprehensive Survey","abstract":"The rapid advancement of large language models (LLMs) has intensified the need for effective mechanisms to transform continuous multimodal data into discrete representations suitable for language-based processing. Discrete tokenization, with vector quantization (VQ) as a central approach, offers both computational efficiency and compatibility with LLM architectures. Despite its growing importance, there is a lack of a comprehensive survey that systematically examines VQ techniques in the context of LLM-based systems. This work fills this gap by presenting the first structured taxonomy and analysis of discrete tokenization methods designed for LLMs. We categorize 8 representative VQ variants that span classical and modern paradigms and analyze their algorithmic principles, training dynamics, and integration challenges with LLM pipelines. Beyond algorithm-level investigation, we discuss existing research in terms of classical applications without LLMs, LLM-based single-modality systems, and LLM-based multimodal systems, highlighting how quantization strategies influence alignment, reasoning, and generation performance. In addition, we identify key challenges including codebook collapse, unstable gradient estimation, and modality-specific encoding constraints. Finally, we discuss emerging research directions such as dynamic and task-adaptive quantization, unified tokenization frameworks, and biologically inspired codebook learning. This survey bridges the gap between traditional vector quantization and modern LLM applications, serving as a foundational reference for the development of efficient and generalizable multimodal systems. A continuously updated version is available at: https://github.com/jindongli-Ai/LLM-Discrete-Tokenization-Survey.","short_abstract":"The rapid advancement of large language models (LLMs) has intensified the need for effective mechanisms to transform continuous multimodal data into discrete representations suitable for language-based processing. Discrete tokenization, with vector quantization (VQ) as a central approach, offers both computational effi...","url_abs":"https://arxiv.org/abs/2507.22920","url_pdf":"https://arxiv.org/pdf/2507.22920v1","authors":"[\"Jindong Li\",\"Yali Fu\",\"Jiahong Liu\",\"Linxiao Cao\",\"Wei Ji\",\"Menglin Yang\",\"Irwin King\",\"Ming-Hsuan Yang\"]","published":"2025-07-21T10:52:14Z","proceeding":"cs.CL","tasks":"[\"cs.CL\",\"cs.AI\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false,"code_links":[{"ID":611966,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2892194,"paper_url":"https://arxiv.org/abs/2507.22920","paper_title":"Discrete Tokenization for Multimodal LLMs: A Comprehensive Survey","repo_url":"https://github.com/jindongli-Ai/LLM-Discrete-Tokenization-Survey","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}