{"ID":2834086,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2512.02901","arxiv_id":"2512.02901","title":"Fairy2i: Training Complex LLMs from Real LLMs with All Parameters in $\\{\\pm 1, \\pm i\\}$","abstract":"Large language models (LLMs) have revolutionized artificial intelligence, yet their massive memory and computational demands necessitate aggressive quantization, increasingly pushing representations toward the theoretical limit of a single bit. While complex-valued LLMs, such as iFairy, offer a superior chance for low-bit representation compared to real-valued counterparts, they require training from scratch, preventing the utilization of the vast ecosystem of pre-trained real-valued foundation models. Here we present Fairy2i, a universal framework that transforms pre-trained real-valued layers into an equivalent widely-linear complex form, enabling extremely low-bit quantization while reusing existing checkpoints. By proving a lossless mathematical equivalence between real and widely-linear maps, we convert standard Transformers into the complex domain and employ a phase-aware quantization scheme with a highly efficient codebook of fourth roots of unity. Furthermore, we introduce a recursive residual quantization mechanism that iteratively minimizes quantization error, allowing inference to proceed via efficient multiplication-free accumulation. We demonstrate that Fairy2i restores the performance of LLaMA-2 7B at an effective 2-bit precision to levels nearly comparable with full-precision baselines, significantly outperforming state-of-the-art real-valued binary and ternary quantization methods. This work bridges the gap between the representational efficiency of complex-valued arithmetic and the practical utility of pre-trained models, paving a new way for efficient inference on commodity hardware. We open-source the Fairy2i model and code at https://huggingface.co/PKU-DS-LAB/Fairy2i-W2 and https://github.com/PKULab1806/Fairy2i-W2.","short_abstract":"Large language models (LLMs) have revolutionized artificial intelligence, yet their massive memory and computational demands necessitate aggressive quantization, increasingly pushing representations toward the theoretical limit of a single bit. While complex-valued LLMs, such as iFairy, offer a superior chance for low-...","url_abs":"https://arxiv.org/abs/2512.02901","url_pdf":"https://arxiv.org/pdf/2512.02901v3","authors":"[\"Feiyu Wang\",\"Xinyu Tan\",\"Bokai Huang\",\"Yihao Zhang\",\"Guoan Wang\",\"Peizhuang Cong\",\"Tong Yang\"]","published":"2025-12-02T16:14:08Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.AI\"]","methods":"[\"Transformer\",\"Large Language Model\",\"Language Model\"]","has_code":false,"code_links":[{"ID":606380,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2834086,"paper_url":"https://arxiv.org/abs/2512.02901","paper_title":"Fairy2i: Training Complex LLMs from Real LLMs with All Parameters in $\\{\\pm 1, \\pm i\\}$","repo_url":"https://github.com/PKULab1806/Fairy2i-W2","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}