{"ID":2825182,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2512.21580","arxiv_id":"2512.21580","title":"Gamayun's Path to Multilingual Mastery: Cost-Efficient Training of a 1.5B-Parameter LLM","abstract":"We present Gamayun, a 1.5B-parameter multilingual language model trained entirely from scratch on 2.5T tokens. Designed for efficiency and deployment in resource-constrained environments, Gamayun addresses the lack of research on small non-English-centric LLMs by adopting a novel two-stage pre-training strategy: balanced multilingual training for cross-lingual alignment, followed by high-quality English enrichment to transfer performance gains across languages. Our model supports 12 languages, with special focus on Russian. Despite a significantly smaller training budget than comparable models, Gamayun outperforms LLaMA3.2-1B (9T tokens) on all considered benchmarks, and surpasses Qwen2.5-1.5B (18T tokens) on a wide range of English and multilingual tasks. It matches or exceeds Qwen3 (36T tokens) on most tasks outside advanced STEM, achieving state-of-the-art results in Russian, including the MERA benchmark, among the models of comparable size (1-2B parameters).","short_abstract":"We present Gamayun, a 1.5B-parameter multilingual language model trained entirely from scratch on 2.5T tokens. Designed for efficiency and deployment in resource-constrained environments, Gamayun addresses the lack of research on small non-English-centric LLMs by adopting a novel two-stage pre-training strategy: balanc...","url_abs":"https://arxiv.org/abs/2512.21580","url_pdf":"https://arxiv.org/pdf/2512.21580v2","authors":"[\"Alexander Podolskiy\",\"Semen Molokov\",\"Timofey Gerasin\",\"Maksim Titov\",\"Alexey Rukhovich\",\"Artem Khrapov\",\"Kirill Morozov\",\"Evgeny Tetin\",\"Constantine Korikov\",\"Pavel Efimov\",\"Polina Lazukova\",\"Yuliya Skripkar\",\"Nikita Okhotnikov\",\"Irina Piontkovskaya\",\"Meng Xiaojun\",\"Zou Xueyi\",\"Zhang Zhenhe\"]","published":"2025-12-25T08:52:23Z","proceeding":"cs.CL","tasks":"[\"cs.CL\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false}