{"ID":2857194,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.13849","arxiv_id":"2510.13849","title":"Language steering in latent space to mitigate unintended code-switching","abstract":"Multilingual Large Language Models (LLMs) often exhibit hallucinations such as unintended code-switching, reducing reliability in downstream tasks. We propose latent-space language steering, a lightweight inference-time method that identifies language directions via Principal Component Analysis (PCA) on parallel translations and steers token embeddings along these axes to control language identity. Our approach mitigates code-switching while preserving semantics with negligible computational overhead and requires only minimal parallel data for calibration. Empirically, we achieve 95-99\\% language classification accuracy using a single principal component and reduce next-token distributional divergence by up to 55\\% across multiple language pairs on Qwen2.5 and Llama-3.2 models. Generation-based evaluation on Llama-3.2 further demonstrates 63--99\\% reduction in Code-Switching Index across four language pairs ($p \u003c 0.001$). We further analyze the layer-wise evolution of language representations, revealing that language identity concentrates in final layers with near-perfect linear separability.","short_abstract":"Multilingual Large Language Models (LLMs) often exhibit hallucinations such as unintended code-switching, reducing reliability in downstream tasks. We propose latent-space language steering, a lightweight inference-time method that identifies language directions via Principal Component Analysis (PCA) on parallel transl...","url_abs":"https://arxiv.org/abs/2510.13849","url_pdf":"https://arxiv.org/pdf/2510.13849v3","authors":"[\"Andrey Goncharov\",\"Nikolai Kondusov\",\"Alexey Zaytsev\"]","published":"2025-10-11T19:49:38Z","proceeding":"cs.CL","tasks":"[\"cs.CL\",\"cs.LG\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false}