{"ID":6024131,"CreatedAt":"2026-07-08T01:00:23.257252134Z","UpdatedAt":"2026-07-09T20:17:26.950452338Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.05615","arxiv_id":"2607.05615","title":"A Coin Flip Per Token: Bernoulli Sparse Steering of Large Language Models","abstract":"Activation steering via sparse autoencoders (SAEs) enables behavioral control of large language models without task-specific fine-tuning, but standard methods apply the steering signal at every generated token, incurring constant per-token perturbation that risks degrading fluency. We ask: is dense intervention necessary? We introduce Stochastic Token Steering (STS), which gates each token independently with probability $p$, and Stochastic Block Steering (SBS), which gates a leading window once per sequence; neither requires a reward model or learned gating policy. Across two model families and two behavioral tasks, steering only 50% of the tokens recovers most of the dense-steering effect while preserving fluency, and steering as few as 30% surpasses prompt-based control. The optimal steering magnitude scales inversely with the intervention ratio, revealing that SAE-mediated control is rate-limited: the behavioral outcome depends on cumulative signal dosage across a sequence.","short_abstract":"Activation steering via sparse autoencoders (SAEs) enables behavioral control of large language models without task-specific fine-tuning, but standard methods apply the steering signal at every generated token, incurring constant per-token perturbation that risks degrading fluency. We ask: is dense intervention necessa...","url_abs":"https://arxiv.org/abs/2607.05615","url_pdf":"https://arxiv.org/pdf/2607.05615v1","authors":"[\"Nima Eshraghi\",\"Lovedeep Gondara\",\"Yuqing Huang\",\"Sagarika Suresh\",\"Leizer Teran\",\"Jithin Pradeep\",\"Xiaotong Xu\",\"Fanny Chevalier\"]","published":"2026-07-06T20:25:27Z","proceeding":"cs.LG","tasks":"[\"cs.LG\"]","methods":"[\"Language Model\"]","has_code":false}