{"ID":2855283,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.13681","arxiv_id":"2510.13681","title":"How Sampling Affects the Detectability of Machine-written texts: A Comprehensive Study","abstract":"As texts generated by Large Language Models (LLMs) are ever more common and often indistinguishable from human-written content, research on automatic text detection has attracted growing attention. Many recent detectors report near-perfect accuracy, often boasting AUROC scores above 99\\%. However, these claims typically assume fixed generation settings, leaving open the question of how robust such systems are to changes in decoding strategies. In this work, we systematically examine how sampling-based decoding impacts detectability, with a focus on how subtle variations in a model's (sub)word-level distribution affect detection performance. We find that even minor adjustments to decoding parameters - such as temperature, top-p, or nucleus sampling - can severely impair detector accuracy, with AUROC dropping from near-perfect levels to 1\\% in some settings. Our findings expose critical blind spots in current detection methods and emphasize the need for more comprehensive evaluation protocols. To facilitate future research, we release a large-scale dataset encompassing 37 decoding configurations, along with our code and evaluation framework https://github.com/BaggerOfWords/Sampling-and-Detection","short_abstract":"As texts generated by Large Language Models (LLMs) are ever more common and often indistinguishable from human-written content, research on automatic text detection has attracted growing attention. Many recent detectors report near-perfect accuracy, often boasting AUROC scores above 99\\%. However, these claims typicall...","url_abs":"https://arxiv.org/abs/2510.13681","url_pdf":"https://arxiv.org/pdf/2510.13681v1","authors":"[\"Matthieu Dubois\",\"François Yvon\",\"Pablo Piantanida\"]","published":"2025-10-15T15:36:45Z","proceeding":"cs.CL","tasks":"[\"cs.CL\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false,"code_links":[{"ID":608234,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2855283,"paper_url":"https://arxiv.org/abs/2510.13681","paper_title":"How Sampling Affects the Detectability of Machine-written texts: A Comprehensive Study","repo_url":"https://github.com/BaggerOfWords/Sampling-and-Detection","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}