{"ID":5675933,"CreatedAt":"2026-07-03T01:40:09.565152011Z","UpdatedAt":"2026-07-04T17:54:59.62573241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.01345","arxiv_id":"2607.01345","title":"TurnNat: Automatic Evaluation of Turn-Taking Naturalness in Dyadic Spoken Dialogue","abstract":"Turn-taking naturalness is central to full-duplex spoken dialogue systems, yet its automatic evaluation remains limited. Existing evaluations often rely on human judgments or behavior-specific timing metrics, making it difficult to compare heterogeneous timing failures within a unified framework. We propose TurnNat, a likelihood-based framework for automatic turn-taking naturalness evaluation in two-channel spoken dialogue. A causal turn-taking prediction model trained on natural conversations estimates future two-speaker voice-activity states, and the negative log-likelihood (NLL) of the observed future activity measures timing atypicality. TurnNat pools frame-level NLLs over turn-taking boundary units (TBUs) extracted from utterance onsets and offsets, and aggregates mean and tail TBU scores into a dialogue-level naturalness score. We further construct a controlled perturbation benchmark of paired natural and perturbed dialogue clips, validated by human naturalness judgments. Experiments on this benchmark show that TurnNat successfully identifies unnatural turn-taking perturbations across heterogeneous timing failures.","short_abstract":"Turn-taking naturalness is central to full-duplex spoken dialogue systems, yet its automatic evaluation remains limited. Existing evaluations often rely on human judgments or behavior-specific timing metrics, making it difficult to compare heterogeneous timing failures within a unified framework. We propose TurnNat, a...","url_abs":"https://arxiv.org/abs/2607.01345","url_pdf":"https://arxiv.org/pdf/2607.01345v1","authors":"[\"Hao Zhang\",\"Thomas Thebaud\",\"Georgi Tinchev\",\"Venkatesh Ravichandran\",\"Laureano Moro-Velazquez\"]","published":"2026-07-01T18:03:09Z","proceeding":"cs.CL","tasks":"[\"cs.CL\",\"cs.AI\"]","methods":"[]","has_code":false}