{"ID":6267107,"CreatedAt":"2026-07-10T01:11:38.759438437Z","UpdatedAt":"2026-07-13T01:02:08.706470581Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.08256","arxiv_id":"2607.08256","title":"Best-of-$N$ TTS Evaluation is Confounded by ASR Family Alignment","abstract":"Best-of-$N$ (BoN) inference improves content consistency in zero-shot text-to-speech by selecting from $N$ candidates with an automatic speech recognition (ASR) verifier. We identify an underexplored evaluation confound: a verifier's apparent quality depends strongly on which ASR family judges it. On LibriSpeech-PC test-clean~\\citep{librispeechpc} with F5-TTS~\\citep{f5tts}, verifier rankings reverse across Whisper, wav2vec~2.0, and HuBERT evaluators, and same-family verifier-evaluator pairs recover 2-3$\\times$ more oracle headroom than cross-family pairs despite near-identical representations (linear CKA $0.978$) -- a pattern consistent with identity- or lineage-level coupling rather than representational overlap. We propose two \\textbf{cross-family rank ensembles} (rank-averaging and conjunctive max-rank) that attain the lowest mean WER across three independent evaluators -- $1.61\\%$ at $N{=}10$ ($-12\\%$ relative to F5-TTS) -- with no measurable degradation under automatic SIM-o/UTMOS metrics; the best single verifier drives WER from $2.06\\%$ to $1.72\\%$ ($-16.5\\%$) under the official F5-TTS evaluator. We recommend cross-evaluator triangulation as default reporting practice.","short_abstract":"Best-of-$N$ (BoN) inference improves content consistency in zero-shot text-to-speech by selecting from $N$ candidates with an automatic speech recognition (ASR) verifier. We identify an underexplored evaluation confound: a verifier's apparent quality depends strongly on which ASR family judges it. On LibriSpeech-PC tes...","url_abs":"https://arxiv.org/abs/2607.08256","url_pdf":"https://arxiv.org/pdf/2607.08256v1","authors":"[\"Taehyung Yu\",\"Seongjae Kang\"]","published":"2026-07-09T09:01:03Z","proceeding":"cs.CL","tasks":"[\"cs.CL\",\"cs.AI\",\"cs.LG\",\"cs.SD\"]","methods":"[]","has_code":false}