{"ID":2882718,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2508.09654","arxiv_id":"2508.09654","title":"Improving Diversity in Language Models: When Temperature Fails, Change the Loss","abstract":"Increasing diversity in language models is a challenging yet essential objective. A common approach is to raise the decoding temperature. In this work, we investigate this approach through a simplistic yet common case to provide insights into why decreasing temperature can improve quality (Precision), while increasing it often fails to boost coverage (Recall). Our analysis reveals that for a model to be effectively tunable through temperature adjustments, it must be trained toward coverage. To address this, we propose rethinking loss functions in language models by leveraging the Precision-Recall framework. Our results demonstrate that this approach achieves a substantially better trade-off between Precision and Recall than merely combining negative log-likelihood training with temperature scaling. These findings offer a pathway toward more versatile and robust language modeling techniques.","short_abstract":"Increasing diversity in language models is a challenging yet essential objective. A common approach is to raise the decoding temperature. In this work, we investigate this approach through a simplistic yet common case to provide insights into why decreasing temperature can improve quality (Precision), while increasing...","url_abs":"https://arxiv.org/abs/2508.09654","url_pdf":"https://arxiv.org/pdf/2508.09654v1","authors":"[\"Alexandre Verine\",\"Florian Le Bronnec\",\"Kunhao Zheng\",\"Alexandre Allauzen\",\"Yann Chevaleyre\",\"Benjamin Negrevergne\"]","published":"2025-08-13T09:37:53Z","proceeding":"cs.CL","tasks":"[\"cs.CL\",\"cs.LG\"]","methods":"[\"Language Model\"]","has_code":false}