{"ID":2888957,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2508.08274","arxiv_id":"2508.08274","title":"Distilling Knowledge from Large Language Models: A Concept Bottleneck Model for Hate and Counter Speech Recognition","abstract":"The rapid increase in hate speech on social media has exposed an unprecedented impact on society, making automated methods for detecting such content important. Unlike prior black-box models, we propose a novel transparent method for automated hate and counter speech recognition, i.e., \"Speech Concept Bottleneck Model\" (SCBM), using adjectives as human-interpretable bottleneck concepts. SCBM leverages large language models (LLMs) to map input texts to an abstract adjective-based representation, which is then sent to a light-weight classifier for downstream tasks. Across five benchmark datasets spanning multiple languages and platforms (e.g., Twitter, Reddit, YouTube), SCBM achieves an average macro-F1 score of 0.69 which outperforms the most recently reported results from the literature on four out of five datasets. Aside from high recognition accuracy, SCBM provides a high level of both local and global interpretability. Furthermore, fusing our adjective-based concept representation with transformer embeddings, leads to a 1.8% performance increase on average across all datasets, showing that the proposed representation captures complementary information. Our results demonstrate that adjective-based concept representations can serve as compact, interpretable, and effective encodings for hate and counter speech recognition. With adapted adjectives, our method can also be applied to other NLP tasks.","short_abstract":"The rapid increase in hate speech on social media has exposed an unprecedented impact on society, making automated methods for detecting such content important. Unlike prior black-box models, we propose a novel transparent method for automated hate and counter speech recognition, i.e., \"Speech Concept Bottleneck Model\"...","url_abs":"https://arxiv.org/abs/2508.08274","url_pdf":"https://arxiv.org/pdf/2508.08274v1","authors":"[\"Roberto Labadie-Tamayo\",\"Djordje Slijepčević\",\"Xihui Chen\",\"Adrian Jaques Böck\",\"Andreas Babic\",\"Liz Freimann\",\"Christiane Atzmüller Matthias Zeppelzauer\"]","published":"2025-07-30T21:50:30Z","proceeding":"cs.CL","tasks":"[\"cs.CL\",\"cs.AI\",\"cs.LG\"]","methods":"[\"Transformer\",\"Large Language Model\",\"Language Model\"]","has_code":false}