{"ID":2891047,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2508.03716","arxiv_id":"2508.03716","title":"FeynTune: Large Language Models for High-Energy Theory","abstract":"We present specialized Large Language Models for theoretical High-Energy Physics, obtained as 20 fine-tuned variants of the 8-billion parameter Llama-3.1 model. Each variant was trained on arXiv abstracts (through August 2024) from different combinations of hep-th, hep-ph and gr-qc. For a comparative study, we also trained models on datasets that contained abstracts from disparate fields such as the q-bio and cs categories. All models were fine-tuned using two distinct Low-Rank Adaptation fine-tuning approaches and varying dataset sizes, and outperformed the base model on hep-th abstract completion tasks. We compare performance against leading commercial LLMs (ChatGPT, Claude, Gemini, DeepSeek) and derive insights for further developing specialized language models for High-Energy Theoretical Physics.","short_abstract":"We present specialized Large Language Models for theoretical High-Energy Physics, obtained as 20 fine-tuned variants of the 8-billion parameter Llama-3.1 model. Each variant was trained on arXiv abstracts (through August 2024) from different combinations of hep-th, hep-ph and gr-qc. For a comparative study, we also tra...","url_abs":"https://arxiv.org/abs/2508.03716","url_pdf":"https://arxiv.org/pdf/2508.03716v2","authors":"[\"Paul Richmond\",\"Prarit Agarwal\",\"Borun Chowdhury\",\"Vasilis Niarchos\",\"Constantinos Papageorgakis\"]","published":"2025-07-24T18:21:03Z","proceeding":"cs.CL","tasks":"[\"cs.CL\",\"cs.LG\",\"hep-th\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false}