{"ID":2848061,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.26510","arxiv_id":"2510.26510","title":"LLMs as In-Context Meta-Learners for Model and Hyperparameter Selection","abstract":"Model and hyperparameter selection are critical but challenging in machine learning, typically requiring expert intuition or expensive automated search. We investigate whether large language models (LLMs) can act as in-context meta-learners for this task. By converting each dataset into interpretable metadata, we prompt an LLM to recommend both model families and hyperparameters. We study two prompting strategies: (1) a zero-shot mode relying solely on pretrained knowledge, and (2) a meta-informed mode augmented with examples of models and their performance on past tasks. Across synthetic and real-world benchmarks, we show that LLMs can exploit dataset metadata to recommend competitive models and hyperparameters without search, and that improvements from meta-informed prompting demonstrate their capacity for in-context meta-learning. These results highlight a promising new role for LLMs as lightweight, general-purpose assistants for model selection and hyperparameter optimization.","short_abstract":"Model and hyperparameter selection are critical but challenging in machine learning, typically requiring expert intuition or expensive automated search. We investigate whether large language models (LLMs) can act as in-context meta-learners for this task. By converting each dataset into interpretable metadata, we promp...","url_abs":"https://arxiv.org/abs/2510.26510","url_pdf":"https://arxiv.org/pdf/2510.26510v3","authors":"[\"Youssef Attia El Hili\",\"Albert Thomas\",\"Malik Tiomoko\",\"Abdelhakim Benechehab\",\"Corentin Léger\",\"Corinne Ancourt\",\"Balázs Kégl\"]","published":"2025-10-30T14:04:25Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"stat.ML\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false}