{"ID":3083551,"CreatedAt":"2026-06-05T06:46:15.197025399Z","UpdatedAt":"2026-06-07T05:49:02.101151534Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.06473","arxiv_id":"2606.06473","title":"MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discovery","abstract":"Large language model (LLM) agents are increasingly applied to long-horizon tasks such as scientific discovery and machine learning engineering (MLE), where sustained self-evolution becomes a key capability. However, existing MLE agents suffer from inter-branch information isolation, memoryless search, and lack of hierarchical control, which together hinder long-horizon optimization. We present MLEvolve, an LLM-based self-evolving multi-agent framework for end-to-end machine learning algorithm discovery. By extending tree search to Progressive MCGS, MLEvolve enables cross-branch information flow through graph-based reference edges and gradually shifts the search from broad exploration to focused exploitation with an entropy-inspired progressive schedule. To allow the agent to evolve with accumulated experience, we introduce Retrospective Memory, which combines a cold-start domain knowledge base with a dynamic global memory for task-specific experience retrieval and reuse. For stable long-horizon iteration, we further decouple strategic planning from code generation with adaptive coding modes. Evaluation on MLE-Bench shows that MLEvolve achieves state-of-the-art performance across multiple dimensions including average medal rate and valid submission rate under a 12-hour budget (half the standard runtime). Moreover, MLEvolve also outperforms specialized algorithm discovery methods including AlphaEvolve on mathematical algorithm optimization tasks, demonstrating strong cross-domain generalization. Our code is available at https://github.com/InternScience/MLEvolve.","short_abstract":"Large language model (LLM) agents are increasingly applied to long-horizon tasks such as scientific discovery and machine learning engineering (MLE), where sustained self-evolution becomes a key capability. However, existing MLE agents suffer from inter-branch information isolation, memoryless search, and lack of hiera...","url_abs":"https://arxiv.org/abs/2606.06473","url_pdf":"https://arxiv.org/pdf/2606.06473v1","authors":"[\"Shangheng Du\",\"Xiangchao Yan\",\"Jinxin Shi\",\"Zongsheng Cao\",\"Shiyang Feng\",\"Zichen Liang\",\"Boyuan Sun\",\"Tianshuo Peng\",\"Yifan Zhou\",\"Xin Li\",\"Jie Zhou\",\"Liang He\",\"Bo Zhang\",\"Lei Bai\"]","published":"2026-06-04T17:55:59Z","proceeding":"cs.AI","tasks":"[\"cs.AI\",\"cs.CL\"]","methods":"[\"Large Language Model\",\"Language Model\",\"LoRA\"]","has_code":false,"code_links":[{"ID":612818,"CreatedAt":"2026-06-05T06:46:15.197025399Z","UpdatedAt":"2026-06-05T06:46:15.197025399Z","DeletedAt":null,"paper_id":3083551,"paper_url":"https://arxiv.org/abs/2606.06473","paper_title":"MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discovery","repo_url":"https://github.com/InternScience/MLEvolve","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}