{"ID":5439514,"CreatedAt":"2026-07-01T01:17:58.482524686Z","UpdatedAt":"2026-07-02T20:59:23.075938969Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.30911","arxiv_id":"2606.30911","title":"Why Solve It Twice? Hierarchical Accumulation of Skills for Transfer-Efficient ML Engineering","abstract":"ML engineering agents waste compute rediscovering known techniques because every competition is a cold start. We present HASTE, a hierarchical multi-agent system that organizes cross-competition knowledge into three scope tiers (global, domain, and competition-specific), each coupled to a matching agent level. An orchestrator coordinates domain specialists and promotes learning between tiers via LLM-driven abstraction. A controlled ablation provides evidence for scoped loading: holding a 159-skill inventory constant across 8 competitions, tiered loading achieves a 100% medal rate while flat loading reaches only 62.5%, the same medal rate as loading no skills, and consumes 2x the output tokens. On the full MLE-Bench Lite benchmark (22 Kaggle competitions), HASTE reaches a medal rate of 77.3% using Claude Sonnet 4.6 at 12h per competition. In a cold-start run, the system begins with no accumulated skills. In warm-start runs, it reloads skills learned from earlier competitions, using only global and domain-level skills for transfer across competitions. Warm starts use 52% fewer refinement iterations, and the fraction of proposed changes kept by the agent rises from 42% at low inventory to 85% once 50+ skills are available. These results suggest that better knowledge organization can partly substitute for model strength and compute budget in ML-engineering agents.","short_abstract":"ML engineering agents waste compute rediscovering known techniques because every competition is a cold start. We present HASTE, a hierarchical multi-agent system that organizes cross-competition knowledge into three scope tiers (global, domain, and competition-specific), each coupled to a matching agent level. An orche...","url_abs":"https://arxiv.org/abs/2606.30911","url_pdf":"https://arxiv.org/pdf/2606.30911v1","authors":"[\"Yongbin Kim\",\"Yashar Talebirad\",\"Osmar R. Zaiane\"]","published":"2026-06-29T20:59:14Z","proceeding":"cs.AI","tasks":"[\"cs.AI\",\"cs.LG\",\"cs.MA\"]","methods":"[\"Large Language Model\",\"Generative Adversarial Network\"]","has_code":false}