{"ID":3050117,"CreatedAt":"2026-06-04T02:13:16.786527022Z","UpdatedAt":"2026-06-07T09:16:17.280914754Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.04703","arxiv_id":"2606.04703","title":"Rethinking Continual Experience Internalization for Self-Evolving LLM Agents","abstract":"Experience internalization converts contextual experience from past interactions into reusable parametric capability, offering a promising path toward continual learning in large language models (LLMs). While prior work has predominantly focused on single-iteration transfer, we discover that under multi-iteration experience learning, existing methods suffer from a progressive capability collapse rather than compounding improvement. We systematically examine this failure through three vital dimensions of experience internalization: (1) Experience Granularity: We find that principle-level experience is more durable than instance-level experience, as it effectively abstracts transferable strategies away from trajectory-specific details. (2) Experience Injection Pattern: Our analysis reveals that step-wise injection significantly outperforms global injection by aligning experience with intermediate decision states, a property that is critical for long-horizon tool use. (3) Internalization Regime: We demonstrate that off-policy context-distillation on high-quality teacher trajectories provides a substantially more stable training signal than on-policy context-distillation, which is inherently limited by local corrections on student-induced flawed states. Together, these insights yield a simple yet robust recipe for stable and sustainable experience internalization, providing concrete guidance for engineering self-evolving and continually learning LLMs.","short_abstract":"Experience internalization converts contextual experience from past interactions into reusable parametric capability, offering a promising path toward continual learning in large language models (LLMs). While prior work has predominantly focused on single-iteration transfer, we discover that under multi-iteration exper...","url_abs":"https://arxiv.org/abs/2606.04703","url_pdf":"https://arxiv.org/pdf/2606.04703v1","authors":"[\"Jingwen Chen\",\"Wenkai Yang\",\"Shengda Fan\",\"Wenbo Nie\",\"Chenxing Sun\",\"Shaodong Zheng\",\"Yangen Hu\",\"Lu Pan\",\"Ke Zeng\",\"Yankai Lin\"]","published":"2026-06-03T10:30:09Z","proceeding":"cs.CL","tasks":"[\"cs.CL\",\"cs.LG\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false}