{"ID":2842367,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2511.10480","arxiv_id":"2511.10480","title":"STAGE: A Symbolic Tensor grAph GEnerator for distributed AI system co-design","abstract":"Optimizing the performance of large language models (LLMs) on large-scale AI training and inference systems requires a scalable and expressive mechanism to model distributed workload execution. Such modeling is essential for pre-deployment system-level optimizations (e.g., parallelization strategies) and design-space explorations. While recent efforts have proposed collecting execution traces from real systems, access to large-scale infrastructure remains limited to major cloud providers. Moreover, traces obtained from existing platforms cannot be easily adapted to study future larger-scale system configurations. We introduce Symbolic Tensor grAph GEnerator(STAGE), a framework that synthesizes high-fidelity execution traces to accurately model LLM workloads. STAGE supports a comprehensive set of parallelization strategies, allowing users to systematically explore a wide spectrum of LLM architectures and system configurations. STAGE demonstrates its scalability by synthesizing high-fidelity LLM traces spanning over 32K GPUs, while preserving tensor-level accuracy in compute, memory, and communication. STAGE is publicly available to facilitate further research in distributed machine learning systems: https://github.com/astra-sim/symbolic tensor graph","short_abstract":"Optimizing the performance of large language models (LLMs) on large-scale AI training and inference systems requires a scalable and expressive mechanism to model distributed workload execution. Such modeling is essential for pre-deployment system-level optimizations (e.g., parallelization strategies) and design-space e...","url_abs":"https://arxiv.org/abs/2511.10480","url_pdf":"https://arxiv.org/pdf/2511.10480v2","authors":"[\"Changhai Man\",\"Joongun Park\",\"Hanjiang Wu\",\"Huan Xu\",\"Srinivas Sridharan\",\"Tushar Krishna\"]","published":"2025-11-13T16:44:56Z","proceeding":"cs.DC","tasks":"[\"cs.DC\",\"cs.AI\"]","methods":"[\"Large Language Model\",\"Language Model\",\"LoRA\"]","has_code":false,"code_links":[{"ID":607122,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2842367,"paper_url":"https://arxiv.org/abs/2511.10480","paper_title":"STAGE: A Symbolic Tensor grAph GEnerator for distributed AI system co-design","repo_url":"https://github.com/astra-sim/symbolic","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}