{"ID":2851567,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.19322","arxiv_id":"2510.19322","title":"Enabling Reconfiguration-Communication Overlap for Collective Communication in Optical Networks","abstract":"Collective communication (CC) is critical for scaling distributed machine learning (DML). The predictable traffic patterns of DML present a great opportunity for applying optical network technologies. Optical networks with reconfigurable topologies promise high bandwidth and low latency for collective communications. However, existing approaches face inherent limitations: static topologies are inefficient for dynamic communication patterns within CC algorithm, while frequent topology reconfiguration matching every step of the algorithm incurs significant overhead. In this paper, we propose SWOT, a demand-aware optical network framework that employs ``intra-collective reconfiguration'' to dynamically align network resources with CC traffic patterns. SWOT hides reconfiguration latency by overlapping it with data transmission through three key techniques: \\textit{Heterogeneous Message Splitting}, \\textit{Asynchronous Overlapping}, and \\textit{Topology Bypassing}. Extensive simulations demonstrate that SWOT reduces communication completion time up to 89.7% across diverse CC algorithm compared to static baselines, demonstrating strong robustness to varying optical resources and reconfiguration delay.","short_abstract":"Collective communication (CC) is critical for scaling distributed machine learning (DML). The predictable traffic patterns of DML present a great opportunity for applying optical network technologies. Optical networks with reconfigurable topologies promise high bandwidth and low latency for collective communications. H...","url_abs":"https://arxiv.org/abs/2510.19322","url_pdf":"https://arxiv.org/pdf/2510.19322v3","authors":"[\"Changbo Wu\",\"Zhuolong Yu\",\"Gongming Zhao\",\"Hongli Xu\"]","published":"2025-10-22T07:34:04Z","proceeding":"cs.NI","tasks":"[\"cs.NI\",\"cs.AI\",\"cs.DC\"]","methods":"[]","has_code":false}