{"ID":5937917,"CreatedAt":"2026-07-07T03:14:33.014478982Z","UpdatedAt":"2026-07-07T08:21:00.942939557Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.03763","arxiv_id":"2607.03763","title":"FedACT: Federated Adaptive Coordinate Trust Modulation for Robust Transformer Training under Data Heterogeneity","abstract":"Federated Transformer training increasingly relies on local AdamW, whose adaptive updates can provide much stronger local progress than SGD-based training. However, under heterogeneous client data, even globally corrected AdamW updates may remain highly uneven in coordinate-wise reliability. We refer to this phenomenon as coordinate trust mismatch. Existing federated adaptive optimizers mainly address mismatch at the client-update or communication-round level, but still apply the corrected adaptive direction densely and uniformly across coordinates. In this paper, we propose FedACT, a global-aware coordinate trust modulation method for federated AdamW training. FedACT first forms a globally corrected adaptive direction and then reallocates update magnitudes according to a coordinate-wise trust score, assigning larger steps to coordinates jointly supported by local gradients and global correction, while preserving smaller non-zero updates on the remaining coordinates. Extensive experiments on federated vision Transformers, CNNs, LLM pre-training, and LLM fine-tuning show that FedACT consistently improves over strong federated adaptive baselines, with the largest gains on Transformer models under stronger data heterogeneity. Mechanism analyses further show that FedACT improves cross-client direction consistency, suggesting that coordinate-level trust allocation effectively complements round-level global-local correction. Code will be released.","short_abstract":"Federated Transformer training increasingly relies on local AdamW, whose adaptive updates can provide much stronger local progress than SGD-based training. However, under heterogeneous client data, even globally corrected AdamW updates may remain highly uneven in coordinate-wise reliability. We refer to this phenomenon...","url_abs":"https://arxiv.org/abs/2607.03763","url_pdf":"https://arxiv.org/pdf/2607.03763v1","authors":"[\"Shuai Li\",\"Qinglin Wang\",\"Ping Luo\",\"Jiahuan Wang\",\"Hongyang Hu\",\"Haotian Mo\",\"Yigui Feng\",\"Ziang Liu\",\"Qisong Xiao\",\"Jie Liu\",\"Tao Sun\"]","published":"2026-07-04T08:31:49Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.AI\"]","methods":"[\"Vision Transformer\",\"Transformer\",\"Large Language Model\",\"Convolutional Neural Network\"]","has_code":false}