{"ID":2864016,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.03298","arxiv_id":"2510.03298","title":"CAFL-L: Constraint-Aware Federated Learning with Lagrangian Dual Optimization for On-Device Language Models","abstract":"We introduce Constraint-Aware Federated Learning with Lagrangian Dual Optimization (CAFL-L), a principled extension of FedAvg that explicitly incorporates device-level resource constraints including energy, communication, memory, and thermal budgets. CAFL-L employs Lagrangian dual optimization to dynamically adapt training hyperparameters -- freezing depth, local steps, batch size, and communication compression -- while preserving training stability through token-budget preservation via gradient accumulation. Experiments on a character-level language model demonstrate that CAFL-L achieves superior constraint satisfaction compared to standard FedAvg (reducing memory usage by 20% and communication by 95%) while maintaining competitive validation performance, making it practical for deployment on resource-constrained edge devices.","short_abstract":"We introduce Constraint-Aware Federated Learning with Lagrangian Dual Optimization (CAFL-L), a principled extension of FedAvg that explicitly incorporates device-level resource constraints including energy, communication, memory, and thermal budgets. CAFL-L employs Lagrangian dual optimization to dynamically adapt trai...","url_abs":"https://arxiv.org/abs/2510.03298","url_pdf":"https://arxiv.org/pdf/2510.03298v2","authors":"[\"Dongqi Zheng\",\"Wenjin Fu\"]","published":"2025-09-29T22:07:20Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.CL\",\"cs.DC\"]","methods":"[\"Language Model\"]","has_code":false}