{"ID":2890603,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2507.19458","arxiv_id":"2507.19458","title":"Hierarchical Deep Reinforcement Learning Framework for Multi-Year Asset Management Under Budget Constraints","abstract":"Budget planning and maintenance optimization are crucial for infrastructure asset management, ensuring cost-effectiveness and sustainability. However, the complexity arising from combinatorial action spaces, diverse asset deterioration, stringent budget constraints, and environmental uncertainty significantly limits existing methods' scalability. This paper proposes a Hierarchical Deep Reinforcement Learning methodology specifically tailored to multi-year infrastructure planning. Our approach decomposes the problem into two hierarchical levels: a high-level Budget Planner allocating annual budgets within explicit feasibility bounds, and a low-level Maintenance Planner prioritizing assets within the allocated budget. By structurally separating macro-budget decisions from asset-level prioritization and integrating linear programming projection within a hierarchical Soft Actor-Critic framework, the method efficiently addresses exponential growth in the action space and ensures rigorous budget compliance. A case study evaluating sewer networks of varying sizes (10, 15, and 20 sewersheds) illustrates the effectiveness of the proposed approach. Compared to conventional Deep Q-Learning and enhanced genetic algorithms, our methodology converges more rapidly, scales effectively, and consistently delivers near-optimal solutions even as network size grows.","short_abstract":"Budget planning and maintenance optimization are crucial for infrastructure asset management, ensuring cost-effectiveness and sustainability. However, the complexity arising from combinatorial action spaces, diverse asset deterioration, stringent budget constraints, and environmental uncertainty significantly limits ex...","url_abs":"https://arxiv.org/abs/2507.19458","url_pdf":"https://arxiv.org/pdf/2507.19458v1","authors":"[\"Amir Fard\",\"Arnold X. -X. Yuan\"]","published":"2025-07-25T17:42:34Z","proceeding":"cs.AI","tasks":"[\"cs.AI\",\"cs.LG\",\"eess.SY\",\"math.OC\"]","methods":"[\"Reinforcement Learning\"]","has_code":false}