{"ID":2867223,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2509.19130","arxiv_id":"2509.19130","title":"Deep Reinforcement Learning for Dynamic Sensing and Communications","abstract":"Environmental sensing can significantly enhance mmWave communications by assisting beam training, yet its benefits must be balanced against the associated sensing costs. To this end, we propose a unified machine learning framework that dynamically determines when to sense and leverages sensory data for beam prediction. Specifically, we formulate a joint sensing and beamforming problem that maximizes the average signal-to-noise ratio under an average sensing budget. Lyapunov optimization is employed to enforce the sensing constraint, while a deep Q-Network determines the sensing slots. A pretrained deep neural network then maps the sensing data to optimal beams in the codebook. Simulations based on the real-world DeepSense dataset demonstrate that the proposed approach substantially reduces sensing overhead while maintaining satisfactory communications performance.","short_abstract":"Environmental sensing can significantly enhance mmWave communications by assisting beam training, yet its benefits must be balanced against the associated sensing costs. To this end, we propose a unified machine learning framework that dynamically determines when to sense and leverages sensory data for beam prediction....","url_abs":"https://arxiv.org/abs/2509.19130","url_pdf":"https://arxiv.org/pdf/2509.19130v1","authors":"[\"Abolfazl Zakeri\",\"Nhan Thanh Nguyen\",\"Ahmed Alkhateeb\",\"Markku Juntti\"]","published":"2025-09-23T15:15:42Z","proceeding":"eess.SP","tasks":"[\"eess.SP\"]","methods":"[\"Reinforcement Learning\"]","has_code":false}