{"ID":2868631,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2509.15637","arxiv_id":"2509.15637","title":"Interplay Between Belief Propagation and Transformer: Differential-Attention Message Passing Transformer","abstract":"Transformer-based neural decoders have emerged as a promising approach to error correction coding, combining data-driven adaptability with efficient modeling of long-range dependencies. This paper presents a novel decoder architecture that integrates classical belief propagation principles with transformer designs. We introduce a differentiable syndrome loss function leveraging global codebook structure and a differential-attention mechanism optimizing bit and syndrome embedding interactions. Experimental results demonstrate consistent performance improvements over existing transformer-based decoders, with our approach surpassing traditional belief propagation decoders for short-to-medium length LDPC codes.","short_abstract":"Transformer-based neural decoders have emerged as a promising approach to error correction coding, combining data-driven adaptability with efficient modeling of long-range dependencies. This paper presents a novel decoder architecture that integrates classical belief propagation principles with transformer designs. We...","url_abs":"https://arxiv.org/abs/2509.15637","url_pdf":"https://arxiv.org/pdf/2509.15637v1","authors":"[\"Chin Wa Lau\",\"Xiang Shi\",\"Ziyan Zheng\",\"Haiwen Cao\",\"Nian Guo\"]","published":"2025-09-19T06:03:42Z","proceeding":"cs.IT","tasks":"[\"cs.IT\",\"eess.SP\"]","methods":"[\"Transformer\"]","has_code":false}