{"ID":5676042,"CreatedAt":"2026-07-03T01:40:09.565152011Z","UpdatedAt":"2026-07-05T00:20:42.556504712Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.01553","arxiv_id":"2607.01553","title":"X-LogSMask: Expand Transformer for Graph-Structured Data","abstract":"Transformers have become general-purpose architectures, but their all-to-all self-attention is poorly matched to graph data, whose interactions are sparse, structured and multi-scale. Existing Graph Transformers address this mismatch through structural encodings, hybrid message-passing modules or learned attention constraints, often introducing additional complexity and limited interpretability. Here we introduce X-LogSMask, an explainable multi-head logarithmic structural mask that injects symmetrically normalized graph topology directly into attention logits. The logarithmic transform converts structural connectivity into a topology-aware gating signal, suppressing unsupported node interactions while preserving feature-dependent attention. By assigning different powers of the normalized adjacency matrix to different attention heads, X-LogSMask gives each head a defined structural radius and supports multi-hop information propagation within a single layer. We further show that a standard Transformer encoder can be interpreted as one-step message passing on a complete graph, motivating X-LogSMask as a topology-constrained alternative to unrestricted self-attention. Across 20 node-, edge- and graph-level benchmarks, Transformers equipped with X-LogSMask achieve state-of-the-art performance on 13 datasets and remain competitive in a lightweight one-layer configuration. These results show that simple, interpretable structural masks can make self-attention an effective graph-learning operator without changing the Transformer architecture. The code is available at https://github.com/LiLeyan-0120/X-LogSMask.","short_abstract":"Transformers have become general-purpose architectures, but their all-to-all self-attention is poorly matched to graph data, whose interactions are sparse, structured and multi-scale. Existing Graph Transformers address this mismatch through structural encodings, hybrid message-passing modules or learned attention cons...","url_abs":"https://arxiv.org/abs/2607.01553","url_pdf":"https://arxiv.org/pdf/2607.01553v1","authors":"[\"Leyan Li\",\"Rennong Yang\",\"Zhenxing Zhang\",\"Liping Hu\"]","published":"2026-07-02T00:20:58Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.AI\"]","methods":"[\"Transformer\"]","has_code":false,"code_links":[{"ID":613914,"CreatedAt":"2026-07-03T01:40:09.565152011Z","UpdatedAt":"2026-07-03T01:40:09.565152011Z","DeletedAt":null,"paper_id":5676042,"paper_url":"https://arxiv.org/abs/2607.01553","paper_title":"X-LogSMask: Expand Transformer for Graph-Structured Data","repo_url":"https://github.com/LiLeyan-0120/X-LogSMask","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}