{"ID":2862945,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2509.26538","arxiv_id":"2509.26538","title":"HilbertA: Hilbert Attention for Image Generation with Diffusion Models","abstract":"Designing sparse attention for diffusion transformers requires reconciling two-dimensional spatial locality with GPU efficiency, a trade-off that current methods struggle to achieve. Existing approaches enforce two-dimensional spatial locality but often incur uncoalesced memory access. We present HilbertA, a 2D-aware and GPU-efficient sparse attention mechanism. HilbertA reorders image tokens along Hilbert curves to achieve a contiguous memory layout while preserving spatial neighborhoods, and employs a sliding schedule across layers to enable long-range information propagation without repeated or uncoalesced memory access. To further enhance cross-tile communication and positional awareness, HilbertA introduces a small central shared region. Implemented in Triton, HilbertA delivers comparable image quality with significant acceleration over prior methods on Flux.1-dev, demonstrating the feasibility of hardware-aligned two-dimensional sparse attention for high-resolution image generation. HilbertA delivers attention speedups of $2.3\\times$ when generating $1024\\times 1024$ images, and up to $4.17\\times$ at $2048\\times 2048$, while achieving image quality comparable to or surpassing baselines.","short_abstract":"Designing sparse attention for diffusion transformers requires reconciling two-dimensional spatial locality with GPU efficiency, a trade-off that current methods struggle to achieve. Existing approaches enforce two-dimensional spatial locality but often incur uncoalesced memory access. We present HilbertA, a 2D-aware a...","url_abs":"https://arxiv.org/abs/2509.26538","url_pdf":"https://arxiv.org/pdf/2509.26538v1","authors":"[\"Shaoyi Zheng\",\"Wenbo Lu\",\"Yuxuan Xia\",\"Haomin Liu\",\"Shengjie Wang\"]","published":"2025-09-30T17:13:22Z","proceeding":"cs.AI","tasks":"[\"cs.AI\"]","methods":"[\"Diffusion Model\",\"Transformer\"]","has_code":false}