{"ID":2895646,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2507.08456","arxiv_id":"2507.08456","title":"Space filling positionality and the Spiroformer","abstract":"Transformers excel when dealing with sequential data. Generalizing transformer models to geometric domains, such as manifolds, we encounter the problem of not having a well-defined global order. We propose a solution with attention heads following a space-filling curve. As a first experimental example, we present the Spiroformer, a transformer that follows a polar spiral on the $2$-sphere.","short_abstract":"Transformers excel when dealing with sequential data. Generalizing transformer models to geometric domains, such as manifolds, we encounter the problem of not having a well-defined global order. We propose a solution with attention heads following a space-filling curve. As a first experimental example, we present the S...","url_abs":"https://arxiv.org/abs/2507.08456","url_pdf":"https://arxiv.org/pdf/2507.08456v1","authors":"[\"M. Maurin\",\"M. Á. Evangelista-Alvarado\",\"P. Suárez-Serrato\"]","published":"2025-07-11T09:56:15Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.AI\",\"math.DG\",\"math.DS\",\"math.SG\"]","methods":"[\"Transformer\"]","has_code":false}