{"ID":2880658,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2508.13773","arxiv_id":"2508.13773","title":"PENGUIN: Enhancing Transformer with Periodic-Nested Group Attention for Long-term Time Series Forecasting","abstract":"Despite advances in the Transformer architecture, their effectiveness for long-term time series forecasting (LTSF) remains controversial. In this paper, we investigate the potential of integrating explicit periodicity modeling into the self-attention mechanism to enhance the performance of Transformer-based architectures for LTSF. Specifically, we propose PENGUIN, a simple yet effective periodic-nested group attention mechanism. Our approach introduces a periodic-aware relative attention bias to directly capture periodic structures and a grouped multi-query attention mechanism to handle multiple coexisting periodicities (e.g., daily and weekly cycles) within time series data. Extensive experiments across diverse benchmarks demonstrate that PENGUIN consistently outperforms both MLP-based and Transformer-based models. Code is available at https://github.com/ysygMhdxw/AISTATS2026_PENGUIN.","short_abstract":"Despite advances in the Transformer architecture, their effectiveness for long-term time series forecasting (LTSF) remains controversial. In this paper, we investigate the potential of integrating explicit periodicity modeling into the self-attention mechanism to enhance the performance of Transformer-based architectur...","url_abs":"https://arxiv.org/abs/2508.13773","url_pdf":"https://arxiv.org/pdf/2508.13773v3","authors":"[\"Tian Sun\",\"Yuqi Chen\",\"Weiwei Sun\"]","published":"2025-08-19T12:19:12Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.AI\"]","methods":"[\"Transformer\"]","has_code":false,"code_links":[{"ID":610698,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2880658,"paper_url":"https://arxiv.org/abs/2508.13773","paper_title":"PENGUIN: Enhancing Transformer with Periodic-Nested Group Attention for Long-term Time Series Forecasting","repo_url":"https://github.com/ysygMhdxw/AISTATS2026_PENGUIN","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}