{"ID":2884180,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2508.07395","arxiv_id":"2508.07395","title":"Parity Requires Unified Input Dependence and Negative Eigenvalues in SSMs","abstract":"Recent work has shown that LRNN models such as S4D, Mamba, and DeltaNet lack state-tracking capability due to either time-invariant transition matrices or restricted eigenvalue ranges. To address this, input-dependent transition matrices, particularly those that are complex or non-triangular, have been proposed to enhance SSM performance on such tasks. While existing theorems demonstrate that both input-independent and non-negative SSMs are incapable of solving simple state-tracking tasks, such as parity, regardless of depth, they do not explore whether combining these two types in a multilayer SSM could help. We investigate this question for efficient SSMs with diagonal transition matrices and show that such combinations still fail to solve parity. This implies that a recurrence layer must both be input-dependent and include negative eigenvalues. Our experiments support this conclusion by analyzing an SSM model that combines S4D and Mamba layers.","short_abstract":"Recent work has shown that LRNN models such as S4D, Mamba, and DeltaNet lack state-tracking capability due to either time-invariant transition matrices or restricted eigenvalue ranges. To address this, input-dependent transition matrices, particularly those that are complex or non-triangular, have been proposed to enha...","url_abs":"https://arxiv.org/abs/2508.07395","url_pdf":"https://arxiv.org/pdf/2508.07395v1","authors":"[\"Behnoush Khavari\",\"Mehran Shakerinava\",\"Jayesh Khullar\",\"Jerry Huang\",\"François Rivest\",\"Siamak Ravanbakhsh\",\"Sarath Chandar\"]","published":"2025-08-10T15:49:44Z","proceeding":"cs.LG","tasks":"[\"cs.LG\"]","methods":"[]","has_code":false}