Parity Requires Unified Input Dependence and Negative Eigenvalues in SSMs

Behnoush Khavari; Fran\c{c}ois Rivest; Jayesh Khullar; Jerry Huang; Mehran Shakerinava; Sarath Chandar; Siamak Ravanbakhsh

REVIEW 4 major objections 5 minor 17 references

Parity Requires Unified Input Dependence and Negative Eigenvalues in SSMs

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper proves that stacking S4D and Mamba layers still cannot solve parity under finite precision; a single recurrence layer must combine input dependence with negative or complex eigenvalues.

desk verdict The hybrid-parity question is the right one and the theorem is likely true, but the proof silently assumes rational eigenvalue phases, which generic finite-precision S4D layers do not satisfy. read the letter →

arxiv 2508.07395 v1 pith:Q27NNAFV submitted 2025-08-10 cs.LG

Behnoush Khavari , Mehran Shakerinava , Jayesh Khullar , Jerry Huang , François Rivest , Siamak Ravanbakhsh , Sarath Chandar This is my paper

classification cs.LG

keywords paritystate-trackingstatespacemodelslinearrecurrentneuralnetworksfiniteprecisionS4DMambanegativeeigenvalues

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Parity—whether a bitstring has an even number of ones—is a minimal state-tracking task that many linear recurrent networks cannot generalize on. Earlier work showed the failure has two separate architectural causes: time-invariant diagonal SSMs like S4D ignore the input, while input-dependent diagonal SSMs like Mamba restrict their eigenvalues to non-negative values. This paper asks whether the weaknesses cancel when the two layer types are stacked. It proves they do not: a finite-precision network made of S4D and Mamba layers, with skip connections and a learnable initial state, still cannot solve parity for arbitrary lengths. The conclusion is that the required properties must coexist inside a single recurrence layer.

What carries the argument

The load-bearing device is the coarse-grained $W$-step transition over the repeating block $s = 1 0^{W-1}$. In an S4D layer, $W$ is chosen so that $W q_j$ is an integer for each eigenvalue phase $q_j$, making $A_j^W$ a real non-negative number and turning a complex time-invariant recurrence into a non-negative block recurrence; in a Mamba layer, the product of the input-dependent transition matrices $A(x_1),\ldots,A(x_W)$ is non-negative by design. Under finite precision, repeated application of either block transition converges to a fixed value, so the state at every phase of the block stops carrying information about the block count. A second mechanism, Lemma 4.3, coarse-grains pairs of in

What would settle it

Simulate a finite-precision S4D layer with $A = r e^{2\pi i q}$ for a rational $q$ with a large denominator on repeated $1 0^{W-1}$ blocks and inspect the hidden state at block boundaries: if it enters a cycle longer than 1 instead of becoming stationary, the convergence step in the proof of Theorem 4.4 is false. A complementary experiment: train a two-layer S4D+Mamba model on length-8 parity and test on $s^T$ with $s = 1 0^{W-1}$; the theorem predicts extrapolation accuracy near 50%.

Watch

Extended reading notes

Core claim

Theorem 4.4: a finite-precision LRNN made of S4D and Mamba layers, with skip connections and a learnable initial state, cannot solve parity. The all-ones input cannot carry the proof, since a single S4D layer with a complex eigenvalue solves modular counting there. The proof repeats the block $1 0^{W-1}$, with $W$ chosen so each S4D eigenvalue $A_j = r_j e^{2\pi i q_j}$ satisfies $A_j^W \ge 0$. Both layer types then act non-negatively at block scale, and finite precision makes their states converge to a fixed pattern, so block-count parity is unreadable. Experiments agree: length-8 training reaches 100%, extrapolation stays near 50%.

Load-bearing premise

The proof needs finite-precision arithmetic to make repeated block transitions settle at a fixed value, and it needs every S4D eigenvalue phase angle to be rational so that one fixed power $W$ turns all complex eigenvalues real and non-negative; if either condition fails—exact arithmetic, irrational phases, or finite precision that only enters a longer cycle—the convergence argument in Section B no longer applies.

Editorial extensions

If this is right

A finite-precision diagonal SSM that solves parity must contain a layer whose transition is input-dependent and has a negative (or genuinely complex) eigenvalue at the same time; no stacking of complementary layers can supply the missing property.
S4D+Mamba hybrids will not extrapolate on parity: the experiments show near-perfect training accuracy on length-8 sequences and near-random accuracy on longer ones, same as S4D-only and Mamba-only stacks.
The standard all-ones input is too weak to demonstrate failure of hybrid models, because a time-invariant S4D layer with complex eigenvalues can already solve modular counting on it; the repeated-block input class is needed to expose the hybrid's limitation.
Skip connections and learnable initial states do not rescue the hybrid within the diagonal finite-precision setting, since the impossibility proof explicitly allows both.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper, the block-coarse-graining mechanism yields a quantitative prediction: at fixed precision, the length at which extrapolation fails should grow with the least common multiple of the denominators of the S4D eigenvalue phases, since $W$ must cover those denominators.
The theorem is a finite-precision statement; in exact arithmetic, a time-invariant diagonal layer with an irrational-phase eigenvalue would keep rotating instead of converging, so the result leaves open whether idealized infinite-precision linear recurrences could solve parity without input dependence.
The authors leave non-diagonal transition matrices for future work, but the same $W$-step folding idea would carry over to any layer whose eigenvalues are scaled rotations by rational phases, suggesting the unified-property conclusion may be a general design rule for efficient SSMs rather than a diagonal-model artifact.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper asks a question that prior work left open: for diagonal SSMs, can you split input dependence and negative eigenvalues across layers and still solve parity? The answer it gives — no — is exactly what the community needs, and if Theorem 4.4 were fully established it would be a compact, useful result. The modular-counting observation (Prop 4.2) is correct and nicely contrasts Mamba's failure mode. The experiments are light, but they are consistent with the theory and are not the main claim. I think the paper deserves a serious referee, but not in its current form.

The soft spot is load-bearing. The proof of Theorem 4.4 in Section B requires choosing an integer W such that every S4D eigenvalue A_j = r_j exp(2πi q_j) has W q_j ∈ N, making (A_j)^W real and non-negative. This is the hinge of the whole convergence argument. But nothing in the definition of a finite-precision LRNN forces the phases q_j to be rational. A perfectly valid finite-precision S4D layer can have A = 3 + i, whose phase is irrational, and then (3+i)^W is never real non-negative for any positive W. For such a layer the construction of W and of the adversarial input s^T breaks down, and the induction has no base case. The stress-test note is right: the block-level dynamics can be a rotation, and finite precision can enter cycles of period greater than 1, which the convergence argument does not rule out. This is not a minor footnoted assumption; the theorem as stated is not established for generic finite-precision parameters.

There is also a small but real error in Appendix E: Lemma E.2 claims the product of two real symmetric PSD matrices has only non-negative entries. That is false — take [[1,0],[0,0]] and [[1,-1],[-1,1]]. The diagonal case needed for Lemma 4.3 holds, so the main lemma survives, but the appendix overgeneralizes.

On the credit side: the contribution over Sarrof, Grazzi, and Merrill is genuine — the hybrid-layer impossibility is new and clearly stated. The writing is honest about the diagonal restriction. The citation pattern looks fair, and the proof structure is otherwise plausible.

My recommendation: send this to peer review, because the question matters and the gap looks fixable. The authors should either restrict the theorem to eigenvalues with rational phase (covering most practical initializations) or find an argument that handles arbitrary finite-precision phases, perhaps via periodicity or compactness. Until then, I would not cite the theorem as proven, but I would cite it as a promising partial result and encourage the follow-up.

Referee Report

4 major / 5 minor

Summary. The paper studies the expressivity of hybrid linear recurrent neural networks / state space models, focusing on the parity task. Its main claim is Theorem 4.4: a finite-precision LRNN composed of S4D and Mamba layers, skip connections, and a learnable initial state cannot solve parity. The proof constructs periodic inputs of the form 10^{W-1} repeated T times and argues that after coarse graining, both S4D and Mamba layers eventually produce states that are stationary or nearly stationary, so parity information is lost. The paper also introduces a failure mode for non-negative SSMs on inputs such as (01)^T, gives a modular-counting construction for S4D, and reports experiments on parity extrapolation and an offset-prediction task.

Significance. If the main theorem were established, the paper would answer a natural open question from Grazzi et al. (2024): whether input dependence and negative eigenvalues can be separated across layers rather than being present in the same recurrent layer. The question is well motivated, and the proof strategy adapts known impossibility arguments to the hybrid setting. The paper also has several useful conceptual contributions, including explicit examples and experiments. However, the proof of the central theorem contains a load-bearing unjustified assumption about eigenvalue phases, and two supporting statements are false or mis-stated as written. The central claim is therefore not currently established, and the paper needs substantial revision.

major comments (4)

[Section B, proof of Theorem 4.4] The proof begins by asserting that every S4D eigenvalue A_j can be written as r_j e^{2π i q_j} with q_j rational, and then chooses a positive integer W such that W q_j ∈ N. This is unjustified and is false even for finite-precision parameters. Finite precision stores real and imaginary parts as dyadic rationals; for example, A = 3 + i is exactly representable in finite precision, but q = atan2(1,3)/(2π) is irrational, so no integer W makes (3+i)^W real and non-negative. The construction of W, the input s = 10^{W-1}, and the key simplification A^{tW} = r^{tW} in the S4D case all depend on this rationality assumption. Consequently the induction over layers fails for generic finite-precision S4D layers. Restricting Theorem 4.4 to rational-phase eigenvalues would be a substantial weakening; alternatively, the authors need a genuine finite-precision eventual-periodicity argument that handles
[Section 4.2, Proposition 4.2] The proof of Proposition 4.2 sets A = exp(2πi/T), where T is the length of the input. This parameter dependence on T is not valid for a fixed model: a single S4D layer must process inputs of all lengths. The construction as written does not show that a fixed S4D layer can solve modular counting or parity, nor that the failure mode of Theorem 4.1 can be circumvented. The intended construction is presumably A = exp(2πi/m) for a fixed modulus m, with B = 0 and h0 = 1; this should be stated and proved. As written, the proposition is false.
[Section E, Lemma E.2] Lemma E.2 claims that for two real symmetric PSD matrices A and B, the product AB has only non-negative entries. This is false. For example, A = [[1,-1],[-1,2]] and B = [[1,-2],[-2,5]] are both PSD, but AB = [[3,-7],[-5,12]] has negative entries. The proof in Section E actually establishes only that the eigenvalues of AB are non-negative, not that its entries are non-negative. This invalidates the attempted generalization in Lemma 4.3 to non-diagonal PSD input-dependent transition matrices; the diagonal case used in Theorem 4.4 is trivially true, but the PSD statement as written is incorrect.
[Section B, finite-precision convergence argument] The proof repeatedly asserts that in finite precision, the terms U1 and U2 'must converge' because they diverge exponentially, diverge linearly, or converge. This is a hand-wavy account: the paper never defines the finite-precision model (rounding mode, overflow/underflow semantics), and it assumes that overflow saturates to a fixed value and underflow becomes exactly zero. Moreover, with irrational-phase eigenvalues, the block-level dynamics can rotate, and finite-precision rounding can enter cycles of period greater than 1. The proof does not rule out cycles whose parity phase could encode the answer. This is a load-bearing gap that compounds the rational-phase issue above.

minor comments (5)

[Section 2 and Appendix A] The background defines Mamba with h_t = A(x_t)h_{t-1} + B(x_t)x_t, but the recurrence in Eq. (2) and the proof of Lemma 4.3 use B(x_t) without the input x_t. Please fix the notation consistently.
[Section 5] The text refers to 'Theorem 4.2' in the Offset Prediction section, but the paper only contains Proposition 4.2. Please correct the cross-reference.
[Section E] There are several typos: 'principar square root', 'R⋉', and 'eignenvalues'. Also the statement 'Theorem E.4' should refer to Lemma E.4.
[Section C] The captions of Figures 7/8 say 'Double ISI outputs' while the text says 'Two ISIs'; the terminology (offset vs onset) is also inconsistent and should be unified.
[Section 6, Limitations] The limitations section acknowledges that the analysis uses a vector-valued hidden state rather than the full matrix representation used in some SSM implementations. Since the diagonal-state theorem is the paper's core claim, a brief note on why this simplification does not change the result would strengthen the presentation.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the central impossibility theorem is a new adversarial construction, and the cited background theorems are external.

full rationale

The paper's central claim (Theorem 4.4) is a theoretical impossibility result. Its proof in Section B constructs, for any finite-precision S4D+Mamba model, an adversarial input (s^T with s=10^(W-1)) and shows the coarse-grained state converges, making parity unreadable for large T. The construction does not fit any parameter to the parity label, does not define parity in terms of the model's states, and does not rename an existing empirical result. The cited results (Sarrof et al. Theorem 2/13, Grazzi et al. Theorem 1) are used as independent background; their authors do not overlap with the present paper, so there is no load-bearing self-citation. Proposition 4.2 is a constructive existence proof, not a fitted prediction, and the experiments merely confirm the claimed failure empirically. The Limitations section acknowledges a simplified vector-valued hidden state; this is a scope assumption, not a circular step. The one notable weakness is in Appendix B: the proof assumes every S4D eigenvalue A_j = r_j exp(2π i q_j) has rational q_j so that a positive integer W with W q_j ∈ N exists. For a genuinely finite-precision complex eigenvalue such as 3+i (dyadic real and imaginary parts), the phase is irrational and no such W exists, so the adversarial input may be undefined and the convergence argument fails as written. That is a soundness/correctness gap in the theorem as stated, but it is not circularity: the conclusion is not equivalent to the assumptions by construction, no parameter fitted to the target is relabeled as a prediction, and the proof does not reduce to a self-citation chain.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new free parameters or invented entities: no new force, particle, dimension, or fitted constant. The central claim rests on the finite-precision saturation assumption, the rational-phase assumption, diagonality of the transition matrices, and a restricted output readout. All of these are architectural or modeling assumptions rather than parameters fitted to data.

assumptions (4)

domain assumption Finite-precision arithmetic causes powers of non-negative transition matrices to converge to a fixed saturated or underflowed value after finitely many steps.
Used in Section B and Section A to conclude that states on the constructed periodic input become constant for large T; this is the core mechanism of the impossibility result.
domain assumption All S4D diagonal eigenvalues have rational phase angles q_j, so some W exists with W q_j integer and A_j^W real non-negative.
Invoked in the first paragraph of the proof of Theorem 4.4 to construct the input s^T = (1 0^{W-1})^T. Unstated as an assumption; holds for finite-precision parameters but not for arbitrary real eigenvalues.
domain assumption Mamba transition matrices are diagonal with non-negative entries, and S4D transition matrices are diagonal.
Restricts the theorem to efficient diagonal SSMs, as stated in the abstract and Section 4.4. The proof uses diagonality to reduce matrix powers to scalar diagonal powers.
domain assumption The output or acceptance decision is a non-recurrent function of the final state and input, so periodic states imply periodic outputs.
Needed at the end of Section B to argue that a periodic state cycle of length W cannot encode the parity of arbitrarily large T; skip connections are handled by concatenation, not by recurrence.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Parity Requires Unified Input Dependence and Negative Eigenvalues in SSMs." pith.science (2026). https://pith.science/paper/Q27NNAFV

@misc{pith2026250807395,
  author       = {Pith},
  title        = {Pith review of: Parity Requires Unified Input Dependence and Negative Eigenvalues in SSMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q27NNAFV}},
  note         = {Machine review of arXiv:2508.07395}
}

read the original abstract

Recent work has shown that LRNN models such as S4D, Mamba, and DeltaNet lack state-tracking capability due to either time-invariant transition matrices or restricted eigenvalue ranges. To address this, input-dependent transition matrices, particularly those that are complex or non-triangular, have been proposed to enhance SSM performance on such tasks. While existing theorems demonstrate that both input-independent and non-negative SSMs are incapable of solving simple state-tracking tasks, such as parity, regardless of depth, they do not explore whether combining these two types in a multilayer SSM could help. We investigate this question for efficient SSMs with diagonal transition matrices and show that such combinations still fail to solve parity. This implies that a recurrence layer must both be input-dependent and include negative eigenvalues. Our experiments support this conclusion by analyzing an SSM model that combines S4D and Mamba layers.

Figures

Figures reproduced from arXiv: 2508.07395 by the authors.

**Figure 1.** (Top) Output of S4D on the offset prediction task, trained for 1k epochs with a hidden size of 8 and a learning rate of 0.005. (Bottom) Output of Mamba on the offset prediction task, trained for 1k epochs with a hidden size of 8 and a learning rate of 0.01. Both models predict the offset correctly and anticipate the onset. 6. Conclusion, Limitations, and Future Directions The core question we addressed in this paper… view at source ↗

**Figure 2.** Finite-state automaton for offset prediction task when n = 4. C.1. Results In this section, we provide the results for the offset prediction task and generalization performance of S4D and Mamba on the offset prediction task. The two setups designed for evaluating their generalization are explained below [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

**Figure 3.** Offset prediction of S4D. We see that S4D correctly predicts the offset for all ISIs [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Offset prediction of Mamba. We see that Mamba correctly predicts the offset for all ISIs [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

**Figure 5.** Figure 5: Two ISI outputs of S4D. We see that S4D correctly predicts the offset for both ISIs. 9 [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

**Figure 6.** Figure 6: Two ISI outputs of Mamba. We see that Mamba correctly predicts the offset for both ISIs. Two ISIs Here we test the model’s generalizability by introducing it to signals where there are only two ISI present, separated by a longer than trained ITI. The objective is to de…

**Figure 7.** Figure 7: Double ISI outputs for S4D. We observe that S4D can anticipate the second ISI [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]

**Figure 8.** Figure 8: Double ISI outputs for Mamba . Mamba does not anticipate the second ISI. Double ISI This tests the model’s ability to count when introduced to two consecutive ISIs. The model should turn off at the first offset and then turn on again before the end of the second offset…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

17 extracted references · 11 canonical work pages

[1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
[2]

Simplicity bias in transformers and their ability to learn sparse boolean functions

Bhattamishra, S., Patel, A., Kanade, V., and Blunsom, P. Simplicity bias in transformers and their ability to learn sparse boolean functions. arXiv preprint arXiv:2211.12316, 2022

arXiv 2022
[3]

K., Catt, E., Cundy, C., Hutter, M., Legg, S., Veness, J., and Ortega, P

Deletang, G., Ruoss, A., Grau-Moya, J., Genewein, T., Wenliang, L. K., Catt, E., Cundy, C., Hutter, M., Legg, S., Veness, J., and Ortega, P. A. Neural networks and the chomsky hierarchy. In International Conference on Learning Representations, 2023

work page 2023
[4]

Y., Dao, T., Saab, K

Fu, D. Y., Dao, T., Saab, K. K., Thomas, A. W., Rudra, A., and R \'e , C. Hungry hungry hippos: Towards language modeling with state space models. arXiv preprint arXiv:2212.14052, 2022

arXiv 2022
[5]

Grazzi, R., Siems, J., Franke, J. K. H., Zela, A., Hutter, F., and Pontil, M. Unlocking state-tracking in linear rnns through negative eigenvalues, 2024

work page 2024
[6]

and Dao, T

Gu, A. and Dao, T. Mamba: Linear-time sequence modeling with selective state spaces. In First Conference on Language Modeling, 2024

work page 2024
[7]

Hippo: Recurrent memory with optimal polynomial projections

Gu, A., Dao, T., Ermon, S., Rudra, A., and R\' e , C. Hippo: Recurrent memory with optimal polynomial projections. In Conference on Advances in Neural Information Processing Systems, 2020

work page 2020
[8]

Efficiently modeling long sequences with structured state spaces

Gu, A., Goel, K., and R \'e , C. Efficiently modeling long sequences with structured state spaces. arXiv preprint arXiv:2111.00396, 2021

arXiv 2021

Show all 17 references

[9]

On the parameterization and initialization of diagonal state space models

Gu, A., Goel, K., Gupta, A., and R \'e , C. On the parameterization and initialization of diagonal state space models. In Conference on Advances in Neural Information Processing Systems, 2022

2022
[10]

Diagonal state spaces are as effective as structured state spaces

Gupta, A., Gu, A., and Berant, J. Diagonal state spaces are as effective as structured state spaces. Advances in Neural Information Processing Systems, 35: 0 22982--22994, 2022

2022
[11]

and Rofin, M

Hahn, M. and Rofin, M. Why are sensitive functions hard for transformers? arXiv preprint arXiv:2402.09963, 2024

2024 arXiv
[12]

T., Goel, S., Krishnamurthy, A., and Zhang, C

Liu, B., Ash, J. T., Goel, S., Krishnamurthy, A., and Zhang, C. Transformers learn shortcuts to automata. In International Conference on Learning Representations, 2023

2023
[13]

The illusion of state in state-space models

Merrill, W., Petty, J., and Sabharwal, A. The illusion of state in state-space models. In International Conference on Machine Learning, 2024

2024
[14]

The expressive capacity of state space models: A formal language perspective

Sarrof, Y., Veitsman, Y., and Hahn, M. The expressive capacity of state space models: A formal language perspective. In Conference on Advances in Neural Information Processing Systems, 2024

2024
[15]

Deltaproduct: Improving state-tracking in linear rnns via householder products

Siems, J., Carstensen, T., Zela, A., Hutter, F., Pontil, M., and Grazzi, R. Deltaproduct: Improving state-tracking in linear rnns via householder products. arXiv preprint arXiv:2502.10297, 2025

2025
[16]

T., Warrington, A., and Linderman, S

Smith, J. T., Warrington, A., and Linderman, S. Simplified state space layers for sequence modeling. In International Conference on Learning Representations, 2023

2023
[17]

Long range arena : A benchmark for efficient transformers

Tay, Y., Dehghani, M., Abnar, S., Shen, Y., Bahri, D., Pham, P., Rao, J., Yang, L., Ruder, S., and Metzler, D. Long range arena : A benchmark for efficient transformers. In International Conference on Learning Representations, 2021

2021

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

[2] [2]

Simplicity bias in transformers and their ability to learn sparse boolean functions

Bhattamishra, S., Patel, A., Kanade, V., and Blunsom, P. Simplicity bias in transformers and their ability to learn sparse boolean functions. arXiv preprint arXiv:2211.12316, 2022

arXiv 2022

[3] [3]

K., Catt, E., Cundy, C., Hutter, M., Legg, S., Veness, J., and Ortega, P

Deletang, G., Ruoss, A., Grau-Moya, J., Genewein, T., Wenliang, L. K., Catt, E., Cundy, C., Hutter, M., Legg, S., Veness, J., and Ortega, P. A. Neural networks and the chomsky hierarchy. In International Conference on Learning Representations, 2023

work page 2023

[4] [4]

Y., Dao, T., Saab, K

Fu, D. Y., Dao, T., Saab, K. K., Thomas, A. W., Rudra, A., and R \'e , C. Hungry hungry hippos: Towards language modeling with state space models. arXiv preprint arXiv:2212.14052, 2022

arXiv 2022

[5] [5]

Grazzi, R., Siems, J., Franke, J. K. H., Zela, A., Hutter, F., and Pontil, M. Unlocking state-tracking in linear rnns through negative eigenvalues, 2024

work page 2024

[6] [6]

and Dao, T

Gu, A. and Dao, T. Mamba: Linear-time sequence modeling with selective state spaces. In First Conference on Language Modeling, 2024

work page 2024

[7] [7]

Hippo: Recurrent memory with optimal polynomial projections

Gu, A., Dao, T., Ermon, S., Rudra, A., and R\' e , C. Hippo: Recurrent memory with optimal polynomial projections. In Conference on Advances in Neural Information Processing Systems, 2020

work page 2020

[8] [8]

Efficiently modeling long sequences with structured state spaces

Gu, A., Goel, K., and R \'e , C. Efficiently modeling long sequences with structured state spaces. arXiv preprint arXiv:2111.00396, 2021

arXiv 2021