Pith. sign in

REVIEW 4 major objections 5 minor 20 references

Algorithm Development in Neural Networks: Insights from the Streaming Parity Task

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read RNNs trained only on short parity strings can, with small initial weights and enough data, cross a sharp phase transition to perfect generalization on sequences of arbitrary length, by merging hidden representations into a finite automaton.

desk verdict A credible, honestly limited mechanism paper for infinite length generalization via implicit state merger in RNNs; the experiments are strong, the theory is a qualitative model with one untested load-bearing ansatz. read the letter →

arxiv 2507.09897 v2 pith:IQ7PFCHV submitted 2025-07-14 cs.LG q-bio.NC

Loek van Rossem , Andrew M. Saxe This is my paper
classification cs.LGq-bio.NC
keywords recurrentneuralnetworksstreamingparityinfinitegeneralizationlengthstatemergingfiniteautomatonextractioneffectivetheoryphasetransition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to explain how a deep network can learn a genuine algorithm from finite data rather than merely interpolate its training examples. The setting is the streaming parity task: a recurrent network sees binary strings and must report whether the running count of ones is even or odd, trained only on strings up to length 10. The authors show that with small weight initialization and enough data, the network suddenly generalizes perfectly to all string lengths, and they trace this to an implicit representational merger: gradient descent, aided by continuity of the output map, pulls together the hidden representations of sequences that agree on all future outputs. Once enough such pairs have merged, the automaton extracted from the hidden states becomes finite, at which point behavior is fixed for every sequence length at once. If the mechanism is right, it reveals a concrete route by which neural networks can achieve infinite generalization from finite experience, and why that generalization arrives as a sharp phase transition rather than gradual improvement.

What carries the argument

The central object is the implicit representational merger effect, analyzed through an effective two-point interaction model. Nearby hidden representations $h_1, h_2$ are treated as freely optimizable vectors and the network's output predictions as freely optimizable smooth maps, which reduces the dynamics to a self-contained three-scalar system for the squared representational distance $\|dh\|^2$, the average squared prediction distance, and an output-alignment metric. Solving this system gives the merger condition $C < N G^{n-m}$ with agreement on all future outputs, which predicts which pairs collapse, why mergers start only for long enough sequences and small enough initial weights, and why the resulting automaton becomes finite exactly when generalization becomes perfect. The paper also uses a second, fixed-expansion-point version of the model to explain why agreeing pairs initially diverge before merging, producing the tree-fitting phase that precedes the generalization phase.

What would settle it

Train a one-layer RNN with 100 hidden units on all parity sequences up to length 10 while sweeping the initial weight scale $G$ and training set size, and measure accuracy on sequences of length 100: the merger condition $C < N G^{n-m}$ predicts a sharp boundary with accuracy at chance (0.5) on one side and perfect (1.0) on the other. A falsifying result would be a gradual, smooth improvement in long-sequence accuracy across this sweep, or a pair of sequences that disagree on some future output yet still merge.

Watch

Extended reading notes

Core claim

On the streaming parity task, an RNN with a single recurrent layer trained on all sequences up to length 10 can generalize flawlessly to sequences thousands of times longer. The paper's central claim is that this happens when hidden representations merge: because the output map is continuous, when two sequences in the training set agree on all future target outputs and their representations come close, gradient descent reduces the loss most efficiently by moving the representations together. The authors derive a merger condition from an effective interaction model: states merge exactly when they agree on all future outputs and the initial weight scale $G$ and dataset size $N$ satisfy $C < N G^{n-m}$, where $m$ and $n$ are the minimal sequence lengths involved. Under this condition, enough mergers eventually render the extracted automaton finite; because the training set contains all sequences up to the task automaton's size, the finitized automaton must agree with the task's two-state parity automaton on every possible string, so generalization to all lengths is instant and complete. The sharpness of the resulting phase transition—chance accuracy before the boundary, perfect accuracy after—follows from the fact that no mergers occur before the condition is met, so the extracted automaton either stays an exponentially large tree or collapses to a finite one.

Load-bearing premise

The whole mechanism assumes the network is expressive and smooth enough that hidden representations and output maps can be optimized as if they were independent, freely movable objects; if real gradient descent cannot move them this way because of shared parameters, global interactions, or architectural constraints, the predicted merger condition and the phase transition to infinite generalization need not occur.

Editorial extensions

If this is right

  • Below a data-size or weight-scale threshold, the RNN converges to a complete tree that fits the training data and shows chance-level accuracy on long sequences; above it, long-sequence accuracy jumps abruptly to perfect.
  • The same two-phase pattern—tree fitting, then merger-driven generalization—holds for any task expressible by a finite automaton, as demonstrated on random regular tasks.
  • The learned finite automaton is typically non-minimal: redundant states are expected from the learning dynamics alone, and after state reduction it is computationally equivalent to the task's minimal automaton.
  • The mechanism is architecture-independent in the sense of any smooth, expressive recurrent map, as shown by repeating the experiments with a hyperbolic-tangent activation.
  • Transformers on modular arithmetic do not show the merger pattern in their hidden representations; the pattern appears in the attention matrix instead, and their transition to generalization is smoother.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the merger condition is the operative mechanism, an intervention that explicitly penalizes representational distance between pairs that agree on all future outputs should trigger infinite generalization earlier in training; this is a direct testable extension the paper does not perform.
  • The bimodal long-sequence accuracy (chance or perfect) suggests that validation loss on long sequences is a poor early-warning signal: monitoring the number of extracted automaton states during training would detect the approach of the phase transition before behavior changes.
  • The redundant-states prediction implies that two networks or animals solving the same task may end up with behaviorally identical but structurally different non-minimal automata, so representational similarity measures that ignore automaton redundancy could misjudge whether two systems implement the same algorithm.
  • The same continuity-based merger argument offers a possible explanation for catastrophic length-generalization failures: if a model has merged most but not all agreeing pairs, its automaton is large but not finite, and accuracy will stay perfect on moderately long sequences before breaking abruptly at some length past the training distribution.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies recurrent neural networks (RNNs) trained on the streaming parity task, where inputs are binary sequences of arbitrary length and the output is the parity of the number of ones. The authors report that, with sufficient training data and sufficiently small weight initialization, RNNs transition sharply from memorizing a tree of training sequences to perfect generalization on arbitrarily long sequences. They extract finite automata from the hidden representations, observe an initial phase of automaton expansion into a complete tree followed by a phase of state mergers, and interpret this as the network building a finite automaton for the task. The theoretical contribution is an effective interaction model in which nearby hidden representations merge when they agree on all future outputs; the central result is the merger condition in Eq. (9), which predicts a sharp boundary between tree fitting and full generalization. The paper supports this with experiments on ReLU and tanh RNNs, random regular tasks, and a discontinuous output map, and it discusses implications for redundant states in brains and for transformers.

Significance. If the proposed mechanism is correct, the paper offers a concrete, interpretable account of how gradient descent can build a computational algorithm from finite training data, and the observed phase transition is striking. The empirical work is a genuine strength: the paper includes public code, tests across multiple architectures and tasks, checks that merged pairs always agree on future outputs, and provides qualitative confirmation of several theoretical predictions (e.g., the length threshold for mergers in Fig. 7 and the sharp phase transition in Fig. 9). The theoretical contribution is, however, more provisional: the derivation rests on strong simplifications that are not directly validated, and the central merger condition contains an unknown architecture-dependent constant. The paper is honest about many of these limitations, but the load-bearing assumptions deserve closer scrutiny.

major comments (4)
  1. [Appendix B.1, Eq. (17)] The closed 3-scalar system in Eq. (4) is derived only after imposing the radial-motion Ansatz d(dh)/dt ∝ dh. This Ansatz is introduced without empirical justification, and it is essential: it is used to replace D_yi(d(dh)/dt) with terms proportional to (d||dh||^2/dt)/||dh||^2 D_yi dh, which is what closes the system. Without this assumption, Eq. (4) does not follow, and therefore Eq. (9), the central merger condition, is not established for real RNNs. The subsequent paragraph in Section 3.3 that calls the solution 'exactly solved' overstates the status of this result. I ask the authors to test the Ansatz directly in trained networks (e.g., measure the angle between d(dh)/dt and dh during training), or alternatively to present Eq. (4) as a heuristic model and soften the claim that the effective theory 'finds' the merger effect.
  2. [Section 3.3, Eq. (9)] The merger condition contains the unknown architecture-dependent constant C. The paper does not estimate C or give bounds for it, so Eq. (9) does not yield a quantitative prediction of the phase boundary in Fig. 9; the specific scaling N G^{n-m} is not falsifiable without fixing or measuring C. The experiments demonstrate the qualitative dependence on N and G, but the theory is not quantitatively coupled to the data. The authors should either calibrate C from the experiments or explicitly state that only the qualitative sign of the predicted boundary is being tested.
  3. [Section 3.2, paragraph on parameter sharing] The model treats h1, h2 and the output maps y1,...,yN as freely optimizable variables and explicitly ignores parameter sharing ('we will choose to ignore potential interaction effects arising from parameter sharing'). In a real RNN, the same recurrent weights produce h1, h2, and all other states, and the output maps share parameters with each other and with the recurrent map. This is not a harmless simplification for the central claim: the gradient updates of these objects are correlated in a way the effective model does not capture. The authors should provide evidence that the independent-update approximation is adequate, for example by comparing the predicted merger behavior in a network with tied parameters to the effective-model prediction, or by showing that the qualitative phase transition persists when parameter sharing is taken into account.
  4. [Appendix A.2 and Appendix C.1, merger threshold] The automaton extraction procedure groups hidden states into the same state when their distance is below an epsilon threshold, set in Appendix C.1 to a fixed fraction of the representational standard deviation. The observed phase transition in the number of automaton states (Fig. 9, left) depends on this threshold: a sufficiently large threshold would merge all states, and a sufficiently small threshold would prevent all mergers. The validation-accuracy transition (Fig. 9, right) is more robust, but the paper's interpretation in terms of the automaton becoming finite relies entirely on the threshold choice. Please include a sensitivity analysis over a range of thresholds to show that the two-phase picture and the boundary in Fig. 9 are not artifacts of the extraction cutoff.
minor comments (5)
  1. [Appendix B.1] The phrase 'gradient decent' should be 'gradient descent'.
  2. [Appendix B.1] The word 'Anzats' should be 'Ansatz'.
  3. [Appendix D.9] The text says 'nosier dynamics'; this should be 'noisier dynamics'.
  4. [Section 3.3] The phrase 'exactly solved' is ambiguous: the solution in Eq. (5) is exact for the reduced scalar system, not for the full RNN dynamics. Consider rephrasing to avoid overclaiming.
  5. [General] Figures 4 and 6 are not explicitly referenced in the body text; please add cross-references for clarity.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the merger condition is derived in-appendix from stated assumptions and tested against un-fitted experiments; the only self-citation is methodological.

full rationale

The central result, Eq. (9), is not imported as a black box: Appendix B.1 derives the 3-scalar system from the stated local-linear model with freely optimizable hidden vectors and output maps, and the radial-motion Ansatz in Eq. (17) is an explicit assumption rather than a hidden reuse of the conclusion. The final-distance solution (Eq. 5) and the merger condition (Eq. 9) follow algebraically from that system, and no parameter is fitted to the parity data: C in Eq. (9) is declared an unknown architecture-dependent constant, so the qualitative checks in Figures 7, 9, 10 and D.3 are genuine predictions of scaling/threshold structure, not post-hoc fits. The phrase in Section 3.2, 'using the modeling approach from (van Rossem & Saxe, 2024), adapted for recurrent networks,' is a methodological self-citation, but the current paper re-derives the interaction model and its conclusions in Appendix B, so the self-citation is not load-bearing. The 'finite automaton then instant generalization' step in Section 4.2 is a logical consequence of DFA equivalence once the training set contains all strings up to the task automaton size, not a restatement of the measured phase transition. The limitations section candidly lists ignored higher-order interactions, global interactions, architectural biases, regularization, and noise; these are scope assumptions that could affect correctness in real RNNs, but they do not make the derivation equivalent to its inputs. Overall, no equation reduces by construction to its own input, and the central phase-transition claim retains independent empirical content.

Assumptions & free parameters 3 free parameters · 7 assumptions · 0 invented entities

The theory contributes no fitted constants to the data, which is a strength, but it introduces an unknown architecture constant C and effective learning rates that are not derived from the network. The central merger mechanism is therefore a qualitative explanation rather than a parameter-free prediction.

free parameters (3)
  • Architecture constant C = unknown, not fitted
    Appears in the merger condition Equation (9), C < N G^(n-m); depends on architecture but is left undetermined, so the theory makes qualitative predictions only.
  • Effective learning rates 1/tau_h and 1/tau_y = not fitted
    Parameters of the interaction model in Equation (4) and Appendix B.1, treated as effective rates of abstract hidden and output maps; no procedure maps them to actual network hyperparameters.
  • Automaton merger threshold epsilon = 0.01 times representational standard deviation
    Used in Appendix A.2 and C.1 to define state identity; the number of automaton states and hence the observed phase boundary depend on this choice.
assumptions (7)
  • ad hoc to paper High expressivity: the network can be treated as arbitrarily optimizable vectors h1, h2 and smooth maps y1,...,yN, with parameter sharing ignored.
    Invoked in Section 3.2 and Figure 5 to replace actual parameter gradients with free smooth-map gradients; the authors require the network to be 'large and complex enough to have the freedom to behave like a smooth map.'
  • domain assumption Local linear approximation of output maps around the representational mean is valid.
    Used in Section 3.2; Appendix D.2 gives an empirical check that first-order terms dominate on average, but the approximation is not proven for all interacting pairs.
  • ad hoc to paper Representational movement ansatz: d/dt dh is proportional to dh.
    Appendix B.1 reduces the dynamics to a scalar system 'considering solutions where representations either move closer or further away from each other'; this restricts the allowed gradient flow.
  • domain assumption Initial representational distances and output sensitivities scale as G^m and G^n for small initialization G<1.
    Used in Section 3.3, Equation (8), to derive the merger condition C < N G^(n-m).
  • ad hoc to paper Global interactions and higher-order terms can be ignored; the system behaves as locally interacting particles.
    Section 4.1 treats the learning dynamics as locally interacting particles, and Section 7 lists the omission of global interactions as a limitation.
  • standard math Universal approximation guarantees enough freedom for independent optimization of the smooth output maps.
    Section 3.2 cites Hornik et al. and Csaji, but the step from approximation capacity to independent gradient-descent freedom is an additional assumption.
  • ad hoc to paper The fixed expansion point interaction model remains qualitatively accurate even when representations drift far from the expansion point.
    Section 5.2 acknowledges the model 'may lose accuracy' as the pair drifts; this model is used to explain the initial divergence before merging.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Algorithm Development in Neural Networks: Insights from the Streaming Parity Task." pith.science (2026). https://pith.science/paper/IQ7PFCHV

@misc{pith2026250709897,
  author       = {Pith},
  title        = {Pith review of: Algorithm Development in Neural Networks: Insights from the Streaming Parity Task},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IQ7PFCHV}},
  note         = {Machine review of arXiv:2507.09897}
}
read the original abstract

Even when massively overparameterized, deep neural networks show a remarkable ability to generalize. Research on this phenomenon has focused on generalization within distribution, via smooth interpolation. Yet in some settings neural networks also learn to extrapolate to data far beyond the bounds of the original training set, sometimes even allowing for infinite generalization, implying that an algorithm capable of solving the task has been learned. Here we undertake a case study of the learning dynamics of recurrent neural networks (RNNs) trained on the streaming parity task in order to develop an effective theory of algorithm development. The streaming parity task is a simple but nonlinear task defined on sequences up to arbitrary length. We show that, with sufficient finite training experience, RNNs exhibit a phase transition to perfect infinite generalization. Using an effective theory for the representational dynamics, we find an implicit representational merger effect which can be interpreted as the construction of a finite automaton that reproduces the task. Overall, our results disclose one mechanism by which neural networks can generalize infinitely from finite training experience.

Figures

Figures reproduced from arXiv: 2507.09897 by the authors.

Figure 1
Figure 1. Sequence length generalization of a recurrent network on a simple task. Left: Example sequences of the streaming parity task, from the training dataset (short sequences) and the validation dataset (long sequences). Right: The mean squared loss of a recurrent neural network during training on the streaming parity task. Note that the loss also goes to zero for sequences much longer than found in the training data. Thi… view at source ↗
Figure 2
Figure 2. Example of a two state DFA solving the streaming parity task. Every time it receives a 1 as an input it alternates between the state that will output 0 and the state that will output 1. When it receives a 0 it remains in its current state. 2.2. Automaton Extraction We will use a relatively simple method for constructing an automaton from the representational space of an RNN, as this will be enough for our purposes. … view at source ↗
Figure 3
Figure 3. Example of the automaton extraction procedure. Before the network has seen any input symbols the internal vector of hidden activations has some initial value h0, which we will call the initial state of the automaton. After seeing a symbol, say e.g. 1, the hidden activations change to a different vector fh(h0, 1). We will call this vector the state corresponding to input sequence 1, and say that the initial state has… view at source ↗
Figures from the paper (17 more)
Figure 4
Figure 4. Figure 4: Automata extracted from the hidden layer of an RNN during training on the streaming parity task. First the model fits the training data with a complete tree. After that states merge until the automaton becomes finite, which is when generalization occurs. The RNN consis…
Figure 5
Figure 5. Figure 5: Schematic illustration of the interaction model be￾tween two nearby representations corresponding to sequences x (1) 1 · · · x (m1) 1 and x (1) 2 · · · x (m2) 2 . The effect of the model as￾signing their representations is abstracted away with arbitrarily optimizable v…
Figure 6
Figure 6. Figure 6: Validation loss for sequences of varying lengths during training on the streaming parity task. The validation loss initially does not change while the training loss goes down, but at some point it suddenly drops for all sequence lengths at the same time. Sequence lengt…
Figure 7
Figure 7. Figure 7: Fraction of pairs agreeing on all future outputs that ended up merging after training on the streaming parity task. We can see that mergers only start to occur once m = min(m1, m2) reaches a certain threshold. Because not all pairs agreeing on all future outputs merge,…
Figure 8
Figure 8. Figure 8: Learned automaton in the RNN from [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: Left: Number of states at convergence as function of the training dataset size and initial weight scale used. Right: Validation accuracy of 30 sequences of length 100. All trials converged on the training set. Note the sudden jump from chance (0.5) to complete generali…
Figure 10
Figure 10. Figure 10: Left: Numerical solutions of the fixed expansion point interaction model. The representational distance of a pair agreeing on all possible future outputs first diverges before finally merging, whereas a pair which disagrees on some outputs only diverges. Right: Normal…
Figure 11
Figure 11. Figure 11: The training loss, validation loss, number of states in the attention matrix, and number of states in the hidden layer of a single layer transformer trained on modular subtraction. Note that the pattern of state mergers after initially state splitting is only present …
Figure 12
Figure 12. Figure 12: The mean pairwise representational distance of training set during training on the streaming parity task. It never decreases during training, hence state mergers cannot be entirely explained by representational contraction. the first-order term ( [PITH_FULL_IMAGE:fig…
Figure 13
Figure 13. Figure 13: The norms of the second-order term divided by the first-order term in the Taylor expansion of the output map during training, averaged over all pairs that ended up below the merging threshold and all pairs that ended up above the merging threshold out of 100 randomly …
Figure 14
Figure 14. Figure 14: Fraction of mergers that first move apart by least a factor 10 of the merging threshold, at some point during training, before merging in the end. Displayed as a function of the sequence lengths of the points in each pair. We can see that pairs with a larger differenc…
Figure 15
Figure 15. Figure 15: The number of pairs of datapoints whose representational distance are apart more than the merging threshold during training on the streaming parity task, divided by whether or not they agree on all possible future outputs. We see that the drop in number of automaton s…
Figure 16
Figure 16. Figure 16: Experimental results of a recurrent neural network trained on a randomly generated regular task. Note that the task automaton and learned automaton are equal, but states are displayed in different locations. 25 [PITH_FULL_IMAGE:figures/full_fig_p025_16.png]
Figure 17
Figure 17. Figure 17: Experimental results of a recurrent neural network with hyperbolic tangent non-linearity trained on the streaming parity task. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_17.png]
Figure 18
Figure 18. Figure 18: The training loss, validation loss, number of states in the attention matrix, and number of states in the hidden layer of a single layer transformer trained on modular addition. Note that as with subtraction we see a pattern of initial attention pattern divergence fol…
Figure 19
Figure 19. Figure 19: The training loss, validation loss, number of states in the key matrix, and number of states in the value matrix of a single layer transformer trained on modular subtraction. Note that some amount of state merging appears to happen in the value matrix, but not for the…
Figure 20
Figure 20. Figure 20: Experimental results of a recurrent neural network with a discontinuity in the output map trained on the streaming parity task. 28 [PITH_FULL_IMAGE:figures/full_fig_p028_20.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

20 extracted references · 16 canonical work pages

  1. [1]

    A finite set of states Q

    work page

  2. [2]

    A finite set of possible input symbols Σ, called the alphabet

    work page

  3. [3]

    A transition function δ : Q × Σ → Q

    work page

  4. [5]

    ISSN: 2640-3498

    URL https://proceedings.mlr.press/ v125/chizat20a.html. ISSN: 2640-3498. Chizat, L., Oyallon, E., and Bach, F. On Lazy Training in Differentiable Programming, January

    work page

  5. [6]

    URL http://arxiv.org/abs/1812. 07956. arXiv:1812.07956 [cs, math]. Cohen-Karlik, E., Menuhin-Gruman, I., Giryes, R., Co- hen, N., and Globerson, A. Learning Low Dimensional State Spaces with Overparameterized Recurrent Neural Nets, March 2023. URL http://arxiv.org/abs/ 2210.14064. arXiv:2210.14064 [cs]. Cs´aji, B. Approximation with Artificial Neural Netw...

    arXiv 2023

  6. [10]

    A recurrent neural network without chaos

    URL https://proceedings.mlr.press/ v235/humayun24a.html. ISSN: 2640-3498. Lake, B. and Baroni, M. Generalization with- out Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks. In Pro- ceedings of the 35th International Conference on Ma- chine Learning , pp. 2873–2882. PMLR, July 2018. URL https://proceedings.mlr.press/v80...

    work page Pith review arXiv 2018

  7. [11]

    URL http://arxiv.org/abs/2309. 11028. arXiv:2309.11028 [cs, q-bio, stat]. Lin, B., Mur, M., Kietzmann, T., and Kriegeskorte, N. Vi- sualizing Representational Dynamics with Multidimen- sional Scaling Alignment, July 2019. URL http:// arxiv.org/abs/1906.09264. arXiv:1906.09264 [cs, q-bio, stat]. Liu, Z., Kitouni, O., Nolte, N., Michaud, E. J., Tegmark, M.,...

    work page Pith review arXiv 2019

  8. [12]

    ISBN 978-0- 19-514822-0

    Oxford University Press, January 2006. ISBN 978-0- 19-514822-0. doi: 10.1093/acprof:oso/9780195148220. 003.0010. URL https://doi.org/10.1093/ acprof:oso/9780195148220.003.0010. Omlin, C. W. and Giles, C. L. Constructing deter- ministic finite-state automata in recurrent neural net- works. Journal of the ACM , 43(6):937–972, Novem- ber 1996. ISSN 0004-5411...

    arXiv 2006

Show all 20 references
  1. [14]

    ISSN: 2640-3498

    URL https://proceedings.mlr.press/ v235/van-rossem24a.html. ISSN: 2640-3498. Wang, K., Variengien, A., Conmy, A., Shlegeris, B., and Steinhardt, J. Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small, Novem- ber 2022. URL http://arxiv.org/...

    2022 arXiv

  2. [18]

    x(n) the automaton is said to accept the string x when there exists a sequence of states r(0),

    A subset of accepting states F Given some string of input symbols x = x(1)x(2) . . . x(n) the automaton is said to accept the string x when there exists a sequence of states r(0), . . . , r(n) ∈ Q such that

  3. [19]

    ∀i r(i+1) = δ(r(i), x(i+1))

  4. [20]

    r(n) ∈ F In the context of the streaming parity task we can take the subset of accepting states to be precisely those for which the model predicts an output 1. A.2. Extraction Algorithm In order to extract one from a recurrent neural network, we define the state corresponding ...

    1971

  5. [1956]

    Google-Books-ID: oL57iECEeEwC

    ISBN 978-0-691-07916-5. Google-Books-ID: oL57iECEeEwC. Atanasov, A., Bordelon, B., Sainathan, S., and Pehlevan, C. The Onset of Variance-Limited Behavior for Networks in the Lazy and Rich Regimes, December 2022. URL https://arxiv.org/abs/2212.12147v1. Brennan, C., Aggarwal, A....

    2022 arXiv

  6. [1989]

    doi: 10.1016/0893-6080(89) 90020-8

    ISSN 08936080. doi: 10.1016/0893-6080(89) 90020-8. URL https://linkinghub.elsevier. com/retrieve/pii/0893608089900208. Humayun, A. I., Balestriero, R., and Baraniuk, R. Deep Networks Always Grok and Here is Why. In Proceedings of the 41st International Conference on Machine Le...

  7. [1992]

    doi: 10.1162/neco.1992.4

    ISSN 0899-7667. doi: 10.1162/neco.1992.4. 3.393. URL https://ieeexplore.ieee.org/ document/6796344. Conference Name: Neural Computation. Goh, G., †, N. C., †, C. V ., Carter, S., Petrov, M., Schubert, L., Radford, A., and Olah, C. Multimodal Neurons in Artificial Neural Networ...

    1992

  8. [2020]

    ISSN: 2640-3498

    URL https://proceedings.mlr.press/ v107/can20a.html. ISSN: 2640-3498. Chizat, L. and Bach, F. Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss. In Proceedings of Thirty Third Confer- ence on Learning Theory, pp. 1305–1338. PMLR, July

  9. [2021]

    Driscoll, L

    URL https://openreview.net/forum? id=pVU7Gp7Nq4k. Driscoll, L. N., Shenoy, K., and Sussillo, D. Flexible mul- titask computation in recurrent networks utilizes shared dynamical motifs. Nature Neuroscience, 27(7):1349– 1363, July 2024. ISSN 1546-1726. doi: 10.1038/ s41593-024-0...

    2024

  10. [2023]

    ISSN: 2640-3498

    URL https://proceedings.mlr.press/ v202/abbe23a.html. ISSN: 2640-3498. Adriaensen, R. and Maene, J. Extracting Fi- nite State Machines from Transformers, October

  11. [2024]

    URL http://arxiv.org/abs/2410. 06045. arXiv:2410.06045 [cs]. Anil, C., Wu, Y ., Andreassen, A., Lewkowycz, A., Misra, V ., Ramasesh, V ., Slone, A., Gur-Ari, G., Dyer, E., and Neyshabur, B. Exploring Length Generalization in Large Language Models, November 2022. URL http:// ar...

  12. [6474]

    Tino, P., Horne, W., Giles, C., and Collingwood, P

    doi: 10.1523/JNEUROSCI.09-07-02382.1989. Tino, P., Horne, W., Giles, C., and Collingwood, P. Finite State Machines and Recurrent Neural Networks – Au- tomata and Dynamical Systems Approaches. March 1999. ISSN 9780125264204. doi: 10.1016/B978-012526420-4/ 50007-0. Turner, E., D...

    1989

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.