Pith. sign in

REVIEW 4 major objections 5 minor 36 references

Temporal Chunking Enhances Recognition of Implicit Sequential Patterns

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that adding a context tag that marks community-entry boundaries lets an RNN achieve optimal prediction with a BPTT window of 1 instead of 7 on a synthetic sequential task.

desk verdict The BPTT-1 gain is likely a confound: the context tag is just the last G-following token, and the missing baseline of feeding that trivial token directly undermines the paper's central claim. read the letter →

arxiv 2506.00588 v2 pith:LI2U3KXU submitted 2025-05-31 cs.LG cs.AI

Jayanta Dey , Nicholas Soures , Miranda Gonzales , Itamar Lerner , Christopher Kanan , Dhireesha Kudithipudi This is my paper
classification cs.LGcs.AI
keywords temporalchunkingcontexttagsrecurrentneuralnetworkstruncatedbackpropagationthroughtimeimplicitsequencelearningsleep-basedmemoryconsolidationtransferserialreactiontask
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that an RNN can solve a sequential-prediction task requiring seven-token memory using a backpropagation window as short as one step, provided the input is augmented with a context tag marking entry into a recurring "community" of tokens. The tags are discovered offline from the hidden-state geometry of a briefly trained network, a process the authors liken to sleep-based consolidation. If the claim holds, temporal chunking offers a lightweight way to compress long-range dependencies into short-window training for recurrent models. A small human pilot study with a serial reaction time task shows that people also extract community boundaries after limited exposure, though they do not learn the long-range direction rule within the same session.

What carries the argument

The central object is the context tag: a binary signal appended to each input token that marks the beginning of a community traversal, i.e., the first token after the hub G. The tag is derived from cosine-distance peaks between hidden states of a briefly trained Layer-1 RNN, which cluster by community; peaks occur at community-entry positions. A separate context RNN is trained on the resulting mask so the tags remain stable when Layer 1 is fine-tuned, and both layers are then trained jointly with the paired (token, tag) inputs during the post-sleep phase. This tag carries the identity of the current community, which is exactly the information a seven-token window normally provides, thereby compressing the dependency into one step.

What would settle it

Train the chunked RNN with context tags assigned at randomly chosen time steps rather than true community boundaries, keeping all other settings identical. If the BPTT-1 model still reaches optimal accuracy, the tags' semantic content is not what drives the result; if performance drops to the naive RNN level, the boundary tags are the cause. A complementary check is to recompute the cosine-distance peaks after Phase 3 fine-tuning: if peaks no longer align with actual community entries while performance stays high, the model may be leveraging the tag pathway in a different way.

Watch

Extended reading notes

Core claim

The central claim is that a two-layer RNN trained in three stages—brief wake training, offline tag discovery, and joint fine-tuning with tagged inputs—reaches optimal prediction accuracy on the synthetic community task with a truncated BPTT window of 1, whereas a naive RNN requires a window of at least 7. The tags are produced by detecting peaks in cosine distance between consecutive hidden states of the first layer, which signal the first token after the hub token G; a separate small RNN is trained to emit these tags so that later fine-tuning of the first layer does not corrupt them. With meaningful tags, the chunked RNN achieves the optimal accuracy of about 79.17 percent (the ceiling set by random community entry); with constant tags, it does not. The same mechanism gives faster forward transfer to a related task with altered transition probabilities.

Load-bearing premise

The load-bearing premise is that cosine-distance peaks in Layer 1's hidden states reliably mark community-entry boundaries both before and after Layer 1 is fine-tuned, so the separately trained context RNN keeps producing correct tags; if the hidden-state geometry shifts, the extra tag input becomes noise and the chunked model loses its advantage.

Editorial extensions

If this is right

  • If the claim is correct, recurrent models can be trained with truncated BPTT windows far shorter than the true dependency length, cutting memory and computation in streaming settings.
  • The three-stage wake–sleep–wake training protocol offers a template for offline consolidation: replay buffered experience to discover stable structure tags before joint fine-tuning.
  • Chunked RNNs transfer faster to a related task, implying that learned context tags function as reusable structural knowledge rather than task-specific memorization.
  • The human pilot suggests that community-boundary recognition and direction-rule learning are dissociable, supporting the paper's hypothesis that a sleep period may be needed to consolidate the long-range rule.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The tag-discovery step could likely be replaced by unsupervised clustering of hidden states, removing the need for a separately trained context RNN; this is a natural next experiment the paper does not run.
  • The same boundary-tag idea might be applied to attention-based models with fixed context windows, where tags could act as compressed memory tokens for far-away structure.
  • A scaling test is implied but not performed: if the community length or required history is increased beyond seven tokens, it would show whether the BPTT-1 advantage persists or degrades, delimiting the mechanism's reach.
  • The paper's future sleep study would provide a direct behavioral falsifier of the scaffolding story: if post-sleep subjects still fail at position 2, the claim that sleep consolidates the long-range rule would be undermined.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a three-stage, neuro-inspired training procedure for RNNs: pre-sleep wake training, an offline sleep phase in which context tags are derived from cosine-distance peaks in the hidden states, and post-sleep wake training in which the input stream is augmented with the context tag. The method is evaluated on a synthetic two-community Markov environment whose transition rule requires memory of the last seven tokens. The central claim is that a chunked two-layer RNN reaches the 79.17% optimal accuracy with a BPTT window as small as 1, whereas a naive RNN requires a window of at least 7, and that a constant-tag ablation fails at short windows. A small human SRTT pilot and a transfer-learning experiment are presented as additional motivation.

Significance. If the BPTT-1 result is robust and is genuinely caused by the sleep-phase chunking mechanism, it would be a useful proof-of-concept that a compact context tag can substitute for long truncated-BPTT windows under resource constraints. The synthetic environment is clean and precisely specified, the optimal-performance ceiling is well defined, and the paper includes an ablation and a transfer experiment. The work is honestly framed as a pilot, and the human SRTT study is carefully described. However, the current evidence does not yet isolate the proposed chunking mechanism from simpler properties of the context tag, and the feasibility of solving a 7-token dependency with BPTT-1 is not explained. The central claims are therefore conditional on missing controls and clarifications.

major comments (4)
  1. [Section 5.1 / Algorithm 1 Phase 3 / Figure 6] The informative context tag is, in this environment, a delayed copy of the last token immediately following G: every community boundary is exactly a token whose predecessor is G, and Algorithm 1 sets c_t to the most recent such token. The constant-tag ablation in Figure 6 (top) controls for the presence of an extra input channel, but it does not control for the information content of that channel. A direct control is needed: train a naive RNN with an auxiliary input equal to this same delayed G-following token, without the Phase-2 sleep procedure. If that control reaches the ceiling at BPTT-1, then the offline sleep-phase chunking is not the cause of the reported gain; if it fails, the paper should state what the Phase-2-derived tag adds beyond a token that is trivially computable from the raw stream.
  2. [Section 5.1 / Section 3 / Algorithm 1] The BPTT-1 result is not accounted for by the information requirements of the task. To predict the direction inside a community, the model needs the current, last, and penultimate community visits. With w=1, the input at a typical time inside a community is (x_t, c_t), where c_t is the current community's entry token; it does not identify the two earlier visits required by the rule in Section 3. The hidden state could in principle carry that history, but truncated BPTT of length 1 does not propagate gradients through the recurrence, so it is unclear how the network learns to store the earlier visits. The paper should report the BPTT window used in Phase 1 of the Figure 6 experiments and, if Phase 1 uses a longer window, state this explicitly; otherwise the BPTT-1 optimal curves may be an artifact of longer-window pretraining rather than of the chunking mechanism.
  3. [Section 5.2 / Figure 7] The transfer-learning comparison has the same confound as the main result: the chunked model receives context tags as an extra input channel, while the naive model does not, and the tags are the same delayed G-following tokens used in Section 5.1. The faster target-task learning could therefore reflect the extra input representation rather than the sleep-phase chunking process. A control experiment using the direct delayed-token auxiliary input on the target task, without Phase 2, is required to support the transfer claim.
  4. [Section 4.2] The sentence 'a traditional RNN does not remember tokens outside its input window' is inaccurate as stated. An RNN's hidden state carries information forward in time even with truncated BPTT; the restriction imposed by a short BPTT window is on gradient-based credit assignment through time, not on forward memory. The empirical result that this specific vanilla RNN needs a 7-step BPTT window to reach ceiling should be reported as an empirical finding for the chosen architecture and training setup, not as a general principle about RNNs.
minor comments (5)
  1. [Abstract] The sentence 'We evaluate this idea in a controlled synthetic environment designed to reveal the limitations...' appears twice in the abstract; the duplication should be removed.
  2. [Figure 3 caption] The caption says 'a 7-step BPTT input window,' but BPTT window and input window are different quantities; the wording should be clarified to distinguish the recurrent training truncation from the number of input tokens observed by the model.
  3. [Algorithm 1 / Section 5] The pseudocode labels Phase 1 as 'Initial Training of Base RNN' but the text says Layer 2 is frozen while Layer 1 is trained; the pseudocode does not reflect the two-layer structure or which weights are updated in each phase. The phases should be described consistently in the text and in the algorithm.
  4. [Algorithm 1, lines 21-31] The timing of the context-tag update is confusing: line 24 uses c_t before line 30 updates it when a boundary is detected. The text should state explicitly that, at a boundary step, the tag used for prediction is the previous boundary token and the tag used for subsequent steps is the current entry token.
  5. [Section 4.1] The human pilot is very small (n=11, with n=5 for the high-performing subgroup) and the statistical claims are based on aggregated generation-task accuracies; the paper should include effect sizes or confidence intervals for the key comparisons and should avoid overinterpreting the null result at position 2 given the small sample.

Circularity Check

1 steps flagged · score 3.0 of 10

The BPTT-1 advantage largely reduces to feeding a delayed copy of the last community-entry token as an extra input channel, not to the proposed sleep-phase chunking mechanism.

  1. other [Section 5 (context-tag construction), Algorithm 1 Phase 3 (lines 21-31), result in Section 5.1 / Figure 6 bottom]
    "We use the detected Position 1 token as a context tag in our next training step. For example, if the sequence is ‘ABC G DEF G CAB G’, the mask will be ‘100 0 100 0 100 0’ and the context tags are ‘AAAA DDDD CCCC’. ... Algorithm 1: c0 ← x0; for t ... ct ← ct−1; ... if ˆm(t) = 1 then ct ← xt."

    In the synthetic environment, mask=1 occurs exactly at the first token after G (community entry). Algorithm 1 therefore defines ct as the last raw token that followed G, held until the next G. This is a delayed identity copy of a raw stream token, not a compressed latent representation. Feeding (xt, ct) supplies the chunked RNN with the current community identity as an explicit second channel, while the naïve RNN must infer that identity from the last 7 tokens. The constant-tag ablation controls only for the presence of a meaningless tag, not for the informative delayed-copy channel.

full rationale

The paper's strongest quantitative claim (BPTT window 1 suffices with chunking, while naïve RNN needs 7) is not a formal derivation from the model, so the circularity is limited. The central confound is that the 'context tag' is implemented as the raw token immediately after G, so the chunked model receives an explicit delayed copy of the community-entry token; the improvement may be due to this extra input channel rather than to the three-stage sleep-phase chunking. The wrong/constant-tag ablation rules out a meaningless tag but does not rule out a trivial delayed raw-token tag. The human pilot and transfer experiments are not circular. No self-citation is load-bearing: Lerner's temporal-scaffolding references motivate but do not justify the machine-learning result. Because the BPTT-1 outcome is not literally equal to the input by construction (the tag alone does not determine the direction-dependent position-2 prediction), the score is moderate rather than high.

Assumptions & free parameters 4 free parameters · 5 assumptions · 1 invented entities

The central claim rests on the synthetic task construction, the empirical hidden-state clustering observation, the standard truncated BPTT setup, the neuro-inspired sleep hypothesis, and the ideal assumption of an offline replay buffer with a separate tag predictor. No new physical entities are introduced; the context tag is an algorithmic construct with evidence only inside this paper.

free parameters (4)
  • Sliding window for online prediction error = win = 1000
    Chosen by the authors because it gave a less noisy estimate (Section 2.2); the reported accuracy curves depend on this window.
  • Context-tag peak detection rule = No numeric threshold; boundary if cosine(h_t, h_{t-1}) > cosine(h_t, h_{t+1})
    Hand-chosen rule in Algorithm 1 Phase 2 for detecting community boundaries; no sensitivity analysis is provided.
  • Number of hidden neurons per layer = Swept 5 to 35 in Figures 3 and 6
    Experimental sweep; the BPTT-1 success is shown for some sizes, but no criterion is given for selecting the reported operating point.
  • Pre-sleep and post-sleep training durations T1 and T2 = Not reported
    Algorithm 1 requires a training length split and a replay buffer size, but the paper does not state them; these choices affect the comparison.
assumptions (5)
  • domain assumption An RNN's hidden states after brief training cluster by community, so cosine-distance peaks mark community-entry boundaries.
    Empirical observation in Figure 4 is generalized to the tag-generation step; no theoretical guarantee is given (Section 5).
  • domain assumption The synthetic environment's next-token rule depends on exactly the last seven tokens, and communities are the only relevant structure.
    The task is constructed this way (Section 3), so results may not transfer to real-world sequence data.
  • standard math Truncated BPTT with hidden-state carryover is a valid training regime, and the hidden state can carry information forward even when gradients are truncated.
    Standard RNN/BPTT assumption used in all experiments (Appendix A).
  • domain assumption The temporal scaffolding hypothesis, that sleep replay compresses experience into chunks, motivates the method.
    Borrowed from the neuroscience literature [7,8]; not tested by the machine experiments, which use an offline replay buffer as a proxy for sleep.
  • ad hoc to paper A separately trained context RNN can predict community boundaries without being overwritten by later fine-tuning of Layer 1.
    The paper explicitly notes the assumption of ideal conditions, including the use of an additional RNN to preserve context tags (Section 7).
invented entities (1)
  • Context tag (c_t)
    purpose: Auxiliary input fed to Layer 2 that encodes the detected community-entry token, compressing each community visit into a stable label during post-sleep training.
    Context tags have no falsifiable handle outside this paper's synthetic environment; their utility is shown only by the within-paper wrong-tag ablation and transfer curves, and they are derived from the model's own hidden states.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Temporal Chunking Enhances Recognition of Implicit Sequential Patterns." pith.science (2026). https://pith.science/paper/LI2U3KXU

@misc{pith2026250600588,
  author       = {Pith},
  title        = {Pith review of: Temporal Chunking Enhances Recognition of Implicit Sequential Patterns},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LI2U3KXU}},
  note         = {Machine review of arXiv:2506.00588}
}
read the original abstract

In this pilot study, we propose a neuro-inspired approach that compresses temporal sequences into context-tagged chunks, where each tag represents a recurring structural unit or``community'' in the sequence. These tags are generated during an offline sleep phase and serve as compact references to past experience, allowing the learner to incorporate information beyond its immediate input range. We evaluate this idea in a controlled synthetic environment designed to reveal the limitations of traditional neural network based sequence learners, such as recurrent neural networks (RNNs), when facing temporal patterns on multiple timescales. We evaluate this idea in a controlled synthetic environment designed to reveal the limitations of traditional neural network based sequence learners, such as recurrent neural networks (RNNs), when facing temporal patterns on multiple timescales. Our results, while preliminary, suggest that temporal chunking can significantly enhance learning efficiency under resource constrained settings. A small-scale human pilot study using a Serial Reaction Time Task further motivates the idea of structural abstraction. Although limited to synthetic tasks, this work serves as an early proof-of-concept, with initial evidence that learned context tags can transfer across related task, offering potential for future applications in transfer learning.

Figures

Figures reproduced from arXiv: 2506.00588 by the authors.

Figure 1
Figure 1. Token sequence generation in the synthetic environment. Top Left: Transition graph showing two communities, {A, B, C} and {D, E, F}, connected via a central hub token G. Solid arrows represent deterministic intra-community transitions and returns to G; dashed arrows indicate uniform random transitions (p = 1 6 ) from G to any community entry token. Top Right: Rule table for determining traversal direction (clockwise… view at source ↗
Figure 2
Figure 2. Humans learn the community structure and successfully utilize the learned structure in a novel environment. Top: Graphs showing connectivity for the Session 1 environment (state transition probability p = 1 4 ) and the Session 2 novel environment (p = 1 2 ). Bottom Column 1: Individual subject performance for sessions 1 and 2 shown as a function of average accuracies for predictions made at positions 0, 2, and 3. Hi… view at source ↗
Figure 3
Figure 3. Performance of RNN on our simulation setup. A näive RNN is trained on the simulation setup described in Section 3 with varying neurons, layers and BPTT window. Studies show that an RNN needs at least 15 neurons, a single layer, and a 7-step BPTT input window to achieve the optimal performance. The error bars (interquartile range) are shown as shaded region on both sides of each solid curve (median prediction error).… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: A naïve RNN learns the community structure. Left: Scree plot showing the optimal embedding dimension (knee) at 2 (red cross). Right: Visualization of the low-dimensional manifold of RNN hidden states, obtained via Multidimensional Scaling (MDS) using cosine distance. C…
Figure 5
Figure 5. Figure 5: Proposed three-step chunked training. (a) In this model a learner first collects experience from the environment in a pre-sleep mode, derives the context tags during sleep and finally, and uses the context tags to chunk the learning in the post-sleep mode. (b) Context …
Figure 6
Figure 6. Figure 6: Chunking enables a two-layer RNN to achieve optimal performance with fewer resources. Top: Ablation study where the context tag is fixed to a constant token. Without meaningful context tags, the proposed RNN architecture fails to reach optimal performance when using fe…
Figure 7
Figure 7. Figure 7: RNN with chunking results in better forward transfer than a naïve RNN. Performance on target task for chunked RNNs (red curve) improves over that of naïve RNNS (yellow curve) as the target task is introduced. In this section, we evaluate chunked RNNs in a transfer lear…
Figure 8
Figure 8. Figure 8: Backpropagation through time. Left: An RNN with input xt and output ot. Right: Unfolded RNN over time steps. A.1 Backpropagation Through Time In an RNN, the recurrent weights are shared across time steps. To optimize these weights, Backpropagation Through Time (BPTT), …
Figure 9
Figure 9. Figure 9: Example sequence explaining our simulation setup. The penultimate and the last visit determine the direction of traversal within the current community. + * + S D F J K L Ring Finger Ring Finger Middle Finger Middle Finger Index Finger Index Finger Right Hand Left Hand …
Figure 10
Figure 10. Figure 10 [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: SRTT Results show Implicit Learning of Community Structure during Training (Session 1). Column 1: Average reaction times per block of SRTT displayed in chronological order for all subjects (n = 11). Column 2: Average response accuracies per block of SRTT for all subje…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

36 extracted references · 21 canonical work pages

  1. [1]

    Motivated by the neuro-inspired temporal scaffolding hypothesis, we propose a temporal chunking-based framework that allows RNNs to detect temporal regularities extending well beyond their input window

    work page

  2. [2]

    We demonstrate that a three-stage learning model (reflecting pre-sleep wake, sleep, and post-sleep wake modes) can efficiently capture complex temporal patterns using far fewer BPTT steps while improving temporal learning performance

    work page

  3. [3]

    chunking

    In a small-scale pilot human study using SRTT, we observe that participants show recognition of community structures after a brief training session. Although limited in scope, the results suggest that some degree of structural abstraction or “chunking” may emerge during wakeful learning alone, providing preliminary motivation for our proposed mechanism. A...

    work page

  4. [4]

    Chunking and data compression in verbal short-term memory

    Dennis Norris and Kristjan Kalm. Chunking and data compression in verbal short-term memory. Cognition, 208, 3 2021. ISSN 18737838. doi: 10.1016/j.cognition.2020.104534. 2

    arXiv 2021

  5. [5]

    When transitioning to the target task, Layer 2 receives context tags learned from the source task

    We use the same number of layers and nodes for the näive and the chunked RNN. When transitioning to the target task, Layer 2 receives context tags learned from the source task. Both layers are trained jointly, as shown in the post-sleep wake mode in Figure 5. Note that layer 2 is frozen only once for the source task and never for the target task. After a ...

    work page

  6. [6]

    Once inside a community, the direction of traversal—clockwise or counterclockwise—is determined by the identity of the two most recent community visits relative to the current one

    The selected token determines both which community to enter (Community 1 for A, B, C; Community 2 for D, E, F) and where the traversal will begin within that community. Once inside a community, the direction of traversal—clockwise or counterclockwise—is determined by the identity of the two most recent community visits relative to the current one. If both...

    work page 2024

  7. [7]

    Long short-term memory

    Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780,

  8. [8]

    Learning long-term dependencies with gradient descent is difficult

    Y oshua Bengio, Patrice Simard, and Paolo Frasconi. Learning long-term dependencies with gradient descent is difficult. IEEE transactions on neural networks , 5(2):157–166, 1994. 2

    work page 1994

Show all 36 references
  1. [9]

    The magical number seven, plus or minus two: Some limits on our capacity for processing information

    George A Miller. The magical number seven, plus or minus two: Some limits on our capacity for processing information. PSYCHOLOGICAL REVIEW, 63:81–97, 3 1956. doi: https://doi.org/10.1037/h0043158. 2

    1956 doi

  2. [10]

    Cortico-hippocampal interaction during up-down states and memory consolidation

    Mayank R Mehta. Cortico-hippocampal interaction during up-down states and memory consolidation. NA- TURE NEUROSCIENCE, 10:13–15, 2007. doi: https://doi.org/10.1038/nn0107-13. 2

    2007 doi

  3. [11]

    About sleep’s role in memory

    Bjorn Rasch and Jan Born. About sleep’s role in memory. Physiol Rev, 93:681–766, 2013. doi: 10.1152/ physrev.00032.2012. 2

    2013

  4. [12]

    Sleep is For the Brain , pages 245–256

    Itamar Lerner. Sleep is For the Brain , pages 245–256. Wiley, 11 2017. doi: 10.1002/9781119159193.ch18. 2

    2017 doi

  5. [13]

    Sleep and the extraction of hidden regularities: a systematic review and the importance of temporal rules

    Itamar Lerner and Mark A Gluck. Sleep and the extraction of hidden regularities: a systematic review and the importance of temporal rules. Sleep Medicine Reviews, 47:39–50, 2019. 2, 5 11

    2019

  6. [14]

    Human associative memory

    John R Anderson and Gordon H Bower. Human associative memory . Psychology press, 2014. 4

    2014

  7. [15]

    Mnemonics in education: Current research and applications

    Adam L Putnam. Mnemonics in education: Current research and applications. T ranslational Issues in Psychological Science, 1(2):130, 2015. 4

    2015

  8. [16]

    The serial reaction time task: implicit motor skill learning? Journal of Neuroscience , 27(38):10073–10075, 2007

    Edwin M Robertson. The serial reaction time task: implicit motor skill learning? Journal of Neuroscience , 27(38):10073–10075, 2007. 5

    2007

  9. [17]

    Interaction between hippocampal and striatal systems predicts subsequent consolidation of motor sequence memory.PloS one, 8(3):e59490, 2013

    Geneviève Albouy, Virginie Sterpenich, Gilles Vandewalle, Annabelle Darsaud, Steffen Gais, Géraldine Rauchs, Martin Desseilles, Mélanie Boly, Thanh Dang-Vu, Evelyne Balteau, et al. Interaction between hippocampal and striatal systems predicts subsequent consolidation of motor ...

    2013

  10. [18]

    Higher-order associative learning in amnesia: Evidence from the serial reaction time task

    Tim Curran. Higher-order associative learning in amnesia: Evidence from the serial reaction time task. Journal of cognitive neuroscience , 9(4):522–533, 1997

    1997

  11. [19]

    The neural correlates of implicit and explicit sequence learning: Interacting networks revealed by the process dissociation procedure

    Arnaud Destrebecqz, Philippe Peigneux, Steven Laureys, Christian Degueldre, Guy Del Fiore, Joël Aerts, André Luxen, Martial Van Der Linden, Axel Cleeremans, and Pierre Maquet. The neural correlates of implicit and explicit sequence learning: Interacting networks revealed by th...

    2005

  12. [20]

    Implicit learning-explicit knowing: A role for sleep in memory system interaction

    Stefan Fischer, Spyridon Drosopoulos, Jim Tsen, and Jan Born. Implicit learning-explicit knowing: A role for sleep in memory system interaction. Journal of Cognitive Neuroscience , 18:311–319, 2006. 6

    2006

  13. [21]

    Searching for memory: The brain, the mind, and the past

    Daniel L Schacter. Searching for memory: The brain, the mind, and the past . Basic books, 2008. 9

    2008

  14. [22]

    Hierarchical recurrent neural networks for long-term dependencies

    Salah Hihi and Y oshua Bengio. Hierarchical recurrent neural networks for long-term dependencies. Ad- vances in neural information processing systems , 8, 1995. 10

    1995

  15. [23]

    A clockwork rnn

    Jan Koutnik, Klaus Greff, Faustino Gomez, and Juergen Schmidhuber. A clockwork rnn. In International conference on machine learning , pages 1863–1871. PMLR, 2014. 10

    2014

  16. [24]

    Hierarchical multiscale recurrent neural networks.arXiv preprint arXiv:1609.01704, 2016

    Junyoung Chung, Sungjin Ahn, and Y oshua Bengio. Hierarchical multiscale recurrent neural networks.arXiv preprint arXiv:1609.01704, 2016. 10

    2016 arXiv

  17. [25]

    Skip rnn: Learning to skip state updates in recurrent neural networks

    Víctor Campos, Brendan Jou, Xavier Giró-i Nieto, Jordi Torres, and Shih-Fu Chang. Skip rnn: Learning to skip state updates in recurrent neural networks. arXiv preprint arXiv:1708.06834 , 2017. 11

    2017 arXiv

  18. [26]

    Dilated recurrent neural networks

    Shiyu Chang, Y ang Zhang, Wei Han, Mo Yu, Xiaoxiao Guo, Wei Tan, Xiaodong Cui, Michael Witbrock, Mark A Hasegawa-Johnson, and Thomas S Huang. Dilated recurrent neural networks. Advances in neural information processing systems , 30, 2017. 11

    2017

  19. [27]

    Unitary evolution recurrent neural networks

    Martin Arjovsky, Amar Shah, and Y oshua Bengio. Unitary evolution recurrent neural networks. In Interna- tional conference on machine learning , pages 1120–1128. PMLR, 2016. 11

    2016

  20. [28]

    Antisymmetricrnn: A dynamical system view on recurrent neural networks

    Bo Chang, Minmin Chen, Eldad Haber, and Ed H Chi. Antisymmetricrnn: A dynamical system view on recurrent neural networks. arXiv preprint arXiv:1902.09689 , 2019. 11

    1902 arXiv

  21. [29]

    Transformer- xl: Attentive language models beyond a fixed-length context

    Zihang Dai, Zhilin Y ang, Yiming Y ang, Jaime Carbonell, Quoc V Le, and Ruslan Salakhutdinov. Transformer- xl: Attentive language models beyond a fixed-length context. arXiv preprint arXiv:1901.02860 , 2019. 11

    1901 arXiv

  22. [30]

    Compressive transformers for long-range sequence modelling

    Jack W Rae, Anna Potapenko, Siddhant M Jayakumar, and Timothy P Lillicrap. Compressive transformers for long-range sequence modelling. arXiv preprint arXiv:1911.05507 , 2019. 11

    1911 arXiv

  23. [31]

    Neural turing machines

    Alex Graves, Greg Wayne, and Ivo Danihelka. Neural turing machines. arXiv preprint arXiv:1410.5401 ,

    arXiv

  24. [32]

    Efficiently modeling long sequences with structured state spaces

    Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces. arXiv preprint arXiv:2111.00396 , 2021. 11 12

    2021 arXiv

  25. [33]

    Gated linear attention trans- formers with hardware-efficient training

    Songlin Y ang, Bailin Wang, Yikang Shen, Rameswar Panda, and Y oon Kim. Gated linear attention trans- formers with hardware-efficient training. arXiv preprint arXiv:2312.06635 , 2023. 11

    2023 arXiv

  26. [34]

    Rwkv: Reinventing rnns for the transformer era

    Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Matteo Grella, et al. Rwkv: Reinventing rnns for the transformer era. arXiv preprint arXiv:2305.13048, 2023. 11

    2023 arXiv

  27. [35]

    Inductive biases for deep learning of higher-level cognition.Proceedings of the Royal Society A , 478(2266):20210068, 2022

    Anirudh Goyal and Y oshua Bengio. Inductive biases for deep learning of higher-level cognition.Proceedings of the Royal Society A , 478(2266):20210068, 2022. 11

    2022

  28. [36]

    Gray, Sol Simpson, Michael MacAskill, Richard Höchenberger, Hiroyuki Sogo, Erik Kastman, and Jonas Kristoffer Lindeløv

    Jonathan Peirce, Jeremy R. Gray, Sol Simpson, Michael MacAskill, Richard Höchenberger, Hiroyuki Sogo, Erik Kastman, and Jonas Kristoffer Lindeløv. Psychopy2: Experiments in behavior made easy. Behavior Research Methods, 51:195–203, 2 2019. ISSN 15543528. doi: 10.3758/s13428-01...

    2019 doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.