Pith. sign in

REVIEW 3 major objections 4 minor 37 references

TextNCA: Neural Cellular Automata for Language Modeling via Hierarchical Local Attention

T0 review · 3 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A staged narrow-to-wide attention schedule, not iteration, drives TextNCA's language-modeling behavior.

desk verdict The staged-schedule result is probably real, but the paper's flagship quantitative claim about iteration rests on a control that doesn't match receptive field or depth; still worth refereeing. read the letter →

arxiv 2608.02050 v1 pith:M5WAGHRJ submitted 2026-08-03 cs.CL cs.AIcs.LG

Avni Mittal , Avinash Anand , Ashutosh Kumar , Dikshant Kukreja , Kritarth Prasad , Sushane Dulloo , Erik Cambria , Timothy Liu
show 2 more authors
Zhengkui Wang Rajiv Ratn Shah
This is my paper
classification cs.CLcs.AIcs.LG
keywords neuralcellularautomatalanguagemodelinglocalattentioniteratedweightsharinghierarchicalwindowscheduleperplexityablationWikiText-103GRUgating
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TextNCA asks whether a neural cellular automaton primitive—strictly local attention, iterated weight-shared updates, and a gated in-place state—can model language, and which of those properties actually does the work. The paper's central claim is that the staged narrow-to-wide window schedule, starting at 8 tokens and widening to 128, accounts for most of the model's behavior on WikiText-103. A non-iterating sliding-window transformer that reuses the same schedule comes within +4.1 perplexity of the full iterated model, while reversing, flattening, or breaking the schedule costs between +16.7 and +70.8 perplexity. Iteration acts as a bounded effective-depth knob with a sharp optimum at four steps per stage, and it only helps when a GRU gate and learned per-step embeddings are present. The model does not beat transformers at this scale, so the paper frames itself as an analytical probe rather than a performance proposal.

What carries the argument

The load-bearing object is the NCA cell: a shared causal windowed-attention module followed by LayerNorm, an FFN, and a GRU gate that combines the perception output with the previous token state; a learned step embedding is added at each iteration so the shared weights can specialise across steps. Three such stages, each with its own parameters and windows w=8, 32, and 128, are applied T_s=4 times each, giving a receptive field that grows from 29 to 153 to 661 tokens. That receptive-field growth is the mechanism that makes the narrow-to-wide schedule matter: dependencies beyond 152 tokens cannot be resolved until the final wide stage. The key control, SWin-TF-Staged, is a non-iterating trans

What would settle it

Retrain SWin-TF-Staged and Hier-TextNCA with exactly matched parameter counts and per-step FLOPs (for instance by adjusting the control's width) under the same 60k-step protocol, and check whether the 4.1-perplexity gap remains, shrinks, or inverts; also test whether the gap is sensitive to the uniform-w=128 control's parameter count.

Watch

Extended reading notes

Core claim

The paper defines TextNCA, a 1D causal windowed-attention realization of the NCA primitive: a token state is updated in place by a shared local rule, iterated T_s times per stage, across three cascaded stages with windows 8, 32, and 128. On WikiText-103 at roughly 30M parameters and 60k training steps, Hier-TextNCA reaches 60.3 perplexity. The central discovery is that the staged narrow-to-wide schedule carries most of the behavior: a non-iterating 6-layer sliding-window transformer with the same scheduled windows reaches 64.5 perplexity, only +4.1 worse, while reversing the schedule costs +70.8, flattening it costs +59.4, and breaking monotonicity costs +16.7. Iteration is a bounded effecti

Load-bearing premise

The load-bearing premise is that the non-iterating control SWin-TF-Staged (35.5M parameters) is a fair comparison for Hier-TextNCA (30.8M) even though matching is on training steps rather than on parameter count or per-step FLOPs, so the +4.1 PPL gap attributed to iteration could change under exact parameter matching.

Editorial extensions

If this is right

  • If the narrow-to-wide schedule is the dominant factor, then NCA-style iteration and weight sharing are not what make local language models work at this scale; the context-width curriculum is.
  • Iteration functions as an effective-depth knob: perplexity is U-shaped in T_s, with a clear optimum at 4, so adding more shared updates eventually hurts rather than converging to a fixed point.
  • The iteration benefit is conditional: it appears only when the GRU gate and learned per-step embeddings are present, so un-gated or unconditioned iterated transformers are expected to behave qualitatively differently.
  • Attention locality, not iteration, limits extractive QA: local-attention variants cluster at SQuAD F1 19–26 while global-attention models reach 27–33, and adding a global readout block does not close the gap.
  • An inference-time iteration-count knob is achievable with stochastic-T_s training and sinusoidal step embeddings, but it costs about 41 perplexity points on this benchmark.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • As an editorial extension: the schedule-dominance result suggests a cheap recipe for Transformer training—warm up the context window from narrow to wide during pretraining—that the paper did not test directly but that its control experiment implies.
  • As an editorial extension: since the non-iterating control is 15% larger than the iterated model (35.5M vs 30.8M) and matching is on training steps rather than parameters or FLOPs, an exact parameter-matched comparison could shrink or enlarge the +4.1 PPL gap; that experiment is outside the paper's current design.
  • As an editorial extension: the prepare-then-decode interpretation—early stages build linear separability, the final wide stage converts it to next-token probability—could be tested as a design principle for hierarchical language models, e.g. by making only the final stage use global attention while early stages remain local.
  • As an editorial extension: the inverse result for GLA kernels (better at single scale, worse in the hierarchy) suggests that kernel choice and window schedule interact; testing other kernels such as linear attention or state-space layers inside the staged hierarchy could map where the interaction comes from.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces TextNCA, a 1D causal windowed-attention realization of the Neural Cellular Automaton primitive, and studies a hierarchical variant with three stages of window sizes [8,32,128] and T_s=4 shared-weight iterations per stage. Training on SlimPajama and evaluating on WikiText-103 at ~30M parameters and 60k steps, the paper reports that Hier-TextNCA reaches 60.3 PPL, behind Transformer-6L (52.8) and Transformer-12L (44.7). The central claim is a decomposition: the staged narrow-to-wide window schedule accounts for most of the architecture's behavior, while iterated weight sharing adds only a bounded ~4.1 PPL benefit on top. Supporting evidence includes schedule perturbations (reversed, uniform, non-monotone costs of +16.7 to +70.8 PPL), a U-shaped T_s sweep with optimum at 4, gating and step-embedding ablations, stochastic-T_s training for an inference-time iteration knob, and extensive interpretability analyses. The paper is positioned as an analytical probe rather than a performance bid.

Significance. If the central attribution is correct, the paper provides a valuable controlled decomposition of NCA-style computation for language modeling: the dominant factor would be the staged context schedule, not iterative weight sharing. The paper is commendably honest about compute-matched gaps, out-of-domain evaluation, and downstream limitations, and it reports extensive ablations and mechanistic analyses with a planned code release. However, the key quantitative claim that iteration adds only ~4.1 PPL rests on a non-iterating control that does not actually match the iterated model's receptive field or sequential depth. Because the flagship configuration is also selected on the test set, the main quantitative conclusions need additional support before they can be accepted.

major comments (3)
  1. [§5.1, Table 2; §3.2 Eq. (R_K)] The central claim that iterated weight sharing contributes only ~4.1 PPL relies on the SWin-TF-Staged control, which is described as matching Hier-TextNCA's receptive field. This is incorrect under the paper's own formula R_K = 1 + Σ_k T_s(w_k−1). Hier-TextNCA with [8,32,128] and T_s=4 has R = 1 + 4·7 + 4·31 + 4·127 = 661, while the 6-layer SWin-TF-Staged with layer windows [8,8,32,32,128,128] has R = 1 + 7 + 7 + 31 + 31 + 127 + 127 = 331. The control also has only 6 sequential attention layers versus 12 NCA steps. The +4.1 PPL gap can therefore be attributed to reduced context and reduced sequential depth, not solely to the absence of iteration/weight sharing. Moreover, the uniformly wide SWin-TF-6L control (R=763) reaches 63.2 PPL, within 2.9 of the flagship, suggesting that a properly matched non-iterating model could match or beat Hier-TextNCA. A 12-layer non-iterating sliding-window
  2. [§5.1, Table 2] The claim that the staged schedule's benefit is 'conditional on the iterated weight-shared body' is also confounded. The comparison SWin-TF-Staged (64.5 PPL) vs SWin-TF-6L uniform w=128 (63.2 PPL) varies both the schedule and the receptive field (331 vs 763). The better performance of the uniform control may simply reflect its larger receptive field, not a negative interaction between staging and non-iterated transformers. The paper reads the uniform control's advantage as evidence that the schedule's benefit requires iteration, but this requires a non-iterating control that matches both the scheduled windows and the receptive field. Without such a control, the 'schedule dominates, iteration is secondary' conclusion is not quantitatively established.
  3. [§5.1, §5.3, Tables 2 and 7] The flagship configuration (K=3, T_s=4) is selected using WikiText-103 test perplexity: the paper states 'We fix K=3 and T_s=4 based on the Ts sweep in §5.3... and the stage-count ablation in §5.1,' and all reported PPL numbers are on the test split. No held-out validation set or selection protocol is described. This selection-on-test procedure inflates the flagship's PPL and biases the magnitude of the schedule and iteration effects. At minimum, the authors should report a validation-based selection or show that the qualitative conclusions are unchanged under a proper validation split.
minor comments (4)
  1. [Abstract; §5.2, Table 3] The abstract and §4.2 describe the comparison as 'parameter-matched,' but Table 3 shows Hier-TextNCA at 30.8M parameters vs Transformer-6L at 35.5M and Transformer-12L at 54.5M. Matching here is on training steps/compute, not parameter count. Please reword to 'matched-step' or 'matched-compute' to avoid ambiguity.
  2. [Table 2 caption] The caption states 'at matched architecture and compute,' while §4.1 refers to both matched-parameter and matched-compute comparisons. These are different protocols; the table caption should specify which one applies to each row or clarify the comparison basis.
  3. [Appendix F, Table 10] The matched-FLOP and wall-clock PPL values for Hier-TextNCA at 31k and 23k steps are interpolated from the eval-loss curve. This is reasonable, but it should be stated in the main text (§5.2) when citing these numbers, not only in the appendix.
  4. [§7 vs Appendix E] The stochastic-T_s model is called 'STOCHASTICT' in §7 and 'Hier-StochasticT' in Appendix E. Please use one consistent name throughout.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found; empirical ablation, though the +4.1 PPL iteration attribution rests on a control that does not match receptive field or depth.

full rationale

The central decomposition is an ablation, not a derivation. The paper's key claim that the narrow-to-wide schedule accounts for most of the effect, with iteration adding ~4 PPL, is supported by interventions run at matched 60k steps: reversing the schedule ([128,32,8]) gives 131.1 PPL (+70.8), uniform [32,32,32] gives 119.7 (+59.4), non-monotone [8,128,32] gives 77.0 (+16.7), and a separately trained 6-layer non-iterating SWin-TF-Staged control reaches 64.5. None of these numbers is a fitted parameter renamed as a prediction; each is an independent training run. The Ts=4 setting is selected from a from-scratch sweep (80.3/60.3/73.9/117.9 PPL for Ts=2/4/6/8), so it is honest model selection rather than a self-fulfilling definition. Self-citations (e.g., Mittal et al. 2025) are background and are not load-bearing; no uniqueness theorem or ansatz is imported from the authors' prior work. The skeptical concern is a control-validity issue, not circularity: by the paper's own formula in §3.2, SWin-TF-Staged's receptive field is 1+7+7+31+31+127+127=331, while Hier-TextNCA's is 661, and the control uses 6 sequential attention applications vs 12 NCA steps, so the +4.1 PPL gap cannot cleanly isolate iteration/weight sharing. The uniform-w=128 SWin-TF-6L at 63.2 PPL does not rescue the attribution because it too is not matched on these dimensions. This weakens the headline conclusion, but the conclusion is not forced by construction, by definition, or by a fitted quantity; it is an under-supported empirical claim. Therefore the circularity score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the chosen schedule, T_s, and K, which are either hand-set or selected on the evaluation metric. No new theoretical entities are introduced.

free parameters (3)
  • T_s (iterations per stage) = 4
    Selected as the minimum of the T_s sweep on WikiText-103 test PPL (80.3, 60.3, 73.9, 117.9). This is test-set model selection.
  • K (number of stages) = 3
    Selected because K=3 beats K=2 and K=4 in the stage-count ablation measured on WikiText-103 test PPL.
  • Window schedule [w1,w2,w3] = [8,32,128]
    Chosen by hand to give a narrow-to-wide cascade; not swept. The schedule itself is the object of study, so it is an input rather than a fit, but its specific values are not justified by a principled rule.
assumptions (3)
  • domain assumption Next-token prediction on WikiText-103 is a valid proxy for language modeling capability at this scale.
    The paper only evaluates on WikiText-103 under an out-of-domain SlimPajama-Mistral protocol, and any conclusions about language modeling rely on this proxy.
  • domain assumption Standard backpropagation through time with gradient checkpointing trains the recurrent updates sufficiently.
    Training dynamics are empirical; the paper does not prove optimization convergence.
  • domain assumption Causal windowed softmax attention correctly implements 'local perception' for the NCA primitive.
    The mapping is a design choice, not a formal equivalence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TextNCA: Neural Cellular Automata for Language Modeling via Hierarchical Local Attention." pith.science (2026). https://pith.science/paper/M5WAGHRJ

@misc{pith2026260802050,
  author       = {Pith},
  title        = {Pith review of: TextNCA: Neural Cellular Automata for Language Modeling via Hierarchical Local Attention},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/M5WAGHRJ}},
  note         = {Machine review of arXiv:2608.02050}
}
abstract

Can a strictly local, iterated, weight-shared computation primitive support language modelling, and which of those three properties actually drives the model's behaviour? We define \textsc{TextNCA}, a 1D causal windowed-attention realisation of the Neural Cellular Automaton primitive, and study a hierarchical variant that cascades three stages with windows $w \in \{8, 32, 128\}$ and $T_s$ shared-weight iterations per stage, all on WikiText-103 at roughly 30M parameters and 60k training steps. The model does not match a parameter-matched Transformer at this scale (Hier-TextNCA $60.3$ vs.\ Transformer-6L $52.8$ and Transformer-12L $44.7$ PPL), so we treat it as an analytical probe rather than a proposed alternative. The behaviour we observe is largely explained by the staged narrow-to-wide schedule: a non-iterating sliding-window Transformer that reuses the same schedule comes within $+4.1$ PPL of the iterated model, while reversing, flattening, or breaking the monotonic ordering of the schedule costs between $+16.7$ and $+70.8$ PPL. Iteration adds a smaller bounded benefit on top of the schedule, with a clear optimum at $T_s{=}4$ and a U-shaped degradation beyond it. The GRU gate and learned per-step embeddings are required for that benefit to appear, and training with random $T_s$ yields an inference-time iteration-count knob at the cost of substantially higher absolute PPL. We position the work as a controlled reading of which parts of NCA-style computation carry the weight in language modelling.

Figures

Figures reproduced from arXiv: 2608.02050 by the authors.

Figure 1
Figure 1. Hierarchical TextNCA (staged) architecture. (a) A single NCA cell: local attention with window wk, LayerNorm, FFN, and a GRU gate that combines the previous state ht−1 (dashed residual) with the transformed signal. A learned step embedding st is added at the input so the shared weights can still specialise across iterations. (b) The staged pipeline. Three stages with windows w ∈ {8, 32, 128} iterate Ts=4 times each … view at source ↗
Figure 2
Figure 2. The bracket marks the +4.1 PPL gap between Hier-TextNCA (60.3) and SWin-TF-Staged (64.5), against which the +59.4 to +70.8 PPL penalties from schedule perturbations should be read. (+16.7). All three controls confirm that narrow￾to-wide staged expansion is essential. The non-monotone variant retains the narrow first stage but inserts a wide middle stage, and its penalty is far smaller than reversing or flattening: s… view at source ↗
Figure 3
Figure 3. Per-step linear probing accuracy across the 12 [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (15 more)
Figure 4
Figure 4. Figure 4: Linear CKA between the embedding state and [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Perception-kernel ablation at single scale [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Downstream evaluation at matched 60k train [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: WikiText-103 PPL (log scale) against training steps for the key TextNCA and Transformer variants. Hier [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Per-step next-token loss of Hier-TextNCA. [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Pairwise cosine similarity between the 15 token hidden states at seven snapshots (embedding + start [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: GRU gate statistics per NCA step. Bars are means over hidden dimensions; error bars are ±1σ. Top: update gate z, a small U-shape in the w=8 stage (0.60, 0.39, 0.52), ∼ 0.30 through the w=32 stage, and rising to 0.62 by step 12. Bottom: reset gate r, falling monotonica…
Figure 11
Figure 11. Figure 11: Within-stage attention maps for the running prompt. Rows are stages ( [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 12
Figure 12. Figure 12: Hidden-state dynamics across the 12 NCA steps. Top: update magnitude ∥∆h∥. Bottom: cosine similarity between consecutive states. States stabilize within stages but shift at stage boundaries; the final update remains nonzero. has no usable inference-time knob. E.1 Why …
Figure 18
Figure 18. Figure 18: Inference-time T ′ s sweep, WikiText-103 PPL (log scale). Shaded band: StochasticT training distribution T ′ s ∈ {2, 4, 6}. StochasticT (green) keeps a working valley at PPL 101–108; the deterministic SinStep counterpart (red dashed) diverges at any off-distribution T…
Figure 17
Figure 17. Figure 17: WikiText-103 PPL (log scale) for the deterministic Hier-TextNCA flagship evaluated at different inference T ′ s (total NCA steps = 3T ′ s). PPL is U-shaped with the minimum at the training horizon (T ′ s=4, 12 steps) [PITH_FULL_IMAGE:figures/full_fig_p018_17.png]
Figure 13
Figure 13. Figure 13: Logit lens across the 13 representations (embedding + 12 NCA steps) for three positions in the relativity [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]
Figure 14
Figure 14. Figure 14: Per-token logit-lens trajectories across 12 [PITH_FULL_IMAGE:figures/full_fig_p020_14.png]
Figure 19
Figure 19. Figure 19: Compute–PPL Pareto frontier on WikiText-103 at 60k training steps. Standard transformers (blue) define the frontier; Hier-TextNCA (green) lies 7.5 PPL behind TF-6L at 1.9× the compute. Dashed line: lower-PPL convex hull. TextNCA is not on the compute–PPL frontier at t…
Figure 16
Figure 16. Figure 16: Top-5 predicted tokens at the final position [PITH_FULL_IMAGE:figures/full_fig_p021_16.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

37 extracted references · 7 linked inside Pith

  1. [1]

    Advances in Neural Information Processing Systems , volume=

    Attention is all you need , author=. Advances in Neural Information Processing Systems , volume=

  2. [2]

    arXiv preprint arXiv:2001.08361 , year=

    Scaling laws for neural language models , author=. arXiv preprint arXiv:2001.08361 , year=

    arXiv 2001

  3. [3]

    Advances in Neural Information Processing Systems , volume=

    Training compute-optimal large language models , author=. Advances in Neural Information Processing Systems , volume=

  4. [4]

    2002 , publisher=

    A New Kind of Science , author=. 2002 , publisher=

    2002

  5. [5]

    Distill , volume=

    Growing neural cellular automata , author=. Distill , volume=

  6. [6]

    arXiv preprint arXiv:2105.07299 , year=

    Texture generation with neural cellular automata , author=. arXiv preprint arXiv:2105.07299 , year=

    arXiv

  7. [7]

    Distill , volume=

    Self-classifying MNIST digits , author=. Distill , volume=

  8. [8]

    International Conference on Learning Representations , year=

    Variational neural cellular automata , author=. International Conference on Learning Representations , year=

Show all 37 references
  1. [9]

    arXiv preprint arXiv:2603.10055 , year=

    Training Language Models via Neural Cellular Automata , author=. arXiv preprint arXiv:2603.10055 , year=

  2. [10]

    International Conference on Learning Representations , year=

    Universal transformers , author=. International Conference on Learning Representations , year=

  3. [11]

    International Conference on Learning Representations , year=

    Neural GPUs learn algorithms , author=. International Conference on Learning Representations , year=

  4. [12]

    Advances in Neural Information Processing Systems , volume=

    Deep equilibrium models , author=. Advances in Neural Information Processing Systems , volume=

  5. [13]

    International Conference on Machine Learning , year=

    Arora, Simran and Eyuboglu, Sabri and Zhang, Michael and Smola, Alex and Dao, Tri and R. International Conference on Machine Learning , year=

  6. [14]

    Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise

    Bae, Sangmin and Fisch, Adam and Harutyunyan, Hrayr and Ji, Ziwei and Kim, Seungyeon and Schuster, Tal , booktitle=. Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise

  7. [15]

    International Conference on Learning Representations , year=

    Universal Transformers , author=. International Conference on Learning Representations , year=

  8. [16]

    Lan, Zhenzhong and Chen, Mingda and Goodman, Sebastian and Gimpel, Kevin and Sharma, Piyush and Soricut, Radu , booktitle=

  9. [17]

    arXiv preprint arXiv:2502.05171 , year=

    Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach , author=. arXiv preprint arXiv:2502.05171 , year=

    arXiv

  10. [18]

    Conference on Language Modeling (COLM) , year=

    Mamba: Linear-Time Sequence Modeling with Selective State Spaces , author=. Conference on Language Modeling (COLM) , year=

  11. [19]

    International Conference on Machine Learning , year=

    Hyena Hierarchy: Towards Larger Convolutional Language Models , author=. International Conference on Machine Learning , year=

  12. [20]

    International Conference on Machine Learning , year=

    Gated linear attention transformers with hardware-efficient training , author=. International Conference on Machine Learning , year=

  13. [21]

    arXiv preprint arXiv:2004.05150 , year=

    Longformer: The long-document transformer , author=. arXiv preprint arXiv:2004.05150 , year=

    2004 arXiv

  14. [22]

    Advances in Neural Information Processing Systems , volume=

    Big Bird: Transformers for longer sequences , author=. Advances in Neural Information Processing Systems , volume=

  15. [23]

    Dao, Tri and Fu, Daniel Y and Ermon, Stefano and Rudra, Atri and R. Flash. Advances in Neural Information Processing Systems , volume=

  16. [24]

    Interpreting

    nostalgebraist , journal=. Interpreting

  17. [25]

    Computational Linguistics , volume=

    Probing classifiers: Promises, shortcomings, and advances , author=. Computational Linguistics , volume=

  18. [26]

    International Conference on Machine Learning , year=

    Similarity of neural network representations revisited , author=. International Conference on Machine Learning , year=

  19. [27]

    SlimPajama: A 627B token cleaned and deduplicated version of RedPajama , author=

  20. [28]

    arXiv preprint arXiv:1609.07843 , year=

    Pointer sentinel mixture models , author=. arXiv preprint arXiv:1609.07843 , year=

    arXiv

  21. [29]

    arXiv preprint arXiv:1604.06174 , year=

    Training deep nets with sublinear memory cost , author=. arXiv preprint arXiv:1604.06174 , year=

    arXiv

  22. [30]

    International Conference on Learning Representations (ICLR) , year=

    Reasoning with Latent Thoughts: On the Power of Looped Transformers , author=. International Conference on Learning Representations (ICLR) , year=

  23. [31]

    Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL-HLT) , pages=

    Learning Word Vectors for Sentiment Analysis , author=. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL-HLT) , pages=

  24. [32]

    Advances in Neural Information Processing Systems (NeurIPS) , year=

    Character-level Convolutional Networks for Text Classification , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=

  25. [33]

    Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages=

    Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , author=. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages=

    2013

  26. [34]

    Rajpurkar, Pranav and Zhang, Jian and Lopyrev, Konstantin and Liang, Percy , booktitle=

  27. [35]

    Kalkhof, John and Gonz. Med-. International Conference on Information Processing in Medical Imaging (IPMI) , pages=. 2023 , organization=

    2023

  28. [36]

    2025 , organization=

    Mittal, Avni and Kalkhof, John and Mukhopadhyay, Anirban and Bhavsar, Arnav , booktitle=. 2025 , organization=

    2025

  29. [37]

    arXiv preprint arXiv:1603.08983 , year=

    Adaptive computation time for recurrent neural networks , author=. arXiv preprint arXiv:1603.08983 , year=

    arXiv

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.