Pith. sign in

REVIEW 3 major objections 2 minor 39 references

pLSTM: parallelizable Linear Source Transition Mark networks

T0 review · 3 major / 2 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper introduces pLSTM, a linear recurrent architecture that moves cell states onto the edges of a directed acyclic graph and processes the whole graph in parallel, with Source, Transition, and Mark gates and two stability modes.

desk verdict Real contribution to multi-dimensional linear RNNs, but the general-DAG D-mode stabilization claim is overclaimed and needs scoping down before acceptance. read the letter →

arxiv 2506.11997 v1 pith:IES4N6HO submitted 2025-06-13 cs.LG stat.ML

Korbinian Pöppel , Richard Freinschlag , Thomas Schmied , Wei Lin , Sepp Hochreiter This is my paper · ORCID
classification cs.LGstat.ML
keywords pLSTMlinearrecurrentnetworksdirectedacyclicgraphslineparallelassociativescanlong-rangestabilityimageextrapolationgraphrepresentationlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to give modern linear recurrent networks what multi-dimensional LSTMs once had: the ability to operate directly on grids, trees, and directed acyclic graphs instead of being forced into a fixed sequential order. The proposal, pLSTM, moves the recurrent cell state from nodes to edges and controls the flow with three learned gates: Source injects input into an edge, Transition moves state between consecutive edges, and Mark reads edge states out to a node. Because every update is linear, the paper shows the computation can be reorganized by divide and conquer into higher-order gates, giving a parallel associative scan over a DAG and logarithmic-time processing on regular grids. The paper also claims two modes that keep long-range activations and gradients bounded, a directed P-mode and a diffusive D-mode, and supports this with an arrow-pointing image task where pLSTM extrapolates to larger resolutions better than Transformer and recurrent baselines. A sympathetic reader would care because the result suggests linear RNNs can escape the sequence-only bottleneck without losing parallel training.

What carries the argument

The load-bearing object is the line graph $G'$ of the DAG: the nodes of $G'$ are the edges of the original DAG, and a line edge connects two original edges that meet at a node. pLSTM places its cell state on these nodes, so the recurrence becomes a power iteration on the adjacency matrix $T$ of $G'$, with Source, Transition, and Mark gates replacing input, forget, and output gates. Associativity of the linear operators is what lets the paper combine them into higher-order Source/Transition/Mark tensors over a recursive decomposition of the graph, turning a node-by-node topological pass into a parallel scan. Stability is a rule on $T$: in P-mode each node's outgoing transitions satisfy the column-sum bound $\sum_e |T^e_{e'}| \le 1$, keeping matrix powers bounded and gradients bounded in the dual norm; in D-mode one diagonal transition family is zeroed so $G'$ becomes a multitree with exactly one path between any two states, removing the exponential path sum that otherwise accumulates on grids.

What would settle it

Build a diamond DAG whose target receives the sum of the two branch inputs, train D-mode and unrestricted pLSTM on random instances of it, and compare accuracy. If the unrestricted network solves the task while D-mode accuracy is systematically lower, the multitree reduction has removed a route the task needs, refuting the claim that D-mode stabilizes general DAGs without sacrificing expressiveness; if D-mode preserves the contribution in every such DAG, the assumption stands.

Watch

Extended reading notes

Core claim

The central claim is that a linear RNN can be reparametrized as a network on a DAG by placing the cell state $C_e$ on each edge $e$, injecting node input through Source gates, propagating through Transition gates along consecutive edges, and reading node outputs through Mark gates. Writing the recurrence this way keeps every operation linear, so the exact node-to-node transfer is a sum over paths of products of Transitions; instead of enumerating the exponentially many paths, the paper merges Source, Transition, and Mark objects hierarchically over a recursive graph decomposition, yielding a parallel associative scan that reduces to einsum, concatenation, and padding steps for 1D and 2D grids. For long-range stability, the paper identifies two regimes: P-mode keeps the absolute column sums of the Transition matrix per node at most one, which bounds matrix powers and limits a signal to a chosen propagation line with only power-law decay, and D-mode zeroes one family of transitions so the line graph becomes a multitree, removing the exponential path-count blowup at the price of directionality. On the arrow-pointing extrapolation task, the claimed result is that pLSTM generalizes to larger image resolutions where Transformers and scanline recurrent models do not, and on ImageNet and molecular-graph benchmarks the paper reports accuracy competitive with established baselines.

Load-bearing premise

The D-mode stability argument assumes that for every DAG one can delete enough Transition edges so the line graph becomes a multitree, at most one path between any two state edges, without cutting information the task needs; the paper demonstrates this reduction for 2D grids by dropping one diagonal Transition, but does not prove it for arbitrary DAGs.

Editorial extensions

If this is right

  • Training on images, volumes, or graph-structured data no longer requires flattening them into a scanline order; one pLSTM layer can propagate information along all edges of a DAG in parallel, with a chunkwise-recurrent option that stops hierarchical merging at any chosen level.
  • With P- and D-modes alternated across layers, a model can both aim a signal along a chosen direction and spread it globally, making the effective receptive field a tunable design choice rather than an accident of the traversal order.
  • The arrow-pointing experiments imply that recurrence over the true data geometry can beat global-attention or positional-embedding shortcuts when the test resolution changes, because the model is not forced to learn distances in pixels.
  • On graph data, pLSTM acts as a message-passing network whose receptive field is a whole connected component per layer instead of one hop, reaching accuracy competitive with standard GNNs at matched parameter counts on the tested small-molecule and bioinformatics datasets.
  • The state-tracking extension with non-diagonal Transition matrices carries the same stability arguments, so a DAG pLSTM can count or track states without losing the P- and D-mode bounds.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not prove that every DAG admits a D-mode transition subset whose line graph is a multitree and still preserves all source-to-target connections; it demonstrates the reduction on 2D grids. A diamond DAG with two informative routes would be the minimal test of whether the D-mode multitree reduction preserves task-relevant connectivity.
  • The exponential path-count argument applies to any linear recurrence on a DAG with in-degree greater than one, so the P-mode/D-mode distinction is likely a general design principle for graph state-space models, not a pLSTM-specific trick.
  • The paper itself notes that the arrow-pointing task tests long-range behavior in a restricted way and that extrapolation is not perfect; a natural next step would be multi-scale medical or remote-sensing benchmarks where input resolution shifts at test time.
  • With non-diagonal Transition matrices, pLSTM may implement counting and state-tracking on graphs; testing it on graph analogues of regular-language tasks would separate the stabilization claim from the representational claim.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 2 minor

Summary. The paper introduces pLSTM, a linear recurrent architecture defined on the line graph of a DAG, with Source, Transition, and Mark gates replacing the usual input, forget, and output gates. The authors propose a hierarchical parallelization scheme for regular grids and, more speculatively, for general DAGs, and derive two stabilization mechanisms: a P-mode that bounds column sums of the Transition matrix and a D-mode that reduces the line graph to a multitree so that path-count explosion is avoided. Experiments cover a new synthetic arrow-pointing extrapolation task, ImageNet-1k, and TUDataset graph benchmarks.

Significance. If the claims are scoped correctly, pLSTM is a valuable bridge between linear RNNs and multi-dimensional RNNs. The P-mode column-sum bound in Section 4.2 and the power-law decay calculation in Appendix C.2 are concrete, checkable derivations. The arrow-pointing experiments include multiple seeds with confidence intervals, compare against several strong baselines, and show a clear extrapolation advantage for pLSTM on the tested setting. The release of code and datasets is a further strength. The main weakness is that the D-mode stabilization, which is advertised as a general result for DAGs, is only rigorously demonstrated for 2D grids, and the graph experiments do not verify the D-mode reduction. These are correctable issues, but they are load-bearing because the general-DAG stabilization claim is a headline contribution.

major comments (3)
  1. [4.2, Section 4.3.1, and contribution bullet in Section 1] The D-mode stabilization claim for general DAGs is not proven and, under a natural reachability-preservation requirement, is false. Section 4.2 states that D-mode 'reduces T or G′ from a DAG to a multitree' and that this yields long-range propagation, but the only concrete case given is the 2D grid in Section 4.3.1, where zeroing one diagonal Transition produces a multitree while preserving reachability. No preservation condition is stated for arbitrary DAGs. Under the natural requirement that every ancestor-descendant reachability relation of the original DAG be preserved, consider the DAG with edges s→a, a→b, a→c, b→d, c→d, d→t. Its line graph contains two distinct paths from the source edge (s→a) to the sink edge (d→t); retaining both transitions preserves both b→d and c→d but violates the unique-path condition, while zeroing either transition severs one of those relations. Thus the general-DAG claim in the introduction ('formally derive the general stabilization ... on general DAGs') is either trivial if no preservation is required, or unsupported if preservation is required. Please state a precise invariant for D-mode and prove that a reachability-preserving multitree reduction exists for the claimed class; otherwise, restrict the general claim to grids or to DAG families for which the reduction is proven.
  2. [5.4 (Graph experiments)] The graph experiments do not provide empirical support for the D-mode stabilization claim. Section 5.4 states that pLSTM alternates between P-mode and D-mode on molecular graphs, but no implementation detail is given for the multitree reduction on these DAG covers. The Transitions are computed from learned node and edge features, and no Transition is reported as being zeroed or otherwise forced into a multitree structure. Therefore the results in Table 3 cannot be read as evidence that the D-mode stabilization works on general graph-structured data. Please specify how D-mode is enforced for graph inputs, or remove the implication that the graph experiments exercise the D-mode mechanism.
  3. [4.1.2 and Appendices A.1/A.2] The hierarchical parallelization is claimed for general DAGs, but the recursion in Appendix A.2 is conditional on a decomposition into 'loosely self-similar' subgraphs, and no theorem or complexity analysis states for which DAGs such decompositions exist with bounded boundary size. The appendix itself acknowledges that arbitrary node subsets always produce a decomposition but that the edge balance may be poor. The graph experiments in Appendix D.4 explicitly use the recurrent, non-parallel form. Consequently, the parallelization claim is demonstrated only for regular grids (Appendices A.3 and A.4). Please either prove the parallelization for the stated class of general DAGs or explicitly scope the claim to the cases for which the construction is provided.
minor comments (2)
  1. [Table 5 and Figure 5] Table 5 is titled 'Test Results on Arrow Pointing Extrapolation' but reports numbers only for pLSTM variants and ViT, while Figure 5 shows learning curves for EfficientNet, ViL, Mamba2D, and 2DMamba. Numeric results for all baselines should be reported in the table for reproducibility and for a complete comparison.
  2. [Appendix A.3] The concatenation notation using ⌋i and ⌉i is nonstandard and easy to misread; a small worked example alongside Eq. (21) would make the recursive construction substantially clearer.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: pLSTM's stability derivations and empirical results are self-contained, with no fitted input renamed as prediction.

full rationale

The paper derives pLSTM from a linearized DAG recurrence (Eq. 5) and an algebraic expansion over paths (Eq. 7). The P-mode bound follows from sub-multiplicative L1 norm constraints on Transitions (Eq. 9), and the D-mode bound follows from the definition of D-mode as a multitree reduction, which by construction leaves a single path term. Neither step assumes the conclusion it is used to prove; the multitree reduction is an architectural constraint, not a fitted quantity. The arrow-pointing extrapolation results and ImageNet/graph benchmarks are empirical comparisons, not derived predictions, so no fitted parameter is relabeled as a prediction. The paper does cite same-group prior work (xLSTM, ViL, Tiled Flash Linear Attention), but those citations supply baselines and implementation context rather than the load-bearing stabilization argument. The D-mode existence claim for arbitrary DAGs is not proven in Section 4.2/4.3.1 (only the 2D-grid case is constructed), and the diamond-DAG counterexample from the skeptic shows the general claim is at least unsupported; however, an unsupported assumption is a correctness gap, not a circular reduction of the kind this pass flags. The limitations paragraph also concedes the arrow task is 'restricted.' No step in the derivation chain reduces to its own input by definition or by self-citation.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The theoretical claims (norm bounds, path-decay) are derived from standard linear algebra and graph theory, with no fitted constants. The main added assumptions are the 'loosely self-similar' decomposition for efficient hierarchical parallelization and the D-mode multitree reduction, which is stated without proof for general DAGs. No new physical entities are postulated.

assumptions (4)
  • standard math The line graph of a DAG is a DAG, and its adjacency matrix is nilpotent.
    Used in Section 4.2 to justify the finite power-series bound on Cell states.
  • standard math Induced matrix norms are sub-multiplicative, so column-sum normalization of T bounds all powers.
    Used in Section 4.2 to convert the node-local constraint into a global stability bound.
  • ad hoc to paper The DAG can be decomposed recursively into balanced 'loosely self-similar' subgraphs (Appendix A.1).
    This defines the class of graphs for which the hierarchical parallelization achieves its intended efficiency; arbitrary DAGs may not admit balanced decompositions.
  • domain assumption For the D-mode, the line graph can be reduced to a multitree by zeroing some Transitions, with at most one path between any two nodes.
    Stated in Section 4.2 without proof for general DAGs; demonstrated only for the 2D grid case.

how reviews work

0 comments
Cite this review

Pith. "Pith review of pLSTM: parallelizable Linear Source Transition Mark networks." pith.science (2026). https://pith.science/paper/IES4N6HO

@misc{pith2026250611997,
  author       = {Pith},
  title        = {Pith review of: pLSTM: parallelizable Linear Source Transition Mark networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IES4N6HO}},
  note         = {Machine review of arXiv:2506.11997}
}
read the original abstract

Modern recurrent architectures, such as xLSTM and Mamba, have recently challenged the Transformer in language modeling. However, their structure constrains their applicability to sequences only or requires processing multi-dimensional data structures, such as images or molecular graphs, in a pre-defined sequential order. In contrast, Multi-Dimensional RNNs (MDRNNs) are well suited for data with a higher level structure, like 2D grids, trees, and directed acyclic graphs (DAGs). In this work, we extend the notion of multi-dimensionality to linear RNNs. We introduce parallelizable Linear Source Transition Mark networks (pLSTMs) using Source, Transition, and Mark gates that act on the line graph of a general DAG. This enables parallelization in analogy to parallel associative scans and the chunkwise-recurrent form of sequential linear RNNs, but for DAGs. For regular grids (1D and 2D), like images, this scheme can be efficiently implemented using einsum operations, concatenations, and padding in logarithmic time. pLSTMs tackle the vanishing/exploding activation/gradient problem for long distances in DAGs via two distinct modes: a directed propagation mode (P-mode) and a diffusive distribution mode (D-mode). To showcase the long-range capabilities of pLSTM, we introduce arrow-pointing extrapolation as a synthetic computer vision task that contains long-distance directional information. We demonstrate that pLSTMs generalize well to larger image sizes, whereas Transformers struggle to extrapolate. On established molecular graph and computer vision benchmarks, pLSTMs also show strong performance. Code and Datasets are available at: https://github.com/ml-jku/plstm_experiments.

Figures

Figures reproduced from arXiv: 2506.11997 by the authors.

Figure 1
Figure 1. Illustration of the receptive fields induced by pLSTM and related architectures (for a single layer). CNNs are locally restricted while ViTs have a global receptive field. Modern recur￾rent architectures, such as ViM, traverse the 2D grid sequentially. pLSTM effectively extends the receptive field via its combination of D-mode and P-mode. Multi-dimensional RNNs (MDRNNs) [Graves et al., 2007] have demonstrated how no… view at source ↗
Figure 2
Figure 2. Illustration of the Ar￾row pointing task. The model has to classify whether an arrow is pointing towards a circle (top left). Models with global receptive fields, such as Vision Transform￾ers (ViTs), can solve this task by leveraging directional information encoded via positional embeddings (top right), but they often strug￾gle to generalize to higher resolu￾tions. In contrast, pLSTMs can ef￾fectively solve this tas… view at source ↗
Figure 3
Figure 3. Transition from input/- forget/output gating in between nodes (and input/output) in (lin￾ear) RNNs towards Source/Transi￾tion/Mark gating between edges and input/output (bottom/top) in pLSTMs. Linear RNNs resemble the structure of the original LSTM. For linearization, they remove the previous hidden state de￾pendency of the gates and cell update. Additionally, they com￾monly include a state expansion dimension or qu… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Illustration of pLSTM on general DAGs (left), and on 1D / 2D grids (right): In the top-left part, a general DAG is visualized, with Sources S n e ′, Transitions Te ′e, and Marks M n e pop￾ulating node-outgoing-edge, incoming-edge-outgoing-edge, and incoming-edge-node p…
Figure 5
Figure 5. Figure 5: Training curves for the Arrow Pointing Extrapolation task, averaged over 5 seeds with 90% CI. ViT and EfficientNet can quickly match the training set (left), and EfficientNet reaches the best validation performance of all models on the samples of the same resolution as…
Figure 6
Figure 6. Figure 6: Training curves for the Arrow Pointing Extrapolation task, averaged over 5 seeds with 90% CI on different model ablations. P-mode by itself performs worse, D-mode by itself is not as general in interpolation, but performs better on extrapolation compared to other model…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 21 canonical work pages

  1. [1]

    + log Γ ((1 − β)∆ + 1)) + ∆ log (α) − ∆ log (1 − α) (37) Stirling ≈ −∂β (β∆ log (β) + (1 − β)∆ log (1 − β)) + ∆ log (α) − ∆ log (1 − α) = ∆ log /pa{enleft.big4 α(1 − β) β(1 − α) /pa{en{ight.big4 21 This implies β = α for the direction of largest propagation. Now, inserting th is direction into the Transition product: T full xy(x+∆ x)(y+∆ y)∣β =α = /pa{enl...

    work page 2024

  2. [5]

    J. Deng, W . Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Im agenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recogni tion, pages 248–255. Ieee,

    work page 2009

  3. [6]

    URL https://www.jstor.org/stable/2371086?origin=crossref. K. Xu, W . Hu, J. Leskovec, and S. Jegelka. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826,

    work page arXiv

  4. [7]

    F. Gers. Learning to forget: continual prediction with LSTM . In 9th International Con- ference on Artificial Neural Networks: ICANN ’99 , volume 1999, pages 850–855, Ed- inburgh, UK,

    work page 1999

  5. [9]

    The orthogonal matrices U , V can be parametrized by the product of House- holder matrices (generated from vectors), or the exponenti al of the Lie-group / generating group of special orthogonal matrices: the skew-symmetric matric es (directly parameterized). With these parametrizations, in turn depending on the network inputs ( at nodes), state-tracking ...

    work page 2024

  6. [10]

    URL https://openreview.net/forum?id=UvTo3tVBk2. A. Gu and T. Dao. Mamba: Linear-time sequence modeling with s elective state spaces. arXiv preprint arXiv:2312.00752,

    arXiv

  7. [15]

    S. Li, H. Singh, and A. Grover. Mamba-ND: Selective State Spa ce Modeling for Multi-dimensional Data. In A. Leonardis, E. Ricci, S. Roth, O. Russakovsky, T. S attler, and G. V arol, editors, Computer Vision – ECCV 2024 , pages 75–92, Cham,

    work page 2024

  8. [17]

    URL https://www.cambridge.org/core/product/identifier/9780511810817/type/book. H. A. Jung. On a class of posets and the corresponding compara bility graphs. Journal of Combinatorial Theory, Series B , 24(2):125–133,

    arXiv

Show all 39 references
  1. [22]

    Schmied, T

    14 T. Schmied, T. Adler, V . Patil, M. Beck, K. Pöppel, J. Brandst etter, G. Klambauer, R. Pascanu, and S. Hochreiter. A large recurrent action model: xlstm enable s fast inference for robotics tasks. arXiv preprint arXiv:2410.22391 ,

    arXiv

  2. [23]

    Siems, T

    J. Siems, T. Carstensen, A. Zela, F. Hutter, M. Pontil, and R. Grazzi. Deltaproduct: Improving state-tracking in linear rnns via householder products. arXiv preprint arXiv:2502.10297 ,

  3. [24]

    Y . Sun, X. Li, K. Dalal, J. Xu, A. Vikram, G. Zhang, Y . Dubois, X . Chen, X. Wang, S. Koyejo, et al. Learning to (learn at test time): Rnns with expressive hidden states. arXiv preprint arXiv:2407.04620,

    arXiv

  4. [25]

    K. S. Tai, R. Socher, and C. D. Manning. Improved semantic rep resentations from tree-structured long short-term memory networks. arXiv preprint arXiv:1503.00075 ,

    arXiv

  5. [26]

    Touvron, M

    H. Touvron, M. Cord, and H. Jégou. DeiT III: Revenge of the ViT. In Computer Vision – ECCV 2022: 17th European Conference, T el Aviv, Israel, October 23–27, 2022, Proceedings, Part XXIV, pages 516–533, Berlin, Heidelberg,

    2022

  6. [27]

    ISBN 978-3-031-20052-6

    Springer-V erlag. ISBN 978-3-031-20052-6. doi: 10.1007/ 978-3-031-20053-3_30. URL https://doi.org/10.1007/978-3-031-20053-3_30 . event- place: Tel Aviv, Israel. A. V an Den Oord, N. Kalchbrenner, and K. Kavukcuoglu. Pixel r ecurrent neural networks. In International conference...

    doi

  7. [33]

    URL https://openreview.net/forum?id=r8H7xhYPwz. J. Zhang, A. T. Nguyen, X. Han, V . Q.-H. Trinh, H. Qin, D. Samaras, and M. S. Hosseini. 2dmamba: Efficient state space model for image representation with ap plications on giga-pixel whole slide image classification. arXiv preprin...

  8. [34]

    X. Zhu, P . Sobhani, and H. Guo. Dag-structured long short-te rm memory for semantic composition- ality. In Proceedings of the 2016 conference of the north american cha pter of the association for computational linguistics: Human language technologies , pages 917–926,

    2016

  9. [35]

    meta-paths

    16 A Method Details A.1 Loosely self-similar graphs Graphs modeling hierarchical order or spatial extensions, as in meshes or grids do typically have low node degree and can be decomposed recursively into subgr aphs of similar structure. We use the term loosely self-similar gr...

    2024

  10. [38]

    3a" → "rand-m9-mstd0.5-inc1

    (T), 0.05 (S), 0.1 (B) LayerScale - Warmup Epochs 5 Peak Learning Rate 4e-3 (T), 4e-3 (S), 3e-3 (B) → 1e-5 Weight Decay 0.2 Gradient Clip Norm 1.0 Optimizer Lamb → AdamW Loss Type Binary Cross Entropy → Cross Entropy MixUp 0.8 CutMix 1.0 Label Smoothing 0.0 → 0.1 Global Batch ...

  11. [39]

    AugmentRepeats 3 → 1 TestCropRatio 1.0 RandomCrop rrc 3https://github.com/facebookresearch/fvcore 23 D.3.1 Ablation Settings For the ablation studies, we use a simplified training settin g, without an additional fine-tuning stage, resembling a DeiT-T training over 400 epochs [To...

    2021

  12. [40]

    rand-m9-mstd0.5-inc1

    LayerScale - Warmup Epochs 5 Peak Learning Rate 1e-3 Weight Decay 0.05 Gradient Clip Norm 1.0 Optimizer AdamW Loss Type Cross Entropy MixUp 0.8 CutMix 1.0 Label Smoothing 0.0 → 0.1 Global Batch Size 2048 ColorJitter 0.0 AutoAugment "rand-m9-mstd0.5-inc1" RandomErasing 0.25 Aug...

    2020

  13. [1932]

    doi: 10.2307/237108

    ISSN 00029327. doi: 10.2307/237108

    doi

  14. [1961]

    doi: 10.1109/JRPROC.1961.287775. C. Morris, N. M. Kriege, F. Bause, K. Kersting, P . Mutzel, and M. Neumann. TUDataset: A collec- tion of benchmark datasets for learning with graphs. In ICML 2020 W orkshop on Graph Repre- sentation Learning and Beyond (GRL+

    1961

  15. [1978]

    doi: https://doi.org/10.1016/0095-8956 (78)90013-8

    ISSN 0095-8956. doi: https://doi.org/10.1016/0095-8956 (78)90013-8. URL https://www.sciencedirect.com/science/article/pii/0095895678900138. N. Kalchbrenner, I. Danihelka, and A. Graves. Grid long shor t-term memory. arXiv, 1507.01526,

  16. [1982]

    doi: 10.1073/p nas.79.8.2554

    ISSN 0027-8424, 1091-6490. doi: 10.1073/p nas.79.8.2554. URL https://pnas.org/doi/full/10.1073/pnas.79.8.2554. R. A. Horn and C. R. Johnson. Matrix Analysis . Cambridge Univer- sity Press, 1 edition, Dec

    doi

  17. [1985]

    doi: 10.1017/CBO97805118108

    ISBN 978-0-521-38632-6 97 8-0-521- 30586-0 978-0-511-81081-7. doi: 10.1017/CBO97805118108

    doi

  18. [1994]

    X. Chen, Z. Shi, X. Qiu, and X. Huang. Dag-based long short-te rm memory for neural word seg- mentation. arXiv preprint arXiv:1707.00248 ,

    arXiv

  19. [1999]

    ISBN 978-0-85296-721-8

    IEE. ISBN 978-0-85296-721-8. doi: 10.10 49/cp:19991218. URL https://digital-library.theiet.org/content/conferen ces/10.1049/cp_19991218. J. Gilmer, S. S. Schoenholz, P . F. Riley, O. Vinyals, and G. E. Dahl. Neural Message Passing for Quantum Chemistry. In D. Precup and Y . W ...

    doi

  20. [2009]

    doi: 10.1109/TNN.2008. 2005605. Y . Schiff, C.-H. Kao, A. Gokaslan, T. Dao, A. Gu, and V . Kulesh ov. Caduceus: Bi-directional equivariant long-range dna sequence modeling. arXiv preprint arXiv:2403.03234 ,

    2008 arXiv

  21. [2013]

    doi: 10.1109/icassp.2013.6638947. A. Graves, S. Fernández, and J. Schmidhuber. Multi-dimensi onal recurrent neural networks. In International Conference on Artificial Neural Networks (IC ANN), volume 4668 of Lecture Notes in Computer Science , pages 6645–6649. Springer, Berlin,...

    2013

  22. [2015]

    Schmidinger, L

    N. Schmidinger, L. Schneckenreiter, P . Seidl, J. Schimunek , P .-J. Hoedt, J. Brandstetter, A. Mayr, S. Luukkonen, S. Hochreiter, and G. Klambauer. Bio-xlstm: G enerative modeling, representation and in-context learning of biological and chemical sequenc es. arXiv preprint ar...

    arXiv

  23. [2016]

    URL http://arxiv.org/abs/1603.07063. Z. Liu, H. Mao, C.-Y . Wu, C. Feichtenhofer, T. Darrell, and S. Xie. A convnet for the 2020s. In Proceedings of the IEEE/CVF conference on computer vision a nd pattern recognition , pages 11976–11986,

    arXiv

  24. [2017]

    Dao and A

    T. Dao and A. Gu. Transformers are ssms: Generalized models a nd efficient algorithms through structured state space duality. arXiv preprint arXiv:2405.21060 ,

    arXiv

  25. [2018]

    C. Wang, W . Zheng, Y . Huang, J. Zhou, and J. Lu. V2M: Visual 2-D imensional Mamba for Image Representation Learning. arXiv preprint arXiv:2410.10382 ,

    arXiv

  26. [2019]

    Y ang, B

    S. Y ang, B. Wang, Y . Shen, R. Panda, and Y . Kim. Gated linear attention transformers with hardware- efficient training. arXiv preprint arXiv:2312.06635 ,

    arXiv

  27. [2021]

    URL https://openreview.net/forum?id=YicbFdNTTy. Y . Duan, W . Wang, Z. Chen, X. Zhu, L. Lu, T. Lu, Y . Qiao, H. Li, J. Dai, and W . Wang. Vision- rwkv: Efficient and scalable visual perception with rwkv-li ke architectures. arXiv preprint arXiv:2403.02308,

    arXiv

  28. [2022]

    Merrill, J

    W . Merrill, J. Petty, and A. Sabharwal. The illusion of state in state-space models. arXiv preprint arXiv:2404.08819,

    arXiv

  29. [2023]

    Y ang, B

    S. Y ang, B. Wang, Y . Zhang, Y . Shen, and Y . Kim. Parallelizinglinear transformers with the delta rule over sequence length. arXiv preprint arXiv:2406.06484 ,

    arXiv

  30. [2024]

    arXiv:2412.16146 [cs]

    URL http://arxiv.org/abs/2412.16146. arXiv:2412.16146 [cs]. M. Beck, K. Pöppel, P . Lippe, and S. Hochreiter. Tiled flash li near attention: More efficient linear rnn and xlstm kernels. arXiv preprint arXiv:2503.14376 , 2025a. M. Beck, K. Pöppel, M. Spanring, A. Auer, O. Prudnik...

  31. [2025]

    arXiv:2502.10818 [cs]

    URL http://arxiv.org/abs/2502.10818. arXiv:2502.10818 [cs]. E. Baron, I. Zimerman, and L. Wolf. A 2-Dimensional State Spa ce Layer for Spatial Induc- tive Bias. In The Twelfth International Conference on Learning Represen tations,

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.