Pith. sign in

REVIEW 4 major objections 6 minor 58 references

SILVA Networks as Structured Implicit Layers and Vector Attractors via Dynamic Interaction Fields

T0 review · 4 major / 6 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SILVA Networks make the internal dynamics of an implicit layer inspectable by separating stimulus, local interaction, global interaction, damping, and readout inside a single fixed-point update.

desk verdict A real method contribution with a clean path-sum identity and honest negative results, but the central attribution claim rests on a convergence precondition its own diagnostics show is not always met. read the letter →

arxiv 2607.28989 v1 pith:ETCGD6LU submitted 2026-07-31 cs.LG cs.NE

Jose Luis Lima de Jesus Silva This is my paper
classification cs.LGcs.NE
keywords implicitneurallayersdeepequilibriummodelsgraphnetworksfixed-pointiterationspectralradiusinteractionfieldslong-rangebenchmarksnodeclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes SILVA Networks, a class of implicit neural layers in which the fixed-point update is explicitly decomposed into an input stimulus, a local interaction operator, a global interaction operator, damping, and a readout. The same damped equilibrium template is instantiated for images, molecules, citation graphs, and long-range graph benchmarks by changing what counts as a node, a neighborhood, and a global summary. The experiments aim to show that this decomposition is more than cosmetic: ablations reveal that local interactions carry the graph tasks, the global term adds little on MNIST and citation networks at the tested capacity, and the global term is load-bearing on the CLUSTER long-range benchmark, where removing it costs 5.49 percentage points. If correct, the paper establishes that the internal dynamics of an implicit layer can be trained, ablated, visualized, and diagnosed rather than treated as an opaque solver.

What carries the argument

The central object is the dynamic interaction field Fθ(z,x)=fθ(z,x)−z, driven by the damped fixed-point iteration z_{k+1}=(1−α)z_k+αfθ(z_k,x). The update map fθ is organized as stimulus Wstim x plus a local interaction map L and a global interaction map G, with domain-specific definitions of nodes, neighborhoods, and global summaries; the settled state z_K is read out as an approximate vector attractor. The local linearization decomposes the one-step operator into self-persistence (1−α)I, local αM_L, and global αM_G, and the finite-solver response is the weighted path sum R_{K,α}=Σ_{r=0}^{K−1} w_r^{(K,α)} M^r with M=M_L+M_G, which connects damping, spectral radius, and solver budget to effec

What would settle it

A direct test: on CLUSTER, a no-global ablation with the same budget and five seeds that matches or exceeds 73.04% would refute the claim that the global interaction term is load-bearing for long-range node classification; on Pubmed, measuring ρ_T at every epoch and finding collapsed seeds with ρ_T below 1 throughout would refute the instability attribution.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that a single fixed-point architecture can separate five computational roles — stimulus, local interaction, global interaction, damping, and readout — inside one equilibrium solve, and that this separation is empirically meaningful. The evidence is the CLUSTER result: the full local+global model reaches 73.04±0.60% versus 67.55±0.84% for the no-global ablation, a 5.49-point gap, while on MNIST the four-arm ablation shows no arm separated by more than 1–2 standard deviations and on Cora/Citeseer removing the local term collapses accuracy. The paper reads these results as task-dependent roles for the interaction terms rather than as a uniformly beneficial glo

Load-bearing premise

The paper's task-dependent attribution assumes the trained models actually operate near the fixed point that the readout uses; on Pubmed the seeds bifurcate into converged and collapsed bands, the three-layer stack collapses uniformly, and the spectral radius can exceed 1 during training, so on some real graphs the reading that the global term is not useful is confounded with convergence instability.

Editorial extensions

If this is right

  • If the decomposition is correct, removing the global term should hurt exactly where long-range aggregation is needed; the CLUSTER gap of 5.49 percentage points is the direct test and it is met.
  • The finite-budget weights w_r^{(K,α)} predict that when the spectral radius of M is close to 1, longer interaction paths are under-resolved by fixed K; near-boundary models should be the ones most sensitive to solver budget.
  • The damping coefficient α cancels in the exact fixed-point gradient, so the implicit adjoint training gradient is independent of the solver trajectory used to reach the fixed point.
  • Stacking composes separately solved attractors with distinct damping timescales rather than deepening one equilibrium solve, giving a hierarchy of equilibria.
  • The four-arm MNIST ablation and the parameter-matched feedforward comparison imply that at easy tasks and tested capacity, the equilibrium and interaction machinery is not earning its computational cost; the paper treats this as consistent with its own premise.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the same decomposition suggests a practical selection rule — measure whether a task's performance moves when the global term is ablated, and disable it when it does not; the paper's own data already show such a rule would simplify the MNIST and citation-network models.
  • Beyond the paper: the Pubmed bimodal convergence and the ρ>1 collapse trace suggest that monitoring the damped-step spectral radius during training could serve as an early-warning signal; a testable extension is to compare models trained with a ρ_T cap against unconstrained models on large graphs.
  • Beyond the paper: the path-sum weights predict a concrete scaling law — for a fixed spectral radius, doubling the solver budget K should change accuracy more for models closer to ρ=1; this could be tested directly by sweeping K on CLUSTER and citation networks.
  • Beyond the paper: because the bounded top-k global variant avoids the uniform-attention collapse observed at graph scale, it offers a candidate slot for long-range tasks where the mean-field broadcast is too coarse; testing node-to-node top-k attention on CLUSTER would extend the regime map the paper begins.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces SILVA Networks, a damped fixed-point layer with update z_{k+1} = (1-α)z_k + α f_θ(z_k, x), in which f_θ explicitly separates a stimulus term, a local interaction term, a global interaction term, and a self-persistence/damping term. The authors derive a locally linearized path-sum expansion (Eq. 59) connecting solver depth, damping, spectral radius, and interaction-path length; propose spectral-radius and energy diagnostics; and instantiate the same template on MNIST, CIFAR-10, ZINC, Cora/Citeseer/Pubmed, and CLUSTER. The main empirical claim is task-dependent roles: local interactions are load-bearing in graph tasks, MNIST gains little from the equilibrium machinery at the tested capacity, and the global term is load-bearing on CLUSTER, where the full model reaches 73.04±0.60% versus 67.55±0.84% for the no-global ablation.

Significance. The architectural idea is valuable: making the internal composition of an implicit layer explicit — stimulus, local, global, self-persistence — and analyzing the finite-budget solver through a path-sum identity is a clean, analytically checkable contribution. The paper also deserves credit for honest negative results: the MNIST parameter-matched feedforward baseline shows the equilibrium machinery is not earning its cost there, and the citation-network experiments show no global benefit. The internal CLUSTER ablation is the strongest evidence for the global-term claim and is falsifiable. However, the central empirical narrative depends on configurations whose fixed-point status is only partially verified; the convergence diagnostics that are present (Pubmed bifurcation, stack collapse, ρ>1 trajectories) show that the attractor assumption cannot be taken for granted. If this is resolved, the paper would be a solid contribution to interpretable implicit layers.

major comments (4)
  1. [Section 3, CLUSTER paragraph and Table 3] The headline global-benefit result lacks convergence diagnostics. Eq. (1) reads out z_K as an approximate fixed point, and the path-sum/attractor interpretation in Eqs. (59)–(62) is meaningful only if the damped iteration is near convergence or ρ<1. Yet on Pubmed the same architecture bifurcates into converged (70–78%) and collapsed (19–56%) seeds (Table 2, Fig. 5c), the three-layer stack collapses uniformly, and Fig. 7 shows trained ρ>1. For CLUSTER, where the 5.49 pp global-vs-no-global difference is reported (n=4 vs n=5), no per-seed residuals, spectral radii, or trajectory diagnostics are given. The difference could be an optimization artifact if convergence status differs between arms. Please report residual norms at K and ρ_T per seed for the full and no-global CLUSTER models, and ideally per-seed accuracy trajectories.
  2. [Section 2.13 and Discussion] Because most experiments use truncated BPTT through K=15 unrolled Picard steps, not an implicit solve, the trained object is a finite recurrent network unless convergence is verified. The paper itself acknowledges this in the Discussion. This undermines the 'vector attractor' framing for non-converged configurations. I recommend either (a) verifying fixed-point convergence (residual below tolerance) for each configuration and reporting the fraction of seeds that converge, or (b) systematically reframing the claims as properties of finite unrolled networks and moving the attractor language to the linearized path-sum diagnostics only. At minimum, the convergence assumption behind Eq. (13) should be checked per experiment, not assumed.
  3. [Tables 1 and 3, baseline comparisons] The CLUSTER claim 'exceeds the best matched local graph baseline by 14.29 percentage points' is made against GCN/GatedGCN baselines that are not parameter-matched (Table 3 note) and may have fewer parameters than SILVA. Similarly, the ZINC comparison (Table 1) gives SILVA bond-type edge input while the baselines are restricted to node features and topology. These comparisons do not isolate the SILVA interaction field from extra capacity or input information. The internal ablation (no-global vs. full) is the cleanest evidence for the global term; consider making it the primary support and adding parameter-matched baselines or at least reporting parameter counts for all rows.
  4. [Section 2.10 and Fig. 10] The interpretability narrative treats ρ<1 as the condition for the Neumann/path-sum analysis, but Fig. 10 reports no correlation between measured ρ and robustness, and training trajectories can cross ρ=1 without collapse (Fig. 8). This does not invalidate the path-sum algebra, but it means ρ is not a reliable indicator that the executed model is in the contractive regime. The claim that finite K 'under-resolves' longer interaction histories (Section 2.10) is only interpretable if ρ<1 is verified for the specific trained model. Please report ρ_T at the terminal state for each seed in the main result tables, or at least for CLUSTER and Pubmed, and state explicitly when ρ>1 for converged-looking models.
minor comments (6)
  1. [Section 2.2, Eq. (26)] The symbol α is used both for the damping coefficient and for attention weights α_{ij}^{(k)}. Please rename one of them to avoid confusion across the paper.
  2. [Section 2.2, Eq. (28)] The displayed Jacobian decomposition is a linearization at fixed attention weights, but the equation omits the Jacobian factors of LayerNorm/ReLU and tanh. State explicitly that the form is schematic rather than the full Jacobian.
  3. [Figure 2 caption] The residual is written as ∥T_θ(z_k;x)−z_k∥₂, but the text defines the update through f_θ. Define T_θ as the damped update map to make the caption self-contained.
  4. [Figure 5c] The per-seed Pubmed panel is difficult to parse because many points overlap. Consider adding a small table of per-seed values or a jittered strip plot with explicit seed labels.
  5. [Data and code availability] The statement that materials 'will be available from the corresponding author upon reasonable request' is not sufficient for reproducibility. Please deposit code, hyperparameters, and seed-level results in a public repository.
  6. [Abstract and Section 2.13] The phrase 'vector attractors' in the abstract overstates the status of z_K for experiments that use only K=15–20 unrolled steps. Qualify the language to 'approximate fixed points of the finite unrolled solver' unless convergence is verified.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the path-sum and adjoint results are algebraic identities from the damped iteration, and the empirical role claims are ablations rather than fitted predictions.

full rationale

The derivation chain is self-contained. Eq. (1) defines damped Picard iteration; linearizing gives Eq. (10); substituting z_0 = 0 and expanding yields Eqs. (24)-(25) and the exact finite path-sum Eq. (59) with weights w_r^(K,alpha) given by Eq. (60) as functions of alpha and K. This is an algebraic identity from repeated substitution, not a quantity fitted to data, and the paper explicitly identifies the K-to-infinity limit as the Neumann series (I-M)^{-1}s. The response operator R_{K,alpha} in Eq. (61) is defined as the same sum, and the implicit-adjoint identity I - J_T = alpha(I - J_f) plus the cancellation of alpha in Eq. (76) follows by rearrangement of the damped update, with the DEQ gradient [4] cited only as prior context. The empirical role claims (global term load-bearing on CLUSTER, neutral on citation networks/MNIST) are ablations of the same trained template; the differences are measured accuracies, not parameters recovered from the experiment. The local descent surrogate in Eq. (45) is an exact equivalence between one gradient step on a frozen quadratic and the damped update, and the paper states it is a local diagnostic, not a global energy proof. The paper's own limitations are explicitly disclosed: truncated BPTT, finite K, approximate linearization, Pubmed seed bifurcation, three-layer collapse, and rho exceeding 1. These affect interpretation and stability, but none is a case where a predicted quantity is identical by construction to an input. No load-bearing self-citation or imported uniqueness theorem appears.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The central theoretical object is the damped fixed-point iteration and its local linearization. Most of the path-sum theory is bare linear algebra once M = M_L + M_G is assumed. The load-bearing non-linear assumptions are that finite-K outputs are approximate attractors, that the evaluated maps are contracting, and that non-differentiable top-k neighborhoods can be frozen at z*. No new physical entities are introduced; the 'dynamic interaction field' and 'vector attractor' are relabelings of the fixed-point residual and fixed point.

free parameters (4)
  • Damping coefficients α_l = α=0.25 (MNIST); α1=0.5, α2=0.2 (CIFAR-10, ZINC, citation, CLUSTER); α=0.5,0.35,0.2 for the three-layer stack
    Chosen by hand, not learned. The damping coefficient does not change the exact fixed point, but it controls the finite-K path-sum weights and the spectral-radius diagnostic, so it is load-bearing for the dynamics claims.
  • Solver iteration budgets K_l = K=20 for MNIST, CIFAR-10, ZINC; K=15 for citation networks and CLUSTER
    Finite truncation of the Picard loop. The paper reads out z_K as an approximate vector attractor, so K determines how much of the infinite Neumann series is actually realized; Pubmed collapsed seeds show convergence is not guaranteed.
  • Vision k-nearest-neighbor size and top-k global attention size = k=4 for the vision channel graph (Table 4); top-k size for the global variant not explicitly reported
    Hand-selected neighborhood sizes control the sparsity of the local and global operators; the global top-k variant's k is not stated, which is a reproducibility gap.
  • Hidden width, learning rate, epochs, patience, gradient clip = d=64; lr 0.002/0.001/0.01; epochs 30/100/50; patience 5/15/10; clip 1.0
    Standard training hyperparameters. They are not part of the theoretical derivation, but they matter for baseline matching; the paper states baselines share the same split and protocol but are not parameter-matched.
assumptions (6)
  • standard math Banach fixed-point theorem and contraction of f_θ
    Invoked in Section 2.1, Eq. (13), to justify existence and convergence of the fixed point. The evaluated LayerNorm-ReLU attention update is not proven to be a contraction; only the static-matrix tanh case has a checkable sufficient condition ||A||_2 < 1 (Eq. 83).
  • domain assumption Local linearization: tanh(z) ≈ z or frozen update h for the path-sum and local descent surrogate
    Sections 2.8 and 2.10 explicitly state that the path-sum, energy surrogate, and response operator are exact only for the local linearization with tanh(z)≈z or with h held fixed. This is a controlled approximation, not an identity for the full nonlinear map.
  • domain assumption Power iteration estimates the dominant eigenvalue of the damped Jacobian
    Algorithm 2 uses vector-Jacobian power iteration with random initialization. The paper notes the Jacobian can be non-normal, so ρ_T is used as an asymptotic stability indicator rather than a singular-value contraction certificate.
  • domain assumption Finite-K solver state z_K is treated as an approximate vector attractor
    Main experiments read out z_K after 15–20 iterations with no general residual criterion. The Pubmed collapsed seeds and the three-layer stack collapse show this assumption can fail in practice.
  • domain assumption Non-differentiable top-k selection is handled by freezing the selected graph at z*
    The Discussion states that top-k selection is not everywhere differentiable and implicit differentiation treats the selected graph as frozen at the fixed point. This applies to the vision channel graph and the top-k global-attention variant.
  • domain assumption Public benchmark datasets and splits are accepted as ground truth
    All empirical claims rest on MNIST, CIFAR-10, ZINC, Cora, Citeseer, Pubmed, and CLUSTER as external benchmarks, with splits and preprocessing as described in Section 2 and the Methods summary.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SILVA Networks as Structured Implicit Layers and Vector Attractors via Dynamic Interaction Fields." pith.science (2026). https://pith.science/paper/ETCGD6LU

@misc{pith2026260728989,
  author       = {Pith},
  title        = {Pith review of: SILVA Networks as Structured Implicit Layers and Vector Attractors via Dynamic Interaction Fields},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ETCGD6LU}},
  note         = {Machine review of arXiv:2607.28989}
}
read the original abstract

Many learning problems require representations that reconcile direct input, nearby structure, and broader context. In implicit neural layers, these influences are usually absorbed into a single fixed-point update, making it hard to identify what enters from the stimulus, what propagates locally, what comes from global context, and what is produced by solver dynamics. Here we introduce SILVA Networks, Structured Implicit Layers and Vector Attractors via Dynamic Interaction Fields. SILVA separates stimulus, local interaction, global interaction, damping, and readout inside one fixed-point architecture. The same template is instantiated for images, molecules, citation networks, and long-range graph benchmarks through domain-specific definitions of nodes, neighborhoods, and global summaries. Experiments and ablations show task-dependent roles for these terms: local interactions are load-bearing in the graph tasks, MNIST gains little from recurrence at the tested capacity, and the clearest global benefit appears in a long-range node-classification benchmark. SILVA therefore provides an implicit representation whose internal interaction dynamics can be trained, ablated, visualized, and diagnosed.

Figures

Figures reproduced from arXiv: 2607.28989 by the authors.

Figure 1
Figure 1. Conceptual mechanism of a SILVA equilibrium layer. (a) Stimulus, local interaction, [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Convergence diagnostics for trained SILVA image classifiers. (a) Damped fixed-point [PITH_FULL_IMAGE:figures/full_fig_p020_2.png] view at source ↗
Figure 3
Figure 3. Input-conditioned equilibrium-state geometry from the converged hidden state [PITH_FULL_IMAGE:figures/full_fig_p021_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Stacked equilibrium hierarchy on CIFAR-10. PCA-projected Picard-iteration trajectories [PITH_FULL_IMAGE:figures/full_fig_p022_4.png]
Figure 5
Figure 5. Figure 5: Citation-network ablations. (a) Cora. (b) Citeseer. Filled markers and whiskers show [PITH_FULL_IMAGE:figures/full_fig_p024_5.png]
Figure 6
Figure 6. Figure 6: MNIST local-global diagnostics. (a–d,f) Operator and solver diagnostics from an adjoint [PITH_FULL_IMAGE:figures/full_fig_p027_6.png]
Figure 7
Figure 7. Figure 7: Loss of local stability in one adjoint-trained MNIST diagnostic trajectory. (a) Validation [PITH_FULL_IMAGE:figures/full_fig_p029_7.png]
Figure 8
Figure 8. Figure 8: Epoch-resolved spectral-radius probe for the configuration-space grid’s largest- [PITH_FULL_IMAGE:figures/full_fig_p030_8.png]
Figure 9
Figure 9. Figure 9: Configuration-space map of ρT at random initialization. (a) Mean ρT for each pairing of global and local interaction modes. Cells report mean ± standard deviation over n = 5 seeds. (b) Individual seed estimates in the GAT local-interaction column, with diamonds and whi…
Figure 10
Figure 10. Figure 10: Robustness and spectral radius for twenty independently-trained MNIST ablation-matrix [PITH_FULL_IMAGE:figures/full_fig_p032_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

58 extracted references · 4 linked inside Pith

  1. [1]

    Battaglia, Razvan Pascanu, Matthew Lai, Danilo Jimenez Rezende, and Koray Kavukcuoglu

    Peter W. Battaglia, Razvan Pascanu, Matthew Lai, Danilo Jimenez Rezende, and Koray Kavukcuoglu. Interaction networks for learning about objects, relations and physics. InAd- vances in Neural Information Processing Systems (NeurIPS), volume 29, pages 4502–4510, 2016

    2016

  2. [2]

    Schoenholz, Patrick F

    Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry. InProceedings of the 34th International Con- ference on Machine Learning, volume 70 ofProceedings of Machine Learning Research, pages 1263–1272, 2017

    2017

  3. [3]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. InAdvances in Neural Infor- mation Processing Systems (NeurIPS), volume 30, pages 5998–6008, 2017

    2017

  4. [4]

    Zico Kolter, and Vladlen Koltun

    Shaojie Bai, J. Zico Kolter, and Vladlen Koltun. Deep equilibrium models. InAdvances in Neural Information Processing Systems (NeurIPS), volume 32, 2019

    2019

  5. [5]

    Implicit graph neural networks

    Fangda Gu, Heng Chang, Wenwu Zhu, Somayeh Sojoudi, and Laurent El Ghaoui. Implicit graph neural networks. InAdvances in Neural Information Processing Systems (NeurIPS), volume 33, pages 11984–11995, 2020

    2020

  6. [6]

    Rumelhart, Geoffrey E

    David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams. Learning representations by back-propagating errors.Nature, 323:533–536, 1986

    1986

  7. [7]

    Hopfield

    John J. Hopfield. Neural networks and physical systems with emergent collective computational abilities.Proceedings of the National Academy of Sciences, 79(8):2554–2558, 1982

    1982

  8. [8]

    A tutorial on energy-based learning

    Yann LeCun, Sumit Chopra, Raia Hadsell, Marc’Aurelio Ranzato, and Fu Jie Huang. A tutorial on energy-based learning. InPredicting Structured Data. MIT Press, 2006

    2006

Show all 58 references
  1. [9]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR), pages 770–778, 2016

    2016

  2. [10]

    Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David K. Duvenaud. Neural ordi- nary differential equations. InAdvances in Neural Information Processing Systems (NeurIPS), volume 31, 2018

    2018

  3. [11]

    The graph neural network model.IEEE Transactions on Neural Networks, 20(1):61–80, 2009

    Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model.IEEE Transactions on Neural Networks, 20(1):61–80, 2009

    2009

  4. [12]

    Hauck, and Bao Wang

    Justin Baker, Qingsong Wang, Cory D. Hauck, and Bao Wang. Implicit graph neural networks: A monotone operator viewpoint. InProceedings of the 40th International Conference on Ma- chine Learning, volume 202 ofProceedings of Machine Learning Research, pages 1521–1548, 2023

    2023

  5. [13]

    Zico Kolter

    Shaojie Bai, Vladlen Koltun, and J. Zico Kolter. Stabilizing equilibrium models by jacobian regularization.arXiv preprint arXiv:2106.14342, 2021

    2021 arXiv

  6. [14]

    Zico Kolter

    Shaojie Bai, Vladlen Koltun, and J. Zico Kolter. Multiscale deep equilibrium models. In Advances in Neural Information Processing Systems (NeurIPS), volume 33, 2020. 42

    2020

  7. [15]

    Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst

    Michael M. Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geometric deep learning: Going beyond Euclidean data.IEEE Signal Processing Magazine, 34(4):18–42, 2017

    2017

  8. [16]

    Kipf and Max Welling

    Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. InInternational Conference on Learning Representations (ICLR), 2017

    2017

  9. [17]

    Graph attention networks

    Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. InInternational Conference on Learning Rep- resentations (ICLR), 2018

    2018

  10. [18]

    On the bottleneck of graph neural networks and its practical implications

    Uri Alon and Eran Yahav. On the bottleneck of graph neural networks and its practical implications. InInternational Conference on Learning Representations (ICLR), 2021

    2021

  11. [19]

    Botvinick, Andrew Zisserman, Oriol Vinyals, and João Carreira

    Andrew Jaegle, Sebastian Borgeaud, Jean-Baptiste Alayrac, Carl Doersch, Catalin Ionescu, David Ding, Skanda Koppula, Daniel Zoran, Andrew Brock, Evan Shelhamer, Olivier Hénaff, Matthew M. Botvinick, Andrew Zisserman, Oriol Vinyals, and João Carreira. Perceiver IO: A general ar...

    2022

  12. [20]

    Graph perceiver IO: A general architecture for graph structured data.arXiv preprint arXiv:2209.06418, 2022

    Seyun Bae, Hoyoon Byun, Changdae Oh, Yoon-Sik Cho, and Kyungwoo Song. Graph perceiver IO: A general architecture for graph structured data.arXiv preprint arXiv:2209.06418, 2022

    2022 arXiv

  13. [21]

    Wilson and Jack D

    Hugh R. Wilson and Jack D. Cowan. Excitatory and inhibitory interactions in localized pop- ulations of model neurons.Biophysical Journal, 12(1):1–24, 1972

    1972

  14. [22]

    Dynamics of pattern formation in lateral-inhibition type neural fields.Bio- logical Cybernetics, 27(2):77–87, 1977

    Shun-ichi Amari. Dynamics of pattern formation in lateral-inhibition type neural fields.Bio- logical Cybernetics, 27(2):77–87, 1977

    1977

  15. [23]

    Mikail Khona and Ila R. Fiete. Attractor and integrator networks in the brain.Nature Reviews Neuroscience, 23:744–766, 2022

    2022

  16. [24]

    Ben-Yishai, R

    R. Ben-Yishai, R. L. Bar-Or, and H. Sompolinsky. Theory of orientation tuning in visual cortex.Proceedings of the National Academy of Sciences, 92(9):3844–3848, 1995

    1995

  17. [25]

    Rajesh P. N. Rao and Dana H. Ballard. Predictive coding in the visual cortex: A functional interpretationofsomeextra-classicalreceptive-fieldeffects.Nature Neuroscience, 2:79–87, 1999

    1999

  18. [26]

    Gilbert and Wu Li

    Charles D. Gilbert and Wu Li. Top-down influences on visual processing.Nature Reviews Neuroscience, 14:350–363, 2013

    2013

  19. [27]

    Irwin, Teague Sterling, Michael M

    John J. Irwin, Teague Sterling, Michael M. Mysinger, Erin S. Bolstad, and Ryan G. Coleman. ZINC: A free tool to discover chemistry for biology.Journal of Chemical Information and Modeling, 52(7):1757–1768, 2012

    2012

  20. [28]

    Joshi, Thomas Laurent, Yoshua Bengio, Xavier Bresson, and Anh Tuan Luu

    Vijay Prakash Dwivedi, Chaitanya K. Joshi, Thomas Laurent, Yoshua Bengio, Xavier Bresson, and Anh Tuan Luu. Benchmarking graph neural networks.Journal of Machine Learning Research, 24(43):1–48, 2023

    2023

  21. [29]

    Collective classification in network data.AI Magazine, 29(3):93–106, 2008

    Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi- Rad. Collective classification in network data.AI Magazine, 29(3):93–106, 2008. 43

    2008

  22. [30]

    Cohen, and Ruslan Salakhutdinov

    Zhilin Yang, William W. Cohen, and Ruslan Salakhutdinov. Revisiting semi-supervised learn- ing with graph embeddings. InProceedings of the 33rd International Conference on Machine Learning, volume 48 ofProceedings of Machine Learning Research, pages 40–48, 2016

    2016

  23. [31]

    Youcef Saad and Martin H. Schultz. GMRES: A generalized minimal residual algorithm for solving nonsymmetric linear systems.SIAM Journal on Scientific and Statistical Computing, 7(3):856–869, 1986

    1986

  24. [32]

    Gradient-based learning applied to document recognition.Proceedings of the IEEE, 86(11):2278–2324, 1998

    Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition.Proceedings of the IEEE, 86(11):2278–2324, 1998

    1998

  25. [33]

    Sur les opérations dans les ensembles abstraits et leur application aux équations intégrales.Fundamenta Mathematicae, 3:133–181, 1922

    Stefan Banach. Sur les opérations dans les ensembles abstraits et leur application aux équations intégrales.Fundamenta Mathematicae, 3:133–181, 1922

    1922

  26. [34]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009

    2009

  27. [35]

    Lei Jimmy Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization.arXiv preprint arXiv:1607.06450, 2016

    2016 arXiv

  28. [36]

    A clockwork rnn

    Jan Koutník, Klaus Greff, Faustino Gomez, and Jürgen Schmidhuber. A clockwork rnn. In Proceedings of the 31st International Conference on Machine Learning, volume32ofProceedings of Machine Learning Research, pages 1863–1871, 2014

    2014

  29. [37]

    Hierarchical recurrent neural networks for long-term depen- dencies

    Salah El Hihi and Yoshua Bengio. Hierarchical recurrent neural networks for long-term depen- dencies. InAdvances in Neural Information Processing Systems (NeurIPS), volume 8, 1995

    1995

  30. [38]

    Anderson

    Donald G. Anderson. Iterative procedures for nonlinear integral equations.Journal of the ACM, 12(4):547–560, 1965

    1965

  31. [39]

    Walker and Peng Ni

    Homer F. Walker and Peng Ni. Anderson acceleration for fixed-point iterations.SIAM Journal on Numerical Analysis, 49(4):1715–1735, 2011

    2011

  32. [40]

    Batch normalization: Accelerating deep network training by reducing internal covariate shift

    Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. InProceedings of the 32nd International Conference on Machine Learning, volume 37 ofProceedings of Machine Learning Research, pages 448–456, 2015

    2015

  33. [41]

    Dropout: A simple way to prevent neural networks from overfitting.Journal of Machine Learning Research, 15(56):1929–1958, 2014

    Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting.Journal of Machine Learning Research, 15(56):1929–1958, 2014

    1929

  34. [42]

    Sarma, Michael M

    Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E. Sarma, Michael M. Bronstein, and Justin M. Solomon. Dynamic graph CNN for learning on point clouds.ACM Transactions on Graphics, 38(5):146, 2019

    2019

  35. [43]

    On the difficulty of training recurrent neural networks

    Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. On the difficulty of training recurrent neural networks. InProceedings of the 30th International Conference on Machine Learning, volume 28 ofProceedings of Machine Learning Research, pages 1310–1318, 2013

    2013

  36. [44]

    Lyapunov-stable deep equi- librium models

    Haoyu Chu, Shikui Wei, Ting Liu, Yao Zhao, and Yuto Miyatake. Lyapunov-stable deep equi- librium models. InProceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 11615–11623, 2024. 44

    2024

  37. [45]

    Real-time computation at the edge of chaos in recurrent neural networks.Neural Computation, 16(7):1413–1436, 2004

    Nils Bertschinger and Thomas Natschläger. Real-time computation at the edge of chaos in recurrent neural networks.Neural Computation, 16(7):1413–1436, 2004

    2004

  38. [46]

    Ex- ponential expressivity in deep neural networks through transient chaos

    Ben Poole, Subhaneil Lahiri, Maithra Raghu, Jascha Sohl-Dickstein, and Surya Ganguli. Ex- ponential expressivity in deep neural networks through transient chaos. InAdvances in Neural Information Processing Systems (NeurIPS), volume 29, 2016

    2016

  39. [47]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. InInterna- tional Conference on Learning Representations (ICLR), 2015

    2015

  40. [48]

    Spectral normaliza- tion for generative adversarial networks

    Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normaliza- tion for generative adversarial networks. InInternational Conference on Learning Representa- tions (ICLR), 2018

    2018

  41. [49]

    Stable architectures for deep neural networks.Inverse Prob- lems, 34(1), 2017

    Eldad Haber and Lars Ruthotto. Stable architectures for deep neural networks.Inverse Prob- lems, 34(1), 2017

    2017

  42. [50]

    Jolliffe.Principal Component Analysis

    Ian T. Jolliffe.Principal Component Analysis. Springer Series in Statistics. Springer, 2 edition, 2002

    2002

  43. [51]

    Visualizing data using t-SNE.Journal of Machine Learning Research, 9(86):2579–2605, 2008

    Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE.Journal of Machine Learning Research, 9(86):2579–2605, 2008

    2008

  44. [52]

    Rousseeuw

    Peter J. Rousseeuw. Silhouettes: A graphical aid to the interpretation and validation of cluster analysis.Journal of Computational and Applied Mathematics, 20:53–65, 1987

    1987

  45. [53]

    Davies and Donald W

    David L. Davies and Donald W. Bouldin. A cluster separation measure.IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-1(2):224–227, 1979

    1979

  46. [54]

    How powerful are graph neural networks? InInternational Conference on Learning Representations (ICLR), 2019

    Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? InInternational Conference on Learning Representations (ICLR), 2019

    2019

  47. [55]

    Principal neighbourhood aggregation for graph nets

    Gabriele Corso, Luca Cavalleri, Dominique Beaini, Pietro Liò, and Petar Veličković. Principal neighbourhood aggregation for graph nets. InAdvances in Neural Information Processing Systems (NeurIPS), volume 33, 2020

    2020

  48. [56]

    Residual gated graph convnets.arXiv preprint arXiv:1711.07553, 2017

    Xavier Bresson and Thomas Laurent. Residual gated graph convnets.arXiv preprint arXiv:1711.07553, 2017

    2017 arXiv

  49. [57]

    Deeper insights into graph convolutional networks for semi-supervised learning

    Qimai Li, Zhichao Han, and Xiao-Ming Wu. Deeper insights into graph convolutional networks for semi-supervised learning. InProceedings of the AAAI Conference on Artificial Intelligence, volume 32, pages 3538–3545, 2018

    2018

  50. [58]

    Reversible deep equilibrium models.arXiv preprint arXiv:2509.12917, 2025

    Sam McCallum, Kamran Arora, and James Foster. Reversible deep equilibrium models.arXiv preprint arXiv:2509.12917, 2025. 45

    2025

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.