Pith. sign in

REVIEW 2 major objections 4 minor 1 cited by

Logic and the $2$-Simplicial Transformer

T0 review · 2 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A Transformer extended with attention over triples of entities outperforms the ordinary Transformer in a deep reinforcement learning environment whose puzzles encode the tensor connective of linear logic.

desk verdict A genuinely novel attention mechanism with sound math, but the empirical comparison does not isolate the 2-simplicial head; without a matched-capacity control the central causal claim remains unestablished. read the letter →

arxiv 1909.00668 v1 pith:ZBTZDYGO submitted 2019-09-02 cs.LG cs.LOstat.ML

James Clift , Dmitry Doryn , Daniel Murfet , James Wallbridge This is my paper · ORCID
classification cs.LGcs.LOstat.ML
keywords 2-simplicialTransformerhigher-dimensionalattentionscalartripleproducttensor-productvaluevectorsinductivebiaslogicalreasoningdeepreinforcementlearningbridgeBoxWorld
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper extends the Transformer block so that it can attend not just to pairs of entities but to triples, with attention logits given by the scalar triple product of three vectors and messages built from tensor products of value vectors. It then embeds this block in a deep reinforcement learning agent and compares it against the ordinary Transformer agent in bridge BoxWorld, a puzzle where the Gem has two locks and opening a 'bridge' box makes the puzzle unsolvable. The central claim is that in this environment the 2-simplicial Transformer block confers a real advantage: the simplicial agent reaches a higher win rate than the relational agent. The logic framing is that an episode's solvable structure corresponds to a proof in a fragment of linear logic containing the tensor connective, so the paper presents the 2-simplicial block as an inductive bias for logical structure. The paper explicitly leaves open whether the agent is doing logical reasoning internally.

What carries the argument

The load-bearing object is the 2-simplicial attention head, a second attention channel in which every query entity $i$ distributes weight over pairs of key entities $(j,k)$ using the unsigned scalar triple product $\langle p_i, \ell^1_j, \ell^2_k \rangle$—the magnitude of a vector combination of three vectors, measuring how close the three vectors are to being linearly dependent—and updates $i$ with a learned linear map $B$ applied to the tensor product $u_j \otimes u_k$ of value vectors. Because enumerating all triples is $O(N^3)$, the paper restricts the pairs $(j,k)$ to a small set of $M=2$ virtual entities, extra learned vectors updated by the ordinary 1-simplicial attention, so the ordinary attention decides which entities can form 2-simplices and the whole block stays at $O(N^2)$ complexity. The scalar triple product plays the role of the dot product in ordinary attention: zero when the three vectors are pairwise orthogonal, maximal when they are linearly dependent, which the paper interprets as the degree of predicted existence of a 2-simplex with vertices $(i,j,k)$.

What would settle it

Train the relational agent with the same number of parameters, the same two virtual entities, and the same total heads but with only ordinary 1-simplicial attention; if its win rate matches the simplicial agent's, then the 2-simplicial attention is not the cause of the advantage. Conversely, ablating the 2-simplicial head in the simplicial agent and observing no drop in win rate would also falsify the claim.

Watch

Extended reading notes

Core claim

The paper's central discovery is an architecture and an experimental result: replacing the ordinary Transformer block inside a deep RL agent with a 2-simplicial Transformer block—which adds a second attention mode over ordered pairs $(j,k)$ with logits $\langle p_i, \ell^1_j, \ell^2_k \rangle$ and message $B(u_j \otimes u_k)$—produces a higher win rate than the ordinary baseline in bridge BoxWorld. Across four independent trials, the simplicial agent's mean win rate rises above the relational agent's and does not fall back after an initial plateau around a 0.85 win rate. Attention analysis shows the trained agent uses the 2-simplicial attention in the second Transformer block: most query entities attend generically to a common pair of virtual entities, while a few task-relevant entities, such as the two locks on the Gem, the player, and inventory entities, use non-generic attention that sends products of value vectors to specific targets. The paper reads the tensor product as a conjunction in the sense of linear logic, but finds the attention structure is not a homomorphic image of a logically correct explicit strategy, so the question of internal logical reasoning is left open.

Load-bearing premise

The paper's conclusion rests on attributing the higher win rate to the 2-simplicial attention head, but the simplicial agent also has two extra virtual entities and a larger model, and no matched-capacity control was run.

Editorial extensions

If this is right

  • If the central claim is right, 2-simplicial attention is a useful inductive bias for tasks whose underlying logic has triadic or conjunctive structure, not just pairwise relations.
  • The architecture keeps the efficiency of ordinary Transformers by restricting 2-simplices to a small set of virtual entities, so the advantage does not require $O(N^3)$ scaling in practice.
  • The paper's attention analysis suggests the extra head is actually used: task-relevant entities receive attention over pairs of virtual entities whose tensor products carry information about conjunctions of facts.
  • A clear planning consequence is that the agent can learn to avoid bridge boxes in bridge BoxWorld, where ordinary relational attention plateaus at a lower win rate.
  • The definition generalises naturally to $n$-simplicial Transformer blocks via $n$-fold scalar products, so the same idea extends to higher-arity logical connectives.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension not run in the paper is a capacity-matched control: a relational agent with the same parameter count and two extra virtual entity slots would separate the benefit of 2-simplicial attention from the benefit of a larger model; the paper's stated interpretation would be supported only if the ordinary agent still lags.
  • The virtual entities act as a bounded scratch pad, and the 2-simplicial head reads products of their states; one would expect the advantage to grow on puzzles requiring more than two premises as $M$ increases, though the quadratic message complexity would eventually dominate.
  • If the scalar triple product is understood as a volume, the 2-simplicial attention may also be relevant to spatial or geometric reasoning tasks where three-vector configurations carry meaning beyond pairwise similarities, although the paper does not test this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes a 2-simplicial Transformer block, an extension of the Transformer in which entity updates include higher-order attention over pairs of virtual entities, with attention logits given by the unsigned scalar triple product and messages formed by a learned tensor product B(u_j⊗u_k). The construction is developed using Clifford-algebra volume formulas, and the paper defines a bridge BoxWorld environment whose solve condition is phrased in linear logic (with the connectives ⊸ and ⊗). The block is embedded in an IMPALA deep reinforcement learning agent, and the paper reports that the simplicial agent reaches a higher mean win rate than a relational agent on bridge BoxWorld (Figure 3), with time-adjusted curves in Appendix C. The paper also analyses attention maps from a representative run, concluding that the agent has learned to use the 2-simplicial attention, while explicitly leaving open whether the agent performs logical reasoning.

Significance. The mathematical formalization is a genuine contribution: Lemma A.10 and Equation (32) provide a clean Clifford-algebra account of the scalar triple product, and the architecture is specified in sufficient detail to be reproduced. The paper also follows good practice by releasing code, trained agent weights, and training notebooks, and the time-adjusted comparison in Appendix C addresses the wall-clock confound. If a controlled experiment shows that the 2-simplicial head itself causes the win-rate gap, this would establish a novel inductive bias with a concrete demonstration environment. At present, however, the experimental design does not separate the new attention mechanism from the added virtual entities and added capacity, and the analysis section is explicitly anecdotal; this weakens the central causal claim.

major comments (2)
  1. [Section 4.2 / Figure 3] The comparison in Figure 3 does not isolate the 2-simplicial attention. The simplicial agent differs from the relational agent in three ways: it adds two virtual entities (N=42 vs. N=40 in Eqs. (18)-(19)), it adds a 2-simplicial head with a learned tensor B and additional weight matrices, and it widens the first feedforward layer by concatenating the 64-dimensional 1-simplicial output with a 48-dimensional 2-simplicial output. Because the virtual entities are updated by ordinary attention before the 2-simplicial head reads them, the observed win-rate gap could be caused by the extra memory-like entities, by the larger model, or by favorable optimization dynamics rather than by the scalar-triple-product attention itself. A matched-capacity control is needed, such as a relational agent with the same two virtual entities and comparable width, or a simplicial agent with the 2-simplicial head ablated; without such a control the causal claim stated in Section 1 is not established.
  2. [Section 6.1.2] The sentence claiming that 'the output of the 2-simplicial head is the only nontrivial difference between the simplicial and relational agent' is inconsistent with Section 4.2, which also adds virtual entities, the B-tensor, an additional layer normalization, and a wider feedforward input. The following sentence, that the performance differences in Figure 3 'suggest that this output is informative,' is therefore an attribution rather than a conclusion forced by the data. The attention analysis in this section is based on one best run and is described by the authors as anecdotal, so it does not close the gap left by the missing ablation.
minor comments (4)
  1. [Equation (18)] Equation (18) contains an indexing typo in the denominator, where `l,m` appear where `s,t` are intended, and the summation range over all standard entities is inconsistent with the numerator's restriction to virtual pairs; please correct or explicitly justify the normalization window.
  2. [Section 5 / Figure 3] Figure 3 would be easier to interpret if the per-trial win-rate gap and confidence intervals on the difference were reported, since the phrase 'clear advantage' is otherwise a visual judgment based on only four runs of each agent.
  3. [Abstract / Section 7] The abstract's phrase 'useful inductive bias for logical reasoning' is broader than the demonstrated result; Section 6 explicitly leaves the question of logical reasoning open, so the claim should be qualified to an environment with linear-logical structure.
  4. [Section 6.2] The per-puzzle analysis is reported for only a small sample of puzzle types; a full per-type table would help the reader see whether the simplicial advantage is concentrated in particular puzzle types or is uniform across the task distribution.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the 2-simplicial attention mechanism is defined from geometric algebra and evaluated against an external environment, not derived from the experimental outcome it predicts.

full rationale

The paper's central claim is an empirical comparison: the 2-simplicial Transformer block is said to confer an advantage over the ordinary Transformer block on bridge BoxWorld (Section 1, Figure 3). The 2-simplicial attention is defined independently in Section 2.2 via the scalar triple product and Clifford algebra (Definition 2.5, Equation (15)), with no parameter fitted to the experimental win rates or to the logical structure of the environment. The environment is an external benchmark (bridge BoxWorld, Section 3.2), and the comparison with the relational agent is a direct empirical evaluation rather than a renamed restatement of the architecture. The only self-citation, [14] (Clift and Murfet on differential linear logic), is used as background motivation for tensor-product semantics and is not load-bearing for the experimental claim; the paper also cites prior literature for tensor-product networks (e.g., [60]) rather than relying on its own prior work. The analysis in Section 6 is explicitly anecdotal and states that the logical-reasoning interpretation is inconclusive, which is an honest limitation rather than a circular move. The skeptic's concern that the simplicial agent differs in virtual entities and parameter count is a confound in causal attribution of the advantage to the 2-simplicial head; this is a correctness or experimental-design issue, not a circularity in the derivation, and it does not make the central empirical claim equivalent to its inputs by construction. Therefore no circular step is exhibited, and the appropriate circularity score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 1 invented entities

The paper introduces no physical entities or fitted constants; the trained weights are learned from data. The hand-chosen hyperparameters (M, dim(H2), number of heads) are architecture choices that affect the result but are not fitted to achieve a target. The linear-logic interpretation is an unverified domain assumption, and the geometric attention function is an ad hoc design choice.

free parameters (3)
  • M = 2
    Number of virtual entities chosen by hand. The paper says taking M of order sqrt(N) gives O(N^2) complexity, but it uses M=2 for N=40 and reports no ablation over M.
  • dim(H2) = 48
    Dimension of the 2-simplicial attention space, chosen for the reported runs without an ablation study.
  • number of 2-simplicial heads = 1
    Chosen as a minimal configuration; Section 7 states multiple 2-simplicial heads were not tested.
assumptions (3)
  • domain assumption Bridge BoxWorld strategies are equivalent to proofs in intuitionistic linear logic, so success in the environment is evidence of logical reasoning ability.
    Stated in Section 1 (pages 5-6). This motivates the title and the tensor-product semantics, but the empirical comparison between agents does not depend on this equivalence.
  • ad hoc to paper The scalar triple product defined via Clifford algebra magnitude is the correct generalization of dot-product attention to triples, and it measures the degree to which entity i should attend to pair (j,k).
    This is a design choice in Definition 2.5 and Equation 15, not derived from first principles. Its effectiveness is tested empirically in only one environment.
  • ad hoc to paper The update rule B(u_j tensor u_k) models conjunction as multiplication, following Boole and Girard.
    Section 2.2 and Section 6.1.2. This is an interpretive semantic link to linear logic, not a formal requirement for the architecture to function.
invented entities (1)
  • virtual entities (e_{N+1}, ..., e_{N+M})
    purpose: A scratch pad onto which ordinary attention writes representations; the 2-simplicial attention only forms triples whose base (j,k) are virtual entities, reducing complexity from O(N^3) to O(NM^2).
    Introduced in Section 2.2 and analyzed in Section 6.1.2. They are engineering devices with no external or falsifiable evidence outside the architecture.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Logic and the $2$-Simplicial Transformer." pith.science (2026). https://pith.science/paper/ZBTZDYGO

@misc{pith2026190900668,
  author       = {Pith},
  title        = {Pith review of: Logic and the $2$-Simplicial Transformer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZBTZDYGO}},
  note         = {Machine review of arXiv:1909.00668}
}
abstract

We introduce the $2$-simplicial Transformer, an extension of the Transformer which includes a form of higher-dimensional attention generalising the dot-product attention, and uses this attention to update entity representations with tensor products of value vectors. We show that this architecture is a useful inductive bias for logical reasoning in the context of deep reinforcement learning.

Figures

Figures reproduced from arXiv: 1909.00668 by the authors.

Figure 1
Figure 1. [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Right: a sample episode of the bridge BoxWorld environment, in which the Gem has two locks and there is a marked bridge. Left: graph representation of the puzzle, with upper and lower solutions paths and the bridge between them. episode of bridge BoxWorld contain a single formula of the form X1 ⊗ X2 ( G where x1, x2 are the colours of the keys on the Gem, and again a strategy is equivalent to a proof of Γ ` G. In co… view at source ↗
Figure 3
Figure 3. Training curve of mean relational and simplicial agents on bridge BoxWorld. [PITH_FULL_IMAGE:figures/full_fig_p017_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Training curves for the relational agent on bridge BoxWorld. [PITH_FULL_IMAGE:figures/full_fig_p018_4.png]
Figure 5
Figure 5. Figure 5: Training curves for the simplicial agent on bridge BoxWorld. [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]
Figure 6
Figure 6. Figure 6: Visualisation of 1-simplicial attention in first Transformer block, between stan [PITH_FULL_IMAGE:figures/full_fig_p020_6.png]
Figure 7
Figure 7. Figure 7: The 1-simplicial attention of the virtual entities in the first iteration (first and [PITH_FULL_IMAGE:figures/full_fig_p021_7.png]
Figure 8
Figure 8. Figure 8: Visualisation of the 2-simplicial attention in the second Transformer block in [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]
Figure 9
Figure 9. Figure 9: Visualisation of the 2-simplicial attention in the second Transformer block in [PITH_FULL_IMAGE:figures/full_fig_p023_9.png]
Figure 10
Figure 10. Figure 10: Simplicial and relational agent win rate on puzzle types (1 [PITH_FULL_IMAGE:figures/full_fig_p024_10.png]
Figure 11
Figure 11. Figure 11: Simplicial and relational agent win rate on puzzle types (3 [PITH_FULL_IMAGE:figures/full_fig_p025_11.png]
Figure 12
Figure 12. Figure 12: Training curve of mean relational and simplicial agents on bridge BoxWorld, [PITH_FULL_IMAGE:figures/full_fig_p033_12.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Fast and Simplex: 2-Simplicial Attention in Triton

    cs.LG 2025-07 conditional novelty 6.0 of 10

    2-simplicial attention, implemented in Triton with a sliding window, is claimed to yield a steeper loss-versus-parameters scaling exponent than dot-product attention on math and reasoning benchmarks.

Reference graph

Works this paper leans on

69 extracted references · 60 canonical work pages · cited by 1 Pith paper

  1. [1]

    Alain and Y

    G. Alain and Y. Bengio, Understanding intermediate layers using linear classifier probes, in Proceedings of the International Conference on Learning Representations (ICLR), 2016. 35

    work page 2016

  2. [2]

    Aristotle, Sophistical refutations , translated by W. A. Pickard-Cambridge, Com- plete works of Aristotle, Volume 1: The revised Oxford translation , J. Barnes (ed.) Princeton University Press, 1984

    work page 1984

  3. [3]

    Lei Ba, J

    J. Lei Ba, J. R. Kiros and G. E. Hinton, Layer normalization , preprint arXiv:1607.06450, 2016

    arXiv 2016

  4. [4]

    Bahdanau, K

    D. Bahdanau, K. Cho and Y. Bengio, Neural machine translation by jointly learning to align and translate , in Proceedings of the International Conference on Learning Representations (ICLR), 2015

    work page 2015

  5. [5]

    D. G. T. Barrett, F. Hill, A. Santoro, A. S. Morcos and T. Lillicrap, Measuring ab- stract reasoning in neural networks, in Proceedings of the International Conference on Machine Learning (ICML), 2018

    work page 2018

  6. [6]

    T. E. J. Behrens, T. H. Muller, J. C. R. Whittington, S. Mark, A. B. Baram, K. L. Stachenfeld and Z. Kurth-Nelson, What is a cognitive map? Organizing knowledge for flexible behavior , Neuron 100, pp. 490–509, 2018

    work page 2018

  7. [7]

    J. L. S. Bellmund, P. Gardenfors, E. I. Moser and C. F. Doeller, Navigating cogni- tion: Spatial codes for human thinking , Science 362, 2018

    work page 2018

  8. [8]

    J. C. W. Billings, M. Hu, G. Lerda, A. N. Medvedev, F. Mottes, A. Onicas, A. San- toro and G. Petri, Simplex2Vec embeddings for community detection in simplicial complexes, preprint arXiv:1906.09068, 2019

    arXiv 1906

Show all 69 references
  1. [9]

    Boole, The Mathematical Analysis of Logic: Being an Essay Towards a Calculus of Deductive Reasoning, Cambridge University Press, 2009 (first published 1847)

    G. Boole, The Mathematical Analysis of Logic: Being an Essay Towards a Calculus of Deductive Reasoning, Cambridge University Press, 2009 (first published 1847)

    2009

  2. [10]

    Brockman, V

    G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang and W. Zaremba, OpenAI Gym, preprint arXiv:1606.01540, 2016

    2016 arXiv

  3. [11]

    Chazal and B

    F. Chazal and B. Michel, An introduction to topological data analysis: fundamental and practical aspects for data scientists , preprint arXiv:1710.04019, 2017

    2017 arXiv

  4. [12]

    Child, S

    R. Child, S. Gray, A. Radford and I. Sutskever, Generating long sequences with Sparse Transformers, preprint arXiv:1904.10509, 2019

    1904 arXiv

  5. [13]

    K. Cho, A. Courville and Y. Bengio, Describing multimedia content using attention- based encoder-decoder networks, in IEEE Transactions on Multimedia 17(11), 2015

    2015

  6. [14]

    Clift and D

    J. Clift and D. Murfet, Cofree coalgebras and differential linear logic , preprint arXiv:1701.01285v2, 2019

    2019 arXiv

  7. [15]

    Clift, D

    J. Clift, D. Doryn, D. Murfet and J. Wallbridge, https://github.com/dmurfet/ 2simplicialtransformer/. 36

  8. [16]

    A. O. Constantinescu, J. X. O’Reilly and T. E. J. Behrens, Organising conceptual knowledge in humans with a gridlike code , Science 352, Issue 6292, pp. 1464–1468, 2016

    2016

  9. [17]

    Dehghani, S

    M. Dehghani, S. Gouws, O. Vinyals, J. Uszkoreit and L. Kaiser, Universal trans- formers, in Proceedings of the International Conference on Learning Representations (ICLR), 2019

    2019

  10. [18]

    H. Dong, J. Mao, T. Lin, C. Wang, L. Li and D. Zhou, Neural logic machines , in Proceedings of the International Conference on Learning Representations (ICLR), 2019

    2019

  11. [19]

    R. A. Epstein, E. Z. Patai, J. B. Julian and H. J. Spiers, The cognitive map in humans: spatial navigation and beyond , Nature Neuroscience, 2017

    2017

  12. [20]

    Espeholt, H

    L. Espeholt, H. Soyer, R. Munos, K. Simonyan, V. Mnih, T. Ward, Y. Doron, V. Firoiu, T. Harley, I. Dunning, S. Legg and K. Kavukcuoglu, IMPALA: scalable distributed deep-RL with importance weighted actor-learner architectures, preprint arXiv:1802.01561, 2018

    2018 arXiv

  13. [21]

    Frege, On sense and denotation ( ¨Uber Sinn und Bedeutung) , Zeitschrift f¨ ur Philosophie und philosophische Kritik, vol

    G. Frege, On sense and denotation ( ¨Uber Sinn und Bedeutung) , Zeitschrift f¨ ur Philosophie und philosophische Kritik, vol. 100 pp. 25–50, 1892

  14. [22]

    C. L. Giles, G. Z. Sun, H. H. Chen, Y. C. Lee and D. Chen, Higher order recurrent networks and grammatical inference, in Advances in Neural Information Processing Systems (NeurIPS), 1989

    1989

  15. [23]

    C. L. Giles, D. Chen, C. B. Miller, H. H. Chen, G. Z. Sun and Y. C. Lee, Second- order recurrent neural networks for grammatical inference , in IJCNN-91-Seattle International Joint Conference on Neural Networks Vol. 2, pp. 273–281 1991

    1991

  16. [24]

    Gilmer, S

    J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals and G. E. Dahl, Neural message passing for quantum chemistry , in Proceedings of the International Conference on Machine Learning (ICML), 2017

    2017

  17. [25]

    J. -Y. Girard, Linear logic, Theoretical Computer Science 50 (1), pp. 1–102, 1987

    1987

  18. [26]

    Goodfellow, Y

    I. Goodfellow, Y. Bengio and A. Courville, Deep learning, MIT Press, 2016

    2016

  19. [27]

    M. W. Goudreau, C. L. Giles, S. T. Chakradhar and D. Chen, First-order versus second-order single-layer recurrent neural networks , IEEE Transactions on Neural Networks, 5(3), pp. 511–513, 1994

    1994

  20. [28]

    Graves, G

    A. Graves, G. Wayne and I. Danihelka, Neural Turing machines , preprint arXiv:1410.5401, 2014. 37

    2014 arXiv

  21. [29]

    Graves, G

    A. Graves, G. Wayne, M. Reynolds, T. Harlye, I. Danihelka, A. Grabska-Barwin´ ska, S. G. Colmenarejo, E. Grefenstette, T. Ramalho, J. Agapiou, A. P. Badia, K. M. Hermann, Y. Zwols, G. Ostrovski, A. Cain, H. King, C. Summerfield, P. Blun- som, K. Kavukcuoglu and D. Hassabis, Hyb...

    2016

  22. [30]

    A. Guez, M. Mirza, K. Gregor, R. Kabra, S. Racanie` ere, T. Weber, D. Raposo, A. Santoro, L. Orseau, T. Eccles, G. Wayne, D. Silver and T. Lillicrap, An inves- tigation of model-free planning , in Proceedings of the International Conference on Machine Learning (ICML), 2019

    2019

  23. [31]

    Henderson, R

    P. Henderson, R. Islam, P. Bachman, J. Pineau, D. Precup and D. Meger, Deep reinforcement learning that matters , preprint arXiv:1709.06560, 2017

    2017 arXiv

  24. [32]

    Hestenes, New foundations for classical mechanics, 2nd edition, Kluwer Academic publishers 2002

    D. Hestenes, New foundations for classical mechanics, 2nd edition, Kluwer Academic publishers 2002

    2002

  25. [33]

    Hewitt and C

    J. Hewitt and C. D. Manning, A structural probe for finding syntax in word repre- sentations, in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp....

    2019

  26. [34]

    J. J. Hopfield, Neural networks and physical systems with emergent collective com- putational abilities, Proceedings of the National Academy of Sciences of the USA, vol. 79 no. 8 pp. 2554–2558, 1982

    1982

  27. [35]

    Hyland, Game semantics, Semantics and logics of computation 14, 1997

    M. Hyland, Game semantics, Semantics and logics of computation 14, 1997

    1997

  28. [36]

    Irsoy and C

    O. Irsoy and C. Cardie, Modeling compositionality with multiplicative recurrent neural networks, in Proceedings of the International Conference on Learning Rep- resentations (ICLR), 2014

    2014

  29. [37]

    Jaderberg, V

    M. Jaderberg, V. Dalibard, S. Osindero, W. M. Czarnecki, J. Donahue, A. Razavi, O. Vinyals, T. Green, I. Dunning, K. Simonyan and C. Fernando, Population based training of neural networks, preprint arXiv:1711.09846, 2017

    2017 arXiv

  30. [38]

    P. W. Koh and P. Liang, Understanding Black-box predictions via influence func- tions, in Proceedings of the International Conference on Machine Learning (ICML), 2017

    2017

  31. [39]

    Lambiotte, M

    R. Lambiotte, M. Rosvall and I. Scholtes, From networks to optimal higher-order models of complex systems , Nature physics, 15 pp. 313–320, 2019

    2019

  32. [40]

    Liang, R

    E. Liang, R. Liaw, R. Nishihara, R. Moritz, R. Fox, K. Goldberg, J. E. Gonzalez, M. I. Jordan and I. Stoica, RLlib: Abstractions for distributed reinforcement learn- ing, in Proceedings of the International Conference on Machine Learning (ICML), 2018. 38

    2018

  33. [41]

    Y. Liu, R. J. Dolan, Z. Kurth-Nelson and T. E. J. Behrens, Human replay sponta- neously reorganizes experience , Cell, 178 pp. 1–13, 2019

    2019

  34. [42]

    Macdonald, Sobczyks simplicial calculus does not have a proper foundation , preprint arXiv:1710.08274, 2017

    A. Macdonald, Sobczyks simplicial calculus does not have a proper foundation , preprint arXiv:1710.08274, 2017

    2017 arXiv

  35. [43]

    Mackay, Information theory, inference and learning algorithms , Cambridge Uni- versity Press, 2003

    D. Mackay, Information theory, inference and learning algorithms , Cambridge Uni- versity Press, 2003

    2003

  36. [44]

    N. J. Mackintosh, Animal learning , Encylopaedia Britannica, https:// www.britannica.com/science/animal-learning/Insight-and-reasoning , ac- cess date July 15, 2019

    2019

  37. [45]

    Martens, Programming interactive worlds with linear logic, Doctoral dissertation, Carnegie Mellon University, 2015

    C. Martens, Programming interactive worlds with linear logic, Doctoral dissertation, Carnegie Mellon University, 2015

    2015

  38. [46]

    Mavreshko (kpot), keras-transformer, https://github.com/kpot/ keras-transformer

    K. Mavreshko (kpot), keras-transformer, https://github.com/kpot/ keras-transformer

  39. [47]

    P-A. Melli` es,Categorical semantics of linear logic, in : Interactive models of compu- tation and program behaviour, Panoramas et Synth` eses 27, Soci´ et´ e Math´ ematique de France, 2009

    2009

  40. [48]

    Nguyen, J

    A. Nguyen, J. Yosinski and J. Clune, Multifaceted feature visualization: uncovering the different types of features learned by each neuron in deep neural networks , in International Conference on Machine Learning (ICML), 2016

    2016

  41. [49]

    Olah and S

    C. Olah and S. Carter, Attention and Augmented Recurrent Neural Networks, Dis- till, http://distill.pub/2016/augmented-rnns, 2016

    2016

  42. [50]

    J. B. Pollack, The induction of dynamical recognizers, Machine Learning, 7(2-3) pp. 227–252, 1991

    1991

  43. [51]

    Ranzato, A

    M. Ranzato, A. Krizhevsky and G. E. Hinton.Factored 3-Way Restricted Boltzmann machines for modeling natural images , AISTATS, 2010

    2010

  44. [52]

    Santoro, D

    A. Santoro, D. Raposo, D. G. Barrett, M. Malinowski, R. Pascanu, P. Battaglia and T. Lillicrap, A simple neural network module for relational reasoning , in Advances in Neural Information Processing Systems (NeurIPS), 2017

    2017

  45. [53]

    Santoro, R

    A. Santoro, R. Faulkner, D. Raposo, J. Rae, M. Chrzanowski, T. Weber, D. Wier- stra, O. Vinyals, R. Pascanu and T. Lillicrap, Relational recurrent neural networks, in Advances in Neural Information Processing Systems (NeurIPS), 2018

    2018

  46. [54]

    Raposo, Personal communication, May 13, 2019

    D. Raposo, Personal communication, May 13, 2019. 39

    2019

  47. [55]

    Serafini and A

    L. Serafini and A. Garcez, Logic tensor networks: Deep learning and logical rea- soning from data and knowledge , in Proceedings of the International Workshop on Neural-Symbolic Learning and Reasoning (NeSy@HLAI), 2016

    2016

  48. [56]

    Shrikumar, P

    A. Shrikumar, P. Greenside and A. Kundaje, Learning important features through propagating activation differences , in Proceedings of the International Conference on Machine Learning (ICML), 2017

    2017

  49. [57]

    Simonyan, A

    K. Simonyan, A. Vedaldi and A. Zisserman, Deep inside convolutional networks: visualising image classification models and saliency maps , in Proceedings of the International Conference on Learning Representations (ICLR), 2013

    2013

  50. [58]

    Smith, Aristotles Logic , The Stanford Encyclopedia of Philosophy (Summer 2019 Edition), Edward N

    R. Smith, Aristotles Logic , The Stanford Encyclopedia of Philosophy (Summer 2019 Edition), Edward N. Zalta (ed.), https://plato.stanford.edu/archives/ sum2019/entries/aristotle-logic/, 2019

    2019

  51. [59]

    G. E. Sobczyk Simplicial calculus with geometric algebra , in: Micali A., Boudet R., Helmstetter J. (eds) Clifford Algebras and their Applications in Mathematical Physics. Fundamental Theories of Physics, vol 47. Springer, Dordrecht, 1992

    1992

  52. [60]

    Socher, D

    R. Socher, D. Chen, C. D. Manning, and A. Ng, Reasoning with neural tensor net- works for knowledge base completion, in Advances in Neural Information Processing Systems (NeurIPS), 2013

    2013

  53. [61]

    P. V. Spade and J. J. Hintikka, History of logic, Encylopaedia Britannica, https:// www.britannica.com/topic/history-of-logic/Aristotle, access date July 15, 2019

    2019

  54. [62]

    Sutskever, J

    I. Sutskever, J. Martens and G. E. Hinton, Generating text with recurrent neu- ral networks, in Proceedings of the International Conference on Machine Learning (ICML), 2011

    2011

  55. [63]

    R. S. Sutton and A. G. Barto, Reinforcement learning: an introduction, 2nd edition, MIT Press, 2018

    2018

  56. [64]

    Szegedy, V

    C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens and Z. Wojna,Rethinking the Inception Architecture for Computer Vision , Conference on Computer Vision and Pattern Recognition (CVPR), 2016

    2016

  57. [65]

    Tieleman and G

    T. Tieleman and G. Hinton, Lecture 6.5 - RmsProp: Divide the gradient by a running average of its recent magnitude, COURSERA: Neural Networks for Machine Learning, 2012

    2012

  58. [66]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser and I. Polosukhin, Attention is all you need , in Advances in Neural Information Processing Systems (NeurIPS), 2017. 40

    2017

  59. [67]

    Vinyals, I

    O. Vinyals, I. Babuschkin, J. Chung, M. Mathieu, M. Jaderberg, W. Czarnecki, A. Dudzik, A. Huang, P. Georgiev, R. Powell, T. Ewalds, D. Horgan, M. Kroiss, I. Danihelka, J. Agapiou, J. Oh, V. Dalibard, D. Choi, L. Sifre, Y. Sulsky, S. Vezh- nevets, J. Molloy, T. Cai, D. Budden,...

    2019

  60. [68]

    J. C. R. Whittington, T. H. Muller, S. Mark, C. Barry and T. E. J. Behrens, Generalisation of structural knowledge in the hippocampal-entorhinal system , in Advances in Neural Information Processing Systems (NeurIPS), 2018

    2018

  61. [69]

    Zambaldi, D

    V. Zambaldi, D. Raposo, A. Santoro, V. Bapst, Y. Li, I. Babuschkin, K. Tuyls, D. Reichert, T. Lillicrap, E. Lockhart, M. Shanahan, V. Langston, R. Pascanu, M. Botvinick, O. Vinyals and P. Battaglia, Deep reinforcement learning with rela- tional inductive biases, in Proceedings...

    2019

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.