Pith. sign in

REVIEW 3 major objections 4 minor 59 references

A Theoretical Analysis of Compositional Generalization in Neural Networks: A Necessary and Sufficient Condition

T0 review · 3 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A model generalizes compositionally exactly when three conditions hold: structural alignment, unambiguous representation, and minimized representation.

desk verdict Theorem 1 is a definitional unpacking: the reference graph set already contains correct test predictions, so the iff does most of its work by assumption; the paper is honest, but the central claim is not an independent result. read the letter →

arxiv 2505.02627 v1 pith:D33WY4OV submitted 2025-05-05 cs.LG cs.AI

Yuanpeng Li This is my paper · ORCID
classification cs.LGcs.AI
keywords compositionalgeneralizationnecessaryandsufficientconditionstructuralalignmentunambiguousrepresentationminimizedreferencegraphsetneuralnetworksinductivebias
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper aims to settle when a trained neural network can handle novel combinations of already-known pieces: it claims the network does so exactly when three conditions hold. The network's computational graph must match the true compositional hierarchy of the task; each component's output must map unambiguously onto the reference representation; and each component must use the fewest distinct output values needed. The proof reasons over a reference graph set, a set of component-level computations that already gives correct predictions and contains every test component input in training. The authors demonstrate the conditions on a tiny XOR-based example and argue they can be used to architect models and devise regularization before training.

What carries the argument

The load-bearing object is the reference graph set $Z$: a collection of computational graphs, one per sample, whose nodes carry the true intermediate values of the task, with correct outputs on every sample and every test component input appearing in training. The proof mechanism is a node-wise mapping from the model's hypothesis graph to this reference graph. Lemma 1 says a well-defined, onto mapping between finite sets is one-to-one exactly when the domain has minimal size $(|A| = |B|$ for a minimal domain$)$, so unambiguous representation plus minimized representation collapses into a one-to-one mapping between hypothesis and reference values in training. An induction over the hierarchical order of the graph then carries this one-to-one condition from input nodes to the output, proving correct test predictions. The minimal example instantiates this machinery with a manually structured two-stage XOR network trained with a regularization that reduces the number of distinct hidden values.

What would settle it

Train a simple network on a two-component task in which the test set recombines the components in a way that introduces a component-input value absent from training, even though the component itself is a known straightforward function. Concretely, in the paper's XOR setup, withhold all training samples with $(x_1, x_2) = (0,1)$ while keeping both input values present at each position separately, then test on that pair; correct compositional prediction there would falsify Assumption 1 and with it the necessity half of Theorem 1.

Watch

Extended reading notes

Core claim

The central claim is Theorem 1: a model enables compositional generalization if and only if it has structural alignment, unambiguous representation, and minimized representation, where each property is defined against a reference graph set. Structural alignment means the hypothesis graph has the same shape as the reference graph for every sample. Unambiguous representation means equal hidden values in training imply equal reference values; minimized representation means each component has no more distinct training outputs than the reference needs. The necessity direction assumes that all test component inputs are seen in training; the sufficiency direction is proved by induction over the graph hierarchy, using a pigeonhole lemma that turns well-defined onto node mappings into bijections when the domain is minimized. A minimal example with three binary inputs and XOR gates illustrates all three conditions.

Load-bearing premise

Assumption 1, that every test component input must have appeared in training, is load-bearing for the necessity half of Theorem 1; the paper defends it only informally through gradient compression, and if a network can generalize to a truly unseen component input, the necessity proof fails.

Editorial extensions

If this is right

  • Any successful compositional system must embed the true compositional hierarchy in its architecture; if the graph shape diverges from the reference graph, generalization fails no matter how the weights are trained.
  • Regularization that compresses each component's distinct outputs, for instance by adding structured noise and an activity penalty, is not an optional trick but a necessary trigger of compositional behavior under the theorem.
  • The theorem gives a pre-training checklist: architecture matching the task hierarchy, data with all component inputs already seen, and training designed so that equal hidden values never split across reference meanings.
  • Because the conditions concern the trained model, the theorem predicts which trained models will generalize without needing to run the full test distribution: inspect graph alignment and the number of distinct values each component produces.
  • For tasks whose true compositional structure is unknown, the theorem says the structure must be inferred or imposed before compositional generalization becomes possible.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the necessity direction inherits its force from Assumption 1, that test component inputs must be seen in training; an experiment showing reliable generalization to a genuinely unseen component input would require weakening the necessity claim.
  • Editorial extension: because the reference graph set is constructed to already give correct outputs, the three conditions function as a per-component certificate: check each node's graph alignment and distinct-value counts, and behavior on recombinations is determined without running the test set.
  • Editorial extension: the same certificate suggests a tractable design heuristic for modular networks: after training, compare each module's number of distinct outputs with the number of distinct reference values it should encode, and add compression or split modules to close the gap.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper claims a necessary and sufficient condition for compositional generalization in neural networks. It defines compositional generalization as correct test predictions whenever training predictions are correct (Definition 5), then states Theorem 1: a model enables compositional generalization if and only if it has structural alignment with a reference graph set (Definition 6), unambiguous representation (Definition 7), and minimized representation (Definition 8). The proof has a necessity direction that invokes Assumption 1 (all test component inputs are seen in training), and a sufficiency direction that inductively transfers values from a reference graph set to the hypothesis graph set. A minimal XOR task and a discussion of the SCAN jump task are given as illustrations.

Significance. If the claimed theorem were a genuine independent characterization, it would be a valuable design principle tying architecture, regularization, and data to compositional generalization. The manuscript is honest that the main contribution is mathematical and that the experiments are illustrative rather than confirmatory. It also has the virtue of a readable proof structure and a reproducible minimal example. However, the central claim is not established: the reference graph set in Definition 4 already requires correct predictions on all test samples, and in the necessity direction the reference graph set is taken to be the hypothesis set itself. The three conditions therefore carry no independent content beyond Assumption 1, which is asserted rather than proved. The paper is best read as a definitional unpacking of 'seen test component inputs plus a correct reference computation graph,' not as a derivation of a governing condition for compositional generalization.

major comments (3)
  1. [Section 3.2 / Proposition 2] The sufficiency direction is circular with respect to Definition 4. Proposition 2 and its proof in Appendix A.3 assume the existence of a reference graph set Z whose defining property 1 is correct predictions on all of D, including all test samples. The induction in Lemma 14 only establishes that the hypothesis value h_B equals the reference value z_B at test nodes; since z_B already yields the correct output by construction, the target output is assumed rather than derived from the three conditions. The proof also uses Definition 4 property 2 as the source of training reference inputs for each test node, so Assumption 1 is effectively imported into the sufficiency argument as well.
  2. [Section 3.1 / Proposition 1] The necessity direction is vacuous. In Appendix A.2, Lemma 9 asserts that under Definition 5 and Assumption 1 the hypothesis graph set H is itself a reference graph set, and Proposition 1 then sets Z = H. With Z = H, structural alignment is the identity isomorphism, unambiguous representation becomes h_A = h_C implies h_A = h_C, and minimized representation holds because every node is in bijection with itself. Consequently any model that already satisfies Definition 5 and Assumption 1 automatically satisfies all three conditions; the conditions impose no independent constraint in the necessity direction.
  3. [Section 2.3 / Sections 5.4-5.6] Assumption 1 is the only substantive load-bearing premise, and it is not proved. Section 2.3 introduces it as an assumption justified by a qualitative gradient-based optimization argument, Section 5.4 supplies only an informal compression story, and Section 5.5 explicitly states that the necessity of structural alignment derives from Assumption 1. Section 5.6 concedes the point by moving Assumption 1 into Definition 11, so Theorem 2 is also a definitional unpacking rather than a proof that the three conditions govern compositional generalization. As written, Theorem 1 restates the seen-test-component-inputs assumption instead of deriving a necessary and sufficient condition.
minor comments (4)
  1. [Section 2.1] The notation paragraph overloads the symbol h for a node, for its value, and for its input vector; on first reading it is difficult to distinguish h, h, and h_i. A typed notation or a small table of symbols would improve clarity.
  2. [Appendix A.3, Lemma 11] The onto property of the mapping from hypothesis values to reference values is asserted from the construction of the reference graph set, but structural alignment alone does not define a node-value mapping. This step needs a proof or an explicit definition of the node-level mapping induced by graph isomorphism.
  3. [Appendix E, Assumption 2] The SCAN analysis relies on the unproved assumption that when representations are minimized, all action words have equal hypothesis syntax embeddings. This is a task-specific stipulation, so the SCAN discussion does not independently test the theoretical conditions.
  4. [Section 5.7] The claim that the conditions can be used to assess compositional generalization before training is not operationalized: for unambiguous representation the text says it 'currently requiring case-by-case analysis,' which effectively concedes that no general pre-training check is provided.

Circularity Check

3 steps flagged · score 8.0 of 10

Theorem 1 is a definitional unpacking: the conditions are defined relative to a reference graph set that already has correct test predictions, and the necessity half chooses H itself as the reference set.

  1. self definitional [Section 3.1 (Necessity), Proposition 1; Appendix A.2, Lemma 9 and Proposition 1 proof]
    "By Lemma 9, the hypothesis graph set is a reference graph set. We set H itself as its reference graph set Z = H. Since Z is H, they have structural alignment (Definition 6), and all nodes have bijective mappings (well-defined, onto, and one-to-one) to themselves. So, H has unambiguous representation (Definition 7)."

    Lemma 9 derives 'H is a reference graph set' from Definition 5 (correct training predictions imply correct test predictions) and Assumption 1 (test component inputs seen in training). By Definition 4, a reference graph set already has correct predictions on all D, including test. With Z = H, structural alignment is the identity H ~= H, unambiguous representation is h_A = h_C => z_A = z_C with z = h, and minimized representation reduces to |A| = |B| with A = B. Thus the three conditions impose no independent constraint: any model satisfying Definition 5 and Assumption 1 satisfies them by construction. The necessity direction is a relabeling, not a derived characterization.

  2. self definitional [Section 3.2 (Sufficiency), Proposition 2 with Definition 4 and Definition 6]
    "Definition 4 (Reference Graph Set). A reference graph set Z is a graph set with the following properties. 1. All graphs have correct predictions. ∀A∈D : ˆY A = Y A ... Definition 6 (Structural Alignment). A hypothesis graph set H has structural alignment property if it has structural alignment with a reference graph set. ∃Z∈ Z :H∼=Z"

    Structural alignment is defined as the existence of a Z in Z, and every Z in Z already computes correct outputs on every sample in D, including test. Sufficiency therefore assumes an oracle reference graph with correct test predictions and proves only that H's node values coincide with that graph on test nodes (Lemma 14 and Proposition 2). The hard part of compositional generalization — producing a correct test output without being handed a correct reference graph — is contained in the existential quantifier of Definition 6. The sufficiency proof is an unpacking of 'reference graph set': if a correct aligned graph exists and H matches it, H is correct.

1 more flagged steps
  1. other [Section 5.6, Definition 11 and Theorem 2]
    "Under this definition, Assumption 1 (the necessity of seen test component inputs) becomes an integral part of the definition rather than a separate premise. ... Theorem 2 (Alternative Necessary and Sufficient Condition). A model enables alternative compositional generalization (Definition 11) if and only if it has structural alignment (Definition 6), unambiguous representation (Definition 7), and minimized representation (Definition 8)."

    This is the paper's own concession that the only substantive premise of the necessity direction is Assumption 1: once the seen-inputs requirement is moved into the definition of compositional generalization, the same necessary-and-sufficient theorem holds unchanged. Consequently Theorem 1 reduces to 'H is a reference graph set' under Assumption 1, and the three named conditions are restatements of that fact rather than independent governing conditions.

full rationale

The central claim is circular in both directions. On the necessity side, Proposition 1 sets Z = H after Lemma 9 has shown H is a reference graph set, so structural alignment, unambiguous representation, and minimized representation hold by construction and carry no independent content. Section 5.5 concedes: 'The necessity of structural alignment derives from Assumption 1'. On the sufficiency side, the conditions only require existence of a reference graph set that already has correct predictions on all D including test; the induction transfers those correct predictions to H. The remaining substantive premise is Assumption 1, asserted rather than proved and supported only by the informal gradient-compression discussion in Section 5.4; Section 5.6 admits that folding it into the definition preserves the theorem. The minimal example and ablation are illustrative rather than independent evidence, and the citation to the author's prior work (Li et al. 2019) is motivational, not load-bearing for the proof. Score 8 rather than 10 because the sufficiency induction and the one-to-one mapping lemma contain some formal content, but the theorem as stated is forced by the definitions.

Assumptions & free parameters 1 free parameters · 4 assumptions · 1 invented entities

The theorem's non-trivial load is carried by Assumption 1 and by the reference graph set construction. The reference graph set is an invented constructive object whose properties include correct test predictions, which makes the sufficiency proof circular in spirit. The minimization condition is not defined against any fixed target, and the SCAN treatment adds an unproved assumption (Assumption 2).

free parameters (1)
  • Regularization noise scale alpha and penalty beta = alpha=0.1, beta=0.1
    Chosen by hand in the minimal example (Appendix D) to induce the minimized representation condition; values are not learned and no sensitivity analysis is given.
assumptions (4)
  • domain assumption Assumption 1: A model enables compositional generalization only if all test component inputs are seen in training.
    Load-bearing for the necessity direction; justified only by informal gradient-compression arguments in Sections 2.3 and 5.4, not by proof. The paper acknowledges it may fail in exceptional cases.
  • ad hoc to paper Definition 4's reference graph set exists and has correct predictions on all test samples.
    Structural alignment is defined as existence of an isomorphic Z in the set of all reference graph sets; this gives the sufficiency proof a graph that already knows the correct test outputs, making the result close to circular.
  • standard math Pigeonhole principle and elementary set cardinality facts.
    Used in Lemma 1 and Appendix A.1.
  • ad hoc to paper Assumption 2 (Appendix E): When representations are minimized, all action words have equal hypothesis syntax embeddings in the SCAN algorithm.
    This unproved assumption is what makes the SCAN analysis satisfy the theorem; it assumes the hard part of the behavior.
invented entities (1)
  • Reference graph set (Definition 4)
    purpose: Serves as an oracle-like graph set that already has correct predictions on all training and test samples and whose component inputs all appear in training; structural alignment is defined as isomorphism to one of these.
    It is not grounded in measurable data; it bundles the target generalization into the condition. This makes the theorem's sufficiency direction a transfer of correctness from a graph that already solves the test set.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Theoretical Analysis of Compositional Generalization in Neural Networks: A Necessary and Sufficient Condition." pith.science (2026). https://pith.science/paper/D33WY4OV

@misc{pith2026250502627,
  author       = {Pith},
  title        = {Pith review of: A Theoretical Analysis of Compositional Generalization in Neural Networks: A Necessary and Sufficient Condition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D33WY4OV}},
  note         = {Machine review of arXiv:2505.02627}
}
read the original abstract

Compositional generalization is a crucial property in artificial intelligence, enabling models to handle novel combinations of known components. While most deep learning models lack this capability, certain models succeed in specific tasks, suggesting the existence of governing conditions. This paper derives a necessary and sufficient condition for compositional generalization in neural networks. Conceptually, it requires that (i) the computational graph matches the true compositional structure, and (ii) components encode just enough information in training. The condition is supported by mathematical proofs. This criterion combines aspects of architecture design, regularization, and training data properties. A carefully designed minimal example illustrates an intuitive understanding of the condition. We also discuss the potential of the condition for assessing compositional generalization before training. This work is a fundamental theoretical study of compositional generalization in neural networks.

Figures

Figures reproduced from arXiv: 2505.02627 by the authors.

Figure 1
Figure 1. An example of notations. Normal fonts (e.g., [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Visual proof of Lemma 1. Left: When |A| is minimized to match |B|, the mapping becomes bijec￾tive. Right: When |A′ | is not minimized, multiple el￾ements in A′ map to the same element in B, violat￾ing one-to-one mapping. This illustrates that one-to-one mapping is equivalent to minimal domain size under well-defined onto mappings. 3 [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Illustrative proof of the induction step [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: The algorithm for the SCAN jump task. Syn [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 41 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address author booktitle chapter doi edition editor eid howpublished institution isbn issn journal key month note number organization pages publisher school series title type url volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Abadi, M.; Agarwal, A.; Barham, P.; Brevdo, E.; Chen, Z.; Citro, C.; Corrado, G. S.; Davis, A.; Dean, J.; Devin, M.; Ghemawat, S.; Goodfellow, I.; Harp, A.; Irving, G.; Isard, M.; Jia, Y.; Jozefowicz, R.; Kaiser, L.; Kudlur, M.; Levenberg, J.; Man\' e , D.; Monga, R.; Moore, S.; Murray, D.; Olah, C.; Schuster, M.; Shlens, J.; Steiner, B.; Sutskever, I.; T...

    work page 2015

  4. [4]

    Ahuja, K.; Mahajan, D.; Wang, Y.; and Bengio, Y. 2023. Interventional causal representation learning. In International conference on machine learning, 372--407. PMLR

    work page 2023

  5. [5]

    Ahuja, K.; and Mansouri, A. 2024. On Provable Length and Compositional Generalization. In ICML 2024 Workshop on Theoretical Foundations of Foundation Models. ://openreview.net/forum?id=xuwtmXiHMT

    work page 2024

  6. [6]

    Aky \"u rek, E.; and Andreas, J. 2023. LexSym: Compositionality as lexical symmetry. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 639--657

    work page 2023

  7. [7]

    Andreas, J.; Rohrbach, M.; Darrell, T.; and Klein, D. 2016. Neural module networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, 39--48

    work page 2016

  8. [8]

    Bengio, Y. 2017. The consciousness prior. arXiv preprint arXiv:1709.08568

    arXiv 2017

Show all 59 references
  1. [9]

    R.; Lachapelle, S.; Bilaniuk, O.; Goyal, A.; and Pal, C

    Bengio, Y.; Deleu, T.; Rahaman, N.; Ke, N. R.; Lachapelle, S.; Bilaniuk, O.; Goyal, A.; and Pal, C. 2020. A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms. In International Conference on Learning Representations. ://openreview.net/forum?id=ryxWIgBFPS

    2020

  2. [10]

    Brady, J.; von Kügelgen, J.; Lachapelle, S.; Buchholz, S.; Kipf, T.; and Brendel, W. 2025. Interaction Asymmetry: A General Principle for Learning Composable Abstractions. In International Conference on Learning Representations (ICLR). ://openreview.net/forum?id=cCl10IU836. Po...

    2025

  3. [11]

    M.; Frith, C.; Ji, X.; et al

    Butlin, P.; Long, R.; Elmoznino, E.; Bengio, Y.; Birch, J.; Constant, A.; Deane, G.; Fleming, S. M.; Frith, C.; Ji, X.; et al. 2023. Consciousness in artificial intelligence: insights from the science of consciousness. arXiv preprint arXiv:2308.08708

    2023 arXiv

  4. [12]

    Chomsky, N. 1957. Syntactic structures. Walter de Gruyter

    1957

  5. [13]

    M.; and Thomas, J

    Cover, T. M.; and Thomas, J. A. 2012. Elements of Information Theory. Hoboken, NJ: John Wiley & Sons, Inc., 2nd edition. ISBN 978-1-118-58577-1

    2012

  6. [14]

    Dasgupta, I.; Wang, J.; Chiappa, S.; Mitrovic, J.; Ortega, P.; Raposo, D.; Hughes, E.; Battaglia, P.; Botvinick, M.; and Kurth-Nelson, Z. 2019. Causal Reasoning from Meta-reinforcement Learning. arXiv preprint arXiv:1901.08162

    2019 arXiv

  7. [15]

    DeepSeek-AI; Guo, D.; Yang, D.; Zhang, H.; Song, J.; Zhang, R.; Xu, R.; Zhu, Q.; Ma, S.; Wang, P.; Bi, X.; Zhang, X.; et al. 2025. DeepSeek - R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv preprint arXiv:2501.12948

    2025 arXiv

  8. [16]

    Diestel, R. 2025. Graph Theory. Springer, sixth edition. ISBN 9783662701065

    2025

  9. [17]

    L.; Jiang, L.; Lin, B

    Dziri, N.; Lu, X.; Sclar, M.; Li, X. L.; Jiang, L.; Lin, B. Y.; Welleck, S.; West, P.; Bhagavatula, C.; Bras, R. L.; Hwang, J. D.; Sanyal, S.; Ren, X.; Ettinger, A.; Harchaoui, Z.; and Choi, Y. 2023. Faith and Fate: Limits of Transformers on Compositionality. In Thirty-seventh...

    2023

  10. [18]

    Elmoznino, E.; Jiralerspong, T.; Bengio, Y.; and Lajoie, G. 2025. A Complexity-Based Theory of Compositionality. ://arxiv.org/abs/2410.14817

    2025 arXiv

  11. [19]

    A.; and Pylyshyn, Z

    Fodor, J. A.; and Pylyshyn, Z. W. 1988. Connectionism and cognitive architecture: A critical analysis. Cognition 28(1-2): 3--71

    1988

  12. [20]

    Fu, J.; Zhang, Z.; Lu, Y.; and Zheng, N. 2024. A General Theory for Compositional Generalization. arXiv preprint arXiv:2405.11743

    2024 arXiv

  13. [21]

    Goyal, A.; and Bengio, Y. 2022. Inductive biases for deep learning of higher-level cognition. Proceedings of the Royal Society A 478(2266): 20210068

    2022

  14. [22]

    R.; Rahaman, N.; Binas, J.; Blundell, C.; Mozer, M.; and Bengio, Y

    Goyal, A.; Didolkar, A.; Lamb, A.; Badola, K.; Ke, N. R.; Rahaman, N.; Binas, J.; Blundell, C.; Mozer, M.; and Bengio, Y. 2021 a . Coordination Among Neural Modules Through a Shared Global Workspace

    2021

  15. [23]

    Goyal, A.; Lamb, A.; Hoffmann, J.; Sodhani, S.; Levine, S.; Bengio, Y.; and Sch \"o lkopf, B. 2021 b . Recurrent Independent Mechanisms. In International Conference on Learning Representations. ://openreview.net/forum?id=mLcmdlEUxy-

    2021

  16. [24]

    Higgins, I.; Amos, D.; Pfau, D.; Racaniere, S.; Matthey, L.; Rezende, D.; and Lerchner, A. 2018. Towards a definition of disentangled representations. arXiv preprint arXiv:1812.02230

    2018 arXiv

  17. [25]

    Higgins, I.; Matthey, L.; Pal, A.; Burgess, C.; Glorot, X.; Botvinick, M.; Mohamed, S.; and Lerchner, A. 2017. beta-vae: Learning basic visual concepts with a constrained variational framework. In International Conference on Learning Representations, volume 3

    2017

  18. [26]

    Hinton , G. E. 1990. Mapping part-whole hierarchies into connectionist networks. Artificial Intelligence 46(1): 47--75

    1990

  19. [27]

    Jarvis, D.; Klein, R.; Rosman, B.; and Saxe, A. M. 2023. On The Specialization of Neural Modules. In The Eleventh International Conference on Learning Representations. ://openreview.net/forum?id=Fh97BDaR6I

    2023

  20. [28]

    J.; Taylor, C

    Jiang, B.; Xie, Y.; Hao, Z.; Wang, X.; Mallick, T.; Su, W. J.; Taylor, C. J.; and Roth, D. 2024. A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasoners. In Al-Onaizan, Y.; Bansal, M.; and Chen, Y.-N., eds., Proceedings of the 2024 Conference on Empirical M...

    2024 doi

  21. [29]

    Keysers, D.; Sch \"a rli, N.; Scales, N.; Buisman, H.; Furrer, D.; Kashubin, S.; Momchev, N.; Sinopalnikov, D.; Stafiniak, L.; Tihon, T.; Tsarkov, D.; Wang, X.; van Zee, M.; and Bousquet, O. 2020. Measuring Compositional Generalization: A Comprehensive Method on Realistic Data...

    2020

  22. [30]

    Kim, N.; and Linzen, T. 2020. COGS : A Compositional Generalization Challenge Based on Semantic Interpretation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 9087--9105. Online: Association for Computational Linguistics. doi...

    2020 doi

  23. [31]

    Lake, B.; and Baroni, M. 2018. Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks. In International Conference on Machine Learning, 2873--2882

    2018

  24. [32]

    M.; and Baroni, M

    Lake, B. M.; and Baroni, M. 2023. Human-like systematic generalization through a meta-learning neural network. Nature 623: 115--121. doi:10.1038/s41586-023-06668-3

    2023 doi

  25. [33]

    Li, Y.; Zhao, L.; Wang, J.; and Hestness, J. 2019. Compositional Generalization for Primitive Substitutions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMN...

    2019

  26. [34]

    Lippe, P.; Cohen, T.; and Gavves, E. 2022. Efficient Neural Causal Discovery without Acyclicity Constraints. In International Conference on Learning Representations. ://openreview.net/forum?id=eYciPrLuUhG

    2022

  27. [35]

    Lippl, S.; and Stachenfeld, K. 2025. When does compositional structure yield compositional generalization? A kernel theory. In The Thirteenth International Conference on Learning Representations. ://openreview.net/forum?id=FPBce2P1er

    2025

  28. [36]

    C.; and Bengio, Y

    Liu, D.; Lamb, A.; Kawaguchi, K.; Goyal, A.; Sun, C.; Mozer, M. C.; and Bengio, Y. 2021. Discrete-Valued Neural Communication. In Beygelzimer, A.; Dauphin, Y.; Liang, P.; and Vaughan, J. W., eds., Advances in Neural Information Processing Systems. ://openreview.net/forum?id=YS...

    2021

  29. [37]

    Marcus, G. F. 1998. Rethinking eliminative connectionism. Cognitive psychology 37(3): 243--282

    1998

  30. [38]

    I.; Alizadeh, K.; Shahrokhi, H.; Tuzel, O.; Bengio, S.; and Farajtabar, M

    Mirzadeh, S. I.; Alizadeh, K.; Shahrokhi, H.; Tuzel, O.; Bengio, S.; and Farajtabar, M. 2025. GSM -Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models. In The Thirteenth International Conference on Learning Representations. ://openreview....

    2025

  31. [39]

    Mittal, S.; Bengio, Y.; and Lajoie, G. 2022. Is a Modular Architecture Enough? In Koyejo, S.; Mohamed, S.; Agarwal, A.; Belgrave, D.; Cho, K.; and Oh, A., eds., Advances in Neural Information Processing Systems, volume 35, 28747--28760. Curran Associates, Inc

    2022

  32. [40]

    Montague, R. 1970. Universal grammar. Theoria 36(3): 373--398

    1970

  33. [41]

    OpenAI. 2023. GPT-4 Technical Report

    2023

  34. [42]

    Painter, M.; Prugel-Bennett, A.; and Hare, J. 2020. Linear Disentangled Representations and Unsupervised Action Estimation. In Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M. F.; and Lin, H., eds., Advances in Neural Information Processing Systems, volume 33, 13297--13307...

    2020

  35. [43]

    Pearl, J. 2003. CAUSALITY: MODELS, REASONING, AND INFERENCE. Cambridge university press

    2003

  36. [44]

    Pearl, J. 2009. Causality. Cambridge university press

    2009

  37. [45]

    Ram, P.; Klinger, T.; and Gray, A. G. 2024. What Makes Models Compositional? A Theoretical View. In Larson, K., ed., Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24 , 4824--4832. International Joint Conferences on Artificial ...

    2024 doi

  38. [46]

    Rebman, K. R. 1979. The Pigeonhole Principle (What It Is, How It Works, and How It Applies to Map Coloring). The Two-Year College Mathematics Journal 10(1): 3--13. ISSN 00494925. ://www.jstor.org/stable/3026807

    1979

  39. [47]

    Russin, J.; Jo, J.; and O'Reilly, R. C. 2019. Compositional generalization in a deep seq2seq model by separating syntax and semantics. arXiv preprint arXiv:1904.09708

    2019 arXiv

  40. [48]

    R.; Kalchbrenner, N.; Goyal, A.; and Bengio, Y

    Sch \"o lkopf, B.; Locatello, F.; Bauer, S.; Ke, N. R.; Kalchbrenner, N.; Goyal, A.; and Bengio, Y. 2021. Toward causal representation learning. Proceedings of the IEEE 109(5): 612--634

    2021

  41. [49]

    Schug, S.; Kobayashi, S.; Akram, Y.; Sacramento, J.; and Pascanu, R. 2025. Attention as a Hypernetwork. In The Thirteenth International Conference on Learning Representations. ://openreview.net/forum?id=V4K9h1qNxE

    2025

  42. [50]

    M.; Oswald, J

    Schug, S.; Kobayashi, S.; Akram, Y.; Wolczyk, M.; Proca, A. M.; Oswald, J. V.; Pascanu, R.; Sacramento, J.; and Steger, A. 2024. Discovering modular solutions that generalize compositionally. In The Twelfth International Conference on Learning Representations. ://openreview.ne...

    2024

  43. [51]

    Shi, K.; Hong, J.; Deng, Y.; Yin, P.; Zaheer, M.; and Sutton, C. 2024. ExeDec: Execution Decomposition for Compositional Generalization in Neural Program Synthesis. In The Twelfth International Conference on Learning Representations. ://openreview.net/forum?id=oTRwljRgiv

    2024

  44. [52]

    Soulos, P.; Conklin, H.; Opper, M.; Smolensky, P.; Gao, J.; and Fernandez, R. 2024. Compositional Generalization Across Distributional Shifts with Sparse Tree Operations. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. ://openreview.net/forum?i...

    2024

  45. [53]

    Tishby, N.; and Zaslavsky, N. 2015. Deep learning and the information bottleneck principle. In 2015 IEEE Information Theory Workshop ( ITW ) , 1--5. doi:10.1109/ITW.2015.7133169

    2015

  46. [54]

    Vaishnav, M.; and Serre, T. 2023. GAMR : A Guided Attention Model for (visual) Reasoning. In The Eleventh International Conference on Learning Representations. ://openreview.net/forum?id=iLMgk2IGNyv

    2023

  47. [55]

    N.; Kaiser, .; and Polosukhin, I

    Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, 5998--6008

    2017

  48. [56]

    Wiedemer, T.; Brady, J.; Panfilov, A.; Juhos, A.; Bethge, M.; and Brendel, W. 2024. Provable Compositional Generalization for Object-Centric Learning. In The Twelfth International Conference on Learning Representations. ://openreview.net/forum?id=7VPTUWkiDQ

    2024

  49. [57]

    Wiedemer, T.; Mayilvahanan, P.; Bethge, M.; and Brendel, W. 2023. Compositional Generalization from First Principles. In Oh, A.; Naumann, T.; Globerson, A.; Saenko, K.; Hardt, M.; and Levine, S., eds., Advances in Neural Information Processing Systems, volume 36, 6941--6960. C...

    2023

  50. [58]

    Wu, B.; Fang, J.; Zeng, X.; Liang, S.; and Zhang, Q. 2023. Adaptive Compositional Continual Meta-Learning. In Krause, A.; Brunskill, E.; Cho, K.; Engelhardt, B.; Sabato, S.; and Scarlett, J., eds., Proceedings of the 40th International Conference on Machine Learning, volume 20...

    2023

  51. [59]

    Xu, Z.; Niethammer, M.; and Raffel, C. A. 2022. Compositional Generalization in Unsupervised Compositional Representation Learning: A Study on Disentanglement and Emergent Language. In Koyejo, S.; Mohamed, S.; Agarwal, A.; Belgrave, D.; Cho, K.; and Oh, A., eds., Advances in N...

    2022

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.