Pith.

Computation and Language · arXiv:2609.13144 v1 · submitted 11 Sep 2026 · 72 references

Type Diversity Enables Transformers to Generalise Compositionally

Anssi Moisio, Mathias Creutz, Mikko Kurimo

PDF · arXiv · The abstract, sentence by sentence · Cite

The paper says

“Compositional generalisation has been divided into lexical and structural generalisation. Previous work has found that structural generalisation is harder than lexical for Transformers.”

more of the abstract

We propose that this difference is not inherent to Transformers, but due to the high diversity of lexical types and low diversity of structural types in the specific datasets of these previous works. By type diversity we mean the number of different constructors of that type, instead of, for example, the specific word combinations that might populate the structure. To test this, we vary the amounts of type diversity of lexical and structural types in previously published datasets. We create linguistically diverse variants of the COGS and SLOG datasets using Grammatical Framework. We find that type diversity correlates with compositional generalisation equally in lexical and structural test cases, supporting our hypothesis. We note a contradiction with the proposition in previous work that compound divergence explains the difficulty in compositional generalisation tasks. We further investigate the effects of other dataset properties on compositional generalisation, such as the diversity of types other than the novel test structure, and surface properties of the logical semantics format.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 14 Sep 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

No verdict was given on this paper.

3 major objections · 2 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 72 references say.

The paper says, at Abstract

The review did not quote this passage.

Pith says: Objection 1 of 3 · major

The central claim that 'type diversity enables Transformers to generalise compositionally' (implied causal language in the title) is supported only by correlational evidence. The abstract reports that type diversity 'correlates with compositional generalisation equally', but correlation does not establish causation.

more of objection 1

The experimental design, as described, does not rule out alternative explanations: varying the number of constructors via Grammatical Framework likely also changes surface form diversity, syntactic tree depth, lexical frequency distributions, or training set size. Without evidence that these confounds are controlled or isolated, the headline causal inference is unsupported.

The paper says, at Abstract

The review did not quote this passage.

Pith says: Objection 2 of 3 · major

The authors note 'a contradiction with the proposition in previous work that compound divergence explains the difficulty in compositional generalisation tasks'. This contradiction is presented as a finding, but it also raises a concern: if the current dataset manipulation was intended to vary type diversity, the fact that it contradicts a well-known prior explanation suggests that the manipulation may have altered more than just type diversity.

more of objection 2

The authors should clarify whether this contradiction arises from a controlled comparison or from incidental differences in the generated datasets.

This review says the paper's mathematics is wrong. Pith did not check that against the paper's source.

The paper says, at Abstract

The review did not quote this passage.

Pith says: Objection 3 of 3 · major

No mention is made of replication details, such as the number of independent dataset variants, the use of multiple random seeds, or whether the observed correlation is robust across different architectural choices (e.g., layer count, embedding size). These are standard controls needed to strengthen a correlational claim. Without them, the degree of support for the hypothesis remains unclear.

2 minor objections

The paper says, at General

Pith says: minor objection 1 of 2

The title 'Type Diversity Enables Transformers to Generalise Compositionally' uses 'enables', which implies a causal mechanism. The abstract itself uses the more cautious language 'supports our hypothesis', but the title overstates the evidence. The authors should consider a more neutral phrasing such as 'Type Diversity Correlates with Compositional Generalisation in Transformers'.

The paper says, at Abstract

Pith says: minor objection 2 of 2

The definition of 'type diversity' as 'the number of different constructors of that type' is given only parenthetically. A more explicit example or formal definition would help the reader assess the metric before seeing the main experiments.

What would settle it

Select two variant datasets from the Grammatical Framework generation that have identical per-sentence length distributions, lexical frequency spectra, syntactic tree-depth distributions, and training set sizes, but differ in the number of distinct constructors for either lexical or structural types. If the compositional generalization performance (e.g., accuracy on COGS or SLOG test sets) significantly correlates with the constructor count in these matched pairs, the causal role of type diversity is strengthened; otherwise the earlier correlation was likely due to confounds.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 14 Sep 2026

  1. Reader no verdict given confidence low
  2. Skeptic agrees 20 s · 1,915 tokens read, 2,044 written
  3. Editor wrote the review in brief 11 s · 2,629 tokens read, 1,010 written
  4. Referee major revision: 3 major objections, 2 minor 17 s · 2,233 tokens read, 1,736 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Here’s the quick read on arXiv:2609.13144. The idea is genuinely novel: instead of blaming Transformers for being bad at structural generalisation, they point to low type diversity in the structural side of existing datasets. That reframes a well-known empirical pattern and is worth testing. The use of Grammatical Framework to generate controlled variants is also a reasonable methodological step.

The abstract reports a correlation between type diversity and compositional generalisation performance that holds equally for lexical and structural test cases. They also flag a contradiction with the compound-divergence explanation from prior work. That is the new result.

The problem is the causal language. The title says “enables” and the abstract says “type diversity correlates … supporting our hypothesis.” Correlation is not causation, and the stress-test concern about confounds (surface form variety, syntactic complexity, frequency distributions) is real—we cannot tell from the abstract whether the GF manipulation actually isolates type diversity or changes other dataset properties known to affect generalisation. The authors’ own note of a “contradiction with compound divergence” suggests their manipulation may not be clean either.

Without seeing the full experimental setup, controls (e.g., holding training set size, surface word diversity, tree depth constant), and ideally a causal identification strategy (e.g., instrumenting type diversity or manipulating it while tracking confounds), the central claim remains an interesting speculation. The paper would benefit from a more cautious framing and, if possible, a demonstration that the effect survives controls for the most obvious confounds.

Who is this for? Readers working on compositional generalisation or dataset design for semantic parsing. It deserves peer review because the hypothesis is timely and the counterpoint to prior work is clear, but the referee should push for stronger evidence before acceptance—in particular, showing that results hold under multiple controls or with a causal design.

My take: send it to review, but expect a big revision on the causal argument.

The strongest objection

The abstract reports a correlation between type diversity (number of distinct constructors) and compositional generalization performance, and interprets this as supporting the hypothesis that type diversity enables generalization. However, the experimental design described (varying type diversity via Grammatical Framework) may inadvertently alter other dataset properties that are known to affect compositional generalization, such as surface form diversity, syntactic tree depth, lexical frequency distributions, or training set size. The authors' own note of a 'contradiction with compound divergence' suggests that their manipulation may not isolate type diversity cleanly. Without showing that the diverse-controlled variants differ only in the number of constructors while keeping all other cognitively relevant variables constant, the correlation could be an artifact of these confounds. The headline's 'enables' implies a causal direction; the evidence presented (correlation alone) cannot establish causation—type diversity might be a marker of other underlying factors rather than a direct cause. This is the single most load-bearing concern because if the confounds are not controlled, the central claim reduces to a spurious correlation.

The reader's weakest assumption identified the same core issue: whether the dataset variants isolate type diversity from other confounds. Our stress-test sharpens this into a concrete causal concern: the abstract's language ('enables') exceeds the correlational evidence reported. Since the full text is absent, we cannot evaluate the actual controls; but the abstract alone leaves a serious gap that warrants maintaining the current low-confidence UNVERDICTED verdict. No verdict change is needed; a future reader should specifically request causal validation tests.

The referee's report

If the result holds—that type diversity, not structural versus lexical distinction per se, drives compositional generalization difficulty—it would shift the focus of dataset design and model evaluation toward controlling for constructor diversity. It would also challenge prior claims that compound divergence is the primary explanatory factor, as noted by the authors. The practical relevance for sequence-to-sequence models is high, but the strength of the claim depends entirely on whether the dataset manipulation isolates type diversity from confounding variables.

Pith's reading of the paper (a machine's summary, not the authors')

This paper argues that the observed difficulty of structural generalisation over lexical generalisation in Transformers is not an inherent limitation, but a consequence of low type diversity in structural types within existing datasets. The authors define type diversity as the number of distinct constructors for a given syntactic category. By generating linguistically diverse variants of the COGS and SLOG datasets using Grammatical Framework, they show that type diversity correlates equally with compositional generalisation performance in both lexical and structural test cases. This contradicts earlier claims that compound divergence is the primary difficulty. If correct, the work implies that controlling for type diversity can lead to better benchmarks and training strategies for compositional generalisation.

What carries the argument. Type diversity, defined as the number of different constructors for a given syntactic type (e.g., the number of different verbs, or the number of different noun phrase constructions). The Grammatical Framework is used to generate dataset variants that control for this diversity while keeping surface forms and other properties constant. The key claim is that this metric is the causally relevant factor driving compositional generalisation performance.

Core claim. The central discovery is that type diversity—operationalised as the count of distinct constructors per type—is a strong and equal predictor of compositional generalisation performance for Transformers, regardless of whether the generalisation is lexical (new words in known structures) or structural (new structures with known words). This was established by systematically varying the diversity of lexical and structural types in the COGS and SLOG datasets using the Grammatical Framework, and observing that performance tracks diversity in both cases. The finding challenges the proposition from prior work that structural generalisation is inherently harder for Transformers.

The premise it rests on. The Grammatical Framework generated dataset variants isolate type diversity as the single varying factor, cleanly separating it from surface form variation, syntactic complexity, and other confounds, and that the metric (number of distinct constructors) captures the causally relevant aspect of diversity rather than some correlated artifact.

If the paper is right (Pith's inferences, not the authors')

Future compositional generalisation benchmarks should control for type diversity across lexical and structural categories to ensure fair comparison of model architectures.

Training data augmentation strategies that increase type diversity (e.g., generating more distinct constructions for each type) may improve model generalisation to novel combinations.

The observed difficulty of structural generalisation can be reinterpreted as a data diversity issue, not an inherent model limitation.

The compound divergence hypothesis from earlier work is weakened, shifting focus to type-level diversity rather than token-level frequency.

This finding likely extends beyond Transformers to other sequence-to-sequence models, as the mechanism is dataset-driven rather than architecture-specific.

Automated measurement of type diversity could serve as a diagnostic tool to predict model generalisation gaps before expensive training runs.

If type diversity is causally relevant, then building datasets with 'diverse structures' may not be enough; one must ensure many different structural types, not just many examples of few structures.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Compositional generalisation has been divided into lexical and structural generalisation. Previous work has found that structural generalisation is harder than lexical for Transformers. We propose that this difference is not inherent to Transformers, but due to the high diversity of lexical types and low diversity of structural types in the specific datasets of these previous works. By type diversity we mean the number of different constructors of that type, instead of, for example, the specific word combinations that might populate the structure. To test this, we vary the amounts of type diversity of lexical and structural types in previously published datasets. We create linguistically diverse variants of the COGS and SLOG datasets using Grammatical Framework. We find that type diversity correlates with compositional generalisation equally in lexical and structural test cases, supporting our hypothesis. We note a contradiction with the proposition in previous work that compound divergence explains the difficulty in compositional generalisation tasks. We further investigate the effects of other dataset properties on compositional generalisation, such as the diversity of types other than the novel test structure, and surface properties of the logical semantics format.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-09-23

The authors' main claim, as the classifier picked it out 0.99

“We find that type diversity correlates with compositional generalisation equally in lexical and structural test cases, supporting our hypothesis.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.05 · states limitation 0.18 · human participants 0.02 · controlled comparison 0.28 · randomized assignment 0.02 · states quantitative result 0.06 · compares with prior results 0.90 · mentions code or data availability 0.03

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Compositional generalisation has been divided into lexical and structural generalisation. background 1.00 0.06 0.02 asserted 1.00
Previous work has found that structural generalisation is harder than lexical for Transformers. background 1.00 0.03 0.02 asserted 1.00
We propose that this difference is not inherent to Transformers, but due to the high diversity of lexical types and low diversity of structural types in the specific datasets of these previous works. contribution 0.63 0.17 0.40 asserted 0.34
By type diversity we mean the number of different constructors of that type, instead of, for example, the specific word combinations that might populate the structure. method 0.44 0.20 0.04 asserted 0.96
To test this, we vary the amounts of type diversity of lexical and structural types in previously published datasets. method 0.85 0.88 0.03 asserted 0.83
We create linguistically diverse variants of the COGS and SLOG datasets using Grammatical Framework. method 0.99 0.95 0.02 asserted 0.99
We find that type diversity correlates with compositional generalisation equally in lexical and structural test cases, supporting our hypothesis. result 0.97 0.97 0.87 asserted 1.00
We note a contradiction with the proposition in previous work that compound divergence explains the difficulty in compositional generalisation tasks. interpretation 0.60 0.50 0.06 asserted 0.95
We further investigate the effects of other dataset properties on compositional generalisation, such as the diversity of types other than the novel test structure, and surface properties of the logical semantics format. contribution 0.79 0.78 0.03 unclear 0.67

What kind of paper the classifier reports this is: empirical study 1.00 · evidence from observation or experiment 0.91

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 6,796 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (7)
Figure 1 of the paper
Figure 1. Accuracies (% ±std) in the SCAN variants per type diversities of the three hierarchical types (Imp(VP(V))). 24
Figure 2 of the paper
Figure 2. Transformer accuracy as a function of atom and compound divergences (V diversity
Figure 3 of the paper
Figure 3. Transformer accuracy as a function of atom and compound divergences (VP diver
Figure 4 of the paper
Figure 4. Transformer accuracy as a function of atom and compound divergences (Imp diver
Figure 5 of the paper
Figure 5. Transformer accuracies, trained on different training set variants of SLOG.
Figure 6 of the paper
Figure 6. Atom divergences of the SCAN variants. 47
Figure 7 of the paper
Figure 7. Compound divergences of the SCAN variants.

Figures are the authors', reproduced from the paper.

References (72)
  1. [1] Jabbar, Ahmad and Condoravdi, Cleo and Potts, Christopher. Distinguishing fair from unfair compositional generalization tasks. Findings of the Association for Computational Linguistics: EMNLP 2025. 2025. doi:10.18653/v1/2025.findings-emnlp.1133 on Pithdoi:10.18653/v1/2025.findings-emnlp.1133
  2. [2] Chen, Zhengyu and Wang, Siqi and Xiao, Teng and Wang, Yudong and Chen, Shiqi and Cai, Xunliang and He, Junxian and Wang, Jingang. Revisiting Scaling Laws for Language Models: The Role of Data Quality and Training Strategies. Proceedings of the 63rd Annual Meeting of the Associ... on Pithdoi:10.18653/v1/2025.acl-long.1163
  3. [3] Yao, Yuekun and Koller, Alexander. Simple and effective data augmentation for compositional generalization. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2... on Pithdoi:10.18653/v1/2024.naacl-long.25
  4. [4] Misra, Kanishka and Mahowald, Kyle. Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANN s. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024. doi:10.18653/v1/2024.emnlp-main.53 on Pithdoi:10.18653/v1/2024.emnlp-main.53
  5. [5] McCurdy, Kate and Soulos, Paul and Smolensky, Paul and Fernandez, Roland and Gao, Jianfeng. Toward Compositional Behavior in Neural Models: A Survey of Current Views. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024. doi:10.18653/v1/... on Pithdoi:10.18653/v1/2024.emnlp-main.524
  6. [6] Moisio, Anssi and Creutz, Mathias and Kurimo, Mikko. LLM s' morphological analyses of complex FST -generated F innish words. Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics. 2024. doi:10.18653/v1/2024.cmcl-1.21 on Pithdoi:10.18653/v1/2024.cmcl-1.21
  7. [7] Wu, Zhengxuan and Manning, Christopher D. and Potts, Christopher. R e COGS : How Incidental Details of a Logical Form Overshadow an Evaluation of Semantic Interpretation. Transactions of the Association for Computational Linguistics. 2023. doi:10.1162/tacl_a_00623 on Pithdoi:10.1162/tacl_a_00623
  8. [8] Li, Bingzhi and Donatelli, Lucia and Koller, Alexander and Linzen, Tal and Yao, Yuekun and Kim, Najoung. SLOG : A Structural Generalization Benchmark for Semantic Parsing. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023. doi:10.1865... on Pithdoi:10.18653/v1/2023.emnlp-main.194
  9. [9] Zhou, Xiang and Jiang, Yichen and Bansal, Mohit. Data Factors for Better Compositional Generalization. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023. doi:10.18653/v1/2023.emnlp-main.898 on Pithdoi:10.18653/v1/2023.emnlp-main.898
  10. [10] Sun, Kaiser and Williams, Adina and Hupkes, Dieuwke. The Validity of Evaluation Results: Assessing Concurrence Across Compositionality Benchmarks. Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL). 2023. doi:10.18653/v1/2023.conll-1.19 on Pithdoi:10.18653/v1/2023.conll-1.19
  11. [11] Akyurek, Ekin and Andreas, Jacob. L ex S ym: Compositionality as Lexical Symmetry. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023. doi:10.18653/v1/2023.acl-long.38 on Pithdoi:10.18653/v1/2023.acl-long.38
  12. [12] Yang, Jingfeng and Zhang, Le and Yang, Diyi. SUBS : Subtree Substitution for Compositional Semantic Parsing. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022. doi:10.18653/v1/20... on Pithdoi:10.18653/v1/2022.naacl-main.12
  13. [13] Qiu, Linlu and Shaw, Peter and Pasupat, Panupong and Nowak, Pawel and Linzen, Tal and Sha, Fei and Toutanova, Kristina. Improving Compositional Generalization with Latent Structure and Data Augmentation. Proceedings of the 2022 Conference of the North American Chapter of the A... on Pithdoi:10.18653/v1/2022.naacl-main.323
  14. [14] Shin, Seongjin and Lee, Sang-Woo and Ahn, Hwijeen and Kim, Sungdong and Kim, HyoungSeok and Kim, Boseop and Cho, Kyunghyun and Lee, Gichang and Park, Woomyoung and Ha, Jung-Woo and Sung, Nako. On the Effect of Pretraining Corpora on In-context Learning by a Large-scale Languag... on Pithdoi:10.18653/v1/2022.naacl-main.380
  15. [15] Yao, Yuekun and Koller, Alexander. Structural generalization is hard for sequence-to-sequence models. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022. doi:10.18653/v1/2022.emnlp-main.337 on Pithdoi:10.18653/v1/2022.emnlp-main.337
  16. [16] Jiang, Yichen and Zhou, Xiang and Bansal, Mohit. Mutual Exclusivity Training and Primitive Augmentation to Induce Compositionality. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022. doi:10.18653/v1/2022.emnlp-main.808 on Pithdoi:10.18653/v1/2022.emnlp-main.808
  17. [17] Patel, Arkil and Bhattamishra, Satwik and Blunsom, Phil and Goyal, Navin. Revisiting the Compositional Generalization Abilities of Neural Sequence Models. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 2022. do... on Pithdoi:10.18653/v1/2022.acl-short.46
  18. [18] Csord \'a s, R \'o bert and Irie, Kazuki and Schmidhuber, Juergen. The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021. doi:10.18653/v1/2021.em... on Pithdoi:10.18653/v1/2021.emnlp-main.49
  19. [19] Kim, Najoung and Linzen, Tal. COGS : A Compositional Generalization Challenge Based on Semantic Interpretation. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020. doi:10.18653/v1/2020.emnlp-main.731 on Pithdoi:10.18653/v1/2020.emnlp-main.731
  20. [20] Andreas, Jacob. Good-Enough Compositional Data Augmentation. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020. doi:10.18653/v1/2020.acl-main.676 on Pithdoi:10.18653/v1/2020.acl-main.676
show 52 more
  1. [21] Dai, Zihang and Yang, Zhilin and Yang, Yiming and Carbonell, Jaime and Le, Quoc and Salakhutdinov, Ruslan. Transformer- XL : Attentive Language Models beyond a Fixed-Length Context. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019. ... on Pithdoi:10.18653/v1/p19-1285
  2. [22] Geiger, Atticus and Cases, Ignacio and Karttunen, Lauri and Potts, Christopher. Posing Fair Generalization Tasks for Natural Language Inference. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on... on Pithdoi:10.18653/v1/d19-1456
  3. [23] Bastings, Jasmijn and Baroni, Marco and Weston, Jason and Cho, Kyunghyun and Kiela, Douwe. Jump to better conclusions: SCAN both left and right. Proceedings of the 2018 EMNLP Workshop B lackbox NLP : Analyzing and Interpreting Neural Networks for NLP. 2018. doi:10.18653/v1/W18-5407 on Pithdoi:10.18653/v1/w18-5407
  4. [24] Linzen, Tal and Dupoux, Emmanuel and Goldberg, Yoav. Assessing the Ability of LSTM s to Learn Syntax-Sensitive Dependencies. Transactions of the Association for Computational Linguistics. 2016. doi:10.1162/tacl_a_00115 on Pithdoi:10.1162/tacl_a_00115
  5. [25] Hockenmaier, Julia and Steedman, Mark. CCG bank: A Corpus of CCG Derivations and Dependency Structures Extracted from the P enn T reebank. Computational Linguistics. 2007. doi:10.1162/coli.2007.33.3.355 on Pithdoi:10.1162/coli.2007.33.3.355
  6. [26] Smith, David and Eisner, Jason. Quasi-Synchronous Grammars: Alignment by Soft Projection of Syntactic Dependencies. Proceedings on the Workshop on Statistical Machine Translation. 2006 on Pith
  7. [27] de Groote, Philippe. Towards Abstract Categorial Grammars. Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics. 2001. doi:10.3115/1073012.1073045 on PitharXiv
  8. [28] Measuring Compositionality in Representation Learning , author=. 2019 , booktitle= on Pith
  9. [29] Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks , author=. International conference on machine learning , pages=. 2018 , organization= on Pith
  10. [30] Cultural evolution creates the statistical structure of language , author=. Scientific Reports , volume=. 2024 , publisher= on Pith
  11. [31] Chan, Stephanie and Santoro, Adam and Lampinen, Andrew and Wang, Jane and Singh, Aaditya and Richemond, Pierre and McClelland, James and Hill, Felix , booktitle =. Data Distributional Properties Drive Emergent In-Context Learning in on Pith
  12. [32] Measuring Compositional Generalization: A Comprehensive Method on Realistic Data , author=. International Conference on Learning Representations , year= on Pith
  13. [33] Learning to transform natural to formal languages , author=. Proceedings of the 20th national conference on Artificial intelligence-Volume 3 , pages= on Pith
  14. [34] Learning to parse database queries using inductive logic programming , author=. Proceedings of the national conference on artificial intelligence , pages= on Pith
  15. [35] Hupkes, Dieuwke and Dankers, Verna and Mul, Mathijs and Bruni, Elia , journal=. Compositionality decomposed: on Pith
  16. [36] Some logical aspects of grammatical structure , author=. Structure of language and its mathematical aspects , volume=. 1961 , publisher= on Pith
  17. [37] Structural generalization of modification in adult learners of an artificial language , author=. Proceedings of the Annual Meeting of the Cognitive Science Society , volume= on Pith
  18. [38] Grammatical framework , author=. Journal of Functional Programming , volume=. 2004 , publisher= on Pith
  19. [39] Computational semantics in type theory , author=. Math. 2004 , publisher= on Pith
  20. [40] Montague Grammar , author=. 1997 , publisher= on Pith
  21. [41] Universal grammar , author=. Theoria , volume= on Pith
  22. [42] Stochastic inversion transduction grammars and bilingual parsing of parallel corpora , author=. Computational linguistics , volume= on Pith
  23. [43] Syntax Directed Translations and the Pushdown Assembler , author=. Journal of computer and system sciences , year= on Pith
  24. [44] Properties of Syntax Directed Translations , author=. Journal of computer and system sciences , year= on Pith
  25. [45] Compositional generalization from first principles , author=. Advances in Neural Information Processing Systems , volume= on Pith
  26. [46] Towards a definition of disentangled representations , author=. arXiv preprint arXiv:1812.02230 , year= on PithPith's reviewarXiv
  27. [47] del Rio, Felipe and Raymond-Saez, Alain and Florea, Daniel and Icarte, Rodrigo Toro and Hurtado, Julio and Calderon, Cristian B. and Soto, Alvaro , booktitle=. Data Distributional Properties As Inductive Bias for Systematic Generalization , year= on Pith
  28. [48] Johnson, Justin and Hariharan, Bharath and van der Maaten, Laurens and Fei-Fei, Li and Zitnick, C Lawrence and Girshick, Ross , booktitle= on Pith
  29. [49] The mechanistic basis of data dependence and abrupt learning in an in-context classification task , author=. International conference on learning representations , volume= on Pith
  30. [50] Pretraining task diversity and the emergence of non-bayesian in-context learning for regression , author=. Advances in neural information processing systems , volume= on Pith
  31. [51] Goddard, Chase and Smith, Lindsay M. and Ngampruetikorn, Vudtiwat and Schwab, David J. , title =. Proceedings of the 42nd International Conference on Machine Learning , articleno =. 2025 , publisher = on Pith
  32. [52] Ramesh, Rahul and Lubana, Ekdeep Singh and Khona, Mikail and Dick, Robert P. and Tanaka, Hidenori , booktitle =. Compositional Capabilities of Autoregressive. 2024 , editor = on Pith
  33. [53] Yin, Yutong and Wang, Zhaoran , booktitle=. Are on Pith
  34. [54] Rich data drive generalization: Lessons from machine learning for linguistics and cognitive science , author=. Behavioral and Brain Sciences , volume=. 2026 , publisher= on Pith
  35. [55] Potts, Christopher , journal=. Characterizing English preposing in. 2024 , publisher= on Pith
  36. [56] Scaling laws for neural language models , author=. arXiv preprint arXiv:2001.08361 , year= on PithPith's reviewarXiv
  37. [57] Language models are few-shot learners , author=. Advances in neural information processing systems , volume= on Pith
  38. [58] The proper treatment of quantification in ordinary English , author=. Approaches to natural language: Proceedings of the 1970 Stanford workshop on grammar and semantics , pages=. 1973 , organization= on Pith
  39. [59] Generalized quantifiers and natural language , author=. Philosophy, language, and artificial intelligence: Resources for processing natural language , pages=. 1981 , publisher= on Pith
  40. [60] Irene Heim and Angelika Kratzer , editor =. Semantics in Generative Grammar , year = on Pith
  41. [61] Break it down: Evidence for structural compositionality in neural networks , author=. Advances in Neural Information Processing Systems , volume= on Pith
  42. [62] Characterizing intrinsic compositionality in transformers with tree projections , author=. 2022 , booktitle= on Pith
  43. [63] Out-of-distribution generalization via composition: a lens through induction heads in transformers , author=. Proceedings of the National Academy of Sciences , volume=. 2025 , publisher= on Pith
  44. [64] When can transformers compositionally generalize in-context? , author=. Proceedings of the 42nd International Conference on Machine Learning , year= on Pith
  45. [65] Scaling can lead to compositional generalization , author=. Advances in Neural Information Processing Systems , volume= on Pith
  46. [66] Compositional generalization in unsupervised compositional representation learning: A study on disentanglement and emergent language , author=. Advances in Neural Information Processing Systems , volume= on Pith
  47. [67] Uncontrolled lexical exposure leads to overestimation of compositional generalization in pretrained models , author=. arXiv preprint arXiv:2212.10769 , year= on PithPith's reviewarXiv
  48. [68] Large language models and the argument from the poverty of the stimulus , author=. Linguistic Inquiry , pages=. 2024 , publisher= on Pith
  49. [69] Faith and fate: Limits of transformers on compositionality , author=. Advances in neural information processing systems , volume= on Pith
  50. [70] Explain me this: Creativity, competition, and the partial productivity of constructions , author=. 2019 , publisher= on Pith
  51. [71] Attention is all you need , author=. Advances in neural information processing systems , volume= on Pith
  52. [72] The syntactic process , author=. 2000 , publisher= on Pith

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment