REVIEW 4 major objections 4 minor 1 cited by
From Dependency to Compositionality: A Neurosymbolic Lifting of LLM Outputs via Combinatory Categorial Grammar
T0 review · 4 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash
Pith's one-line read The paper argues that LLM output can be lifted into typed compositional derivations because the prefix-by-prefix way LLMs generate text matches CCG's type-completing design.
desk verdict Worth reading as a position paper, but the central alignment claim is asserted, not measured; without that measurement the framework reduces to ordinary CCG parsing of LLM output. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is CCG itself — a lexicalized grammar in which every word carries both a syntactic category and a lambda term, and in which every prefix has a typed partial meaning, so 'John loves' is the category S/NP with the partial term λy.love(john,y). The argument leans on three properties: semantically interpreted incrementality (every prefix is a first-class typed object), derivational equivalence modulo composition (alternative derivations converge to the same lambda term, making the lifting well-defined), and the ability of modern supertaggers to assign categories to unseen words from context. These allow CCG to serve as a post-hoc lifting mechanism applied to observable LLM out
What would settle it
Run a wide-coverage CCG parser over a corpus of fluent LLM-generated text sampled from non-newswire registers, and measure the fraction of sentences that receive any typed derivation with acceptable parse confidence. If a substantial fraction fail to lift, or if the n-best analyses yield conflicting lambda terms on simple declaratives beyond known ambiguity, the central claim is refuted. A benchmark showing parse coverage on LLM output comparable to newswire coverage would support it.
Extended reading notes
Core claim
The central claim is that the prefix-driven, type-completing dynamics of autoregressive generation align closely with the incremental processing model that CCG was designed to support, so LLM outputs admit a principled, incremental, and auditable CCG reconstruction. The paper carefully does not claim LLMs implement CCG internally; it claims the outputs can be lifted post hoc into typed lambda terms, with every prefix carrying a category whose unsaturated argument matches the model's next-token expectation. Two consequences follow. First, the lifting extends beyond natural language to formal languages, because the operation of assigning a typed term to an expression is the same as type-checki
Load-bearing premise
The load-bearing premise is that existing wide-coverage CCG parsers and supertaggers can correctly assign typed derivations to LLM-generated text across registers and domains even though they are mainly trained on about a million words of newswire English; if parser coverage collapses on real LLM output, the lifting and every downstream check collapse.
Editorial extensions
If this is right
- Every fluent declarative sentence produced by an LLM can be represented as an auditable typed lambda term whose components trace back to lexical entries and combinatory steps.
- Hallucination detection can be formulated at the earliest structurally meaningful prefix, before the sentence is complete, by querying external knowledge sources with the predications the lifted prefix exposes.
- Compositional failures in generated code become visible as type weakenings — for example, a generated smart-contract function that omits input validation has a lifted type strictly weaker than its contract's expected type.
- The framework extends to any output language with a typed compositional standard, holding the LLM fixed and varying the type system, so the same architecture covers natural language, code, query languages, and formal mathematics.
- A synchronous coupling of an incremental CCG parser with decoding could mask or reweight grammatically inadmissible tokens, although the paper presents this as a sketch rather than a worked proposal.
Reading between the lines
- I infer that the framework's value depends less on which LLM is used than on the availability of wide-coverage CCG parsing, so the most decisive next experiment is a parser-coverage benchmark on in-the-wild LLM text.
- A testable extension: implement the lifted typing for smart-contract code and check whether the predicted type weakenings correlate with the empirically observed low functional-correctness rates of generated contracts.
- The paper's 'types as interfaces' idea suggests content checking could be typed by the query form each predication licences — membership, referent resolution, subsumption — which could be formalized into a typed protocol between the lifted term and the knowledge source.
- I infer that the alignment claim is robust at word boundaries but becomes fuzzy at subword token boundaries; a full synchronous implementation will need a buffering layer, as the paper itself notes.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a neurosymbolic framework for lifting LLM outputs into typed CCG derivations. Its central claim is that autoregressive, prefix-driven generation and CCG's incremental, type-completing processing are closely aligned, and that this alignment makes the lifting 'principled, incremental, and auditable.' The framework is developed in two layers: a compositional CCG layer that produces lambda terms, and a content layer that checks the lifted structure against external knowledge sources, enabling early flagging of hallucinations. The paper extends the lifting, via the Curry-Howard correspondence, to formal languages such as Solidity, SQL, and OWL, and sketches a synchronous LLM-CCG coupling. It repeatedly disclaims any claim that LLMs internally implement CCG and candidly identifies implementation and parser-coverage limitations.
Significance. If the alignment claim held, the framework would provide a principled basis for incremental compositional auditing of LLM output, with substantial potential for hallucination detection and verification of LLM-generated code and formal-language artifacts. The paper is valuable for articulating a clear layered architecture, for correctly positioning existing machinery (ccg2lambda, incremental CCG parsers, constrained decoding), and for stating its limitations honestly rather than overclaiming. However, the central empirical premise is currently supported only by a single toy example with hand-supplied categories; the contribution is therefore best read as a research program or a conceptual proposal rather than a demonstrated result. The paper's careful non-claim about LLM internals and its explicit acknowledgment of the parser-coverage gap are strengths, but they do not by themselves establish the framework's operational viability.
major comments (4)
- [§3.1, Table 1 and note 2] The central claim that LLM next-token expectations and CCG typed expectations 'agree' is not measured; the only evidence is 'John loves Mary' with hand-supplied lexical categories. This agreement is the basis for calling the lifting 'principled' and for the synchronous early-flagging architecture of §6. The granularity caveat (subword vs word) and the n-best policy acknowledge complications but do not connect them to an actual LLM distribution. If the alignment is absent or weaker than claimed, the lifting reduces to ordinary CCG parsing of LLM text, and the distinctive 'operational alignment' and prevention claims lose their basis. The paper should either provide measurements of agreement between LLM continuations and CCG-admissible continuations on real LLM outputs, or explicitly reframe 'alignment' as a hypothesis and separate it from the framework's usefulness.
- [§7, Limitations and next steps] The framework's applicability depends on wide-coverage CCG parsers handling LLM-generated text across registers, domains, and degrees of formality. The paper acknowledges this, citing supertagger generalization to biomedical text as partial mitigation, but provides no evaluation on actual LLM output. The residual concern is structural and register-level rather than lexical, and it is most pressing in the synchronous regime where parser failures would directly constrain generation. Because the lifting is load-bearing for both checking layers, a demonstration on a small but non-toy corpus of LLM-generated text parsed with an existing CCG parser is needed to establish feasibility. Without such a demonstration, the framework remains a plausible but untested proposal.
- [§4.5 and §6] The 'earliest possible flagging' claim relies on maintaining incremental parser state and issuing an external query at mid-prefix. In the Zeus example, the flag at the fourth prefix depends on a specific lexical treatment ('The Aztec god Zeus' as an NP with a conjunctive modifier) and on an ontology lookup whose mapping is not specified. The synchronous regime in §6 is explicitly a sketch, and the paper itself names the parser-induced garden-path hazard. The text should clearly separate the conceptual possibility of early flagging from an operational claim; as written, §4.5 reads as stronger than the acknowledged implementation status supports.
- [§5.3, Figure 1] The Solidity lifting is a hand-constructed type sketch: no lexicon, parser, or algorithm is given, and the type weakening from (A) to (B) is asserted rather than derived. The caption does label it 'hypothesized,' which is appropriate, but the surrounding text uses this example to support the cross-domain claim that type-checking can verify generated code. The paper should state more explicitly that the formal-language extension is an unverified research direction and should not be read as empirical evidence for the framework's viability. The acknowledgment that 'we do not develop the Solidity lifting in detail here' is honest, but it means the section functions as a research agenda rather than a demonstration.
minor comments (4)
- [References] In the Wijnholds et al. entry, 'Proceedingsofthe24thConferenceonComputationalNaturalLanguageLearning' lacks spaces between words; please fix the bibliography formatting.
- [§3.1, Table 1] The row for t=3 says 'sentence is complete; punctuation expected' but 'punctuation expected' is not a CCG type and is not part of the typed state. Clarify that this is an informal LLM-side expectation, not part of the CCG lifting.
- [§2.2] The statement that alternative derivations yield 'equivalent terms' could be more precise: the lambda terms are equivalent up to beta-eta conversion. Adding this would preempt a technical objection.
- [§4.5] The 'brother of Esau was Elijah' example is attributed to a personal communication. If the episode is not publicly documented, consider anonymizing it or adding a public reference, so that the claim is verifiable.
Circularity Check
No significant circularity: the lifting is a post-hoc architecture; the central alignment claim is an asserted empirical analogy, not a fit, and the worked examples are explicitly hand-constructed sketches.
full rationale
The paper's central claim is that autoregressive LLM generation and CCG share a prefix-driven, type-completing dynamic. This is not derived from the framework's inputs by construction. In §3.1 the 'John loves Mary' table shows a hand-chosen CCG derivation alongside asserted LLM next-token expectations; the CCG categories are assigned by lexical entry, not fitted to the LLM distribution, and the paper repeatedly disclaims any claim that LLMs implement CCG (§3.1: 'does not claim that LLMs internally store symbolic grammatical rules'). The coherence property used to make the lifting well-defined rests on the associativity of CCG composition cited to Pareschi & Steedman 1987; this is an external mathematical fact about combinatory rules, not an input-to-output reduction of the paper's own prediction. The Solidity lifting in Figure 1 is explicitly labeled a 'hypothesized typed lifting' and 'a sketch... presenting them by hand'; the content-check examples are synthetic minimal-pair illustrations. The paper's own §7 limitations state that no end-to-end implementation exists and that parser robustness on LLM text is unassessed, and §6 declines to present the synchronous regime as more than a sketch. These are gaps in empirical support, not circular derivations. No load-bearing step equates a fitted parameter with a prediction, imports a uniqueness theorem to forbid alternatives, or smuggles an ansatz via self-citation. Self-citations (Pareschi & Steedman 1987; Salzano et al. 2026) support secondary or motivational claims and are not the sole justification of the central alignment thesis.
Assumptions & free parameters
assumptions (5)
- standard math CCG combinatory rules and the Curry-Howard correspondence are the correct compositional semantics for natural and formal languages.
- domain assumption LLM outputs are prefix-driven at the granularity of words once subword tokens are buffered.
- domain assumption Existing wide-coverage CCG parsers/supertaggers generalize to LLM-generated text across registers and unseen vocabulary.
- domain assumption Reliable external knowledge sources exist and can be queried for the reference of lifted predications.
- standard math The propositions-as-types correspondence licenses identification of CCG derivations with type-checking in programming languages.
Cite this review
Pith. "Pith review of From Dependency to Compositionality: A Neurosymbolic Lifting of LLM Outputs via Combinatory Categorial Grammar." pith.science (2026). https://pith.science/paper/HE62UTLD
@misc{pith2026260718961,
author = {Pith},
title = {Pith review of: From Dependency to Compositionality: A Neurosymbolic Lifting of LLM Outputs via Combinatory Categorial Grammar},
year = {2026},
howpublished = {\url{https://pith.science/paper/HE62UTLD}},
note = {Machine review of arXiv:2607.18961}
}
read the original abstract
Large language models (LLMs) generate fluent text by incrementally predicting the next token from a prefix. Critics in the generative tradition argue that such systems lack genuine grammar; influential replies from the dependency-grammar perspective hold that LLM behavior is well described by local head-dependent structure built word by word. We argue that a sharper observation has been overlooked: the prefix-driven, type-completing dynamics of autoregressive generation align closely with the incremental processing model that Combinatory Categorial Grammar (CCG) was originally designed to support. On this basis we propose a neurosymbolic framework in which LLM outputs are lifted into typed compositional derivations -- not claiming that LLMs implement CCG internally, but that their outputs admit a principled, incremental, and auditable CCG reconstruction. Two consequences follow. First, through the Curry-Howard correspondence the lifting extends beyond natural language to the formal languages LLMs also produce -- programming languages such as Solidity, description-logic and query languages such as OWL and SQL -- with the type system varying and the architecture held fixed. Second, the lifting supports two layers of checking: a compositional layer that catches structural failures directly, and a content layer that checks the lifted structure against external knowledge sources, enabling the earliest possible flagging of hallucinated content. The account thereby requires of a producer not cognition but a prefix-driven generative profile. We close with a sketch of synchronous LLM-CCG coupling as one direction the framework opens.
Figures
Forward citations
Cited by 1 Pith paper
-
Eigenius: A Typed Knowledge-Graph DBMS with Epistemic Stratification and Institution-Mediated Reasoning
A typed knowledge-graph database that makes provenance and proof checking structural commit-time invariants, demonstrated by re-encoding a Nature study.
Reference graph
Works this paper leans on
-
[1]
Linguist Philos 4(4):517--558
Ades AE, Steedman MJ (1982) On the order of words. Linguist Philos 4(4):517--558
1982
-
[2]
Borghoff UM, Bottoni P, Pareschi R (2025) Human--artificial interaction in the age of agentic AI : a system-theoretical approach. Front Hum Dyn 7:1579166. https://doi.org/10.3389/fhumd.2025.1579166
arXiv 2025
-
[3]
In: Semantics in Text Processing (STEP), pp 277--286
Bos J (2008) Wide-coverage semantic analysis with Boxer . In: Semantics in Text Processing (STEP), pp 277--286
2008
-
[4]
Comput Linguist 47(2):255--308
de Marneffe MC, Manning CD, Nivre J, Zeman D (2021) Universal Dependencies . Comput Linguist 47(2):255--308
2021
-
[5]
MIT Press, Cambridge, MA
Dennett DC (1987) The Intentional Stance. MIT Press, Cambridge, MA
1987
-
[6]
In: Proceedings of the ACM SIGPLAN International Conference on Functional Programming (ICFP), pp 48--59
Findler RB, Felleisen M (2002) Contracts for higher-order functions. In: Proceedings of the ACM SIGPLAN International Conference on Functional Programming (ICFP), pp 48--59
2002
-
[7]
Language 96(2):371--412
Futrell R, Levy RP, Gibson E (2020) Dependency locality as an explanatory principle for word order. Language 96(2):371--412
2020
-
[8]
Cognition 68(1):1--76
Gibson E (1998) Linguistic complexity: locality of syntactic dependencies. Cognition 68(1):1--76
1998
Show all 40 references
-
[9]
MIT Press, Cambridge, MA
Gibson E (2025) Syntax: A Cognitive Approach. MIT Press, Cambridge, MA
2025
-
[10]
In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 427--443
Grenander M, Cohen S, Steedman M (2022) Sentence-incremental neural coreference resolution. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 427--443
2022
-
[11]
Dissertation, University of Edinburgh
Grenander M (2025) Explicit discourse modelling for coreference and summarization. Dissertation, University of Edinburgh
2025
-
[12]
Transformer Circuits Thread
Gurnee W, Sofroniew N, Pearce A et al (2026) Verbalizable representations form a global workspace in language models. Transformer Circuits Thread. https://transformer-circuits.pub/2026/workspace/index.html. Accessed 21 July 2026
2026
-
[13]
In: Aizawa M et al (eds) New Frontiers in Artificial Intelligence
Kubota Y (2014) Combinatory categorial grammar as a substructural logic. In: Aizawa M et al (eds) New Frontiers in Artificial Intelligence. Springer, pp 25--38
2014
-
[14]
Am Math Mon 65(3):154--170
Lambek J (1958) The mathematics of sentence structure. Am Math Mon 65(3):154--170
1958
-
[15]
In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 990--1000
Lewis M, Steedman M (2014) A* CCG parsing with a supertag-factored model. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 990--1000
2014
-
[16]
In: Proceedings of ACL-2016 System Demonstrations, pp 85--90
Mart\' i nez-G\' o mez P, Mineshima K, Miyao Y, Bekki D (2016) ccg2lambda: a compositional semantics system. In: Proceedings of ACL-2016 System Demonstrations, pp 85--90
2016
-
[17]
In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 12076--12100
Min S, Krishna K, Lyu X et al (2023) FActScore : fine-grained atomic evaluation of factual precision in long form text generation. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 12076--12100
2023
-
[18]
In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 2055--2061
Mineshima K, Mart\' i nez-G\' o mez P, Miyao Y, Bekki D (2015) Higher-order logical inference with compositional semantics. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 2055--2061
2015
-
[19]
Theoria 36(3):373--398
Montague R (1970a) Universal grammar. Theoria 36(3):373--398. Reprinted in: Thomason R (ed) Formal Philosophy: Selected Papers of Richard Montague. Yale University Press, 1974
1974
-
[20]
In: Visentini B et al (eds) Linguaggi nella Societ\`a e nella Tecnica
Montague R (1970b) English as a formal language. In: Visentini B et al (eds) Linguaggi nella Societ\`a e nella Tecnica. Edizioni di Comunit\`a, Milan, pp 189--224. Reprinted in: Thomason R (ed) Formal Philosophy: Selected Papers of Richard Montague. Yale University Press, 1974
1974
-
[21]
In: Hintikka J, Moravcsik J, Suppes P (eds) Approaches to Natural Language
Montague R (1973) The proper treatment of quantification in ordinary E nglish. In: Hintikka J, Moravcsik J, Suppes P (eds) Approaches to Natural Language. Reidel, Dordrecht, pp 221--242. Reprinted in: Thomason R (ed) Formal Philosophy: Selected Papers of Richard Montague. Yale...
1973
-
[22]
In: van Benthem J, ter Meulen A (eds) Handbook of Logic and Language
Moortgat M (1997) Categorial type logics. In: van Benthem J, ter Meulen A (eds) Handbook of Logic and Language. Elsevier, pp 93--177
1997
-
[23]
Kluwer, Dordrecht
Morrill GV (1994) Type Logical Grammar: Categorial Logic of Signs. Kluwer, Dordrecht
1994
-
[24]
In: Proceedings of the 25th Annual Meeting of the Association for Computational Linguistics, pp 81--88
Pareschi R, Steedman M (1987) A lazy way to chart-parse with categorial grammars. In: Proceedings of the 25th Annual Meeting of the Association for Computational Linguistics, pp 81--88
1987
-
[25]
In: International Conference on Learning Representations (ICLR)
Poesia G, Polozov A, Le V et al (2022) Synchromesh: reliable code generation from pre-trained language models. In: International Conference on Learning Representations (ICLR)
2022
-
[26]
J Biomed Inform 42(5):852--865
Rimell L, Clark S (2009) Porting a lexicalized-grammar parser to the biomedical domain. J Biomed Inform 42(5):852--865
2009
-
[27]
Manuscript under review
Salzano F, Scalabrino S, Oliveto R, Pareschi R (2026) Beyond code similarity: benchmarking the plausibility, efficiency, and complexity of LLM -generated smart contracts. Manuscript under review
2026
-
[28]
In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 9895--9901
Scholak T, Schucher N, Bahdanau D (2021) PICARD : parsing incrementally for constrained auto-regressive decoding from language models. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 9895--9901
2021
-
[29]
In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pp 228--239
Stanojevi\' c M, Steedman M (2019) CCG parsing algorithm with incremental tree rotation. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pp 228--239
2019
-
[30]
In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp 4111--4122
Stanojevi\' c M, Steedman M (2020) Max-margin incremental CCG parsing. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp 4111--4122
2020
-
[31]
Cogn Sci 47(7):e13312
Stanojevi\' c M, Brennan JR, Dunagan D, Steedman M, Hale JT (2023) Modeling structure-building in the brain with CCG parsing and large language models. Cogn Sci 47(7):e13312
2023
-
[32]
MIT Press, Cambridge, MA
Steedman M (2000) The Syntactic Process. MIT Press, Cambridge, MA
2000
-
[33]
MIT Press, Cambridge, MA
Steedman M (2011) Taking Scope: The Natural Semantics of Quantifiers. MIT Press, Cambridge, MA
2011
-
[34]
Mind 59(236):433--460
Turing AM (1950) Computing machinery and intelligence. Mind 59(236):433--460
1950
-
[35]
Technical Report 83-29, Department of Mathematics, Simon Fraser University
van Benthem J (1983) The semantics of variety in categorial grammar. Technical Report 83-29, Department of Mathematics, Simon Fraser University
1983
-
[36]
Commun ACM 58(12):75--84
Wadler P (2015) Propositions as types. Commun ACM 58(12):75--84
2015
-
[37]
In: Proceedings of the 24th Conference on Computational Natural Language Learning (CoNLL), pp 313--324
Wijnholds G, Sadrzadeh M, Clark S (2020) Representation learning for type-driven composition. In: Proceedings of the 24th Conference on Computational Natural Language Learning (CoNLL), pp 313--324
2020
-
[38]
arXiv preprint arXiv:2307.09702
Willard BT, Louf R (2023) Efficient guided generation for large language models. arXiv preprint arXiv:2307.09702
2023 arXiv
-
[39]
In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp 277--287
Yoshikawa M, Noji H, Matsumoto Y (2017) A* CCG parsing with a supertag and dependency factored model. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp 277--287
2017
-
[40]
In: Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI), pp 658--666
Zettlemoyer LS, Collins M (2005) Learning to map sentences to logical form: structured classification with probabilistic categorial grammars. In: Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI), pp 658--666
2005
Reviewed August 1, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.