Pith. sign in

REVIEW 3 major objections 5 minor 37 references

Investigating the (De)Composition Capabilities of Large Language Models in Natural-to-Formal Language Conversion

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Large language models are deficient in both decomposition and composition when converting natural language to a formal language, and the decomposition deficiency is consistently larger across all five models tested.

desk verdict Genuinely new N2F evaluation framework and a plausible empirical finding, but the headline decomposition metric rests on an unvalidated matching assumption. read the letter →

arxiv 2501.14649 v2 pith:L5JNTRQU submitted 2025-01-24 cs.CL

Ziyao Xu , Houfeng Wang This is my paper · ORCID
classification cs.CL
keywords decompositioncompositionnatural-to-formalconversionin-contextlearningcompositionalgeneralizationsymbolicnameslargelanguagemodelsevaluationframework
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether large language models can learn an unfamiliar formal language from a few examples, a prerequisite for reliable natural-to-formal conversion. To answer it, the authors propose a framework that measures two abilities separately: decomposition, working out what each symbol means from examples, and composition, combining the known symbols into new expressions. In tests with five leading LLMs on a tabular-reasoning formal language, every model made more errors from decomposition than from composition, and all models produced a wide range of error types. The paper further shows that compositional gaps between examples and test questions and misleading symbol names worsen both abilities. A sympathetic reader would care because these two abilities are what an LLM must possess to convert natural language into any formal language it has not been trained on.

What carries the argument

The DEDC framework carries the argument. It semi-automatically builds 323 samples over a fixed formal language of 10 tabular-reasoning primitives; each sample is a directed acyclic graph of expressions plus a natural-language question. For each test sample it selects three demonstration samples that cover the primitives, then measures an LLM in two conditions: $P_{dc}$, with only demonstrations, and $P_c$, with demonstrations plus a simplest worked example for each primitive. The difference defines decomposition deficiency $D_d = P_c - P_{dc}$ and composition deficiency $D_c = 100 - P_c$, making the two capabilities comparable on the same scale. Additional settings (0% vs 100% compositional gap, anomalous vs cross-mapped names) are scored by the same protocol to yield delta measures for each ability.

What would settle it

A model evaluated on the DEDC benchmark that produces $P_{dc} \geq P_c$ — that is, $D_d \leq 0$ — would refute the claim that decomposition deficiencies are consistently larger than composition deficiencies.

Watch

Extended reading notes

Core claim

The paper's core discovery is that LLMs' weakness in natural-to-formal conversion is concentrated in decomposition rather than composition. In the base evaluation, every one of the five models showed $D_d > D_c$: Claude-3.5 had $D_d = 7.74$ vs $D_c = 1.24$, GPT-4o had $D_d = 13.31$ vs $D_c = 5.26$, and the open-weight models ranged from $D_d = 15.79$ to $19.20$. The paper also finds that compositional gaps between demonstration samples and the test question degrade both abilities, and that counter-intuitive symbolic names impose an even larger penalty, adding more than 20 points to decomposition error for every model. Error analysis shows six recurring error types, with primitive confusion most frequent, pointing to deficiencies in both natural language understanding and in learning and using a symbolic system.

Load-bearing premise

The headline numbers assume that seeing a simplest sample of each primitive fully removes the need for decomposition, leaving the entire gap between the two conditions ($P_c - P_{dc}$) to measure decomposition alone, with no interaction between decomposition and composition.

Editorial extensions

If this is right

  • Eliminating the decomposition deficiency alone would raise accuracy by 7.7 to 19.2 percentage points depending on the model, making decomposition the highest-leverage target for improvement.
  • Demonstration selection matters: replacing demonstrations so that every test sample has zero compositional gap with them improves both $P_{dc}$ and $P_c$ for all five models, so graph-matching demonstration samplers are a concrete fix.
  • Counter-intuitive symbolic names are a severe robustness threat; any deployment that renames primitives without retraining should expect decomposition error to rise by more than 20 points for every model.
  • The DEDC framework is reusable: for any N2F task with a defined primitive set and question templates, the same two-condition protocol yields directly comparable $D_d$ and $D_c$ scores.
  • Primitive confusion, the most common error, indicates that symbolic-system learning should be a focal point alongside natural-language understanding in future N2F work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the paper's decoupling assumes additive, non-interacting capabilities; a direct test is to present the primitive demonstrations without the three full demonstration samples, which, if $P_c$ drops, would show $D_d$ overstates the decomposition-only burden.
  • Editorial inference: the benchmark uses one template-generated formal language, so the finding that decomposition dominates composition may not transfer to messier real-world formalisms like SQL; replicating DEDC across other N2F tasks would reveal whether the bottleneck is general.
  • Editorial inference: because evaluation is at temperature zero, the error-type distribution may be a property of greedy decoding; sampling could shift the balance between primitive confusion and omission, which the paper does not explore.
  • Editorial inference: the authors' next-token-probability hypothesis for counter-intuitive names suggests a concrete intervention—explicit name-to-meaning mapping prompts or lightweight fine-tuning should shrink the more-than-20-point decomposition penalty, which the paper leaves untested.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes DEDC, a semi-automatic framework for constructing samples and tasks that aim to evaluate LLMs' decomposition and composition capabilities in natural-to-formal-language (N2F) conversion under in-context learning. Using a simple tabular-reasoning formal language with ten primitives, the framework creates 323 test samples from six base graphs, and for each test sample builds a task in which the model sees either three full demonstration samples (Pdc condition) or one simplest sample per primitive used in the test sample (Pc condition). The authors define composition deficiency as Dc = 100 - Pc and decomposition deficiency as Dd = Pc - Pdc. They evaluate five LLMs and report that all of them are deficient in both capabilities, that decomposition deficiency is systematically larger than composition deficiency, and that compositional gaps and counter-intuitive symbolic names degrade both capabilities. The paper also provides an error-type taxonomy and a detailed analysis of error distributions across the five models.

Significance. If the decoupling assumption is valid, the paper makes a useful empirical contribution: it is one of the first attempts to separately measure decomposition and composition abilities for N2F in an unfamiliar formal language, and its finding that decomposition is the weaker capability across five diverse models is striking and practically relevant. The framework is reusable, the dataset and code are released, and the error taxonomy (primitive confusion, primitive fiction, variable misuse, redundancy, omission, incorrect meaning) is a sensible qualitative contribution. The additional settings for compositional gaps and counter-intuitive names are also well-motivated. The main value of the paper is therefore as an evaluation methodology plus a set of reproducible headline findings; both currently hinge on an identification assumption that the paper does not validate.

major comments (3)
  1. [Section 2.4, Table 1] The decoupling metric Dd = Pc - Pdc rests on an unvalidated identification assumption. The Pc condition and the Pdc condition differ not only in whether the model must infer primitive meanings and formats, but also in the number of demonstrations (up to four additional exemplars in Pc), in the length and syntactic form of the demonstrations (single-step versus multi-step expressions), and in whether the model ever sees compositional use of the primitives before the test item. The paper asserts that Pc - Pdc isolates the additional error caused by decomposition, but provides no ablation, filler-example control, or human baseline to show that the other differences are negligible. Since the margins Dd - Dc are as small as 4.65 points (DeepSeek-2.5) and 6.50 points (Claude-3.5 and Llama-3.1), a small confound could change the ordering for several models. I would like to see a control that matches demonstration count and format across the two conditions, for example by adding the primitive demonstrations to the three full demonstrations, or by replacing the three full demonstrations with an equal number of single-step demonstrations in the Pdc condition.
  2. [Section 2.2, Section 3.2] The results are point estimates with no reported variance, even though demonstration samples are randomly selected. Setting temperature to 0 removes decoding stochasticity but does not remove the variance induced by the random choice of demonstrations. With 323 test samples and Dd-Dc margins as low as 4.65 points, the claim that all five models show Dd > Dc would be more convincing if the authors reported repeated runs with different random demonstration draws, or at least a bootstrap confidence interval. This concern also propagates to the Section 4 setting-effect estimates, which compare point estimates across conditions.
  3. [Section 4, Eqs. (1)-(2)] The setting-effect estimates reuse the same additive decoupling assumption without checking for interactions between decomposition and composition. Specifically, Delta_c^s = Pc^s - Pc and Delta_d^s = (Pdc^s - Pdc) - Delta_c^s assume that the total change in Pdc decomposes into independent composition and decomposition components. If the demonstrations in the 0% gap or 100% gap settings also change the difficulty of primitive inference (for example, by altering which primitives are demonstrated or how they are used in context), then the reported Delta_d^s values are biased. The paper should either validate the additive assumption with targeted controls or present the raw Pdc^s and Pc^s values with a sensitivity analysis that shows the qualitative conclusions are robust to alternative decompositions.
minor comments (5)
  1. [Section 3.3, Table 3] The error-type counts are small in several cells (e.g., Claude-3.5 has only two errors in the Pc condition), so statements about the 'wide coverage' of error types should be phrased as descriptive observations rather than as strong generalizations, or accompanied by confidence intervals.
  2. [Section 2.2] The paper should state how many random demonstration selections were used for each test sample and whether a fixed seed was used; this would help readers interpret the reproducibility of the point estimates.
  3. [Section 4.1.1] The reference to Kim and Linzen (2020) is duplicated in the third sentence; one of the two occurrences should be removed.
  4. [Figure 2 caption] The caption says 'the sample construction of the STD framework,' but the framework is called DEDC; this appears to be a typo.
  5. [Appendix B.1] The grammatical-divergence re-experiment is run only on DeepSeek and changes both Pc and Pdc by 0.31 points; this is a useful sanity check but should be reported as a single anecdotal data point rather than as evidence that grammatical divergence has no effect on other models.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity is present: the DEDC metrics are direct measurements of model accuracy, the Section 4 residual effects are empirical differences, and the only self-citation is non-load-bearing background.

full rationale

The paper's claimed chain is measurement, not derivation: Dc = 100 - Pc and Dd = Pc - Pdc (Section 2.4) are arithmetic transforms of accuracies measured on the constructed tasks. The headline finding Dd > Dc is an empirical ordering of those measured values, not an identity forced by the definitions; nothing in the definitions constrains Pc - Pdc to exceed 100 - Pc. The Section 4 additional-setting effects, △s_c = P^s_c - P_c and △s_d = (P^s_dc - P_dc) - △s_c, are likewise differences of independent measurements, and no equation in the paper defines a construct in terms of the conclusion it is used to support. The main threat to the paper's interpretation is the identification assumption that the Pc condition removes decomposition while leaving composition unchanged: Pdc uses three full multi-step demonstrations while Pc uses one simplest sample per primitive, so Pc - Pdc can also absorb differences in demonstration count, format, and composition practice. That is a validity or measurement concern, not circularity under the required standard: the paper does not fit a parameter to a subset of data and then predict a closely related quantity, and the assumption is not secretly encoded as a result. The only self-citation (Xu and Wang 2024, SPOR) appears in Section 4.1.1 as one of several citations for the importance of compositional generalization and is not used to justify the DEDC decoupling or to rule out alternative metrics. Consequently there is no load-bearing self-citation, imported uniqueness theorem, or ansatz-by-citation in the paper.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claim rests on three domain assumptions: the unfamiliarity of f0-f9 names, the sufficiency of three demonstrations, and the additivity of Dd. None are verified externally, but none are fitted parameters either.

assumptions (4)
  • ad hoc to paper The Pc condition removes the need for decomposition, so Dd = Pc - Pdc isolates decomposition deficiency.
    Section 2.4 defines Dd and Dc from Pdc and Pc. This is a modeling assumption specific to the DEDC framework; it is not independently verified.
  • domain assumption Primitive names f0-f9 carry no prior meaning to the LLMs.
    Appendix A.1: 'we use primitive names without any meaning information attached to them to avoid that LLMs have ever seen the same or similar primitive name-meaning mappings.' If false, Pdc is inflated by prior knowledge.
  • domain assumption The 323 generated samples and templates are representative of N2F decomposition/composition.
    Section 2.1 constructs samples from 6 base graphs and 323 valid schemes. Generalization of the findings beyond tabular reasoning depends on this representativeness.
  • domain assumption Three demonstration samples are sufficient for LLMs to learn the primitives and composition rules.
    Section 2.2: 'Demonstration samples are enough for LLMs to learn desired primitives and basic rules of composition.' If insufficient, Pdc underestimates both abilities.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Investigating the (De)Composition Capabilities of Large Language Models in Natural-to-Formal Language Conversion." pith.science (2026). https://pith.science/paper/L5JNTRQU

@misc{pith2026250114649,
  author       = {Pith},
  title        = {Pith review of: Investigating the (De)Composition Capabilities of Large Language Models in Natural-to-Formal Language Conversion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/L5JNTRQU}},
  note         = {Machine review of arXiv:2501.14649}
}
read the original abstract

To achieve generalized and robust natural-to-formal language conversion (N2F), large language models (LLMs) need to have strong capabilities of decomposition and composition in N2F when faced with an unfamiliar formal language and be able to cope with compositional gaps and counter-intuitive symbolic names. To investigate whether LLMs have this set of basic capabilities in N2F, we propose the DEDC framework. This framework semi-automatically performs sample and task construction, allowing decoupled evaluation of the set of decomposition and composition capabilities of LLMs in N2F. Based on this framework, we evaluate and analyze the most advanced LLMs, and the main findings include that: (1) the LLMs are deficient in both decomposition and composition; (2) the LLMs show a wide coverage of error types that can be attributed to deficiencies in natural language understanding and the learning and use of symbolic systems; (3) compositional gaps and counter-intuitive symbolic names both affect the decomposition and composition of the LLMs. Our work provides a new perspective for investigating the basic capabilities of decomposition and composition of LLMs in N2F. The detailed analysis of deficiencies and attributions can help subsequent improvements of LLMs.

Figures

Figures reproduced from arXiv: 2501.14649 by the authors.

Figure 1
Figure 1. An illustration of the sample and task construction of the DEDC framework. An example of the sample [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Six types of base graphs we identify for the [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. An illustration of the compositional gap be [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: An illustration of the two types of settings for [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 11 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Shengnan An, Zeqi Lin, Qiang Fu, Bei Chen, Nanning Zheng, Jian-Guang Lou, and Dongmei Zhang. 2023. https://doi.org/10.18653/v1/2023.acl-long.618 How do in-context examples affect compositional generalization? In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11027--11052, Toronto, Can...

    doi:10.18653/v1/2023.acl-long.618 2023

  4. [4]

    Anthropic. 2024. https://www.anthropic.com/news/claude-3-5-sonnet Claude 3.5 sonnet

    2024

  5. [5]

    Chamberlin and Raymond F

    Donald D. Chamberlin and Raymond F. Boyce. 1974. https://doi.org/10.1145/800296.811515 SEQUEL: A structured english query language . In Proceedings of 1974 ACM-SIGMOD Workshop on Data Description, Access and Control, Ann Arbor, Michigan, USA, May 1-3, 1974, 2 Volumes , pages 249--264. ACM

    work page arXiv 1974

  6. [6]

    Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li, Xiyou Zhou, and William Yang Wang. 2020 a . https://openreview.net/forum?id=rkeJRhNYDH Tabfact: A large-scale dataset for table-based fact verification . In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020 . OpenReview.net

    2020

  7. [7]

    Zhiyu Chen, Wenhu Chen, Hanwen Zha, Xiyou Zhou, Yunkai Zhang, Sairam Sundaresan, and William Yang Wang. 2020 b . https://doi.org/10.18653/v1/2020.findings-emnlp.190 L ogic2 T ext: High-fidelity natural language generation from logical forms . In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2096--2111, Online. Association fo...

    doi:10.18653/v1/2020.findings-emnlp.190 2020

  8. [8]

    DeepSeek-AI, :, Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, Huazuo Gao, Kaige Gao, Wenjun Gao, Ruiqi Ge, Kang Guan, Daya Guo, Jianzhong Guo, Guangbo Hao, Zhewen Hao, Ying He, Wenjie Hu, Panpan Huang, Erhang Li, Guowei Li, Jiashi Li, Yao Li, Y. K. Li, Wenfeng Liang, Fangyun Lin, A. ...

    arXiv 2024

Show all 37 references
  1. [9]

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, Baobao Chang, Xu Sun, Lei Li, and Zhifang Sui. 2024. https://arxiv.org/abs/2301.00234 A survey on in-context learning . Preprint, arXiv:2301.00234

    2024 arXiv

  2. [10]

    Timothy Dozat and Christopher D. Manning. 2017. https://openreview.net/forum?id=Hk95PK9le Deep biaffine attention for neural dependency parsing . In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceed...

    2017

  3. [11]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zha...

    2024 arXiv

  4. [12]

    Dieuwke Hupkes, Verna Dankers, Mathijs Mul, and Elia Bruni. 2020. https://doi.org/10.1613/JAIR.1.11674 Compositionality decomposed: How do neural networks generalise? J. Artif. Intell. Res., 67:757--795

    2020 doi

  5. [13]

    Dieuwke Hupkes, Mario Giulianelli, Verna Dankers, Mikel Artetxe, Yanai Elazar, Tiago Pimentel, Christos Christodoulopoulos, Karim Lasri, Naomi Saphra, Arabella Sinclair, Dennis Ulmer, Florian Schottmann, Khuyagbaatar Batsuren, Kaiser Sun, Koustuv Sinha, Leila Khalatbari, Maria...

    2022 doi arXiv

  6. [14]

    Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne...

    2024 arXiv

  7. [15]

    Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. 2024 b . https://arxiv.org/abs/2406.00515 A survey on large language models for code generation . Preprint, arXiv:2406.00515

    2024 arXiv

  8. [16]

    George Katsogiannis - Meimarakis and Georgia Koutrika. 2023. https://doi.org/10.1007/S00778-022-00776-8 A survey on deep learning approaches for text-to-sql . VLDB J. , 32(4):905--936

    2023 doi

  9. [17]

    Daniel Keysers, Nathanael Sch \" a rli, Nathan Scales, Hylke Buisman, Daniel Furrer, Sergii Kashubin, Nikola Momchev, Danila Sinopalnikov, Lukasz Stafiniak, Tibor Tihon, Dmitry Tsarkov, Xiao Wang, Marc van Zee, and Olivier Bousquet. 2020. https://openreview.net/forum?id=SygcCn...

    2020

  10. [18]

    Najoung Kim and Tal Linzen. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.731 COGS : A compositional generalization challenge based on semantic interpretation . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9087--9...

    2020 doi

  11. [19]

    Brenden Lake and Marco Baroni. 2018. https://proceedings.mlr.press/v80/lake18a.html Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks . In Proceedings of the 35th International Conference on Machine Learning, volume 80...

    2018

  12. [20]

    Itay Levy, Ben Bogin, and Jonathan Berant. 2023. https://doi.org/10.18653/v1/2023.acl-long.78 Diverse demonstrations improve in-context compositional generalization . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pap...

    2023 doi

  13. [21]

    Jinxin Liu, Shulin Cao, Jiaxin Shi, Tingjian Zhang, Lunyiu Nie, Linmei Hu, Lei Hou, and Juanzi Li. 2024. https://doi.org/10.18653/V1/2024.FINDINGS-ACL.45 How proficient are large language models in formal languages? an in-depth insight for knowledge base question answering . I...

    2024 doi

  14. [22]

    Yubo Ma, Zhibin Gou, Junheng Hao, Ruochen Xu, Shuohang Wang, Liangming Pan, Yujiu Yang, Yixin Cao, Aixin Sun, Hany Awadalla, and Weizhu Chen. 2024. https://arxiv.org/abs/2402.11451 Sciagent: Tool-augmented language models for scientific reasoning . Preprint, arXiv:2402.11451

    2024 arXiv

  15. [23]

    Anssi Moisio, Mathias Creutz, and Mikko Kurimo. 2024. https://arxiv.org/abs/2407.08269 Llms' morphological analyses of complex fst-generated finnish words . Preprint, arXiv:2407.08269

    2024 arXiv

  16. [24]

    Allen Newell and Herbert A. Simon. 2007. https://doi.org/10.1145/1283920.1283930 Computer science as empirical inquiry: symbols and search , page 1975. Association for Computing Machinery, New York, NY, USA

    2007

  17. [25]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff ...

    2024 arXiv

  18. [26]

    Arthur N Prior. 1963. https://academic.oup.com/book/5158?searchresult=1&utm_source=TrendMD&utm_medium=cpc&utm_campaign=Oxford_Academic_Books_TrendMD_0 Formal logic . Oxford University Press

    1963

  19. [27]

    Ivan A Sag, Thomas Wasow, Emily M Bender, and Ivan A Sag. 1999. https://wiki.eecs.yorku.ca/course_archive/2013-14/W/6339/_media/sag_i.a._wasow_t._bender_e.m._syntax.._a_formal_introduction_book_draft_2ed._csli_2003_472s_lf_.pdf Syntactic theory: A formal introduction , volume ...

    1999

  20. [28]

    Arto Salomaa. 1987. https://dl.acm.org/doi/abs/10.5555/27647 Formal languages . Academic Press Professional, Inc

    1987 doi

  21. [29]

    Peter Shaw, Ming-Wei Chang, Panupong Pasupat, and Kristina Toutanova. 2021. https://doi.org/10.18653/v1/2021.acl-long.75 Compositional generalization and natural language variation: Can a semantic parsing approach handle both? In Proceedings of the 59th Annual Meeting of the A...

    2021 doi

  22. [30]

    Freda Shi, Kevin Gimpel, and Karen Livescu. 2024 a . https://doi.org/10.18653/v1/2024.acl-long.666 Structured tree alignment for evaluation of (speech) constituency parsing . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: ...

    2024 doi

  23. [31]

    Liang Shi, Zhengju Tang, Nan Zhang, Xiaotong Zhang, and Zhi Yang. 2024 b . https://arxiv.org/abs/2407.15186 A survey on employing large language models for text-to-sql tasks . Preprint, arXiv:2407.15186

    2024 arXiv

  24. [32]

    Meina Song, Zecheng Zhan, and Haihong E. 2019. https://doi.org/10.1109/ACCESS.2019.2931464 Hierarchical schema representation for text-to-sql parsing with decomposing decoding . IEEE Access, 7:103706--103715

    2019

  25. [33]

    Yike Wu, Jiatao Zhang, Nan Hu, LanLing Tang, Guilin Qi, Jun Shao, Jie Ren, and Wei Song. 2024. https://arxiv.org/abs/2403.18760 Mldt: Multi-level decomposition for complex long-horizon robotic task planning with open-source large language model . arXiv preprint arXiv:2403.18760

    2024 arXiv

  26. [34]

    Jundong Xu, Hao Fei, Liangming Pan, Qian Liu, Mong - Li Lee, and Wynne Hsu. 2024. https://doi.org/10.18653/V1/2024.ACL-LONG.720 Faithful logical reasoning via symbolic chain-of-thought . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics...

    2024 doi

  27. [35]

    Ziyao Xu and Houfeng Wang. 2024. https://doi.org/10.18653/v1/2024.acl-long.36 SPOR : A comprehensive and practical evaluation method for compositional generalization in data-to-text generation . In Proceedings of the 62nd Annual Meeting of the Association for Computational Lin...

    2024 doi

  28. [36]

    Yunhu Ye, Binyuan Hui, Min Yang, Binhua Li, Fei Huang, and Yongbin Li. 2023. https://doi.org/10.1145/3539618.3591708 Large language models are versatile decomposers: Decomposing evidence and questions for table-based reasoning . In Proceedings of the 46th International ACM SIG...

    2023

  29. [37]

    Meishan Zhang. 2020. https://arxiv.org/abs/2006.11056 A survey of syntactic-semantic parsing based on constituent and dependency structures . CoRR, abs/2006.11056

    2020 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.