Pith. sign in

REVIEW 2 major objections 5 minor 49 references

Stacking every NL2SQL pipeline fix does not win; NatSQL and a final-beam reranker help most, and only for the right backbone.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 08:21 UTC pith:QZYGY5AR

load-bearing objection Solid systems paper: interactions matter more than stacking, with a reusable beam reranker and honest negative results; Shapley is approximate but the raw Spider gains hold. the 2 major comments →

arxiv 2607.10911 v1 pith:QZYGY5AR submitted 2026-07-12 cs.CL cs.AIcs.DB

The Nuts and Bolts of Natural Language to SQL Translation: A Systematic Analysis of Model Pipeline Optimisation Approaches and their Interactions

Filip Klubicka , Vasudevan Nedumpozhimana , Sneha Rautmare , Bora Caglayan , Mingxue Wang , John D. Kelleher This is my paper
classification cs.CL cs.AIcs.DB
keywords NL2SQLSQLrerankerSpiderNatSQLsynthetic datapipeline optimisationShapley analysis
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that lightweight natural-language-to-SQL systems still have headroom if their pipelines are tuned carefully, rather than by throwing every popular extension into the same model. Working with two small baselines (SmBoP and RASAT), the authors add NatSQL as an intermediate representation, synthetic-data fine-tuning (GAZP), a token-boundary preprocessing step, and a new final-beam reranker (SQL Verifier and Selector models). Ablations plus approximate Shapley values show that the same extension can help one backbone and hurt the other, and that stacking everything is not best. On the Spider development set the strongest systems lift execution and exact-match accuracy by roughly four and a half points over vanilla RASAT; NatSQL and the reranker are the largest positive contributors for RASAT-based systems. The practical message is that pipeline design for NL2SQL is an interaction problem, not a “more modules is better” problem, and that a good, model-agnostic reranker can raise the ceiling of modest generators.

Core claim

Simply combining all studied NL2SQL pipeline extensions does not produce the best system; each extension’s value depends on its interactions with the baseline architecture and with the other extensions. On Spider, the best RASAT-based combinations raise EX/EM by about 4.5/4.4 points over the vanilla RASAT baseline, with NatSQL and a novel final-beam SQL Verifier/Selector as the strongest positive Shapley contributors for those systems.

What carries the argument

Approximate Shapley values computed over modular pipeline combinations (NatSQL, GAZP synthetic fine-tuning, token preprocessing, and final-beam rerankers built from SQL Verifier and Selector classifiers) that quantify each component’s marginal contribution under different SmBoP- and RASAT-based frameworks.

Load-bearing premise

That Shapley values approximated from an incomplete set of component combinations still correctly rank which extensions help or hurt, even though many combinations were never run.

What would settle it

Train and evaluate the missing combinations (especially NatSmBoP and vanilla RASAT with every extension and both families of rerankers) and recompute the Shapley table; if NatSQL or the reranker change sign or lose rank, the interaction story fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Lightweight NL2SQL pipelines should be designed by measuring pairwise and higher-order interactions, not by stacking every published module.
  • A model-agnostic final-beam SQL Verifier/Selector can raise accuracy of modest generators without enlarging the base model.
  • NatSQL is not a universal free lunch; it helps unconstrained sequence-to-sequence models such as RASAT but can degrade systems already using a relational-algebra intermediate representation.
  • The same reranker and data-augmentation recipes can be dropped onto other generators, including LLMs that emit candidate beams, provided beam coverage is high enough for the reranker to act.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The cascaded design that dropped many weak-baseline combinations may hide cases where a “bad” intermediate representation plus a strong reranker still beats a strong baseline without one.
  • Because the reranker is model-agnostic, the same Verifier/Selector training recipe could be reused as a cheap post-processor for commercial LLM SQL endpoints that already return n-best lists.
  • Negative interactions between preprocessing and GAZP suggest that surface tokenisation changes and synthetic data generation may need to be co-designed rather than applied sequentially.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper studies interactions among four NL2SQL pipeline extensions—NatSQL intermediate representation, GAZP synthetic-data fine-tuning, token-level preprocessing, and a novel final-beam reranker (SQL Verifier / Selector / hybrid)—on two lightweight backbones (SmBoP and RASAT). Components are integrated modularly, evaluated by exact-match and execution accuracy on Spider, and ranked via approximate Shapley values. The central claim is that simply stacking all extensions is suboptimal: impact depends on baseline–extension and extension–extension interactions. Best reported systems (NatRASAT+PP+schema Verifier trained on SmBoP+GAZP beams; NatRASAT+GAZP+schema Verifier trained on NatRASAT+GAZP beams) improve EX/EM by ~4.5/4.4 points over vanilla RASAT, with NatSQL and the reranker identified as the strongest positive contributors for RASAT-based systems.

Significance. If the interaction findings hold, the work supplies a concrete, model-agnostic recipe for raising the ceiling of non-LLM NL2SQL systems without simply scaling parameters. Strengths include a large factorial ablation (Appendix E tables), honest reporting of negative interactions (NatSmBoP underperforms; PP+GAZP fails to compound), a novel model-agnostic reranker with both intrinsic and extrinsic evaluation, and an explicit Shapley framing of component contributions. The NatSQL integration with SmBoP (grammar, binary parser, three variants) and the cumulative-gain beam analyses are useful engineering contributions that other groups can reuse. The results remain relevant for production settings that cannot deploy large LLMs.

major comments (2)
  1. §5.1 and Table 4: the ranking of NatSQL and the reranker as the strongest positive contributors rests on approximate Shapley values computed from an incomplete set of combinations (authors note “all such scores were not available”). Because the experimental design is also cascaded (Limitations: many NatSmBoP and vanilla-RASAT + GAZP/PP/reranker cells were never run), missing cells could reverse signs or relative magnitudes. Either complete the factorial for the four components on the two backbones, or replace the Shapley claim with a more limited statement that is strictly supported by the observed cells.
  2. §3.1 / Appendix A and Figure 3: NatSmBoP variants show higher beam coverage than vanilla SmBoP yet lower top-1 accuracy; none of the developed rerankers close the gap. The paper leaves open whether this is an inherent incompatibility between relational algebra and NatSQL or simply a reranker-training mismatch. A short diagnostic (e.g., oracle top-k selection or a NatSmBoP-specific selector) is needed to decide whether the negative interaction is architectural or merely optimisational; without it the interaction narrative for SmBoP remains incomplete.
minor comments (5)
  1. Table 3 and the surrounding text: the two “best” systems are labelled (A) and (B) but the precise reranker variant (schema vs no-schema, Verifier vs hybrid) is only fully specified later; a single clarifying sentence would help.
  2. §4.3–4.4: label distributions and geometric-reordering probability (0.7) are given, yet no sensitivity analysis is reported; a one-sentence note on robustness would strengthen the reranker claims.
  3. Appendix D (LLaMA zero-shot): the prompt and post-processing effort are acknowledged, but the section is tangential; consider moving it to a short “future work” paragraph or dropping it to keep the main narrative focused.
  4. Typos / formatting: “THENUTS ANDBOLTS”, “SotA”, occasional missing spaces after periods, and inconsistent capitalisation of “Reranker/reranker” appear throughout.
  5. Figure 1 caption and pipeline diagram: the four optimisation points are clear, but the figure itself is not reproduced in the supplied text; ensure it is high-resolution and that arrows match the component names used later.

Circularity Check

0 steps flagged

No circularity: empirical ablations and approximate Shapley on external Spider metrics; components are modular add-ons, not definitional of the reported EX/EM.

full rationale

The paper is a standard empirical systems study. Baselines (SmBoP, RASAT) are trained/evaluated on the public Spider train/dev split using the official evaluation script for Exact Match and Execution accuracy. Pipeline extensions (NatSQL intermediate representation, GAZP synthetic fine-tuning, token preprocessing, and the novel final-beam Verifier/Selector rerankers) are integrated modularly; performance is measured by held-out extrinsic scores and by approximate Shapley values computed from the observed combination table. Training the Verifier/Selector inserts gold SQL when absent from a beam solely to create supervised labels for the binary/multi-class classifiers; this is ordinary dataset construction and does not force the downstream EX/EM numbers (which remain free to rise or fall when the reranker is applied). The single self-citation (Klubička et al. 2018) merely supplies the cumulative-gain charting technique used for beam-coverage analysis and is not load-bearing for any claimed result. Approximate Shapley values and the cascaded experimental design are methodological limitations, not circular reductions of outputs to inputs. No equation, uniqueness claim, or fitted free parameter is redefined as a prediction. Consequently the derivation chain is self-contained against an external benchmark and exhibits zero circularity of the kinds enumerated.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 4 invented entities

Empirical systems paper. Load-bearing content is experimental design choices and domain assumptions about Spider and the two backbones, not free physical constants or new particles. Free parameters are discrete design knobs (synthetic data size, beam sizes, geometric reordering probability, which beams train the reranker). Axioms are standard ML/NLP practice plus the incomplete-combination Shapley approximation. Invented entities are the new model modules (NatSmBoP variants, SQL Verifier, SQL Selector, hybrid reranker), which have independent handles via accuracy metrics but no external physical evidence requirement.

free parameters (4)
  • GAZP synthetic training size (22,417 deduplicated samples) = 22417 unique pairs
    Chosen after size/distribution experiments as the largest available set; performance claims for GAZP fine-tuning depend on this choice.
  • Geometric beam reordering probability (0.7 keep-first) = 0.7
    Heuristic used to match position distribution of correct SQLs when building SQL Selector training data; affects selector training distribution.
  • NatSmBoP initial beam size (100) and linear decay to 10 = 100→10
    Hand-set relative to vanilla SmBoP beam 30 because NatSQL expands the token inventory; affects NatSmBoP coverage and reported underperformance.
  • Reranker training beam source (SmBoP+GAZP vs NatRASAT+GAZP top-5)
    Which model’s beams supply verifier/selector training data is a discrete design choice that defines RR1 vs RR2 and the winning systems A/B.
axioms (5)
  • domain assumption Spider exact-match and execution accuracy on the English Spider dev set are adequate proxies for NL2SQL system quality and for ranking component interactions.
    All main claims and Shapley values are computed only on Spider; Limitations notes missing WikiSQL/SParC/CoSQL and non-English queries.
  • ad hoc to paper Approximate Shapley values from the available (incomplete) combination scores preserve the ranking of component usefulness.
    Stated explicitly in §5.1 footnote: exact Shapley requires all combinations, which were not available.
  • domain assumption Removing PICARD from RASAT yields a fair baseline against which the authors’ own reranker can be compared.
    §2.1: PICARD is stripped so the new reranker is not confounded; this weakens the published RASAT baseline.
  • ad hoc to paper Cascaded omission of low-performing combinations (e.g., NatSmBoP+GAZP, vanilla RASAT+PP/GAZP) does not hide superior systems.
    Limitations admits these were skipped under the assumption weaker bases cannot overtake stronger ones when extended.
  • standard math Standard supervised classification losses and RoBERTa/GraPPa encoders are appropriate for SQL verification and selection.
    Background ML practice used for Verifier and Selector training without new theoretical claims.
invented entities (4)
  • SQL Verifier (RoBERTa sequence classifier over NLQ+schema+SQL) independent evidence
    purpose: Binary correctness prediction used to filter beams for final-beam reranking.
    New module introduced in §4.3; independent_evidence via precision/recall on held-out beams and extrinsic EX/EM when used as reranker.
  • SQL Selector and Binary SQL Selector (GraPPa + linear head) independent evidence
    purpose: Direct multi-class or pairwise selection of best SQL from a fixed candidate list.
    New module in §4.4; evaluated intrinsically by accuracy and extrinsically as reranker.
  • Hybrid Verifier+Binary-Selector reranker independent evidence
    purpose: Exploit high top-1 accuracy by building two-candidate beams then selecting.
    Compositional reranker design in §4.5; measured only within this paper’s pipelines.
  • NatSmBoP V.0/V.1/V.2 (NatSQL grammar + binary parser + macro-loss / beam decay variants) independent evidence
    purpose: Enable SmBoP to generate NatSQL intermediate trees.
    Engineering invention required for NatSQL+SmBoP; all variants underperform vanilla SmBoP on selection despite higher beam coverage.

pith-pipeline@v1.1.0-grok45 · 26685 in / 3971 out tokens · 40541 ms · 2026-07-14T08:21:09.360138+00:00 · methodology

0 comments
read the original abstract

In the age of large language models, Natural Language to SQL (NL2SQL) translation remains an open problem with many useful applications. We explore interactions between several NL2SQL pipeline extensions to inspire development of more lightweight models. Specifically, we integrate the NatSQL intermediate representation, include a preprocessing step and a fine-tuning step based on synthetic data, and develop a novel reranker model to improve SQL selection in the final beam. We perform an ablation study supplemented by a Shapley analysis of these different components integrated with two backbone architectures, SmBoP and RASAT. We find that simply combining all of them does not lead to best results, but that their impact depends on their interactions with the baseline system, as well as each other.

Figures

Figures reproduced from arXiv: 2607.10911 by Bora Caglayan, Filip Klubicka, John D. Kelleher, Mingxue Wang, Sneha Rautmare, Vasudevan Nedumpozhimana.

Figure 1
Figure 1. Figure 1: Points of optimisation in an NL2SQL pipeline. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Cumulative gain plot depicting the relationship between beam size and candidate accuracy for the SmBoP [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Cumulative gain plot depicting the relationship between beam size and candidate accuracy for the SmBoP [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: SQL difficulty distributions in Spider dev and train set, as well our largest synthetic GAZP dataset, as well as [PITH_FULL_IMAGE:figures/full_fig_p016_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

49 extracted references · 3 canonical work pages

  1. [1]

    Aho and Jeffrey D

    Alfred V. Aho and Jeffrey D. Ullman , title =. 1972

    1972

  2. [2]

    Publications Manual , year = "1983", publisher =

    1983

  3. [3]

    Chandra and Dexter C

    Ashok K. Chandra and Dexter C. Kozen and Larry J. Stockmeyer , year = "1981", title =. doi:10.1145/322234.322243

    doi:10.1145/322234.322243 1981

  4. [4]

    Scalable training of

    Andrew, Galen and Gao, Jianfeng , booktitle=. Scalable training of

  5. [5]

    Dan Gusfield , title =. 1997

    1997

  6. [6]

    Tetreault , title =

    Mohammad Sadegh Rasooli and Joel R. Tetreault , title =. Computing Research Repository , volume =. 2015 , url =

    2015

  7. [7]

    A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =

    Ando, Rie Kubota and Zhang, Tong , Issn =. A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =. Journal of Machine Learning Research , Month = dec, Numpages =

  8. [8]

    Advances in neural information processing systems , volume=

    Sequence to sequence learning with neural networks , author=. Advances in neural information processing systems , volume=

  9. [9]

    arXiv preprint arXiv:1409.0473 , year=

    Neural machine translation by jointly learning to align and translate , author=. arXiv preprint arXiv:1409.0473 , year=

    Pith/arXiv arXiv

  10. [10]

    Advances in neural information processing systems , volume=

    Attention is all you need , author=. Advances in neural information processing systems , volume=

  11. [11]

    Advances in neural information processing systems , volume=

    Language models are few-shot learners , author=. Advances in neural information processing systems , volume=

  12. [12]

    arXiv preprint arXiv:1909.00786 , year=

    Editing-based SQL query generation for cross-domain context-dependent questions , author=. arXiv preprint arXiv:1909.00786 , year=

    Pith/arXiv arXiv 1909

  13. [13]

    arXiv preprint arXiv:2009.07396 , year=

    Grounded adaptation for zero-shot executable semantic parsing , author=. arXiv preprint arXiv:2009.07396 , year=

    Pith/arXiv arXiv 2009

  14. [14]

    2022 IEEE Spoken Language Technology Workshop (SLT) , pages=

    N-best hypotheses reranking for text-to-SQL systems , author=. 2022 IEEE Spoken Language Technology Workshop (SLT) , pages=. 2023 , organization=

    2022

  15. [15]

    2023 , eprint=

    Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation , author=. 2023 , eprint=

    2023

  16. [16]

    2023 , eprint=

    DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction , author=. 2023 , eprint=

    2023

  17. [17]

    2023 , eprint=

    C3: Zero-shot Text-to-SQL with ChatGPT , author=. 2023 , eprint=

    2023

  18. [18]

    2023 , eprint=

    A comprehensive evaluation of ChatGPT's zero-shot Text-to-SQL capability , author=. 2023 , eprint=

    2023

  19. [19]

    2023 , eprint=

    SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL , author=. 2023 , eprint=

    2023

  20. [20]

    Proceedings of the national conference on artificial intelligence , pages=

    Learning to parse database queries using inductive logic programming , author=. Proceedings of the national conference on artificial intelligence , pages=

  21. [21]

    Proceedings of the VLDB Endowment , volume=

    ATHENA: an ontology-driven system for natural language querying over relational data stores , author=. Proceedings of the VLDB Endowment , volume=. 2016 , publisher=

    2016

  22. [22]

    arXiv preprint arXiv:2009.13845 , year=

    Grappa: Grammar-augmented pre-training for table semantic parsing , author=. arXiv preprint arXiv:2009.13845 , year=

    Pith/arXiv arXiv 2009

  23. [23]

    The Journal of Machine Learning Research , volume=

    Exploring the limits of transfer learning with a unified text-to-text transformer , author=. The Journal of Machine Learning Research , volume=. 2020 , publisher=

    2020

  24. [24]

    Proceedings of the AAAI Conference on Artificial Intelligence , author=

    RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL , volume=. Proceedings of the AAAI Conference on Artificial Intelligence , author=. 2023 , month=. doi:10.1609/aaai.v37i11.26535 , abstractNote=

    doi:10.1609/aaai.v37i11.26535 2023

  25. [25]

    arXiv preprint arXiv:1709.00103 , year=

    Seq2sql: Generating structured queries from natural language using reinforcement learning , author=. arXiv preprint arXiv:1709.00103 , year=

    Pith/arXiv arXiv

  26. [26]

    Kim, Hyeonji and So, Byeong-Hoon and Han, Wook-Shin and Lee, Hongrae , title =. Proc. VLDB Endow. , month =. 2021 , issue_date =. doi:10.14778/3401960.3401970 , abstract =

    doi:10.14778/3401960.3401970 2021

  27. [27]

    arXiv preprint arXiv:1711.04436 , year=

    Sqlnet: Generating structured queries from natural language without reinforcement learning , author=. arXiv preprint arXiv:1711.04436 , year=

    Pith/arXiv arXiv

  28. [28]

    arXiv preprint arXiv:1902.01069 , year=

    A comprehensive exploration on wikisql with table-aware word contextualization , author=. arXiv preprint arXiv:1902.01069 , year=

    Pith/arXiv arXiv 1902

  29. [29]

    arXiv preprint arXiv:2103.04399 , year=

    Improving text-to-sql with schema dependency learning , author=. arXiv preprint arXiv:2103.04399 , year=

    Pith/arXiv arXiv

  30. [30]

    2019 , eprint=

    RoBERTa: A Robustly Optimized BERT Pretraining Approach , author=. 2019 , eprint=

    2019

  31. [31]

    2023 , eprint=

    The Falcon Series of Open Language Models , author=. 2023 , eprint=

    2023

  32. [32]

    Contributions to the Theory of Games , volume=

    A value for n-person games , author=. Contributions to the Theory of Games , volume=

  33. [33]

    2025 , eprint=

    AGENTIQL: An Agent-Inspired Multi-Expert Framework for Text-to-SQL Generation , author=. 2025 , eprint=

    2025

  34. [34]

    2025 , eprint=

    MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQL , author=. 2025 , eprint=

    2025

  35. [35]

    Language to Logical Form with Neural Attention

    Dong, Li and Lapata, Mirella. Language to Logical Form with Neural Attention. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2016. doi:10.18653/v1/P16-1004

    doi:10.18653/v1/p16-1004 2016

  36. [36]

    and Manning, Christopher D

    See, Abigail and Liu, Peter J. and Manning, Christopher D. Get To The Point: Summarization with Pointer-Generator Networks. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2017. doi:10.18653/v1/P17-1099

    doi:10.18653/v1/p17-1099 2017

  37. [37]

    Bridging Textual and Tabular Data for Cross-Domain Text-to- SQL Semantic Parsing

    Lin, Xi Victoria and Socher, Richard and Xiong, Caiming. Bridging Textual and Tabular Data for Cross-Domain Text-to- SQL Semantic Parsing. Findings of the Association for Computational Linguistics: EMNLP 2020. 2020. doi:10.18653/v1/2020.findings-emnlp.438

    doi:10.18653/v1/2020.findings-emnlp.438 2020

  38. [38]

    RAT-SQL : Relation-Aware Schema Encoding and Linking for Text-to- SQL Parsers

    Wang, Bailin and Shin, Richard and Liu, Xiaodong and Polozov, Oleksandr and Richardson, Matthew. RAT-SQL : Relation-Aware Schema Encoding and Linking for Text-to- SQL Parsers. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020. doi:10.18653/v1/2020.acl-main.677

    doi:10.18653/v1/2020.acl-main.677 2020

  39. [39]

    LGESQL : Line Graph Enhanced Text-to- SQL Model with Mixed Local and Non-Local Relations

    Cao, Ruisheng and Chen, Lu and Chen, Zhi and Zhao, Yanbin and Zhu, Su and Yu, Kai. LGESQL : Line Graph Enhanced Text-to- SQL Model with Mixed Local and Non-Local Relations. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Paper...

    doi:10.18653/v1/2021.acl-long.198 2021

  40. [40]

    PICARD : Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models

    Scholak, Torsten and Schucher, Nathan and Bahdanau, Dzmitry. PICARD : Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021. doi:10.18653/v1/2021.emnlp-main.779

    doi:10.18653/v1/2021.emnlp-main.779 2021

  41. [41]

    RASAT : Integrating Relational Structures into Pretrained S eq2 S eq Model for Text-to- SQL

    Qi, Jiexing and Tang, Jingyao and He, Ziwei and Wan, Xiangpeng and Cheng, Yu and Zhou, Chenghu and Wang, Xinbing and Zhang, Quanshi and Lin, Zhouhan. RASAT : Integrating Relational Structures into Pretrained S eq2 S eq Model for Text-to- SQL. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022. doi:10.18653/v1/2022...

    work page doi:10.18653/v1/2022.emnlp-main.211 2022

  42. [42]

    S m B o P : Semi-autoregressive Bottom-up Semantic Parsing

    Rubin, Ohad and Berant, Jonathan. S m B o P : Semi-autoregressive Bottom-up Semantic Parsing. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021. doi:10.18653/v1/2021.naacl-main.29

    work page doi:10.18653/v1/2021.naacl-main.29 2021

  43. [43]

    S pider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to- SQL Task

    Yu, Tao and Zhang, Rui and Yang, Kai and Yasunaga, Michihiro and Wang, Dongxu and Li, Zifan and Ma, James and Li, Irene and Yao, Qingning and Roman, Shanelle and Zhang, Zilin and Radev, Dragomir. S pider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to- SQL Task. Proceedings of the 2018 Conference on Empirical...

    doi:10.18653/v1/d18-1425 2018

  44. [44]

    and Drake, John and Zhang, Qiaofu

    Gan, Yujian and Chen, Xinyun and Xie, Jinxia and Purver, Matthew and Woodward, John R. and Drake, John and Zhang, Qiaofu. Natural SQL : Making SQL Easier to Infer from Natural Language Specifications. Findings of the Association for Computational Linguistics: EMNLP 2021. 2021. doi:10.18653/v1/2021.findings-emnlp.174

    doi:10.18653/v1/2021.findings-emnlp.174 2021

  45. [45]

    Deduplicating Training Data Makes Language Models Better

    Lee, Katherine and Ippolito, Daphne and Nystrom, Andrew and Zhang, Chiyuan and Eck, Douglas and Callison-Burch, Chris and Carlini, Nicholas. Deduplicating Training Data Makes Language Models Better. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022. doi:10.18653/v1/2022.acl-long.577

    doi:10.18653/v1/2022.acl-long.577 2022

  46. [46]

    Improving Generalization in Language Model-based Text-to- SQL Semantic Parsing: Two Simple Semantic Boundary-based Techniques

    Rai, Daking and Wang, Bailin and Zhou, Yilun and Yao, Ziyu. Improving Generalization in Language Model-based Text-to- SQL Semantic Parsing: Two Simple Semantic Boundary-based Techniques. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 2023. doi:10.18653/v1/2023.acl-short.15

    work page doi:10.18653/v1/2023.acl-short.15 2023

  47. [47]

    and Kelleher, John D

    Klubi c ka, Filip and Salton, Giancarlo D. and Kelleher, John D. Is it worth it? Budget-related evaluation metrics for model selection. Proceedings of the Eleventh International Conference on Language Resources and Evaluation ( LREC 2018). 2018

    2018

  48. [48]

    C o SQL : A Conversational Text-to- SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases

    Yu, Tao and Zhang, Rui and Er, Heyang and Li, Suyi and Xue, Eric and Pang, Bo and Lin, Xi Victoria and Tan, Yi Chern and Shi, Tianze and Li, Zihan and Jiang, Youxuan and Yasunaga, Michihiro and Shim, Sungrok and Chen, Tao and Fabbri, Alexander and Li, Zifan and Chen, Luyao and Zhang, Yuwen and Dixit, Shreya and Zhang, Vincent and Xiong, Caiming and Socher...

    doi:10.18653/v1/d19-1204 2019

  49. [49]

    SP ar C : Cross-Domain Semantic Parsing in Context

    Yu, Tao and Zhang, Rui and Yasunaga, Michihiro and Tan, Yi Chern and Lin, Xi Victoria and Li, Suyi and Er, Heyang and Li, Irene and Pang, Bo and Chen, Tao and Ji, Emily and Dixit, Shreya and Proctor, David and Shim, Sungrok and Kraft, Jonathan and Zhang, Vincent and Xiong, Caiming and Socher, Richard and Radev, Dragomir. SP ar C : Cross-Domain Semantic Pa...

    doi:10.18653/v1/p19-1443 2019