Pith. sign in

REVIEW 5 major objections 5 minor 47 references

Can the Rookies Cut the Tough Cookie? Exploring the Use of LLMs for SQL Equivalence Checking

T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Large language models can classify SQL query equivalence on realistic, complex queries that formal tools cannot handle, with GPT-4 reaching about 82% accuracy on Spider+DIN and about 60% on the new SQLEquiQuest benchmark, while most other…

desk verdict A genuine new benchmark and a robust equivalence-bias finding; the headline accuracy numbers are upper bounds because ground truth is execution-based and prompts were tuned on the test benchmarks. read the letter →

arxiv 2412.05561 v2 pith:HYSLG7FX submitted 2024-12-07 cs.DB cs.LG

Rajat Singh , Srikanta Bedathur This is my paper · ORCID
classification cs.DBcs.LG
keywords SQLqueryequivalencelargelanguagemodelsSQLEquiQuestpromptingstrategieslogicalplanbiasSpider+DINtext-to-SQL
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that large language models can serve as practical SQL query equivalence checkers for realistic, complex queries that formal verification tools cannot process. It reports that GPT-4 reaches up to 82% accuracy on Spider+DIN and about 60% accuracy on SQLEquiQuest, a new benchmark built from graduate-level database assignment submissions, whereas formal tools support only about 30% of SQLEquiQuest pairs. The paper also claims that prompting strategies, in-context examples, and unoptimized logical plans improve LLM performance. A central caveat is that most tested LLMs are biased toward predicting equivalence, performing well on equivalent pairs but poorly on non-equivalent ones.

What carries the argument

The central object is the prompt-to-classifier pipeline: four prompting strategies (basic, chain-of-thought, few-shot, and multi-stage chain-of-thought) feed SQL pairs plus optional unoptimized logical plans from Apache Calcite into an LLM, and a separate GPT classifier maps verbose outputs to 'Equivalent', 'Non-Equivalent', or 'Unknown'. The load-bearing mechanism is the in-context reasoning scaffold: the logical plan makes query semantics explicit, the step-by-step prompts force intermediate explanations, and few-shot examples provide ground-truth demonstrations, together letting the LLM align its equivalence judgment with the execution-based labels. The benchmark SQLEquiQuest is the other central object: a dataset of student assignment submissions against instructor-written ground-truth queries, labeled by execution on a large test database instance.

What would settle it

Take any pair SQLEquiQuest labels equivalent and search exhaustively over all small database instances for the schema, or use a solver like SQLSolver on the formal all-instances definition; if any instance yields different outputs, that label is wrong and the reported accuracy is measured against a flawed ground truth.

Watch

Extended reading notes

Core claim

The central claim is that LLMs, treated as black-box experts, go beyond the coverage of formal SQL equivalence checkers: whereas SQLSolver supports only about 30% of SQLEquiQuest pairs and VeriEQL supports 2.8%, GPT-4 with the best prompting strategy correctly classifies roughly 82% of Spider+DIN pairs and about 60% of SQLEquiQuest pairs. The paper also finds that this capability is uneven: most tested LLMs classify almost all equivalent pairs correctly but systematically mislabel non-equivalent pairs as equivalent, whereas GPT-4 maintains much stronger performance on non-equivalent pairs. Adding an unoptimized logical plan from Apache Calcite to the prompt generally improves results, and fine-tuning Code Llama-13B substantially improves non-equivalent detection.

Load-bearing premise

The ground-truth equivalence labels in both main benchmarks come from running queries on one large test database or test suite, and the paper provides no proof that this instance separates every pair of queries that differ on some database.

Editorial extensions

If this is right

  • GPT-4 can classify SQL equivalence on essentially all query pairs in these benchmarks, where formal tools support at most 30%, so LLM-based checking is a viable first-line tool for realistic query pairs.
  • On Spider+DIN, adding a logical plan to the prompt improves geometric-mean accuracy for most models and prompting strategies, with the largest gains for the smallest model, Code Llama-7B.
  • Most LLMs are biased toward predicting equivalence, with near-zero accuracy on non-equivalent pairs, so any practical deployment must guard against this by calibration or fine-tuning.
  • Fine-tuning Code Llama-13B on labeled equivalence pairs raises non-equivalent accuracy substantially (for example, from 0.5% to 23.8% under basic prompting) while roughly preserving equivalent-pair performance.
  • GPT-4 correctly classifies more than 80% of the SQLEquiQuest pairs SQLSolver supports and about 52% of the pairs SQLSolver does not support, indicating the LLM approach extends rather than merely replaces formal tools.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because SQLEquiQuest and Spider+DIN labels come from execution on one test database or test suite rather than all possible databases, the reported accuracies likely overstate true logical equivalence; a more rigorous benchmark would certify labels by exhaustive search or formal proofs on small schemas.
  • The pervasive equivalence bias suggests a testable extension: ask the LLM to generate a counterexample database instance for each predicted non-equivalence, then execute both queries on it; if it cannot produce one, the prediction should be downgraded.
  • A hybrid pipeline is a natural next step: use a fast biased LLM to filter obviously equivalent pairs and route uncertain or non-equivalent pairs to a formal tool like SQLSolver, potentially recovering formal guarantees where the LLM alone cannot.
  • The paper's finding that GPT-4 performs poorly on Calcite despite strong performance elsewhere hints that training-data overlap and syntactic style, not just semantic difficulty, drive LLM equivalence judgments; a contamination-controlled benchmark would be needed to separate these factors.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces SQLEquiQuest, a new benchmark of SQL query equivalence pairs drawn from DBMS assignment submissions, and evaluates five LLMs with four prompting strategies on Calcite, Spider+DIN, and SQLEquiQuest. It reports that LLMs, particularly GPT-4, achieve accuracy up to 82% on Spider+DIN and 61% on SQLEquiQuest, considerably extending coverage over formal tools, but that most LLMs are strongly biased toward predicting equivalence. The paper also studies logical-plan augmentation and fine-tuning of Code Llama-13B.

Significance. The benchmark is a useful resource and the paper is among the first to systematically study LLMs on this task; the NEQ-bias finding is robust and actionable, and the code is publicly available. However, the central accuracy claims are compromised by execution-based ground truth, in-sample prompt tuning, and few-shot examples drawn from the test set. The formal-tool comparison is interesting but inherits these issues.

major comments (5)
  1. [§3.1, §C.1, §2] The ground-truth labels in both benchmarks are not the formal all-instance equivalence defined in Section 2. SQLEquiQuest labels were assigned by executing queries on 'a large database instance designed to account for all potential corner cases,' and Spider+DIN labels were assigned with test-suite-sql-eval. A finite execution check is sound only for non-equivalence; agreement on a single instance or test suite does not prove equivalence. Because the majority of pairs are labeled equivalent (307/499 in SQLEquiQuest, 385/574 in Spider+DIN), any false-equivalent labels inflate the EQ accuracies and the GM numbers in Tables 2 and 3, including the headline 82% and 61%. The manuscript itself exposes this gap: Section 3.1 asserts, but does not demonstrate, that the instance covers all corner cases. Please provide formal validation of the equivalent labels (e.g., by running a formal prover on supported pairs or by giving a separation argument), or restrict all claims to execution-based equivalence and make the EQ/NEQ distinction the primary reporting dimension.
  2. [§5.4] The authors state that all prompts were 'carefully designed and refined through iterative testing with each LLM used.' Since the same benchmarks are used both for prompt development and for final scoring, the reported numbers are in-sample. This is particularly problematic for comparing prompting strategies: the differences among P1-P4 may reflect overfitting to the specific test distribution rather than general capability. Please use a held-out development set for prompt tuning and report results on a separate test split, or explicitly describe the reported numbers as upper bounds.
  3. [§4.1 (P3), §B.3] Few-shot examples were 'randomly sampled from the dataset and then fixed for all test samples.' Drawing in-context demonstrations from the same benchmark that is being scored leaks information about the test distribution (and potentially about the exact query pairs), inflating few-shot accuracy. Use few-shot examples from a disjoint development set or from query pairs that are excluded from the evaluation set.
  4. [§4.2, §B.5] The final labels are produced by a GPT-family classifier with no reported validation; the 'Unknown' class is treated as a negative result. Classification errors propagate directly into Tables 2 and 3. Please report the classifier's accuracy on a human-annotated sample of outputs, and state explicitly how 'Unknown' is counted in the accuracy definition.
  5. [Tables 2 and 3] No repeated runs, confidence intervals, or significance tests are reported. All API LLMs are sampled with temperature 0.2, so the single-run differences (e.g., GPT-4 vs. GPT-3.5 on Question 3 with CoT) may be within run-to-run variation. Please provide multiple runs or bootstrap intervals and avoid drawing conclusions from small numerical gaps.
minor comments (5)
  1. [§3.1, Table 1] The text says 'for each question in Spider+DIN' but the table is for SQLEquiQuest; correct the reference.
  2. [§D.1] The hyperparameter 'do sample' should be 'do_sample' to match standard PyTorch/HuggingFace naming.
  3. [References] The Brown et al. 2020a and 2020b entries are duplicate citations of the same work and should be merged.
  4. [Figure 2 caption] The radial axis is described as 'the average number of corresponding feature'; this should be 'the average number of corresponding features per query' for clarity.
  5. [§5.2] The phrase 'full coverage' should be qualified: LLMs produce an output for every pair, but 'coverage' in Table 4 refers to the tools' ability to verify, not to the correctness of the LLM's predictions; a direct coverage comparison is therefore misleading.

Circularity Check

0 steps flagged · score 2.0 of 10

No material circularity: the paper is an empirical LLM benchmark evaluation, and its accuracy claims do not reduce by construction to its own inputs.

full rationale

The paper contains no derivation chain in which a prediction is algebraically or definitionally identical to an input. The central claims are empirical: LLMs achieve up to 82% accuracy on Spider+DIN and 61% on SQLEquiQuest, with a bias toward predicting equivalence. These numbers are computed by comparing LLM outputs to benchmark labels, not by fitting a parameter and renaming it a prediction. The closest concerns are label-validity issues, not circularity. Section 2 formally defines equivalence over all instances, while Appendix C.1 states that Spider+DIN labels were assigned with test-suit-sql-eval (finite test-suite execution) and Section 3.1 states that SQLEquiQuest labels came from teaching assistants executing queries on a "large database instance designed to account for all potential corner cases," with no proof that this instance separates all inequivalent pairs. This means the benchmark's "equivalent" classes may be a superset of true formal equivalence, and reported EQ accuracy could be inflated. That is a correctness and benchmark-validity limitation, not a circular reduction: the LLM predictions are not used to construct the labels, and the labels are not derived from the model. The few-shot examples are "randomly sampled from the dataset and then fixed for all test samples," which is standard in-context evaluation practice, though the handful of example items could trivially leak their labels; the GPT-family classifier used to convert free-text outputs into labels is an evaluation-pipeline choice and does not define the target quantity. The only reference with author overlap with this paper is Agarwal et al. (2024), cited once in future-work suggestions on RAG; it is not load-bearing. Under the stated rules, unproven ground-truth assumptions and non-load-bearing self-citation do not constitute circularity, so the appropriate finding is no significant circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

No new physical or mathematical entities are postulated; this is an empirical benchmark study whose central claims rest on benchmark labels, prompt choices, and classifier behavior rather than on derived constants.

free parameters (5)
  • Prompt template set (P1-P4) = iteratively tuned per LLM
    Section 5.4 states prompts were refined through iterative testing with each LLM; scores depend on these hand-chosen templates.
  • Few-shot example selection = 4 examples sampled from benchmark and fixed
    Section B.3; examples drawn from the evaluation dataset may leak structure into prompts.
  • Output classifier = GPT-family LLM; Unknown treated as negative
    Section B.5; final accuracy depends on this second LLM and on treating Unknown as non-equivalence.
  • Inference hyperparameters = temperature 0.2, max_new_tokens 500-10000, top_k 1, num_beams 1
    Appendix D.1; chosen by authors, results may vary across seeds and API versions.
  • LoRA fine-tuning hyperparameters = r=16, alpha=16, dropout=0.05, lr=3e-4, steps=400
    Appendix D.2; used for the fine-tuning results in Table D.2, not the main benchmark table.
assumptions (5)
  • domain assumption Execution equivalence on one database instance or test suite matches semantic equivalence over all instances.
    Section 3.1 for SQLEquiQuest labels and Appendix C.1 for Spider+DIN via test-suit-sql-eval; the paper's own formal definition in Section 2 requires all instances, so this is an unverified reduction.
  • domain assumption Apache Calcite logical plans are faithful and useful for equivalence reasoning.
    Section 4.2; plans generated by Calcite, syntax errors replaced with a placeholder, with no validation that the LP is correct or beneficial beyond observed scores.
  • domain assumption A GPT-family classifier faithfully extracts the LLM's conclusion.
    Section B.5; output classification into Equivalent, Non-Equivalent, or Unknown can introduce errors that propagate into all reported accuracies.
  • domain assumption LLM responses are stable enough for single-run evaluation.
    Section 5.4 documents prompt sensitivity and unpredictable outputs; no repeats or variance are reported.
  • standard math Equivalence checking of arbitrary SQL queries is undecidable in general.
    Section 2 cites Chandra and Merlin and other standard references; this is background for why heuristic LLM baselines are interesting.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Can the Rookies Cut the Tough Cookie? Exploring the Use of LLMs for SQL Equivalence Checking." pith.science (2026). https://pith.science/paper/HYSLG7FX

@misc{pith2026241205561,
  author       = {Pith},
  title        = {Pith review of: Can the Rookies Cut the Tough Cookie? Exploring the Use of LLMs for SQL Equivalence Checking},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HYSLG7FX}},
  note         = {Machine review of arXiv:2412.05561}
}
read the original abstract

Equivalence checking of SQL queries is an intractable problem often encountered in settings ranging from grading SQL submissions to debugging query optimizers. Despite recent work toward developing practical solutions, only simple queries written using a small subset of SQL are supported, leaving the equivalence checking of sophisticated SQL queries at the mercy of intensive, potentially error-prone, manual analysis. In this paper, we explore how LLMs can be used to reason with SQL queries to address this challenging problem. Towards this, we introduce a novel, realistic, and sufficiently complex benchmark called SQLEquiQuest for SQL query equivalence checking that reflects real-world settings. We establish strong baselines for SQL equivalence checking by leveraging the ability of LLMs to reason with SQL queries. We conduct a detailed evaluation of several state-of-the-art LLMs using various prompting strategies and carefully constructed in-context learning examples, including logical plans generated by SQL query processors. Our empirical evaluation shows that LLMs go well beyond the current capabilities of formal models for SQL equivalence, going from a mere 30% supported query pairs to full coverage, achieving up to 82% accuracy on Spider+DIN. However, a critical limitation of LLMs revealed by our analysis is that they exhibit a strong bias for equivalence predictions, with consistently poor performance over non-equivalent pairs, opening a new direction for potential future research.

Figures

Figures reproduced from arXiv: 2412.05561 by the authors.

Figure 1
Figure 1. Example of a non-equivalent pair of SQL queries from a database assignment [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Distribution of various SQL features in Calcite, Spider+DIN, and SQLEquiQuest dataset. The radial axis represents the average number of corresponding feature in a single query in the benchmark. Also note that Calcite contains only equivalent pairs of SQL queries. instance of the schema D, executing both queries produces identical results. That is, for all valid database instances I conforming to D, the outputs of Q1… view at source ↗
Figure 3
Figure 3. Prompting strategies employed (See Section [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Performance of different LLMs under different prompting methods on [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Performance of GPT-4 on Spider+DIN dataset split based on difficulty level. 5.3 Impact of Query Complexity We examined the impact of query complexity on the performance of LLMs using the Spi￾der+DIN dataset, categorizing queries based on complexity levels (Easy, Medium…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 25 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    F oundations of D atabases: T he L ogical L evel

    Serge Abiteboul, Richard Hull, and Victor Vianu. F oundations of D atabases: T he L ogical L evel . Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1995. URL https://api.semanticscholar.org/CorpusID:60457482

    work page 1995

  3. [3]

    Sym KGQA : Few-shot knowledge graph question answering via symbolic program generation and execution

    Prerna Agarwal, Nishant Kumar, and Srikanta Bedathur. Sym KGQA : Few-shot knowledge graph question answering via symbolic program generation and execution. In The 62nd Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2024. URL https://openreview.net/forum?id=nwlIPR4NwR

    work page 2024

  4. [4]

    Exploring large language models for code explanation

    Paheli Bhattacharya, Manojit Chakraborty, Kartheek N S N Palepu, Vikas Pandey, Ishan Dindorkar, Rakesh Rajpurohit, and Rishabh Gupta. Exploring large language models for code explanation. ArXiv, abs/2310.16673, 2023. URL https://api.semanticscholar.org/CorpusID:264451660

    arXiv 2023

  5. [6]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeff Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, S...

    arXiv 2005

  6. [7]

    Sadga: Structure-aware dual graph aggregation network for text-to-sql

    Ruichu Cai, Jinjie Yuan, Boyan Xu, and Zhifeng Hao. Sadga: Structure-aware dual graph aggregation network for text-to-sql. ArXiv, abs/2111.00653, 2021. URL https://api.semanticscholar.org/CorpusID:240353884

    arXiv 2021

  7. [8]

    Calcite Test Suite , 2021

    Apache C alcite. Calcite Test Suite , 2021. URL https://ipads.se.sjtu.edu.cn:1312/opensource/wetune/-/blob/main/ wtune_data/calcite/calcite_tests

    work page 2021

  8. [9]

    Chandra and Philip M

    Ashok K. Chandra and Philip M. Merlin. Optimal Implementation of Conjunctive Queries in Relational Data Bases . In Proceedings of the ACM Symposium on Theory of Computing (STOC), 1977

    work page 1977

Show all 47 references
  1. [10]

    Bikash Chandra, Amol Bhangadia, Bhupesh Chawda, Biplab Kar, K. V. Maheshwara Reddy, Shetal Shah, and S. Sudarshan. Data generation for testing and grading SQL queries. The VLDB Journal, 24: 0 731 -- 755, 2014. URL https://api.semanticscholar.org/CorpusID:16787573

    2014

  2. [11]

    Surajit Chaudhuri and Moshe Y. Vardi. Optimization of real conjunctive queries. Proceedings of the twelfth ACM SIGACT-SIGMOD-SIGART symposium on Principles of database systems, 1993. URL https://api.semanticscholar.org/CorpusID:115572

    1993

  3. [12]

    Conjunctive query containment revisited

    Chandra Chekuri and Anand Rajaraman. Conjunctive query containment revisited. In Theoretical Computer Science, 1997. URL https://api.semanticscholar.org/CorpusID:8506799

    1997

  4. [13]

    Hottsql: proving query rewrites with univalent sql semantics

    Shumo Chu, Konstantin Weitz, Alvin Cheung, and Dan Suciu. Hottsql: proving query rewrites with univalent sql semantics. Proceedings of the 38th ACM SIGPLAN Conference on Programming Language Design and Implementation, 2016. URL https://api.semanticscholar.org/CorpusID:644867

    2016

  5. [14]

    Cosette: An A utomated P rover for SQL

    Shumo Chu, Chenglong Wang, Konstantin Weitz, and Alvin Cheung. Cosette: An A utomated P rover for SQL . In Conference on Innovative Data Systems Research, 2017. URL https://api.semanticscholar.org/CorpusID:12408033

    2017

  6. [15]

    Axiomatic foundations and algorithms for deciding semantic equivalences of SQL queries

    Shumo Chu, Brendan Murphy, Jared Roesch, Alvin Cheung, and Dan Suciu. Axiomatic foundations and algorithms for deciding semantic equivalences of SQL queries. Proc. VLDB Endow., 11 0 (11), 2018. ISSN 2150-8097. doi:10.14778/3236187.3236200. URL https://doi.org/10.14778/3236187.3236200

    2018

  7. [16]

    Equivalence of queries combining set and bag-set semantics

    Sara Cohen. Equivalence of queries combining set and bag-set semantics. Proceedings of the twenty-fifth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, 2006. URL https://api.semanticscholar.org/CorpusID:12028668

    2006

  8. [17]

    P roving Q uery E quivalence U sing L inear I nteger A rithmetic

    Haoran Ding, Zhaoguo Wang, Yicun Yang, Dexin Zhang, Zhenglin Xu, Haibo Chen, Ruzica Piskac, and Jinyang Li. P roving Q uery E quivalence U sing L inear I nteger A rithmetic. Proceedings of the ACM on Management of Data, 1: 0 1 -- 26, 2023. URL https://api.semanticscholar.org/C...

    2023

  9. [18]

    Text-to-sql empowered by large language models: A benchmark evaluation

    Dawei Gao, Haibin Wang, Yaliang Li, Xiuyu Sun, Yichen Qian, Bolin Ding, and Jingren Zhou. Text-to-sql empowered by large language models: A benchmark evaluation. ArXiv, abs/2308.15363, 2023 a . URL https://api.semanticscholar.org/CorpusID:261276437

    2023 arXiv

  10. [19]

    Shuzheng Gao, Xinjie Wen, Cuiyun Gao, Wenxuan Wang, and Michael R. Lyu. What makes good in-context demonstrations for code intelligence tasks with llms? 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE), pp.\ 761--773, 2023 b . URL https://api...

    2023

  11. [20]

    Gemini: A family of highly capable multimodal models

    Google Gemini Team. Gemini: A family of highly capable multimodal models. ArXiv, abs/2312.11805, 2023. URL https://api.semanticscholar.org/CorpusID:266361876

    2023 arXiv

  12. [21]

    Large language models are few-shot summarizers: Multi-intent comment generation via in-context learning

    Mingyang Geng, Shangwen Wang, Dezun Dong, Hao Wang, Ge Li, Zhi Jin, Xiaoguang Mao, and Xiangke Liao. Large language models are few-shot summarizers: Multi-intent comment generation via in-context learning. 2024 IEEE/ACM 46th International Conference on Software Engineering (IC...

    2024

  13. [22]

    GE q O : ML - A ccelerated S emantic E quivalence D etection

    Brandon Haynes, Rana Alotaibi, Anna Pavlenko, Jyoti Leeka, Alekh Jindal, and Yuanyuan Tian. GE q O : ML - A ccelerated S emantic E quivalence D etection. Proceedings of the ACM on Management of Data, 1: 0 1 -- 25, 2023. URL https://api.semanticscholar.org/CorpusID:266192443

    2023

  14. [23]

    Verieql: Bounded equivalence verification for complex sql queries with integrity constraints

    Yang He, Pinhan Zhao, Xinyu Wang, and Yuepeng Wang. Verieql: Bounded equivalence verification for complex sql queries with integrity constraints. ArXiv, abs/2403.03193, 2024. URL https://api.semanticscholar.org/CorpusID:268249011

    2024 arXiv

  15. [24]

    Ioannidis and Raghu Ramakrishnan

    Yannis E. Ioannidis and Raghu Ramakrishnan. C ontainment of conjunctive queries: beyond relations as sets. ACM Trans. Database Syst., 20: 0 288--324, 1995. URL https://api.semanticscholar.org/CorpusID:9431903

    1995

  16. [25]

    Query optimization in database systems

    Matthias Jarke and Jurgen Koch. Query optimization in database systems. ACM Comput. Surv., 16 0 (2): 0 111–152, June 1984. ISSN 0360-0300. doi:10.1145/356924.356928. URL https://doi.org/10.1145/356924.356928

    1984

  17. [26]

    Johnson and Anthony C

    David S. Johnson and Anthony C. Klug. Optimizing conjunctive queries that contain untyped variables. SIAM J. Comput., 12: 0 616--640, 1983. URL https://api.semanticscholar.org/CorpusID:39006673

    1983

  18. [27]

    Large language models are zero-shot reasoners

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. ArXiv, abs/2205.11916, 2022. URL https://api.semanticscholar.org/CorpusID:249017743

    2022 arXiv

  19. [28]

    Aiwei Liu, Xuming Hu, Lijie Wen, and Philip S. Yu. A comprehensive evaluation of chatgpt's zero-shot text-to-sql capability. ArXiv, abs/2303.13547, 2023 a . URL https://api.semanticscholar.org/CorpusID:257757019

    2023 arXiv

  20. [29]

    G-eval: Nlg evaluation using gpt-4 with better human alignment

    Yang Liu, Dan Iter, Yichong Xu, Shuo Wang, Ruochen Xu, and Chenguang Zhu. G-eval: Nlg evaluation using gpt-4 with better human alignment. In Conference on Empirical Methods in Natural Language Processing, 2023 b . URL https://api.semanticscholar.org/CorpusID:257804696

    2023

  21. [30]

    R eview: B

    Andrzej Wlodzimierz Mostowski. R eview: B . A . T rahtenbrot, I mpossibility of an A lgorithm for the D ecision P roblem in F inite C lasses. Journal of Symbolic Logic, 15: 0 229 -- 229, 1950. URL https://api.semanticscholar.org/CorpusID:122740460

    1950

  22. [31]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff ...

    2024 arXiv

  23. [32]

    Din-sql: Decomposed in-context learning of text-to-sql with self-correction

    Mohammad Reza Pourreza and Davood Rafiei. Din-sql: Decomposed in-context learning of text-to-sql with self-correction. ArXiv, abs/2304.11015, 2023. URL https://api.semanticscholar.org/CorpusID:258291425

    2023 arXiv

  24. [33]

    Rasat: Integrating relational structures into pretrained seq2seq model for text-to-sql

    Jiexing Qi, Jingyao Tang, Ziwei He, Xiangpeng Wan, Chenghu Zhou, Xinbing Wang, Quanshi Zhang, and Zhouhan Lin. Rasat: Integrating relational structures into pretrained seq2seq model for text-to-sql. ArXiv, abs/2205.06983, 2022. URL https://api.semanticscholar.org/CorpusID:248810824

    2022 arXiv

  25. [34]

    Baptiste Rozi \`e re, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Tan, Yossi Adi, Jingyu Liu, Tal Remez, J \'e r \'e my Rapin, Artyom Kozhevnikov, I. Evtimov, Joanna Bitton, Manish P Bhatt, Cristian Cant \'o n Ferrer, Aaron Grattafiori, Wenhan Xiong, Alexan...

    2023 arXiv

  26. [35]

    A prompt learning framework for source code summarization

    Weisong Sun, Chunrong Fang, Yudu You, Yuchen Chen, Yi Liu, Chong Wang, Jian Zhang, Quanjun Zhang, Hanwei Qian, Wei-Li Zhao, Yang Liu, and Zhenyu Chen. A prompt learning framework for source code summarization. ArXiv, abs/2312.16066, 2023. URL https://api.semanticscholar.org/Co...

    2023 arXiv

  27. [36]

    T owards P ractical SQL E quivalence R easoning

    Shuxian Wang. T owards P ractical SQL E quivalence R easoning. Technical Report UCB/EECS-2023-188, Electrical Engineering and Computer Sciences, University of California, Berkeley, 2023

    2023

  28. [37]

    We T une: A utomatic D iscovery and V erification of Q uery R ewrite R ules

    Zhaoguo Wang, Zhou Yang Zhou, Yicun Yang, Haoran Ding, Gansen Hu, Ding Ding, Chuzhe Tang, Haibo Chen, and Jinyang Li. We T une: A utomatic D iscovery and V erification of Q uery R ewrite R ules. Proceedings of the 2022 International Conference on Management of Data, 2022. URL ...

    2022

  29. [38]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. ArXiv, abs/2201.11903, 2022. URL https://api.semanticscholar.org/CorpusID:246411621

    2022 arXiv

  30. [39]

    Wang, Victor Zhong, Bailin Wang, Chengzu Li, Connor Boyle, Ansong Ni, Ziyu Yao, Dragomir R

    Tianbao Xie, Chen Henry Wu, Peng Shi, Ruiqi Zhong, Torsten Scholak, Michihiro Yasunaga, Chien-Sheng Wu, Ming Zhong, Pengcheng Yin, Sida I. Wang, Victor Zhong, Bailin Wang, Chengzu Li, Connor Boyle, Ansong Ni, Ziyu Yao, Dragomir R. Radev, Caiming Xiong, Lingpeng Kong, Rui Zhang...

    2022 arXiv

  31. [40]

    Tabert: Pretraining for joint understanding of textual and tabular data

    Pengcheng Yin, Graham Neubig, Wen tau Yih, and Sebastian Riedel. Tabert: Pretraining for joint understanding of textual and tabular data. ArXiv, abs/2005.08314, 2020. URL https://api.semanticscholar.org/CorpusID:218674345

    2005 arXiv

  32. [41]

    Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task

    Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, et al. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task . arXiv preprint arXiv:1809.08887, 2018

    2018 arXiv

  33. [42]

    Llm-sql-solver: Can llms determine sql equivalence? ArXiv, abs/2312.10321, 2023

    Fuheng Zhao, Lawrence Lim, Ishtiyaque Ahmad, Divyakant Agrawal, and Amr El Abbadi. Llm-sql-solver: Can llms determine sql equivalence? ArXiv, abs/2312.10321, 2023. URL https://api.semanticscholar.org/CorpusID:266348267

    2023 arXiv

  34. [43]

    Semantic evaluation for text-to-sql with distilled test suite

    Ruiqi Zhong, Tao Yu, and Dan Klein. Semantic evaluation for text-to-sql with distilled test suite. In The 2020 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2020

    2020

  35. [44]

    Navathe, William R

    Qi Zhou, Joy Arulraj, Shamkant B. Navathe, William R. Harris, and Dong Xu. Automated V erification of Q uery E quivalence U sing S atisfiability M odulo T heories. Proc. VLDB Endow., 12: 0 1276--1288, 2019. URL https://api.semanticscholar.org/CorpusID:201625543

    2019

  36. [45]

    Navathe, William Harris, and Jinpeng Wu

    Qi Zhou, Joy Arulraj, Shamkant B. Navathe, William Harris, and Jinpeng Wu. Spes: A symbolic approach to proving query equivalence under bag semantics. 2022 IEEE 38th International Conference on Data Engineering (ICDE), pp.\ 2735--2748, 2022. URL https://api.semanticscholar.org...

    2022

  37. [46]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  38. [47]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  39. [48]

    Batting" -- Columns:

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

    2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.