Pith. sign in

REVIEW 4 major objections 5 minor 32 references

LogicCat: A Chain-of-Thought Text-to-SQL Benchmark for Complex Reasoning

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper introduces LogicCat, a text-to-SQL benchmark whose questions require physics, arithmetic, commonsense, and hypothetical reasoning, and reports that the strongest evaluated system reaches only 33.20% execution accuracy.

desk verdict LogicCat is a genuinely useful reasoning-heavy text-to-SQL benchmark, but its headline numbers rest on gold SQL whose semantic correctness is not established, so the difficulty claim is indicative rather than definitive. read the letter →

arxiv 2505.18744 v3 pith:WN6XJN6G submitted 2025-05-24 cs.CL

Tao Liu , Xutao Mao , Hongying Zan , Dixuan Zhang , Yifan Li , Haixin Liu , Lulu Kong , Jiaming Hou
show 7 more authors
Rui Li YunLong Li aoze zheng Zhiqiang Zhang Luo Zhewei Kunli Zhang Min Peng
This is my paper
classification cs.CL
keywords text-to-SQLchain-of-thoughtbenchmarkdatasetcomplexreasoningexecutionaccuracyphysicalknowledgecommonsensehypothetical
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that existing text-to-SQL benchmarks miss the multi-step reasoning that real data analysis requires, and that this blind spot is measurable. It introduces LogicCat, a benchmark of 4,038 English questions paired with 12,114 chain-of-thought reasoning steps across 45 databases, built so that answering questions requires physics formulas, arithmetic, commonsense, and hypothetical scenarios. In zero-shot evaluation, the best system reaches 33.20% execution accuracy, and accuracy on hard questions falls to at most 14.96%. The paper's ablation shows that removing the chain-of-thought annotations lowers execution accuracy on every tested model, with the largest drops above 19 points. The point of the benchmark is to give systems that claim to handle real-world analytical SQL a test where the reasoning demand is explicit.

What carries the argument

The carrying mechanism is the benchmark design itself. Each LogicCat question is paired with a gold SQL query and a chain-of-thought annotation that decomposes the reasoning into numbered steps, and the questions are constructed to force four reasoning types — physical knowledge (35.0% of questions), mathematical logic (78%), commonsense (59%), and ideal hypothetical scenarios (25%) — often in combination. Difficulty tiers are defined by SQL token length and the number of symbolic arithmetic operators, and the dataset averages 4.42 arithmetic operators and 3.1 joins per query. Evaluation uses VES, which only checks that a predicted query executes, and execution accuracy, which requires exact match of the executed result; the gap between the two is what lets the paper attribute failures to reasoning rather than syntax. The quality gate is an executability check on the gold SQL, which rose to a 95.05% valid-execution rate after expert review.

What would settle it

Independently recompute the answers to a random sample of LogicCat questions — solving the physics and arithmetic by hand — and compare them with the outputs of the gold SQL; a mismatch rate above a few percent would mean the execution-accuracy numbers are not a clean measure of model reasoning.

Watch

Extended reading notes

Core claim

LogicCat is presented as the first text-to-SQL benchmark designed around chain-of-thought parsing. The central discovery is a performance ceiling: the strongest evaluated approach reaches only 33.20% execution accuracy overall, and hard questions cap near 14.96%, while syntactically valid execution rates reach about 70% for the best models. The gold queries come with 12,114 explicit reasoning steps, and those steps are load-bearing: an ablation shows that removing chain-of-thought annotations lowers execution accuracy in every tested case, with the largest drops above 19 points. The paper reads this as evidence that current models can parse schemas but cannot yet chain external knowledge, unit conversions, and hypothetical assumptions into correct SQL.

Load-bearing premise

The accuracy numbers are only meaningful if each gold SQL query actually returns the answer to the question it accompanies; the paper verifies that the gold queries are executable (95.05% VES) but does not verify that their results are semantically correct answers.

Editorial extensions

If this is right

  • If LogicCat's difficulty numbers hold, text-to-SQL systems aimed at real analytics must be tested on chained computations and domain knowledge, not just retrieval over business schemas.
  • Because removing the chain-of-thought annotations consistently degrades accuracy, stepwise reasoning is a load-bearing part of the benchmark and a practical requirement for deployment.
  • The category breakdown — commonsense easiest, physics and math hardest — gives a concrete target: improving formula application and unit handling should move overall execution accuracy more than general SQL tuning.
  • Releasing the dataset lets any future system be measured against the same 33.20% ceiling, making progress on complex reasoning a trackable quantity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: a natural next step is to measure how often a model's stated chain-of-thought steps are actually consistent with its final SQL, turning the annotations into a new 'reasoning fidelity' metric.
  • Inference: the paper does not report human performance, so an independent sample of human accuracy on a subset of questions would clarify how much of the 33.20% ceiling reflects model limits rather than ambiguity in the questions themselves.
  • Inference: the large gap between valid-execution and exact-match scores suggests errors concentrate after syntax generation, so a targeted error taxonomy could guide training data that fixes wrong-query and incorrect-knowledge failures first.
  • Inference: perturbing database values, adding distractor columns, or changing unit systems would test whether models generalize the underlying reasoning or rely on surface patterns in the schema.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces LogicCat, a new text-to-SQL benchmark containing 4,038 English questions paired with chain-of-thought annotations across 45 databases, organized around four reasoning categories: physical knowledge, mathematical logic, commonsense, and hypothetical reasoning. The authors describe a multi-stage construction and quality-control process with expert review, report dataset complexity statistics, and evaluate a wide range of general-purpose, coder, and specialized text-to-SQL models in a zero-shot setting. Their headline result is that the best method, Chess-SQL with GPT-4o, reaches only 33.20% execution accuracy, and that removing chain-of-thought annotations degrades performance substantially across all tested models.

Significance. If the gold SQL queries are semantically correct and the evaluation protocol is free of leakage, LogicCat would be a useful and timely resource: it targets a genuine gap in text-to-SQL evaluation, provides reasoning-category breakdowns, includes a large annotation effort, and ships with released code and data. The broad model coverage and the chain-of-thought ablation are strengths, as is the explicit reporting of per-category and per-difficulty results. However, the central difficulty claim depends on the correctness of the gold SQL and on a fair comparison to existing benchmarks; both are currently not established to the standard needed for a benchmark paper.

major comments (4)
  1. [Section 3.1, Table 1] The quality-control narrative is internally inconsistent: the text states that 'automated scripts confirmed the executability of all queries,' yet Table 1 reports a post-revision VES of only 95.05%, and the table caption defines VES as 'syntactically correct SQL, regardless of result correctness.' Since Section 4.3 defines EX as matching the gold result, any gold query that is not semantically correct—not merely non-executable—makes the reported 33.20% upper bound uninterpretable as a measure of model reasoning. The paper must either report a semantic-agreement metric (e.g., independent gold SQL generation with execution-match agreement) or otherwise demonstrate that the gold queries answer their questions, not merely that they parse.
  2. [Figure 4, Section 4.1] The prompt description is ambiguous and potentially leaks the answer structure: the Figure 4 caption says the template provides 'a high-level reasoning plan including the necessary formula,' and the shown assistant output contains both the SQL and stepwise reasoning. If the formula or the chain-of-thought steps are supplied in the model input during the zero-shot evaluation, then the Physical and Mathematical categories test formula application and prompt following rather than knowledge retrieval or自主 reasoning, which materially changes the difficulty interpretation. The authors should specify exactly what is in the model input for both the with-CoT and without-CoT conditions, and if formulas are given, add an unassisted evaluation where the model must retrieve the formula itself.
  3. [Section 5.4, Table 5 vs Table 4] The CoT ablation contains an unexplained discrepancy: Table 5 lists 'Deepseek-Coder' with 4.74 EX without CoT and 16.11 EX with CoT, but Table 4 reports Deepseek-Coder-V2 at 23.70 overall EX in what appears to be the with-CoT setting. If these are different models or different evaluation conditions, the table should say so; if they are the same, the numbers contradict each other. This inconsistency undermines the specific claim that CoT produces a 16.11-point gain for this model.
  4. [Section 1, Section 5.1, Table 3] The claim that LogicCat 'substantially increases the task difficulty' relative to Spider, BIRD, and Archer is not supported by controlled evidence. Table 3 reports aggregate dataset statistics, but no same-model, same-prompt baseline is provided on Archer or Spider under identical prompting and evaluation conditions. Because the benchmark was constructed with explicit instructions to include arithmetic, physics, and hypothetical reasoning, the observed low accuracy is partly by construction. Adding a small controlled comparison on Archer and Spider with the same models and prompts would let the reader separate benchmark difficulty from dataset-content effects.
minor comments (5)
  1. [Abstract and Section 1] The number of chain-of-thought annotations is given as 12,114 in the Abstract and 12,144 in the contributions list; these should be reconciled.
  2. [Section 3.1] The annotation effort is reported as '800-person-hour' and later as 'approximately 500 person-hours' plus 200 person-hours for expert review; the total should be stated consistently.
  3. [Section 5.2, Figure 5] The text refers to 'the left chart in Figure 1' for per-category performance, but Figure 1 is the benchmark overview; the intended reference is likely Figure 5 or Table 4.
  4. [Table 4 caption] There are typographical errors in the caption ('broekn' instead of 'broken') and in Section 5.4 ('wtih' instead of 'with', 'Din-SQL' for DIN-SQL); a careful proofreading pass is needed.
  5. [Table 5] The model label 'Deepseek-Coder' is ambiguous; use the exact identifier from Table 4 (Deepseek-Coder-V2) to avoid confusion.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: LogicCat's difficulty result is an empirical measurement, not a quantity fitted from its own inputs.

full rationale

The paper's central claim is that existing models achieve at most 33.20% execution accuracy on LogicCat. This is an empirical measurement on a newly constructed benchmark, not a quantity derived from the benchmark's own definitions. The construction guidelines intentionally require arithmetic, physics, commonsense, and hypothetical reasoning, but that makes the dataset difficult by design rather than making the accuracy result circular: the models' scores are not fitted parameters, and no equation in the paper defines the reported accuracy in terms of the construction choices. The CoT ablation is a controlled comparison, and its outcome is not forced by construction, although the prompt's inclusion of a 'high-level reasoning plan including the necessary formula' (Figure 4) is a possible confound for interpreting the Physical Knowledge category. The internal tension between the statement that 'automated scripts confirmed the executability of all queries' and the reported 95.05% post-revision VES in Table 1 is a data-quality and validity concern about EX, not a circularity in the derivation chain. There is no load-bearing self-citation, no imported uniqueness theorem, and no fitted input renamed as a prediction. I therefore find no significant circularity.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

No free parameters or invented entities are used. The benchmark relies on two domain assumptions about what the annotations and metrics actually certify.

assumptions (2)
  • domain assumption Execution accuracy on LogicCat, defined by exact match with the gold SQL result, is a valid measure of complex reasoning ability.
    The paper uses EX as the primary metric (Section 4.3) but does not establish that the gold queries are the only correct answers or that exact match captures reasoning quality.
  • domain assumption Annotator-written SQL queries are semantically correct if they execute without error.
    Section 3.1 uses VES, a syntactic validity check, as the quality measure and states a 95.05% 'execution accuracy' after revision, without verifying that query outputs answer the original questions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LogicCat: A Chain-of-Thought Text-to-SQL Benchmark for Complex Reasoning." pith.science (2026). https://pith.science/paper/WN6XJN6G

@misc{pith2026250518744,
  author       = {Pith},
  title        = {Pith review of: LogicCat: A Chain-of-Thought Text-to-SQL Benchmark for Complex Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WN6XJN6G}},
  note         = {Machine review of arXiv:2505.18744}
}
read the original abstract

Text-to-SQL is a critical task in natural language processing that aims to transform natural language questions into accurate and executable SQL queries. In real-world scenarios, these reasoning tasks are often accompanied by complex mathematical computations, domain knowledge, and hypothetical reasoning scenarios. However, existing large-scale Text-to-SQL datasets typically focus on business logic and task logic, neglecting critical factors such as vertical domain knowledge, complex mathematical reasoning, and hypothetical reasoning, which are essential for realistically reflecting the reasoning demands in practical applications and completing data querying and analysis. To bridge this gap, we introduce LogicCat, the first Text-to-SQL benchmark dataset specifically designed for complex reasoning and chain-of-thought parsing, encompassing physics, arithmetic, commonsense, and hypothetical reasoning scenarios. LogicCat comprises 4,038 English questions paired 12,114 detailed chain-of-thought reasoning steps, spanning 45 databases across diverse domains, significantly surpassing existing datasets in complexity. Experimental results demonstrate that LogicCat substantially increases the task difficulty for current state-of-the-art models to at most 33.20% execution accuracy, indicating that this task remains exceptionally challenging. The advancement of LogicCat represents a crucial step toward developing systems suitable for real-world enterprise data analysis and autonomous query generation. We have released our dataset code at https://github.com/Ffunkytao/LogicCat.

Figures

Figures reproduced from arXiv: 2505.18744 by the authors.

Figure 1
Figure 1. LogicCat evaluates LLMs on text-to-SQL with [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 3
Figure 3. Two doctoral students generated 75–100 questions per database, adhering to strict guidelines [PITH_FULL_IMAGE:figures/full_fig_p002_3.png] view at source ↗
Figure 2
Figure 2. Overview of LogicCat benchmarking pipeline. Our pipeline includes three main parts: (1) database and question [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figures from the paper (3 more)
Figure 3
Figure 3. Figure 3: The LogicCat database domain distribution across [PITH_FULL_IMAGE:figures/full_fig_p003_3.png]
Figure 4
Figure 4. Figure 4: An illustration of the prompting template used [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Execution Accuracy (EX) of selected models by [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

32 extracted references · 6 canonical work pages

  1. [1]

    Ruisheng Cao, Lu Chen, Zhi Chen, Yanbin Zhao, Su Zhu, and Kai Yu. 2021. https://doi.org/10.18653/v1/2021.acl-long.198 LGESQL : Line graph enhanced text-to- SQL model with mixed local and non-local relations . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural La...

    doi:10.18653/v1/2021.acl-long.198 2021

  2. [2]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, and Bing Xue and. 2025 a . https://arxiv.org/abs/2501.12948 Deepseek-r1: Incentivizing reasoning capability in llms via r...

    arXiv 2025

  3. [3]

    DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, and Chengda Lu. 2025 b . https://arxiv.org/abs/2412.19437 Deepseek-v3 technical report . Preprint, arXiv:2412.19437

    arXiv 2025

  4. [4]

    Longxu Dou, Yan Gao, Xuqi Liu, Mingyang Pan, Dingzirui Wang, Wanxiang Che, Dechen Zhan, Min-Yen Kan, and Jian-Guang Lou. 2023. https://arxiv.org/abs/2301.01067 Towards knowledge-intensive text-to-sql semantic parsing with formulaic knowledge . Preprint, arXiv:2301.01067

    work page Pith review arXiv 2023

  5. [5]

    Dawei Gao, Haibin Wang, Yaliang Li, Xiuyu Sun, Yichen Qian, Bolin Ding, and Jingren Zhou. 2024. https://doi.org/10.14778/3641204.3641221 Text-to-sql empowered by large language models: A benchmark evaluation . Proc. VLDB Endow., 17(5):1132–1145

    arXiv 2024

  6. [6]

    Yingqi Gao, Yifu Liu, Xiaoxia Li, Xiaorong Shi, Yin Zhu, Yiming Wang, Shiqi Li, Wei Li, Yuntao Hong, Zhiling Luo, Jinyang Gao, Liyu Mou, and Yu Li. 2025. https://arxiv.org/abs/2411.08599 A preview of xiyan-sql: A multi-generator ensemble framework for text-to-sql . Preprint, arXiv:2411.08599

    arXiv 2025

  7. [7]

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y. Wu, Y. K. Li, Fuli Luo, Yingfei Xiong, and Wenfeng Liang. 2024. https://arxiv.org/abs/2401.14196 Deepseek-coder: When the large language model meets programming -- the rise of code intelligence . Preprint, arXiv:2401.14196

    arXiv 2024

  8. [8]

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, Kai Dang, Yang Fan, Yichang Zhang, An Yang, Rui Men, Fei Huang, Bo Zheng, Yibo Miao, Shanghaoran Quan, Yunlong Feng, Xingzhang Ren, Xuancheng Ren, Jingren Zhou, and Junyang Lin. 2024. https://arxiv.org/abs/2409.12186 Qwen2.5-coder technica...

    arXiv 2024

Show all 32 references
  1. [9]

    Wuwei Lan, Zhiguo Wang, Anuj Chauhan, Henghui Zhu, Alexander Li, Jiang Guo, Sheng Zhang, Chung-Wei Hang, Joseph Lilien, Yiqun Hu, Lin Pan, Mingwen Dong, Jun Wang, Jiarong Jiang, Stephen Ash, Vittorio Castelli, Patrick Ng, and Bing Xiang. 2023. https://arxiv.org/abs/2305.16265 ...

    2023 arXiv

  2. [10]

    Chia-Hsuan Lee, Oleksandr Polozov, and Matthew Richardson. 2021. https://doi.org/10.18653/v1/2021.acl-long.176 K aggle DBQA : Realistic evaluation of text-to- SQL parsers . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th ...

    2021 doi

  3. [11]

    Fangyu Lei, Jixuan Chen, Yuxiao Ye, Ruisheng Cao, Dongchan Shin, Hongjin SU, ZHAOQING SUO, Hongcheng Gao, Wenjing Hu, Pengcheng Yin, Victor Zhong, Caiming Xiong, Ruoxi Sun, Qian Liu, Sida Wang, and Tao Yu. 2025. https://openreview.net/forum?id=XmProj9cPs Spider 2.0: Evaluating...

    2025

  4. [12]

    Haoyang Li, Shang Wu, Xiaokang Zhang, Xinmei Huang, Jing Zhang, Fuxin Jiang, Shuai Wang, Tieying Zhang, Jianjun Chen, Rui Shi, Hong Chen, and Cuiping Li. 2025. https://arxiv.org/abs/2503.02240 Omnisql: Synthesizing high-quality text-to-sql data at scale . Preprint, arXiv:2503.02240

    2025 arXiv

  5. [13]

    Haoyang Li, Jing Zhang, Hanbing Liu, Ju Fan, Xiaokang Zhang, Jun Zhu, Renjie Wei, Hongyan Pan, Cuiping Li, and Hong Chen. 2024. https://doi.org/10.1145/3654930 Codes: Towards building open-source language models for text-to-sql . Proc. ACM Manag. Data, 2(3)

    2024 doi

  6. [14]

    Bingchang Liu, Chaoyu Chen, Cong Liao, Zi Gong, Huan Wang, Zhichao Lei, Ming Liang, Dajun Chen, Min Shen, Hailian Zhou, Hang Yu, and Jianguo Li. 2023. https://arxiv.org/abs/2311.02303 Mftcoder: Boosting code llms with multitask fine-tuning . Preprint, arXiv:2311.02303

    2023 arXiv

  7. [15]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, and Sam Altman. 2024. https://arxiv.org/abs/2303.08774 Gpt-4 technical report . Preprint, arXiv:2303.08774

    2024 arXiv

  8. [16]

    Mohammadreza Pourreza, Hailong Li, Ruoxi Sun, Yeounoh Chung, Shayan Talaei, Gaurav Tarlok Kakkar, Yu Gan, Amin Saberi, Fatma Ozcan, and Sercan O Arik. 2025 a . https://openreview.net/forum?id=CvGqMD5OtX CHASE - SQL : Multi-path reasoning and preference optimized candidate sele...

    2025

  9. [17]

    Mohammadreza Pourreza and Davood Rafiei. 2023. https://openreview.net/forum?id=p53QDxSIc5 DIN - SQL : Decomposed in-context learning of text-to- SQL with self-correction . In Thirty-seventh Conference on Neural Information Processing Systems

    2023

  10. [18]

    Mohammadreza Pourreza, Shayan Talaei, Ruoxi Sun, Xingchen Wan, Hailong Li, Azalia Mirhoseini, Amin Saberi, and Sercan "O. Arik. 2025 b . https://arxiv.org/abs/2503.23157 Reasoning-sql: Reinforcement learning with sql tailored partial rewards for reasoning-enhanced text-to-sql ...

    2025 arXiv

  11. [19]

    Bowen Qin, Binyuan Hui, Lihan Wang, Min Yang, Jinyang Li, Binhua Li, Ruiying Geng, Rongyu Cao, Jian Sun, Luo Si, Fei Huang, and Yongbin Li. 2022. https://arxiv.org/abs/2208.13629 A survey on text-to-sql parsing: Concepts, methods, and future directions . Preprint, arXiv:2208.13629

    2022 arXiv

  12. [20]

    Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, ...

    2025 arXiv

  13. [21]

    Shayan Talaei, Mohammadreza Pourreza, Yu-Chen Chang, Azalia Mirhoseini, and Amin Saberi. 2024. https://arxiv.org/abs/2405.16755 Chess: Contextual harnessing for efficient sql synthesis . Preprint, arXiv:2405.16755

    2024 arXiv

  14. [22]

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, Soroosh Mariooryad, Yifan Ding, Xinyang Geng, Fred Alcober, Roy Frostig, Mark Omernick, Lexi Walker, and Cosmin Paduraru. 2024. https://ar...

    2024 arXiv

  15. [23]

    Lijie Wang, Ao Zhang, Kun Wu, Ke Sun, Zhenghua Li, Hua Wu, Min Zhang, and Haifeng Wang. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.562 D u SQL : A large-scale and pragmatic C hinese text-to- SQL dataset . In Proceedings of the 2020 Conference on Empirical Methods in Nat...

    2020 doi

  16. [24]

    Xiangjin Xie, Guangwei Xu, Lingyan Zhao, and Ruijie Guo. 2025. https://arxiv.org/abs/2502.14913 Opensearch-sql: Enhancing text-to-sql with dynamic few-shot and consistency alignment . Preprint, arXiv:2502.14913

    2025 arXiv

  17. [25]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, and Fan Zhou. 2025. https://arxiv.org/abs/2505.09388 Qwen3 technical report . Preprint, arXiv:2505.09388

    2025 arXiv

  18. [26]

    Semih Yavuz, Izzeddin Gur, Yu Su, and Xifeng Yan. 2018. https://doi.org/10.18653/v1/D18-1197 What it takes to achieve 100 \ In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1702--1711, Brussels, Belgium. Association for Computati...

    2018 doi

  19. [27]

    Tao Yu, Rui Zhang, Heyang Er, Suyi Li, Eric Xue, Bo Pang, Xi Victoria Lin, Yi Chern Tan, Tianze Shi, Zihan Li, Youxuan Jiang, Michihiro Yasunaga, Sungrok Shim, Tao Chen, Alexander Fabbri, Zifan Li, Luyao Chen, Yuwen Zhang, Shreya Dixit, Vincent Zhang, Caiming Xiong, Richard So...

    2019 doi

  20. [28]

    Tao Yu, Rui Zhang, Michihiro Yasunaga, Yi Chern Tan, Xi Victoria Lin, Suyi Li, Heyang Er, Irene Li, Bo Pang, Tao Chen, Emily Ji, Shreya Dixit, David Proctor, Sungrok Shim, Jonathan Kraft, Vincent Zhang, Caiming Xiong, Richard Socher, and Dragomir Radev. 2019 b . https://doi.or...

    2019 doi

  21. [29]

    Danna Zheng, Mirella Lapata, and Jeff Pan. 2024. https://aclanthology.org/2024.eacl-long.6/ Archer: A human-labeled text-to- SQL dataset with arithmetic, commonsense and hypothetical reasoning . In Proceedings of the 18th Conference of the European Chapter of the Association f...

    2024

  22. [30]

    Yuhang Zhou, Yu He, Siyu Tian, Yuchen Ni, Zhangyue Yin, Xiang Liu, Chuanjun Ji, Sen Liu, Xipeng Qiu, Guangnan Ye, and Hongfeng Chai. 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.800 r^3 - NL 2 GQL : A model coordination and knowledge graph alignment approach for NL 2 ...

    2024 doi

  23. [31]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  24. [32]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.