Pith. sign in

REVIEW 3 major objections 5 minor 38 references

Text-to-SQL for Enterprise Data Analytics

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that a production enterprise Text-to-SQL chatbot can be built by combining a knowledge graph of enterprise semantics with retrieval, ranking, and query-fixing agents, and reports 53% expert-rated correctness on an…

desk verdict Detailed and honest enterprise Text-to-SQL experience report; headline numbers are likely inflated by benchmark–knowledge-graph overlap. read the letter →

arxiv 2507.14372 v1 pith:POTXBWY7 submitted 2025-07-18 cs.CL cs.AIcs.DBcs.HC

Albert Chen , Manas Bundele , Gaurav Ahlawat , Patrick Stetz , Zhitao Wang , Qiang Fei , Donghoon Jung , Audrey Chu
show 10 more authors
Bharadwaj Jayaraman Ayushi Panth Yatin Arora Sourav Jain Renjith Varma Alexey Ilin Iuliia Melnychuk Chelsea Chueh Joyan Sil Xiaofeng Wang
This is my paper
classification cs.CLcs.AIcs.DBcs.HC
keywords text-to-sqlknowledgegraphenterprisedataanalyticslargelanguagemodelsretrieval-augmentedgenerationquerylakeschemalinking
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a usable enterprise Text-to-SQL system is an engineering and data problem, not just a model problem, and that the missing piece is a knowledge graph capturing the semantics of a company's data. It reports on a chatbot deployed inside a large company's data platform, serving over 300 weekly users, and evaluates it on an internal 133-question benchmark. The central numbers are that expert reviewers rate 53% of responses correct or close to correct, and that adding example queries, table clusters, and table/column attributes raises the correct-or-close rate from 9% with schemas alone to 49% in the full system. The paper presents this as a practical path for other enterprises building similar assistants.

What carries the argument

The central object is the knowledge graph: nodes for tables, columns, product areas, users, example queries, and domain knowledge records, with edges capturing ownership, joins, popularity, and cluster membership. It is organized by a user-dataset clustering step that applies independent component analysis to three months of user-table access logs, producing 200 soft clusters of up to 20 top tables each. The other half of the machinery is the Query Writer Agent pipeline — retrieve context from the graph, rank tables and columns with separate LLM rankers, write the SQL, then fix errors in a loop that uses Trino EXPLAIN validation and a Researcher LLM Agent that can search for missing tables and columns.

What would settle it

Take the 133 benchmark questions and measure how many have a near-duplicate question or query already present in the example-query index, wiki pages, or historical query logs used to build the knowledge graph; if a large share do, rerun the evaluation on a freshly written, held-out set of questions and check whether the 53% and the ablation improvements survive.

Watch

Extended reading notes

Core claim

The claim is that a multi-agent Text-to-SQL chatbot for a large, dynamic data lake can work if the model is grounded in a knowledge graph that is continuously refreshed from database metadata, historical query logs, wikis, and user-supplied domain knowledge. Tables are soft-clustered by access patterns so that each user or product area sees relevant candidate tables first; a query writer then retrieves and ranks context, generates SQL, and runs a validation loop in which a researcher agent searches the knowledge graph to fix table and column hallucinations. On the internal benchmark, this configuration yields 78% table recall, 1% schema hallucination, 4% compilation errors, and a 53% expert-rated score of 4 or higher, with the knowledge graph components — especially example queries, clusters, and attributes — delivering most of the quality gain. The paper also reports that the full system runs in under 60 seconds per question on average.

Load-bearing premise

The whole evaluation rests on the internal 133-question benchmark being representative of real enterprise requests and not overlapping with the historical query logs and wiki examples indexed in the knowledge graph; if the benchmark questions resemble already-indexed examples, the reported 53% and the 9%-to-49% ablation gains would partly measure retrieval rather than generation.

Editorial extensions

If this is right

  • If the reported numbers hold, organizations should treat enterprise Text-to-SQL as a knowledge-capture project: the biggest quality lever is institutional context, not prompt engineering or a stronger base model.
  • The ablation order gives a concrete build sequence: start with schema retrieval, then add example queries, then table/column attributes and clusters, and only then invest in rankers and fixers.
  • A validation loop with EXPLAIN-based checking can reduce schema hallucination from 23% to 1%, suggesting that structure-aware error fixing is a practical substitute for execution-based feedback when data access is restricted.
  • The 53% correct-or-close rate with 24% of failures attributed to incorrect filters suggests that the remaining gap is largely about value-level semantics — date ranges, filters, and parameter values — which the paper explicitly leaves to future work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The strongest number, 53%, is measured on a benchmark written by the same domain experts whose wikis and example queries populate the knowledge graph; the paper does not report the overlap, so a fair test would hold out fresh questions and re-measure.
  • The clustering method based on access-log ICA is company-specific in detail but should transfer to any organization whose query logs reveal team structure; a testable extension is to run the same pipeline on a public enterprise-database benchmark and compare against the roughly 31% best reported execution accuracy there.
  • Because the system already surfaces assumptions and lets users fix queries interactively, the practical bar for deployment may be lower than 53% autonomous accuracy: the paper's own survey data, with 95% of users rating queries at least passable, suggests human-in-the-loop correction absorbs some of the residual errors.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper reports on the design and deployment of a production enterprise Text-to-SQL chatbot at LinkedIn. The system combines a knowledge graph (table/column metadata, historical query logs, wikis, code, domain knowledge, example queries) with a multi-stage agent that retrieves and ranks context, writes a query, and fixes syntax or hallucination errors, all exposed through an interactive chat UI. The central quantitative claims are that expert review rated 53% of responses as correct or close to correct on an internal benchmark of 133 questions, and that ablations show knowledge-graph components improve the correct-or-close-to-correct rate from 9% to 48–49% compared to using table schemas alone. The paper also reports deployment signals such as over 300 weekly active users, 33% of chat sessions leading to code pasted into the SQL editor, and a set of negative results in Appendix A.5.

Significance. If the central claims hold, this is a valuable experience report for practitioners building enterprise Text-to-SQL systems. The paper's strengths are its detailed description of a deployed system at scale, the inclusion of negative results (Appendix A.5), explicit pseudocode for the user–dataset clustering, latency and cost proxies, and the transparency about single-run ablations. The deployment metrics and the 33% code-paste rate are useful adoption signals. However, the evaluation is entirely internal, and the independence of the benchmark from the knowledge graph contents is not established. Because the headline numbers may partly reflect retrieval of benchmark-like examples rather than generative Text-to-SQL capability, the paper is currently stronger as a system description than as a measured evaluation. The unresolved leakage concern is load-bearing for the claim that the system 'works for a large company.'

major comments (3)
  1. [§3.1, §2.1.2, §2.2.1] The quantitative headline depends on the internal 133-question benchmark being independent of the knowledge graph contents, but that independence is not established. The knowledge graph indexes historical query logs, code repos, wikis, and human-authored example queries (§2.1.2), and the retrieval stage injects examples semantically similar to the user's question (§2.2.1). Table 2 shows that removing example queries (A.4→A.3) is the largest single ablation step, dropping %4+ from 49% to 24%. If the domain experts who wrote the benchmark questions drew on the same wikis, query logs, or example repositories, the reported scores could largely reflect retrieval or near-copying of indexed content rather than generation for unseen requests. The paper provides no overlap analysis (e.g., token/embedding similarity between benchmark questions or ground-truth SQL and indexed examples), no temporal split, and no retrieval-only baseline. This is not circular by construction because the ground-truth queries were authored externally to the model, but the missing analysis is load-bearing for the claim that the system works for a large company; I would ask for a leakage analysis before the numbers can be interpreted.
  2. [§3.3] Each ablation configuration was run once ('We did a single run for each configuration,' §3.3). With LLM-based components and no reported variance, differences of 1–7 percentage points in Table 2 cannot be distinguished from noise; for example, Full (48%) differs from B.3 (47%) and B.2 (47%) by one point, and the claim that adding domain knowledge decreases quality rests on a single-run 49% (A.4) versus 42% (A.5) comparison. Multiple runs with confidence intervals, or an explicit noise analysis, are needed before the component-attribution conclusions in §3.3.1–3.3.3 are accepted.
  3. [§3.3.4] The headline 53% expert-review figure is not directly comparable to the ablation table. Ratings were received for only 124 of 133 questions, and the production model was 'slightly modified' from the Full configuration, with the modification unspecified. The paper does not state whether the 9 missing questions are missing at random or concentrated in particular product areas, and it does not report inter-rater reliability for the human experts or agreement between the LLM-as-a-judge and human raters. These gaps should be addressed so that the 53% can be interpreted as a stable estimate rather than a single convenience sample.
minor comments (5)
  1. [Abstract and §3.3.1] The abstract and §3.3.1 report 49% for the full configuration, but Table 2 lists 48% for Full; the conclusion repeats 49%. Please reconcile the number and identify the exact configuration to which it refers.
  2. [Table 2] The quality column is labeled 'Score (% 4+)', but §3.3 does not state whether these scores come from human evaluators or LLM-as-a-judge; the rubric in Appendix A.1 is used for both, so the provenance should be explicit.
  3. [§1] There are minor grammatical issues in the introduction, such as 'Uber built a internal Text-to-SQL application and report 50% overlap'; please copyedit the manuscript.
  4. [§2.2.1] The text says examples are retrieved via semantic similarity and 'discarding those that do not use a candidate table,' but it is unclear whether this discarding happens before or after the top-K selection; please clarify the order of operations.
  5. [Appendix A.1] The rubric in Figure 4 describes a score of 2 as '90% of the columns are wrong,' while §3.2 describes a score of 2 as 'has the wrong columns'; please align the rubric wording with the prose.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: benchmark ground truth is authored by domain experts independently of the system and no prediction reduces to its inputs by construction.

full rationale

The paper is a systems/engineering report rather than a derivation from first principles. Its headline numbers (53% expert-rated correct/close, ablation from 9% to 49%) are measured against an internal benchmark defined in Section 3.1, where product-area domain experts provided questions and ground-truth queries. These are external to the model and are not fitted parameters renamed as predictions. The knowledge graph indexes historical query logs, wikis, and code, and example retrieval is a pipeline component; the reported gains from ablating example queries are empirical effects of removing a context source, not an identity between the measured quantity and an input. The only substantive concern is a possible benchmark-overlap or leakage risk: benchmark questions may resemble queries already indexed as examples, and the paper does not report a leakage analysis or a retrieval-only baseline. That is an evaluation-validity threat, not a circular reduction. The paper does not import a load-bearing result from a self-citation chain, invoke a uniqueness theorem from the authors' prior work, or redefine a known result in new coordinates. Under the requirement to flag only demonstrated circularity, no circular step is established.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The system introduces no new theoretical entities. Its results rest on engineering choices and internal infrastructure assumptions: ICA clustering over access logs, manual hyperparameter selection, the internal benchmark, and the reliability of LLM-as-a-judge. These are domain assumptions rather than free parameters fitted to a mathematical target, but they still bound the generality of the conclusions.

free parameters (5)
  • N_comp = 200
    Number of ICA components/clusters, chosen by manual review of the table clusters generated under different settings (Section 2.1.1).
  • T_c = 20
    Tables per cluster, chosen by manual review; high enough to include closely related tables (Section 2.1.1).
  • K_ret = 20
    Tables retrieved before ranking, set to maximize recall while limiting LLM calls (Section 2.2.1).
  • K_rnk = 7
    Top-ranked tables passed to the query writer, balancing precision, recall, and token usage (Section 2.2.2).
  • K_table = 7
    Number of tables passed directly to the writer when rankers are disabled in the B.1 baseline (Section 3.3.2).
assumptions (4)
  • domain assumption Historical user-table access counts are a reliable proxy for table relevance and for mapping users to product areas.
    The clustering and personalization (Algorithms 1-4) assume access patterns generalize to future queries and that team email groups identify representative employees.
  • domain assumption The internal benchmark of 133 questions with expert-authored ground truth is representative of real enterprise user requests and is not contaminated by knowledge graph content.
    All quantitative claims in Section 3 rest on this benchmark; no overlap analysis is provided between benchmark questions and historical query logs or wiki examples indexed in the knowledge graph.
  • domain assumption LLM-as-a-judge using gpt-4o reliably reproduces human expert judgments on the 1-5 rubric.
    Section 3.2 uses LLM-as-a-judge alongside human evaluators but reports no agreement statistics, so the ablation table scores (A.1-C.4) depend on this unverified assumption.
  • domain assumption Trino EXPLAIN in VALIDATE mode detects all syntactic and semantic errors that matter for compilation success.
    The query fixer loop (Section 2.2.3) relies on Trino EXPLAIN to validate syntax, function usage, tables, and columns; the paper states it returns only one error at a time but then uses a separate hallucination validator.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Text-to-SQL for Enterprise Data Analytics." pith.science (2026). https://pith.science/paper/POTXBWY7

@misc{pith2026250714372,
  author       = {Pith},
  title        = {Pith review of: Text-to-SQL for Enterprise Data Analytics},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/POTXBWY7}},
  note         = {Machine review of arXiv:2507.14372}
}
read the original abstract

The introduction of large language models has brought rapid progress on Text-to-SQL benchmarks, but it is not yet easy to build a working enterprise solution. In this paper, we present insights from building an internal chatbot that enables LinkedIn's product managers, engineers, and operations teams to self-serve data insights from a large, dynamic data lake. Our approach features three components. First, we construct a knowledge graph that captures up-to-date semantics by indexing database metadata, historical query logs, wikis, and code. We apply clustering to identify relevant tables for each team or product area. Second, we build a Text-to-SQL agent that retrieves and ranks context from the knowledge graph, writes a query, and automatically corrects hallucinations and syntax errors. Third, we build an interactive chatbot that supports various user intents, from data discovery to query writing to debugging, and displays responses in rich UI elements to encourage follow-up chats. Our chatbot has over 300 weekly users. Expert review shows that 53% of its responses are correct or close to correct on an internal benchmark set. Through ablation studies, we identify the most important knowledge graph and modeling components, offering a practical path for developing enterprise Text-to-SQL solutions.

Figures

Figures reproduced from arXiv: 2507.14372 by the authors.

Figure 1
Figure 1. Tables are the central organizing entity. Given a table, we [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Query Writer Agent architecture. The Agent re [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Multi-agent architecture supports various intents [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: shows the overall scoring rubric for human evaluation and LLM-as-a-judge. In addition to this overall score, we ask reviewers if any table, column, filter, aggregation, join, etc. is incorrect. Overall rating of the query between 1-5 where higher score indicates higher…
Figure 5
Figure 5. Figure 5: Chatbot is available in sidebar next to the SQL editor. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Rich display elements in query output help users understand responses and ask follow-up questions. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Fix-with-AI helps users debug query execution errors. Users can certify example queries from the query editor. [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Researcher LLM Agent architecture. The Researcher is used within query fixing to search for tables to resolve [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

38 extracted references · 18 canonical work pages

  1. [1]

    Vincent Claveau. 2021. Neural text generation for query expansion in information retrieval. In IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology. 202–209

    work page 2021

  2. [2]

    datahub-project. 2024. DataHub. https://github.com/datahub-project/datahub

    work page 2024

  3. [3]

    Venkatesh Emani, Vivek Pandit, Victor Shnayder, Wenjing Wang, and Carlo Curino

    Avrilia Floratou, Fotis Psallidas, Fuheng Zhao, Shaleen Deep, Gunther Hagleither, Wangda Tan, Joyce Cahoon, Rana Alotaibi, Jordan Henkel, Abhik Singla, Alex Van Grootel, Brandon Chow, Kai Deng, Katherine Lin, Marcos Campos, K. Venkatesh Emani, Vivek Pandit, Victor Shnayder, Wenjing Wang, and Carlo Curino. 2024. NL2SQL is a solved problem... Not!. In Confe...

    work page 2024

  4. [4]

    Trino Software Foundation. 2025. EXPLAIN. https://trino.io/docs/current/sql/ explain.html. Accessed: 2025-04-25

    work page 2025

  5. [5]

    Trino Software Foundation. 2025. Trino: The Distributed SQL Query Engine. https://trino.io. Version 475

    work page 2025

  6. [6]

    Dawei Gao, Haibin Wang, Yaliang Li, Xiuyu Sun, Yichen Qian, Bolin Ding, and Jingren Zhou. 2023. Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation. http://arxiv.org/abs/2308.15363 arXiv:2308.15363 [cs]

    arXiv 2023

  7. [7]

    Glean. 2024. Glean Chat API. https://developers.glean.com/docs/client_api/ chat_api/

    work page 2024

  8. [8]

    Aapo Hyvärinen and Erkki Oja. 2000. Independent component analysis: algo- rithms and applications.Neural Networks 13, 4 (2000), 411–430. doi:10.1016/S0893- 6080(00)00026-5

    doi:10.1016/s0893- 2000

Show all 38 references
  1. [9]

    Rolf Jagerman, Honglei Zhuang, Zhen Qin, Xuanhui Wang, and Michael Bendersky. 2023. Query Expansion by Prompting Large Language Models. arXiv:2305.03653 [cs.IR] https://arxiv.org/abs/2305.03653

    2023 arXiv

  2. [10]

    Abhi Khune, Callie Busch, Jeffrey Johnson, Pradeep Chakka, Saketh Chinta- palli, Adarsh Nagesh, Gaurav Paul, and Ben Carroll. 2024. QueryGPT - Natural Language to SQL Using Generative AI. https://www.uber.com/blog/query-gpt/

    2024

  3. [11]

    LangChain. 2024. LangChain. https://www.langchain.com/

    2024

  4. [12]

    Chia-Hsuan Lee, Oleksandr Polozov, and Matthew Richardson. 2021. KaggleD- BQA: Realistic Evaluation of Text-to-SQL Parsers. In ACL:2021:long, Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (Eds.). Online, 2261–2273. doi:10. 18653/v1/2021.acl-long.176

    2021

  5. [13]

    Dongjun Lee, Choongwon Park, Jaehyuk Kim, and Heesoo Park. 2024. Mcs- sql: Leveraging multiple prompts and multiple-choice selection for text-to-sql generation. arXiv preprint arXiv:2405.07467 (2024)

    2024 arXiv

  6. [14]

    Fangyu Lei, Jixuan Chen, Yuxiao Ye, Ruisheng Cao, Dongchan Shin, Hongjin Su, Zhaoqing Suo, Hongcheng Gao, Wenjing Hu, Pengcheng Yin, Victor Zhong, Caiming Xiong, Ruoxi Sun, Qian Liu, Sida Wang, and Tao Yu. 2024. Spider 2.0: Evaluating Language Models on Real-World Enterprise T...

    2024 arXiv

  7. [15]

    Jinyang Li, Binyuan Hui, Ge Qu, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Rongyu Cao, Ruiying Geng, Nan Huo, Xuanhe Zhou, Chenhao Ma, Guoliang Li, Kevin C. C. Chang, Fei Huang, Reynold Cheng, and Yongbin Li. 2023. Can LLM Already Serve as A Database Interface? A...

    2023 arXiv

  8. [16]

    Karime Maamari, Fadhil Abubaker, Daniel Jaroslawicz, and Amine Mhedhbi

  9. [17]

    Microsoft. 2025. Azure OpenAI in Azure AI Foundry Models. https://learn. microsoft.com/en-us/azure/ai-foundry/openai/concepts/models

    2025

  10. [18]

    OpenAI. 2022. text-embedding-ada-002. https://platform.openai.com/docs/ models/text-embedding-ada-002

    2022

  11. [19]

    OpenAI. 2023. GPT-3.5 Turbo. https://platform.openai.com/docs/models/gpt- 3.5-turbo

    2023

  12. [20]

    OpenAI. 2024. GPT-4o. https://platform.openai.com/docs/models/gpt-4o

    2024

  13. [21]

    OpenAI. 2024. GPT-4o mini. https://platform.openai.com/docs/models/gpt-4o- mini

    2024

  14. [22]

    Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. 2011. Scikit-learn: Machine learning in Python. Journal of machine learning research 12, Oct (2011), ...

    2011

  15. [23]

    Mohammadreza Pourreza, Hailong Li, Ruoxi Sun, Yeounoh Chung, Shayan Talaei, Gaurav Tarlok Kakkar, Yu Gan, Amin Saberi, Fatma Ozcan, and Sercan O. Arik

  16. [24]

    Mohammadreza Pourreza and Davood Rafiei. 2023. DIN-SQL: Decomposed In- Context Learning of Text-to-SQL with Self-Correction. doi:10.48550/arXiv.2304. 11015 arXiv:2304.11015 [cs]

    2023 doi arXiv

  17. [25]

    http://arxiv.org/abs/2410.01943 arXiv:2410.01943

    CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQL. http://arxiv.org/abs/2410.01943 arXiv:2410.01943

    arXiv

  18. [26]

    Shayan Talaei, Mohammadreza Pourreza, Yu-Chen Chang, Azalia Mirhoseini, and Amin Saberi. 2024. CHESS: Contextual Harnessing for Efficient SQL Synthesis. http://arxiv.org/abs/2405.16755 arXiv:2405.16755 [cs]

    2024 arXiv

  19. [27]

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: language agents with verbal reinforcement learn- ing. In Advances in Neural Information Processing Systems , A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine ...

    2023

  20. [28]

    Spider Team. 2025. Spider 1.0: Yale Semantic Parsing and Text-to-SQL Challenge. https://yale-lily.github.io/spider. Accessed: 2025-04-25

    2025

  21. [29]

    BIRD Team. 2025. BIRD-SQL: A Big Bench for Large-Scale Database Grounded Text-to-SQLs. https://bird-benchmark.github.io/. Accessed: 2025-04-25

    2025

  22. [30]

    Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2024. Text Embeddings by Weakly-Supervised Contrastive Pre-training. arXiv:2212.03533 [cs.CL] https://arxiv.org/abs/2212. 03533

    2024 arXiv

  23. [31]

    Spider Team. 2025. Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows. https://spider2-sql.github.io/. Accessed: 2025-04-25

    2025

  24. [32]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35 (2022), 24824–24837

    2022

  25. [33]

    Liang Wang, Nan Yang, and Furu Wei. 2023. Query2doc: Query Expansion with Large Language Models. In EMNLP:2023:main, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Singapore, 9414–9423. doi:10.18653/v1/2023.emnlp-main.585

    2023 doi

  26. [34]

    Xing, Hao Zhang, Joseph E

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. arXiv:2306.05685 [cs.CL] https://ar...

    2023 arXiv

  27. [35]

    Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir Radev

  28. [38]

    Fix with AI

    Victor Zhong, Mike Lewis, Sida I. Wang, and Luke Zettlemoyer. 2020. Grounded Adaptation for Zero-shot Executable Semantic Parsing. In EMNLP:2020:main, Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (Eds.). Online, 6869–6882. doi:10.18653/v1/2020.emnlp-main.558 7 AAE ’25, A...

    2020 doi

  29. [2018]

    In EMNLP:2018:1, Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii (Eds.)

    Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross- Domain Semantic Parsing and Text-to-SQL Task. In EMNLP:2018:1, Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii (Eds.). Brussels, Belgium, 3911–3921. doi:10.18653/v1/D18-1425

    2018 doi

  30. [2024]

    http://arxiv.org/abs/2408.07702 arXiv:2408.07702

    The Death of Schema Linking? Text-to-SQL in the Age of Well-Reasoned Language Models. http://arxiv.org/abs/2408.07702 arXiv:2408.07702

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.