Pith. sign in

REVIEW 4 cited by

The Dawn of Natural Language to SQL: Are We Fully Ready?

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2406.01265 v3 pith:EONIPVQT submitted 2024-06-03 cs.DB

Boyan Li , Yuyu Luo , Chengliang Chai , Guoliang Li , Nan Tang This is my paper
classification cs.DB
keywords nl2sqlnl2sql360languagemethodsaccuracydesignexecutionfully
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Translating users' natural language questions into SQL queries (i.e., NL2SQL) significantly lowers the barriers to accessing relational databases. The emergence of Large Language Models has introduced a novel paradigm in NL2SQL tasks, enhancing capabilities dramatically. However, this raises a critical question: Are we fully prepared to deploy NL2SQL models in production? To address the posed questions, we present a multi-angle NL2SQL evaluation framework, NL2SQL360, to facilitate the design and test of new NL2SQL methods for researchers. Through NL2SQL360, we conduct a detailed comparison of leading NL2SQL methods across a range of application scenarios, such as different data domains and SQL characteristics, offering valuable insights for selecting the most appropriate NL2SQL methods for specific needs. Moreover, we explore the NL2SQL design space, leveraging NL2SQL360 to automate the identification of an optimal NL2SQL solution tailored to user-specific needs. Specifically, NL2SQL360 identifies an effective NL2SQL method, SuperSQL, distinguished under the Spdier dataset using the execution accuracy metric. Remarkably, SuperSQL achieves competitive performance with execution accuracy of 87% and 62.66% on the Spider and BIRD test sets, respectively.

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Towards Probabilistic Question Answering Over Tabular Data

    cs.CL 2025-06 reject novelty 6.0 of 10

    LUCARIO is a large-scale benchmark for probabilistic table QA, but Auto-BN's reported gains are circular because ground-truth answers come from the same Bayesian network the method uses for inference.

  2. MasHost Builds It All: Autonomous Multi-Agent System Directed by Reinforcement Learning

    cs.MA 2025-06 conditional novelty 6.0 of 10

    MasHost uses reinforcement learning to autonomously construct query-adaptive multi-agent graphs, and its authors report the best average accuracy across six LLM benchmarks.

  3. MCTuner: Spatial Decomposition-Enhanced Database Tuning via LLM-Guided Exploration

    cs.DB 2025-09 reject novelty 5.0 of 10

    MCTuner reports up to 19.2% performance gains and roughly 1.4x faster discovery of good configurations by combining LLM knob selection with recursive space decomposition and Bayesian optimization.

  4. CogniSQL-R1-Zero: Lightweight Reinforced Reasoning for Efficient SQL Generation

    cs.AI 2025-07 reject novelty 3.0 of 10

    A 7B model trained with GRPO and a sparse execution-correctness reward reaches 59.97% execution accuracy on BIRD dev, though the evaluation protocol and baseline numbers contain inconsistencies.

Pith tools