REVIEW 5 cited by
The Dawn of Natural Language to SQL: Are We Fully Ready?
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Translating users' natural language questions into SQL queries (i.e., NL2SQL) significantly lowers the barriers to accessing relational databases. The emergence of Large Language Models has introduced a novel paradigm in NL2SQL tasks, enhancing capabilities dramatically. However, this raises a critical question: Are we fully prepared to deploy NL2SQL models in production? To address the posed questions, we present a multi-angle NL2SQL evaluation framework, NL2SQL360, to facilitate the design and test of new NL2SQL methods for researchers. Through NL2SQL360, we conduct a detailed comparison of leading NL2SQL methods across a range of application scenarios, such as different data domains and SQL characteristics, offering valuable insights for selecting the most appropriate NL2SQL methods for specific needs. Moreover, we explore the NL2SQL design space, leveraging NL2SQL360 to automate the identification of an optimal NL2SQL solution tailored to user-specific needs. Specifically, NL2SQL360 identifies an effective NL2SQL method, SuperSQL, distinguished under the Spdier dataset using the execution accuracy metric. Remarkably, SuperSQL achieves competitive performance with execution accuracy of 87% and 62.66% on the Spider and BIRD test sets, respectively.
Forward citations
Cited by 5 Pith papers
-
Towards Probabilistic Question Answering Over Tabular Data
LUCARIO is a large-scale benchmark for probabilistic table QA, but Auto-BN's reported gains are circular because ground-truth answers come from the same Bayesian network the method uses for inference.
-
MasHost Builds It All: Autonomous Multi-Agent System Directed by Reinforcement Learning
MasHost uses reinforcement learning to autonomously construct query-adaptive multi-agent graphs, and its authors report the best average accuracy across six LLM benchmarks.
-
MCTuner: Spatial Decomposition-Enhanced Database Tuning via LLM-Guided Exploration
MCTuner reports up to 19.2% performance gains and roughly 1.4x faster discovery of good configurations by combining LLM knob selection with recursive space decomposition and Bayesian optimization.
-
Pi-SQL: Enhancing Text-to-SQL with Fine-Grained Guidance from Pivot Programming Languages
Generating Python programs as intermediate guidance before SQL, then voting on Python execution results to select the fastest matching SQL, improves text-to-SQL execution accuracy and efficiency on BIRD and Archer.
-
CogniSQL-R1-Zero: Lightweight Reinforced Reasoning for Efficient SQL Generation
A 7B model trained with GRPO and a sparse execution-correctness reward reaches 59.97% execution accuracy on BIRD dev, though the evaluation protocol and baseline numbers contain inconsistencies.
Discussion (0). Sign in to comment.