Pith. sign in

REVIEW 13 cited by

MCS-SQL: Leveraging Multiple Prompts and Multiple-Choice Selection For Text-to-SQL Generation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2405.07467 v1 pith:IK3S44IR submitted 2024-05-13 cs.CL

Dongjun Lee , Choongwon Park , Jaehyuk Kim , Heesoo Park This is my paper
classification cs.CL
keywords promptsqueriesbirdmultipleschemabenchmarkscandidatellms
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Recent advancements in large language models (LLMs) have enabled in-context learning (ICL)-based methods that significantly outperform fine-tuning approaches for text-to-SQL tasks. However, their performance is still considerably lower than that of human experts on benchmarks that include complex schemas and queries, such as BIRD. This study considers the sensitivity of LLMs to the prompts and introduces a novel approach that leverages multiple prompts to explore a broader search space for possible answers and effectively aggregate them. Specifically, we robustly refine the database schema through schema linking using multiple prompts. Thereafter, we generate various candidate SQL queries based on the refined schema and diverse prompts. Finally, the candidate queries are filtered based on their confidence scores, and the optimal query is obtained through a multiple-choice selection that is presented to the LLM. When evaluated on the BIRD and Spider benchmarks, the proposed method achieved execution accuracies of 65.5\% and 89.6\%, respectively, significantly outperforming previous ICL-based methods. Moreover, we established a new SOTA performance on the BIRD in terms of both the accuracy and efficiency of the generated queries.

Discussion (0). Sign in to comment.

Forward citations

Cited by 13 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. EcoTable: Cost-effective Table Integration in Data Lakes for Natural Language Queries

    cs.DB 2026-06 unverdicted novelty 6.0 of 10

    Query-driven table integration that uses Steiner-tree search to choose which joins LLMs must verify, reporting 30%+ accuracy gains at 5x lower LLM cost.

  2. RASL: Retrieval Augmented Schema Linking for Massive Database Text-to-SQL

    cs.CL 2025-07 conditional novelty 6.0 of 10

    RASL retrieves relevant tables and columns for text-to-SQL by decomposing schemas into semantic entities, calibrating entity-type importance on training data, and using an LLM to rank candidates, beating baselines on ...

  3. Text-to-SQL for Enterprise Data Analytics

    cs.CL 2025-07 conditional novelty 6.0 of 10

    An enterprise Text-to-SQL chatbot augmented with a knowledge graph, context ranking, and query-fixing agents achieves 53% correct or nearly correct answers on LinkedIn's internal benchmark, with knowledge graph compon...

  4. SEED: Enhancing Text-to-SQL Performance and Practical Usability Through Automatic Evidence Generation

    cs.CL 2025-06 conditional novelty 6.0 of 10

    SEED automatically generates evidence from database schemas, descriptions, and sampled values, improving text-to-SQL accuracy in no-evidence settings.

  5. SDE-SQL: Enhancing Text-to-SQL Generation in Large Language Models via Self-Driven Exploration with SQL Probes

    cs.CL 2025-06 conditional novelty 6.0 of 10

    SDE-SQL improves text-to-SQL accuracy by having the model generate and execute exploratory SQL probes to learn database contents before and while writing the final query.

  6. SQLens: An End-to-End Framework for Error Detection and Correction in Text-to-SQL

    cs.CL 2025-06 conditional novelty 6.0 of 10

    SQLens uses weak supervision over database and LLM error signals to detect clause-level semantic errors in text-to-SQL and fixes them iteratively, improving execution accuracy on BIRD and Spider.

  7. RAISE: Reasoning Agent for Interactive SQL Exploration

    cs.AI 2025-06 conditional novelty 6.0 of 10

    An interactive reasoning agent that explores databases with live queries raises text-to-SQL execution accuracy from 44.8% to 56.5% and achieves 81.8% best-of-N coverage on BIRD.

  8. Knowledge Base Construction for Knowledge-Augmented Text-to-SQL

    cs.CL 2025-05 conditional novelty 6.0 of 10

    KAT-SQL constructs a reusable knowledge base for text-to-SQL by expanding training data with LLM-generated knowledge and retrieving/refining the best entries for each query.

  9. Automatic Metadata Extraction for Text-to-SQL

    cs.DB 2025-05 conditional novelty 6.0 of 10

    LLM-generated summaries of database profiling statistics outperform human-written metadata for text-to-SQL on BIRD, and a SQL-generation-based schema linker adds further gains.

  10. Pi-SQL: Enhancing Text-to-SQL with Fine-Grained Guidance from Pivot Programming Languages

    cs.CL 2025-06 conditional novelty 5.0 of 10

    Generating Python programs as intermediate guidance before SQL, then voting on Python execution results to select the fastest matching SQL, improves text-to-SQL execution accuracy and efficiency on BIRD and Archer.

  11. SHARE: An SLM-based Hierarchical Action CorREction Assistant for Text-to-SQL

    cs.CL 2025-05 conditional novelty 5.0 of 10

    A three-stage small-model assistant corrects text-to-SQL output by converting SQL into pandas-like action trajectories and refining schema and logic before the LLM rewrites the query.

  12. SLM-SQL: An Exploration of Small Language Models for Text-to-SQL

    cs.CL 2025-07 conditional novelty 4.0 of 10

    A post-training pipeline combining SFT, GRPO reinforcement learning, and corrective self-consistency lifts 0.5B-1.5B models to 56.9%-67.1% execution accuracy on BIRD dev.

  13. Exploring the Landscape of Text-to-SQL with Large Language Models: Progresses, Challenges and Opportunities

    cs.CL 2025-05 conditional novelty 4.0 of 10

    A systematic review organizing LLM-based text-to-SQL methods into pre-processing, in-context learning, fine-tuning, and post-processing paradigms, with a catalog of datasets, metrics, challenges, and future directions.

Pith tools