Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

PSM-SQL: Progressive Schema Learning with Multi-granularity Semantics for Text-to-SQL

T0 review · 4 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read PSM-SQL treats schema linking as a progressive, multi-granularity filtering task: by pruning redundant tables and columns in a chain loop before SQL generation, it raises execution accuracy on Spider and Bird by 1 to 3 percentage points…

desk verdict The paper's own ablation contradicts its central claim: the full MSL method performs worse than the ablated variant that is actually reported as PSM-SQL. read the letter →

arxiv 2502.05237 v1 pith:UB5W44HF submitted 2025-02-07 cs.DB cs.AI

Zhuopan Yang , Yuanzhen Xie , Ruichao Zhong , Yunzhi Tan , Enjie Liu , Zhenguo Yang , Mochi Gao , Bo Hu
show 1 more author
Zang Li
This is my paper · ORCID
classification cs.DBcs.AI
keywords text-to-SQLschemalinkingmulti-granularitysemanticschainloopstrategypruningLLMfine-tuningexecutionaccuracy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper attempts to establish that text-to-SQL accuracy improves when the database schema given to the SQL generator is progressively pruned of redundant tables and columns, rather than passed in full. It claims that learning schema semantics at three granularities—columns, tables, and whole databases—and applying this schema linker iteratively in a chain loop keeps the schemas the gold SQL needs while discarding the rest. If true, this matters because redundant schemas interfere with both fine-tuned and prompted large language models, so a reliable pruning method could make text-to-SQL systems cheaper and more accurate. The paper reports execution-accuracy gains of 1 to 3 percentage points over existing methods on the Spider and Bird benchmarks.

What carries the argument

The load-bearing component is the multi-granularity schema linking (MSL) module, which operates at three levels: column level, using a fine-tuned embedding model (BGE) trained with triplet loss to rank column relevance; table level, using a RoBERTa-LSTM cross-encoder with a disentangled question embedding, multi-head attention over columns, and both cosine-similarity and classifier scores to select schemas; and database level, using a LoRA-fine-tuned LLM (Llama3-8B) that takes an instruction and the current schema set and outputs relevant schemas. The MSL outputs are unioned into a filtered schema set, which is fed back into the next round of the chain loop. The chain loop itself is the second load-bearing idea: it trades a lower theoretical upper bound on accuracy for a reduced task difficulty by shrinking the schema set each cycle.

What would settle it

Take the Bird development set and, after each PSM-SQL cycle, check whether every table and column required by the gold SQL is still present in the filtered schema set. If any query whose required schema was pruned in an earlier cycle yields a wrong execution result—while the same query with that schema restored yields the correct result—then the chain loop's irreversible pruning is the cause. A direct test: run Cycle_2 with the gold schemas forcibly added back after pruning; if execution accuracy does not increase over standard Cycle_2, then the pruning loss is not the binding constraint.

Watch

Extended reading notes

Core claim

PSM-SQL's central claim is that schema linking for text-to-SQL should be treated as a progressive, multi-granularity filtering task rather than a one-shot selection. The method trains a column-level embedding model with triplet loss to rank candidate columns, a table-level cross-encoder with a classifier and cosine-similarity scores to select tables and columns, and a database-level fine-tuned LLM to reason about which schemas are relevant. These three filters are applied in a chain loop: each cycle's filtered schema set becomes the next cycle's input, so the task difficulty decreases as redundant schemas are removed. The paper argues that this continuous reduction in schema count, combined with the complementary semantic signals from the three granularities, is what yields higher matching accuracy, lower redundancy, and better execution accuracy than methods that link schemas only at the table level and only once.

Load-bearing premise

The weakest load-bearing premise is that the schema filter in each cycle, trained on ground-truth schema labels, never prunes away a table or column that the gold SQL actually needs, and that those ground-truth labels are complete and correct; if either fails, the correct SQL can become unreachable.

Editorial extensions

If this is right

  • If the central claim holds, replacing one-shot schema linking with iterative multi-granularity pruning reduces the number of tokens and irrelevant columns in prompts, lowering inference cost and helping LLMs focus attention on the relevant schema.
  • The reported gains on both Bird and Spider suggest the method transfers across different dataset distributions, from smaller manually annotated schemas in Spider to large, noisy, evidence-augmented databases in Bird.
  • Because the paper shows that plugging PSM-SQL's predicted schemas into an existing method (MAC-SQL) improves that method's accuracy, the schema linker could be used as a drop-in component in other text-to-SQL pipelines.
  • The chain loop explanation implies that schema-pruning quality, not just SQL-generation capability, is a limiting factor for current text-to-SQL systems.
  • Larger LLMs (Llama3-70B) benefit more from the pruned schemas than smaller ones, suggesting the method's value grows as base models get stronger.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the approach is correct, the schema-linking module could be trained once and reused across different SQL generators (fine-tuned or prompted), because it is decoupled from the parse step; this would make schema pruning a reusable preprocessing service for text-to-SQL.
  • The chain loop's irreversible pruning suggests a testable extension: add a fallback that, when final SQL generation fails, re-expands the schema set from an earlier cycle; if this recovers accuracy, it would show the loop's trade-off can be softened.
  • The ground-truth schema labels are derived from gold SQL; a natural extension would be to generate pseudo-labels from a trained SQL model's attention or from execution feedback, which could remove the dependency on annotated schema labels.
  • The multi-granularity idea could transfer to other retrieval-augmented tasks with hierarchical structure, such as API selection or function-signature matching, not just database schemas.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. PSM-SQL proposes a progressive schema linking framework with a multi-granularity schema linking (MSL) module operating at column, table, and database levels, combined with a chain loop that iteratively prunes redundant schemas before SQL generation. The paper evaluates on Spider and Bird, reporting schema-linking metrics (MA, IA, RE) and SQL-generation metrics (EX, EM, VES). The central claim is that the multi-granularity schema linking and the chain loop improve execution accuracy by 1–3 percentage points over existing methods.

Significance. If the claimed gains were supported, the idea of iteratively pruning schemas at multiple granularities would be a useful contribution to the text-to-SQL literature. The paper has some strengths: it includes a dedicated ablation study (Tables 4–5), reports schema-linking metrics in addition to end-to-end EX/VES, and explicitly acknowledges the chain-loop trade-off in Section 7. However, as detailed below, the paper's own ablation contradicts the attribution of the reported gains to the full MSL configuration, so the central claim is not currently supported.

major comments (4)
  1. [§5.5, §5.6, Tables 1 and 5] The reported PSM-SQL results are identical to the Cycle_2 Cross_encoder+Gen_LLM ablation, not to the full MSL (All) configuration. Table 5 reports Cycle_2 Cross_encoder+Gen_LLM as EX 59.71 and VES 65.15 on the Bird dev set, and Table 1 reports PSM-SQL+Llama3-8B as EX 59.71 and VES 65.15. In contrast, the full MSL (All) configuration in Cycle_2 gives lower EX (58.80) and much lower VES (53.28). Thus the model named PSM-SQL in the headline experiments is not the full method described in Section 4, and the headline results do not demonstrate any benefit from the multi-granularity column-level component (Emb_LLM). This is a load-bearing inconsistency because the paper's central claim attributes the gains to the multi-granularity module.
  2. [§5.6, Table 4] The ablation also shows that MSL (All) is worse than Cross_encoder+Gen_LLM on the schema-linking metrics themselves: in Cycle_2, table-level MA is 32.79 versus 56.65 and table-level RE is 43.07 versus 25.43. The text in §5.6 states that 'MSL achieves better performance than Cross_encoder, Emb_LLM, and Gen_LLM', but the table does not support this for the combined configuration; only IA is higher. This further undermines the mechanism claimed for the reported improvements.
  3. [Tables 1–2] No test-set results are reported for PSM-SQL; the Test columns are '-' for all PSM-SQL rows on both Bird and Spider. Since the abstract and introduction claim 1–3 percentage point improvements over existing methods, and the tables include test-set numbers for baseline methods such as CHESS and SFT CODES, the claimed superiority is not substantiated on held-out test sets. The paper should state explicitly why test results are omitted, or the claims should be limited to dev-set performance.
  4. [§4.2 and §7] The chain loop can discard correct schemas, and the paper acknowledges this in Section 7 but does not quantify or mitigate it. Because each cycle's filter is trained using ground-truth schema labels derived from gold SQL, an incorrect prune in an early cycle cannot be recovered by the later SQL generator. The paper should report per-cycle recall/IA and, if possible, an oracle experiment that re-injects gold schemas, to establish that iterative pruning does not systematically lose information needed for SQL generation.
minor comments (5)
  1. [§5.4] The threshold of 0.5 and the Top-8/Top-4 column selection counts are stated without a sensitivity analysis; since these hyperparameters interact with the chain loop, a brief sensitivity study would strengthen the empirical claims.
  2. [§4.2] The ground-truth schema labels d_gt are used before being formally defined; please state explicitly how d_gt is derived from gold SQL and whether it includes both tables and columns at each granularity level.
  3. [§5.2, Eq. (22)] The VES formula appears to compute an average of time ratios multiplied by EX, but the text describes it as measuring 'execution efficiency of accurately generated SQL queries'; please clarify whether VES is computed only on correctly executed queries or over all predictions.
  4. [Tables 1–2 and §5.5] The row labels 'PSM-SQL+DeepSeek-7B' and 'PSM-SQL+Llama3-8B' are confusing because Section 4 describes Llama3-8B as the database-level reasoner and SQL generator; please clarify which backbone is used for which component in each reported row.
  5. [Throughout] There are several typos and minor wording issues, including 'Trubo' for 'Turbo', 'chemas' for 'schemas', and 'employes' for 'employs'; these should be corrected in a revision.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: PSM-SQL is an empirical supervised-learning pipeline; the reported method/ablation inconsistency is a correctness and attribution issue, not a self-referential derivation.

full rationale

No equation-level circularity is present. The schema-linking losses (Eqs. 4, 11-13, 15) supervise column, table, and database-level modules with ground-truth schema labels d_gt extracted from gold SQL, and the resulting filters are then applied to held-out dev sets. This is standard supervised learning, not a prediction that is forced by construction. SQL generation (Eqs. 16-17) consumes predicted schemas but is evaluated on the same external benchmarks used by all baselines, so no fitted parameter is renamed as a prediction. The reference list contains no prior work by the present authors, so there is no load-bearing self-citation, no imported uniqueness theorem, and no ansatz smuggled in via self-citation. The Section 7 limitation, stating that the chain loop 'may discard some correct schemas related to SQL generation during cycles,' is an acknowledged trade-off rather than circular reasoning. One notable internal inconsistency exists: the PSM-SQL+Llama3-8B row in Tables 1-3 reports dev EX 59.71 / VES 65.15 and table/column schema-linking numbers identical to the Cycle_2 'Cross_encoder+Gen_LLM' ablation in Tables 4-5, while the full 'MSL (All)' configuration scores lower (EX 58.80, VES 53.28). The paper therefore attributes its headline gains to a configuration that, by its own ablation, omits the column-level Emb_LLM contribution. This is a serious correctness and attribution problem, but it is not circular: the reported numbers were not derived from the claimed mechanism by definition or by a self-citation chain.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim relies on supervised labels extracted from gold SQL, on the safety of iterative pruning, and on the transferability of pretrained representations. The hand-set thresholds and top-k counts are free parameters with no sensitivity analysis.

free parameters (4)
  • Schema retention threshold = 0.5
    Used in Eq. (3) and Section 5.4 to retain schemas at column and table levels. Hand-set, with no sensitivity analysis.
  • Top-K column selection counts = Top-8 columns for Top-2 tables, Top-4 columns for remaining tables
    Inference rule in Section 5.4. Hand-set and not ablated.
  • Triplet margin beta = Not reported
    Slack margin in the column-level triplet loss L_c in Eq. (4). Value is unspecified.
  • Chain loop cycle count = 2
    All reported configurations use Cycle_2 in Tables 4-5. No study of more than two cycles is provided.
assumptions (3)
  • domain assumption Ground-truth schema labels d_gt are complete and correctly derived from gold SQL.
    Section 4.2 defines positives as schemas in d_gt. If these labels are incomplete or noisy, all three levels of MSL learn from bad supervision.
  • domain assumption Pruning schemas in earlier cycles preserves all schemas needed by the gold SQL.
    The chain loop feeds filtered schemas into the next cycle, so a discarded correct schema cannot be recovered. Section 7 explicitly admits correct schemas may be discarded.
  • domain assumption Pre-trained embeddings and cross-encoders transfer to schema semantics.
    BGE-large-en-v1.5 and ROBERTA-large are assumed suitable for question-schema matching. No failure analysis or per-domain breakdown is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PSM-SQL: Progressive Schema Learning with Multi-granularity Semantics for Text-to-SQL." pith.science (2026). https://pith.science/paper/UB5W44HF

@misc{pith2026250205237,
  author       = {Pith},
  title        = {Pith review of: PSM-SQL: Progressive Schema Learning with Multi-granularity Semantics for Text-to-SQL},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UB5W44HF}},
  note         = {Machine review of arXiv:2502.05237}
}
read the original abstract

It is challenging to convert natural language (NL) questions into executable structured query language (SQL) queries for text-to-SQL tasks due to the vast number of database schemas with redundancy, which interferes with semantic learning, and the domain shift between NL and SQL. Existing works for schema linking focus on the table level and perform it once, ignoring the multi-granularity semantics and chainable cyclicity of schemas. In this paper, we propose a progressive schema linking with multi-granularity semantics (PSM-SQL) framework to reduce the redundant database schemas for text-to-SQL. Using the multi-granularity schema linking (MSL) module, PSM-SQL learns the schema semantics at the column, table, and database levels. More specifically, a triplet loss is used at the column level to learn embeddings, while fine-tuning LLMs is employed at the database level for schema reasoning. MSL employs classifier and similarity scores to model schema interactions for schema linking at the table level. In particular, PSM-SQL adopts a chain loop strategy to reduce the task difficulty of schema linking by continuously reducing the number of redundant schemas. Experiments conducted on text-to-SQL datasets show that the proposed PSM-SQL is 1-3 percentage points higher than the existing methods.

Figures

Figures reproduced from arXiv: 2502.05237 by the authors.

Figure 1
Figure 1. Illustration of an example of Text-to-SQL [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The framework of PSM-SQL. of language models ranging from 1B to 15B pa￾rameters using their collected SQL-focused cor￾pus. However, they often focus on optimizing SQL generation while neglecting the optimization of schema linking. 3 Problem Definition In this section, we formalize the notations of Text￾to-SQL, which aims to convert the NL questions to SQL queries. Without loss of generality, Text￾to-SQL can be divid… view at source ↗
Figure 3
Figure 3. Details of the Multi-granularity schema link [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. PaVeRL-SQL: Text-to-SQL via Partial-Match Rewards and Verbal Reinforcement Learning

    cs.AI 2025-09 reject novelty 5.0 of 10

    PaVeRL-SQL reports SOTA execution accuracy on Spider2.0-SQLite using partial-match rewards and verbal RL, but overclaims SOTA on Spider and BIRD.

Reference graph

Works this paper leans on

26 extracted references · 16 canonical work pages · cited by 1 Pith paper

  1. [1]

    Ruichu Cai, Jinjie Yuan, Boyan Xu, and Zhifeng Hao. 2021. SADGA: structure-aware dual graph aggregation network for text-to-sql. In NeurIPS, pages 7664--7676

    work page 2021

  2. [2]

    Ruisheng Cao, Lu Chen, Zhi Chen, Yanbin Zhao, Su Zhu, and Kai Yu. 2021. LGESQL: line graph enhanced text-to-sql model with mixed local and non-local relations. In ACL, pages 2541--2555

    work page 2021

  3. [3]

    Naihao Deng, Yulong Chen, and Yue Zhang. 2022. Recent advances in text-to-sql: A survey of what we have and what we expect. In COLING, pages 2166--2187

    work page 2022

  4. [4]

    Xuemei Dong, Chao Zhang, Yuhang Ge, Yuren Mao, Yunjun Gao, lu Chen, Jinshu Lin, and Dongfang Lou. 2023. https://arxiv.org/abs/2307.07306 C3: Zero-shot text-to-sql with chatgpt . Preprint, arXiv:2307.07306

    arXiv 2023

  5. [5]

    Dawei Gao, Haibin Wang, Yaliang Li, Xiuyu Sun, Yichen Qian, Bolin Ding, and Jingren Zhou. 2024. Text-to-sql empowered by large language models: A benchmark evaluation. In VLDB, page 1132–1145

    work page 2024

  6. [6]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. Lora: Low-rank adaptation of large language models. In ICLR, pages 1--13

    work page 2022

  7. [7]

    Dongjun Lee, Choongwon Park, Jaehyuk Kim, and Heesoo Park. 2024. https://arxiv.org/abs/2405.07467 Mcs-sql: Leveraging multiple prompts and multiple-choice selection for text-to-sql generation . Preprint, arXiv:2405.07467

    arXiv 2024

  8. [8]

    Haoyang Li, Jing Zhang, Cuiping Li, and Hong Chen. 2023 a . RESDSQL: decoupling schema linking and skeleton parsing for text-to-sql. In AAAI, pages 13067--13075

    work page 2023

Show all 26 references
  1. [9]

    Haoyang Li, Jing Zhang, Hanbing Liu, Ju Fan, Xiaokang Zhang, Jun Zhu, Renjie Wei, Hongyan Pan, Cuiping Li, and Hong Chen. 2024. Codes: Towards building open-source language models for text-to-sql. ACM on Management of Data, 2(3):1--28

    2024

  2. [10]

    Jinyang Li, Binyuan Hui, Reynold Cheng, Bowen Qin, Chenhao Ma, Nan Huo, Fei Huang, Wenyu Du, Luo Si, and Yongbin Li. 2023 b . Graphix-t5: Mixing pre-trained transformers with graph-aware layers for text-to-sql parsing. In AAAI, pages 13076--13084

    2023

  3. [11]

    Jinyang Li, Binyuan Hui, Ge Qu, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Ruiying Geng, Nan Huo, Xuanhe Zhou, Chenhao Ma, Guoliang Li, Kevin Chen - Chuan Chang, Fei Huang, Reynold Cheng, and Yongbin Li. 2023 c . Can LLM already serve as A database interface? A b...

    2023

  4. [12]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. https://arxiv.org/abs/1907.11692 Roberta: A robustly optimized bert pretraining approach . Preprint, arXiv:1907.11692

    2019 arXiv

  5. [13]

    Ana - Maria Popescu, Oren Etzioni, and Henry A. Kautz. 2003. Towards a theory of natural language interfaces to databases. In IUI, pages 149--157

    2003

  6. [14]

    Mohammadreza Pourreza and Davood Rafiei. 2023. DIN-SQL: decomposed in-context learning of text-to-sql with self-correction. In NeurIPS, pages 1--10

    2023

  7. [15]

    Mohammadreza Pourreza and Davood Rafiei. 2024. https://arxiv.org/abs/2402.01117 Dts-sql: Decomposed text-to-sql with small large language models . Preprint, arXiv:2402.01117

    2024 arXiv

  8. [16]

    Bowen Qin, Lihan Wang, Binyuan Hui, Bowen Li, Xiangpeng Wei, Binhua Li, Fei Huang, Luo Si, Min Yang, and Yongbin Li. 2022. SUN: exploring intrinsic uncertainties in text-to-sql parsers. In COLING, pages 5298--5308

    2022

  9. [17]

    Ge Qu, Jinyang Li, Bowen Li, Bowen Qin, Nan Huo, Chenhao Ma, and Reynold Cheng. 2024. https://arxiv.org/abs/2405.15307 Before generation, align it! a novel and effective strategy for mitigating hallucinations in text-to-sql generation . Preprint, arXiv:2405.15307

    2024 arXiv

  10. [18]

    Shayan Talaei, Mohammadreza Pourreza, Yu-Chen Chang, Azalia Mirhoseini, and Amin Saberi. 2024. https://arxiv.org/abs/2405.16755 Chess: Contextual harnessing for efficient sql synthesis . Preprint, arXiv:2405.16755

    2024 arXiv

  11. [19]

    Tang and Raymond J

    Lappoon R. Tang and Raymond J. Mooney. 2001. Using multiple clause constructors in inductive logic programming for semantic parsing. In ECML, pages 466--477

    2001

  12. [20]

    Bing Wang, Changyu Ren, Jian Yang, Xinnian Liang, Jiaqi Bai, Linzheng Chai, Zhao Yan, Qian-Wen Zhang, Di Yin, Xing Sun, and Zhoujun Li. 2024. https://arxiv.org/abs/2312.11242 Mac-sql: A multi-agent collaborative framework for text-to-sql . Preprint, arXiv:2312.11242

    2024 arXiv

  13. [21]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In NeurIPS, pages 24824--24837

    2022

  14. [22]

    Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, Defu Lian, and Jian-Yun Nie. 2024. https://arxiv.org/abs/2309.07597 C-pack: Packaged resources to advance general chinese embedding . Preprint, arXiv:2309.07597

    2024 arXiv

  15. [23]

    Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir R. Radev. 2018. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. I...

    2018

  16. [24]

    Zelle and Raymond J

    John M. Zelle and Raymond J. Mooney. 1996. Learning to parse database queries using inductive logic programming. In AAAI, pages 1050--1055

    1996

  17. [25]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  18. [26]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.