Pith. sign in

REVIEW 4 major objections 7 minor 21 references

UNJOIN: Enhancing Multi-Table Text-to-SQL Generation via Schema Simplification

T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read UNJOIN lifts multi-table SQL accuracy by flattening schemas

desk verdict Clean, prompt-only flatten-and-translate method with modest gains; the SOTA claim overreaches and the join-inference step is untested on opaque schemas. read the letter →

arxiv 2505.18122 v1 pith:VIY2OGXW submitted 2025-05-23 cs.CL

Poojah Ganesan , Rajat Aayush Jha , Dan Roth , Vivek Gupta This is my paper · ORCID
classification cs.CL
keywords text-to-SQLmulti-tabledatabasesschemasimplificationLLMpromptingquerytranslationretrievalSPIDERbenchmarkBIRD
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

UNJOIN is a prompt-only, two-stage framework for multi-table text-to-SQL translation. It first flattens every table in a database into a single virtual table by renaming each column as TableName.ColumnName, and asks an LLM to write SQL against this simplified schema. A second stage translates that intermediate query back to the original schema, reconstructing the necessary JOINs, UNIONs, and relational logic. Data-free and fine-tuning-free, it is evaluated on SPIDER and BIRD multi-table queries and reported to match or exceed the baselines it compares against, including prompting, in-context-learning, fine-tuned, and reasoning-model approaches. A sympathetic reader would take the central claim to be that decomposing retrieval from relational logic reduces compounding errors in multi-table SQL generation.

What carries the argument

The central object is the simplified single-table schema: every column of every table is renamed as TableName.ColumnName and pooled under one virtual table named after the database. This makes the schema a list of unambiguous columns instead of several connected tables. The intermediate SQL query written against this virtual schema is the carrier of the argument: it lets the LLM perform retrieval and aggregation without choosing join paths. The translation stage then converts that query back to the original schema, using the table-name prefixes to infer where joins belong; an edit-distance post-processing step corrects invented or abbreviated table and column names without changing SQL logic. UNJOIN has two variants, a single-prompt joint version (SP) and a multi-prompt staged version (MP).

What would settle it

Take a set of SPIDER-style databases, replace table and column names with random tokens while preserving the relation graph, run UNJOIN's translation stage, and compare exact match to the named-schema version. If exact match falls to the level of direct prompting, then the claimed improvement comes from name-based join inference rather than from decoupling retrieval and logic.

Watch

Extended reading notes

Core claim

The central claim is that multi-table text-to-SQL becomes more accurate when the task is split so that schema-element retrieval and SQL logic construction no longer compete in one step. UNJOIN's schema simplification merges all columns of a database into one flat schema with table-name prefixes; the LLM then generates an intermediate single-table query, and a later translation step maps it back to the original schema by reintroducing joins and set operations. On the multi-table subset of SPIDER, the multi-prompt variant UNJOIN MP reports exact-match accuracy 77.13 and execution accuracy 99.9 with Llama 3.3, against 75.28 and 98.87 for the strongest standard-prompt baseline; on BIRD it reports 56.35 exact match against 53.55 for chain-of-thought prompting. The paper argues that schema simplification alone improves retrieval precision and recall, but that the translation step is essential, since generating final SQL directly on the simplified schema without translation collapses exact-match accuracy.

Load-bearing premise

The load-bearing premise is that a language model can reconstruct the correct JOINs, UNIONs, and relational logic from table and column name prefixes alone, without explicit foreign-key constraints; if the schema names carry no join information, the translation step has nothing reliable to work with and the whole gain from simplification collapses.

Editorial extensions

If this is right

  • On the paper's evidence, a frozen general-purpose LLM can reach or exceed fine-tuned SQL models on multi-table queries by changing only the prompt's schema representation.
  • Because the method uses only schema names, not row-level data, it can be applied to a new database without data access or training.
  • The translation step, not just flattened retrieval, is necessary; skipping it drastically lowers exact-match accuracy, so any follow-up should preserve that second stage.
  • In open-book settings, UNJOIN acts as a plug-in generator: over several retrievers it raises exact match by roughly 15 to 20 percentage points.
  • The reported gains concentrate in table and column selection, with column recall above 95 percent for GPT-4o on SPIDER under the single-prompt variant.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The method's success likely depends on LLMs having memorized common join keys such as customer_id and account_id from training; on schemas with opaque or randomized column names the translation stage would have no names to lean on, which is a testable failure mode the paper does not address.
  • The framework could be extended to include explicit foreign-key metadata in the translation prompt without sacrificing the flat retrieval stage, which would isolate whether name-based inference is what actually carries the second step.
  • Because the simplification is deterministic and prompt-only, the same two-stage idea may transfer to other structured output tasks where retrieval and composition compete, such as multi-document extraction or graph query generation.
  • The paper's own failure analysis suggests column-name ambiguity in the question-to-schema mapping is a residual error source; future work could add a disambiguation step before translation rather than after generation.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes UNJOIN, a two-stage prompt-only framework for multi-table text-to-SQL. Stage 1 deterministically flattens each multi-table schema into a single virtual table whose columns are named table.column. Stage 2 asks an LLM to first generate a SQL query over this simplified schema and then translate it back to the original schema by reconstructing joins, unions, and relational logic, followed by an edit-distance post-processing step that corrects table/column names. Two variants are studied: UNJOIN SP (single joint prompt) and UNJOIN MP (separate prompts). The method is evaluated on filtered multi-table subsets of SPIDER and BIRD against standard prompting, ICL, SFT, table-QA, and reasoning-LLM baselines, and in an open-book setting with ARM, JAR, DTR, and Contriever retrievers. The central claim is that decoupling schema-element retrieval from SQL-join construction improves accuracy, scalability, and generalizability without data access or fine-tuning.

Significance. If the claims hold, the paper offers a simple and useful prompt-only alternative to fine-tuned multi-table SQL systems: deterministic schema simplification plus LLM-based query translation. The strengths include a broad model and retriever evaluation, the open-book results showing consistent improvements over external retrievers, and the CoT-SS ablation that cleanly isolates the contribution of the translation stage. However, the empirical support is weaker than the abstract suggests: the strongest ICL baseline, RSL-SQL, outperforms both UNJOIN variants on BIRD, and the SPIDER differences are within a fraction of a point. The load-bearing translation mechanism is also not tested on schemas where join keys cannot be inferred from column names, which is a serious gap for the generalization claim.

major comments (4)
  1. [Section 2, Appendix A.1/A.3, Table 9] The translation stage is load-bearing for the method's success. Table 9 shows that GPT-4o CoT-SS (direct SQL generation on the simplified schema) achieves only 11.68 EM on SPIDER, while UNJOIN SP reaches 76.13 EM, so nearly all of the accuracy comes from translating the simplified query back to the original schema. Yet the translation prompt in Appendix A.1 and A.3 supplies only 'An original schema: a set of multiple related tables' and gives no explicit primary-key/foreign-key constraints or join-path hints. The paper nowhere states whether foreign-key information is included in the model's input, and Algorithm 1 constructs the simplified schema from table and column names alone. The method therefore relies on the LLM inferring joins from name overlap or semantic similarity, which is plausible on SPIDER/BIRD but is exactly the assumption that needs testing for the claimed generalizability to 'databases of any size or content.' Please state explicitly what relational metadata is provided to the model, and add experiments with opaque or non-descriptive schema names; without them, the main mechanism of the approach is not empirically established.
  2. [Abstract and Section 4.1, Table 2] The abstract's claim that 'UNJOIN matches or exceeds the state-of-the-art baselines' is not supported by Table 2. RSL-SQL achieves 90.80 QE / 54.30 EM on BIRD, outperforming both UNJOIN SP (88.55 / 51.74) and UNJOIN MP (89.75 / 50.36). On SPIDER, UNJOIN SP exceeds RSL-SQL by only 0.09 EM points (76.13 vs. 76.04). The claim should be restricted to the specific comparison settings and should acknowledge that RSL-SQL remains stronger on BIRD. This is not merely a wording issue; it changes what the reader can conclude from the paper's main quantitative statement.
  3. [Section 4, Datasets paragraph, and Table 2] The paper filters both datasets to multi-table queries (443 SPIDER queries across 81 databases; 1095 BIRD queries across 77 databases), but it does not state whether the ICL baselines in Table 2, especially RSL-SQL, were re-run on the same filtered subsets or taken from their original publications on the full datasets. If the latter, the comparison is not apples-to-apples: UNJOIN is evaluated on a subset, while the baseline numbers may come from a different evaluation set. Please clarify, and ideally re-evaluate the baselines under identical filtering and prompting conditions, or clearly label transferred numbers as full-set results.
  4. [Tables 1, 2, and 6] No variance or statistical significance information is reported anywhere, despite the stochastic nature of LLM decoding. Several headline comparisons are within a fraction of a point (e.g., 76.13 vs. 76.04, 76.00 vs. 76.04), and the open-book gains in Table 6 are reported as single runs. The paper should report multiple runs or seeds with means and standard deviations, or at least confidence intervals, for the central closed-book and open-book comparisons, so that the claimed improvements can be distinguished from noise.
minor comments (7)
  1. [Section 3] There is a typo: 'approches' should be 'approaches'.
  2. [Section 2] 'UNJOIN Varaints' should be 'UNJOIN Variants'; the spelling of 'variants' is also inconsistent elsewhere.
  3. [Section 2] The text refers to 'Appendix 4', but the appendices are lettered (A-F); this should be 'Appendix B' for the Schema Simplification algorithm and example.
  4. [Table 3] The table header says 'Table QA (non SQL-based) baselines EM' and reports only EM. Please clarify whether QE is omitted intentionally and whether these baselines were evaluated on the same filtered multi-table subset as UNJOIN.
  5. [Ethics Statement] The ethics statement promises 'publicly available code, datasets, and relevant resources', but no repository link or resource URL appears in the manuscript.
  6. [Figure 3] Figure 3 would benefit from labeled axes, legend details, and error bars; as presented, the claimed robustness trend of UNJOIN with increasing table counts is hard to quantify.
  7. [Limitations] The limitations section does not mention the method's dependence on descriptive schema names for join inference, even though this is the main failure mode identified by the experimental design and is central to the generalization claim.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: UNJOIN's pipeline is a fixed, parameter-free schema flattening plus LLM generation/translation scored against held-out gold SQL; same-group retriever citations are not load-bearing.

full rationale

UNJOIN is a fixed, parameter-free pipeline: Algorithm 1 deterministically flattens each column into table.column form; Stage 1 and Stage 2 SQL are produced by the LLM prompts in Appendix A and then translated back to the original schema; QE and EM are computed against held-out SPIDER/BIRD gold queries. No quantity in the pipeline is fitted to the target metric, and no output is equal to an input by construction. The CoT-SS ablation (Table 9) is a genuine negative control: directly generating final SQL on the simplified schema collapses EM, so the translation stage carries real work rather than being cosmetic. The main self-referential element is the use of ARM and JAR (Chen et al. 2025a,b, involving co-author Dan Roth) as open-book retrievers; however, Table 6 shows comparable gains with external Contriever and DTR retrievers, so those citations are not load-bearing, and no uniqueness or ansatz is imported from them. The concern that the translation prompt supplies only 'a set of multiple related tables' and no explicit foreign-key constraints is a generalization/correctness risk on opaque schemas, not a circularity: the method's success on SPIDER/BIRD is measured against external gold SQL and would be falsifiable. Hence no circular step is exhibited; the score of 2 merely reflects the presence of minor same-group citations that do not support the central claim.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

No parameters are fitted to data. The method is a fixed prompt pipeline. The main unstated choices are the edit-distance correction rule and the example set. The load-bearing domain assumptions are that LLMs do better on flat schemas and that column names are enough to reconstruct joins. No new physical or conceptual entities are introduced.

free parameters (2)
  • edit_distance_threshold = not reported
    The schema-consistency post-processing step in Section 2 uses an edit-distance-based correction; the threshold or matching rule is not specified, so its contribution to EM accuracy cannot be independently assessed.
  • few_shot_example_count = not reported
    The prompts depend on hand-crafted examples; the exact set is not fully disclosed (Listings 2 and 3 show '<examples>' placeholders), making the method sensitive to prompt choice.
assumptions (4)
  • domain assumption LLMs generate SQL over single-table schemas more reliably than over multi-table schemas
    Stated in Section 1 as the key intuition; it motivates the entire pipeline and is not proven in the paper.
  • domain assumption Table and column names alone suffice for the LLM to reconstruct JOINs and UNIONs in the translation step
    Section 2 Query Translation relies on LLM reasoning without rule-based logic or explicit foreign-key constraints; the paper does not state whether key information is included in the schema.
  • domain assumption The SPIDER and BIRD multi-table subsets are representative enough to support cross-domain generalization claims
    Evaluation in Section 4 is on filtered subsets (443 SPIDER, 1095 BIRD queries), and generalization claims are made from these subsets.
  • ad hoc to paper The edit-distance correction changes only names, not SQL logic
    Section 2 states this; no ablation isolates the effect, so the assumption is load-bearing for attributing gains to the two-stage design rather than to the correction.

how reviews work

0 comments
Cite this review

Pith. "Pith review of UNJOIN: Enhancing Multi-Table Text-to-SQL Generation via Schema Simplification." pith.science (2026). https://pith.science/paper/VIY2OGXW

@misc{pith2026250518122,
  author       = {Pith},
  title        = {Pith review of: UNJOIN: Enhancing Multi-Table Text-to-SQL Generation via Schema Simplification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VIY2OGXW}},
  note         = {Machine review of arXiv:2505.18122}
}
read the original abstract

Recent advances in large language models (LLMs) have greatly improved Text-to-SQL performance for single-table queries. But, it remains challenging in multi-table databases due to complex schema and relational operations. Existing methods often struggle with retrieving the right tables and columns, generating accurate JOINs and UNIONs, and generalizing across diverse schemas. To address these issues, we introduce UNJOIN, a two-stage framework that decouples the retrieval of schema elements from SQL logic generation. In the first stage, we merge the column names of all tables in the database into a single-table representation by prefixing each column with its table name. This allows the model to focus purely on accurate retrieval without being distracted by the need to write complex SQL logic. In the second stage, the SQL query is generated on this simplified schema and mapped back to the original schema by reconstructing JOINs, UNIONs, and relational logic. Evaluations on SPIDER and BIRD datasets show that UNJOIN matches or exceeds the state-of-the-art baselines. UNJOIN uses only schema information, which does not require data access or fine-tuning, making it scalable and adaptable across databases.

Figures

Figures reproduced from arXiv: 2505.18122 by the authors.

Figure 1
Figure 1. Baseline vs UNJOIN Early approaches use syntax trees or query sketches to guide query generation (Xu et al., 2017; Guo et al., 2019), while more recent methods rely on sequence-to-sequence models (Colin, 2020; Scholak et al., 2021a). Recent advances leverage large language models (LLMs), either through in￾context learning with powerful proprietary models or by fine-tuning smaller open-source alternatives, leading to… view at source ↗
Figure 2
Figure 2. Our Proposed Method. After schema conversion, the resulting single-table schema contains a total of 54 columns obtained by combining the columns from all shown tables: order, account, district, disp, card, client, loan, and trans. Due to space constraints, the complete single-table representation is not shown. prompt the LLM to generate a semantically ac￾curate intermediate (simplified) SQL query. By abstracting awa… view at source ↗
Figure 3
Figure 3. Retrieval accuracy Performance with Increas [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Schema Simplification Model Table Precision Table Recall CoT CoT-SS UNJOIN SP UNJOIN MP CoT CoT-SS UNJOIN SP UNJOIN MP GPT-4o 59.30 57.20 73.60 74.70 58.60 58.00 73.40 74.20 GPT-4o Mini 57.40 58.80 75.00 74.80 56.00 58.30 72.40 73.30 Gemini 1.5 Flash 53.00 62.90 65.30 …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 18 canonical work pages

  1. [1]

    Understand the Core Objective from the Question : - Identify the goal of the query ( e . g . , aggregate data , filter specific rows , join information across tables )

    work page

  2. [2]

    Map Simplified Schema Columns to the Original Schema : - Identify how the columns in the simplified schema correspond to tables and columns in the original schema

    work page

  3. [3]

    Construct Necessary Joins : - If the original schema splits data across multiple tables , determine the joins needed to recreate the relationships

    work page

  4. [4]

    Translate Filters and Conditions : - Map WHERE clauses , conditions , and filters in the simplified query to the original schema

    work page

  5. [5]

    Adapt Query Logic ( Aggregation , Sorting , etc .) : - Match aggregations , grouping , or ordering logic from the simplified query to the original schema

    work page

  6. [6]

    < examples > 12 Key Considerations : - Use the Question as a Guide : Align the query logic with the intent expressed in the question

    Validate the Final Query Against the Question : - Review the final query to ensure it satisfies the question and produces the intended result . < examples > 12 Key Considerations : - Use the Question as a Guide : Align the query logic with the intent expressed in the question . - Simplified Schema as a Mapping Tool : Treat the simplified schema as a bridg...

    work page

  7. [10]

    The query should strictly adhere to the schema provided

    work page

  8. [11]

    Ensure correct SQL syntax with SELECT , FROM , WHERE , GROUP BY , and ORDER BY clauses as needed

    work page

Show all 21 references
  1. [12]

    Output : A.3 Prompt for UNJOIN MP : Step2 : Query Translation Listing 3: SQL Query Generation Prompt: UNJOIN MP Step2 You are an expert at semantic parsing

    The output query must be structured , readable , and executable in a standard SQL database . Output : A.3 Prompt for UNJOIN MP : Step2 : Query Translation Listing 3: SQL Query Generation Prompt: UNJOIN MP Step2 You are an expert at semantic parsing . You will be provided : - A...

  2. [13]

    Generate a SQL query based on a simplified schema

  3. [14]

    Translate the simplified query into a query compatible with the original schema

  4. [15]

    ### Steps for Translation :

    Ensure the translated query aligns with the intent described in the question . ### Steps for Translation :

  5. [16]

    ** Understand the Core Objective from the Question **: - Identify the goal of the query ( e . g . , aggregate data , filter specific rows , join information across tables )

  6. [17]

    ** Map Simplified Schema Columns to the Original Schema **: - Identify how the columns in the simplified schema correspond to tables and columns in the original schema

  7. [18]

    ** Construct Necessary Joins **: - If the original schema splits data across multiple tables , determine the joins needed to recreate the relationships

  8. [19]

    ** Translate Filters and Conditions **: - Map WHERE clauses , conditions , and filters in the simplified query to the original schema

  9. [20]

    ** Adapt Query Logic ( Aggregation , Sorting , etc .) **: - Match aggregations , grouping , or ordering logic from the simplified query to the original schema

  10. [21]

    < examples > ### Key Considerations : - ** Use the Question as a Guide **: - Align the query logic with the intent expressed in the question

    ** Validate the Final Query Against the Question **: - Review the final query to ensure it satisfies the question and produces the intended result . < examples > ### Key Considerations : - ** Use the Question as a Guide **: - Align the query logic with the intent expressed in ...

  11. [2017]

    Which customers have an account balance greater than 10 ,000?

    Seq2sql: Generating structured queries from natural language using reinforcement learning. Preprint, arXiv:1709.00103. 11 A Appendix: LLM Prompts and Examples A.1 Prompt for UNJOIN SP Listing 1: SQL Query Generation Prompt for UNJOIN SP You are an expert at semantic parsing . ...

    arXiv

  12. [2023]

    The VLDB Journal, 32(4):905–936

    A survey on deep learning approaches for text- to-SQL. The VLDB Journal, 32(4):905–936. Dongjun Lee, Choongwon Park, Jaehyuk Kim, and Heesoo Park. 2025. MCS-SQL: Leveraging mul- tiple prompts and multiple-choice selection for text- to-SQL generation. In Proceedings of the 31st...

    2025 arXiv

  13. [2025]

    Preprint, arXiv:2501.12948

    Deepseek-r1: Incentivizing reasoning capa- bility in llms via reinforcement learning. Preprint, arXiv:2501.12948. Xuemei Dong, Chao Zhang, Yuhang Ge, Yuren Mao, Yunjun Gao, lu Chen, Jinshu Lin, and Dongfang Lou. 2023. C3: Zero-shot text-to-sql with chatgpt. Preprint, arXiv:230...

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.