{"id":"25e0edcb-3375-412d-a0d6-687eb06c31cf","arxiv_id":"2505.18122","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Flattening a multi-table schema into one table before SQL generation, then reconstructing JOINs in a second pass, improves multi-table text-to-SQL accuracy on SPIDER and BIRD.","lead":"UNJOIN simplifies multi-table SQL generation by flattening all database tables into a single virtual table before asking an LLM to write a query, then translating that query back to the real schema. On SPIDER and BIRD, it beats standard prompting and matches or comes close to stronger in-context learning baselines without fine-tuning or data access.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Translation step must infer JOINs from column-name prefixes without documented foreign-key input; Appendix prompts supply only 'a set of multiple related tables,' leaving the method's core mechanism untested on opaque schemas.","rationale":"I read the paper as arguing that decoupling retrieval from SQL logic, by flattening all columns into a single virtual table and then translating back, improves multi-table text-to-SQL. The strongest internal evidence is the comparison between CoT-SS and UNJOIN: CoT-SS (generating final SQL on the simplified schema without translation) achieves poor EM (Table 9), while UNJOIN's two-stage process yields large EM gains. This demonstrates that the translation step is essential, not a minor add-on. Consequently, the success of the method hinges on the LLM's ability to reconstruct correct JOINs and UNIONs from the simplified query using only the original table and column names. The reader identified this exact assumption as weakest, and the manuscript does not document whether foreign-key constraints are included in the translation prompt; the appendix text suggests they are not. This is an internal gap between the method's claims of generalizability and its tested conditions, not merely a disagreement with external consensus. The concern is concrete and testable: the pseudorandom renaming experiment directly probes whether the translation relies on semantic name overlap or memorized benchmark schemas. If it does, the central claim of applicability to arbitrary databases is undermined. The paper otherwise provides useful ablations and a clear decomposition, and the reader's CONDITIONAL verdict already reflects this uncertainty. I therefore see no reason to change the verdict; the remaining task is to run the proposed test, which would settle whether the concern is decisive or refutable.","tokens_in":16966,"tokens_out":5121,"duration_ms":45049,"concrete_test":"Take the SPIDER dev multi-table subset used by the paper and generate a deterministic pseudorandom renaming of all table and column names (seeded, preserving join structure and data types), then run the UNJOIN MP pipeline with the renamed schema and compare EM against the original. If EM drops by more than ~5 points, the method relies on semantic/memorized names rather than on the simplified-schema representation; if EM is stable, the translation step is robust and the concern is refuted. As a secondary check, include explicit foreign-key constraints in the translation prompt on the original schema and compare; a large improvement would indicate undocumented reliance on FK information.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing claim is that LLMs can recover the relational structure of the original multi-table schema in the translation stage. The paper's own CoT-SS ablation (Table 9) shows that generating the final SQL directly on the simplified schema collapses EM to 11.68 (GPT-4o, SPIDER), so the translation step carries nearly all of the method's accuracy. Yet the translation prompt (Appendix A.1 Step 2; A.3) provides only 'An original schema: a set of multiple related tables,' with no explicit primary-key/foreign-key constraints or join-path hints, and Algorithm 1 constructs the simplified schema from table and column names alone. The LLM must therefore guess join keys from name overlap or semantic similarity. This works on SPIDER/BIRD because those benchmarks use descriptive, shared key names and are likely present in LLM pretraining, so name-based inference and memorization can mask the absence of explicit relational metadata. The paper nowhere reports whether foreign-key information is fed into the model, and it does not test on schemas with opaque or non-descriptive column names, despite claiming generalizability to 'databases of any size or content.' If the LLM cannot infer joins from names alone on a novel schema, the entire improvement over direct prompting disappears, because the ablation shows translation is what fixes the retrieval errors that CoT-style prompts make.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes UNJOIN, a two-stage prompt-only framework for multi-table text-to-SQL. Stage 1 deterministically flattens each multi-table schema into a single virtual table whose columns are named table.column. Stage 2 asks an LLM to first generate a SQL query over this simplified schema and then translate it back to the original schema by reconstructing joins, unions, and relational logic, followed by an edit-distance post-processing step that corrects table/column names. Two variants are studied: UNJOIN SP (single joint prompt) and UNJOIN MP (separate prompts). The method is evaluated on filtered multi-table subsets of SPIDER and BIRD against standard prompting, ICL, SFT, table-QA, and reasoning-LLM baselines, and in an open-book setting with ARM, JAR, DTR, and Contriever retrievers. The central claim is that decoupling schema-element retrieval from SQL-join construction improves accuracy, scalability, and generalizability without data access or fine-tuning.","tokens_in":17240,"tokens_out":5344,"duration_ms":41637,"significance":"If the claims hold, the paper offers a simple and useful prompt-only alternative to fine-tuned multi-table SQL systems: deterministic schema simplification plus LLM-based query translation. The strengths include a broad model and retriever evaluation, the open-book results showing consistent improvements over external retrievers, and the CoT-SS ablation that cleanly isolates the contribution of the translation stage. However, the empirical support is weaker than the abstract suggests: the strongest ICL baseline, RSL-SQL, outperforms both UNJOIN variants on BIRD, and the SPIDER differences are within a fraction of a point. The load-bearing translation mechanism is also not tested on schemas where join keys cannot be inferred from column names, which is a serious gap for the generalization claim.","major_comments":[{"comment":"The translation stage is load-bearing for the method's success. Table 9 shows that GPT-4o CoT-SS (direct SQL generation on the simplified schema) achieves only 11.68 EM on SPIDER, while UNJOIN SP reaches 76.13 EM, so nearly all of the accuracy comes from translating the simplified query back to the original schema. Yet the translation prompt in Appendix A.1 and A.3 supplies only 'An original schema: a set of multiple related tables' and gives no explicit primary-key/foreign-key constraints or join-path hints. The paper nowhere states whether foreign-key information is included in the model's input, and Algorithm 1 constructs the simplified schema from table and column names alone. The method therefore relies on the LLM inferring joins from name overlap or semantic similarity, which is plausible on SPIDER/BIRD but is exactly the assumption that needs testing for the claimed generalizability to 'databases of any size or content.' Please state explicitly what relational metadata is provided to the model, and add experiments with opaque or non-descriptive schema names; without them, the main mechanism of the approach is not empirically established.","section":"Section 2, Appendix A.1/A.3, Table 9"},{"comment":"The abstract's claim that 'UNJOIN matches or exceeds the state-of-the-art baselines' is not supported by Table 2. RSL-SQL achieves 90.80 QE / 54.30 EM on BIRD, outperforming both UNJOIN SP (88.55 / 51.74) and UNJOIN MP (89.75 / 50.36). On SPIDER, UNJOIN SP exceeds RSL-SQL by only 0.09 EM points (76.13 vs. 76.04). The claim should be restricted to the specific comparison settings and should acknowledge that RSL-SQL remains stronger on BIRD. This is not merely a wording issue; it changes what the reader can conclude from the paper's main quantitative statement.","section":"Abstract and Section 4.1, Table 2"},{"comment":"The paper filters both datasets to multi-table queries (443 SPIDER queries across 81 databases; 1095 BIRD queries across 77 databases), but it does not state whether the ICL baselines in Table 2, especially RSL-SQL, were re-run on the same filtered subsets or taken from their original publications on the full datasets. If the latter, the comparison is not apples-to-apples: UNJOIN is evaluated on a subset, while the baseline numbers may come from a different evaluation set. Please clarify, and ideally re-evaluate the baselines under identical filtering and prompting conditions, or clearly label transferred numbers as full-set results.","section":"Section 4, Datasets paragraph, and Table 2"},{"comment":"No variance or statistical significance information is reported anywhere, despite the stochastic nature of LLM decoding. Several headline comparisons are within a fraction of a point (e.g., 76.13 vs. 76.04, 76.00 vs. 76.04), and the open-book gains in Table 6 are reported as single runs. The paper should report multiple runs or seeds with means and standard deviations, or at least confidence intervals, for the central closed-book and open-book comparisons, so that the claimed improvements can be distinguished from noise.","section":"Tables 1, 2, and 6"}],"minor_comments":[{"comment":"There is a typo: 'approches' should be 'approaches'.","section":"Section 3"},{"comment":"'UNJOIN Varaints' should be 'UNJOIN Variants'; the spelling of 'variants' is also inconsistent elsewhere.","section":"Section 2"},{"comment":"The text refers to 'Appendix 4', but the appendices are lettered (A-F); this should be 'Appendix B' for the Schema Simplification algorithm and example.","section":"Section 2"},{"comment":"The table header says 'Table QA (non SQL-based) baselines EM' and reports only EM. Please clarify whether QE is omitted intentionally and whether these baselines were evaluated on the same filtered multi-table subset as UNJOIN.","section":"Table 3"},{"comment":"The ethics statement promises 'publicly available code, datasets, and relevant resources', but no repository link or resource URL appears in the manuscript.","section":"Ethics Statement"},{"comment":"Figure 3 would benefit from labeled axes, legend details, and error bars; as presented, the claimed robustness trend of UNJOIN with increasing table counts is hard to quantify.","section":"Figure 3"},{"comment":"The limitations section does not mention the method's dependence on descriptive schema names for join inference, even though this is the main failure mode identified by the experimental design and is central to the generalization claim.","section":"Limitations"}],"recommendation":"major_revision","confidential_remarks":"The use of ARM and JAR as retrievers is a potential overlap concern since both come from the same group as a co-author, but this is mitigated by the fact that Table 6 also shows consistent gains over external retrievers (Contriever and DTR). Still, the authors should disclose the relationship clearly. The more important issue for the editor is the gap between the abstract's broad 'state-of-the-art' claim and the actual numbers in Table 2; the paper should not be accepted without the authors either re-running the ICL baselines on the filtered subsets or substantially revising the claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know: UNJOIN is a clean, simple idea—flatten every table.column into one virtual table, generate SQL against that, then translate it back to real joins—and it delivers real but modest gains on SPIDER and BIRD. The two-stage combination is genuinely new, and the CoT-SS ablation (directly generating SQL on the simplified schema collapses EM to 11.68 on SPIDER with GPT-4o) is strong evidence that the translation step, not the flattening, carries the accuracy. Being prompt-only and data-free, the method is also attractive as a plug-in over retrievers; the open-book improvements (+14.9 to +19.6 EM over CoT across ARM, JAR, Contriever, DTR) are the most convincing numbers here.  \n\nWhat I'd push back on:  \n\nFirst, the abstract says 'matches or exceeds state-of-the-art baselines,' but RSL-SQL beats both UNJOIN variants on BIRD EM (54.3 vs. 51.7 and 50.4), and on SPIDER the best UNJOIN edge over RSL-SQL is 0.09 EM. That is competitive, not SOTA. The claim needs to be softened.  \n\nSecond, the stress-test concern is legitimate. The translation prompt hands the model 'a set of multiple related tables' with no explicit foreign-key constraints, and Algorithm 1 builds the simplified schema from names alone. So the LLM is reconstructing joins from column-name overlap and semantic similarity. That works for SPIDER and BIRD, whose schemas use descriptive shared key names, but there is no test on opaque or non-descriptive schemas, despite the paper's claim of generalizing to 'databases of any size or content.' Including foreign-key info in the translation prompt or testing on a harder schema set would address this directly.  \n\nThird, minor but real: there are no error bars, and the edit-distance post-processing is not ablated. The top SPIDER scores differ by tenths of a point; without variance estimates, the ranking between UNJOIN and RSL-SQL is not something to over-read.  \n\nThe paper does well on internal consistency, and the failure analysis (misaligned names, ambiguous queries) is honest. The method is simple enough that a serious referee can check the main claims without wondering about hidden hyperparameters.  \n\nWho it's for: text-to-SQL and table QA researchers who want a lightweight, no-training alternative to fine-tuned systems. It's not a breakthrough, but it's a solid, useful contribution. I'd send it to review, with the request that the abstract be toned down and the translation step's assumption about join inference be made explicit and tested.","headline":"Clean, prompt-only flatten-and-translate method with modest gains; the SOTA claim overreaches and the join-inference step is untested on opaque schemas.","tokens_in":17794,"tokens_out":4131,"would_cite":true,"duration_ms":30769,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"UNJOIN lifts multi-table SQL accuracy by flattening schemas","keywords":["text-to-SQL","multi-table databases","schema simplification","LLM prompting","query translation","schema retrieval","SPIDER benchmark","BIRD benchmark"],"falsifier":"Take a set of SPIDER-style databases, replace table and column names with random tokens while preserving the relation graph, run UNJOIN's translation stage, and compare exact match to the named-schema version. If exact match falls to the level of direct prompting, then the claimed improvement comes from name-based join inference rather than from decoupling retrieval and logic.","tokens_in":16763,"feed_emoji":"🗄️","tokens_out":6246,"duration_ms":46402,"temperature":0.7,"pith_summary":"UNJOIN is a prompt-only, two-stage framework for multi-table text-to-SQL translation. It first flattens every table in a database into a single virtual table by renaming each column as TableName.ColumnName, and asks an LLM to write SQL against this simplified schema. A second stage translates that intermediate query back to the original schema, reconstructing the necessary JOINs, UNIONs, and relational logic. Data-free and fine-tuning-free, it is evaluated on SPIDER and BIRD multi-table queries and reported to match or exceed the baselines it compares against, including prompting, in-context-learning, fine-tuned, and reasoning-model approaches. A sympathetic reader would take the central claim to be that decomposing retrieval from relational logic reduces compounding errors in multi-table SQL generation.","feed_headline":"UNJOIN lifts multi-table SQL accuracy by flattening schemas","feed_subtitle":"Rewrite each column as table.column, generate SQL on one virtual table, then reconstruct joins—no data access or fine-tuning.","key_machinery":"The central object is the simplified single-table schema: every column of every table is renamed as TableName.ColumnName and pooled under one virtual table named after the database. This makes the schema a list of unambiguous columns instead of several connected tables. The intermediate SQL query written against this virtual schema is the carrier of the argument: it lets the LLM perform retrieval and aggregation without choosing join paths. The translation stage then converts that query back to the original schema, using the table-name prefixes to infer where joins belong; an edit-distance post-processing step corrects invented or abbreviated table and column names without changing SQL logic. UNJOIN has two variants, a single-prompt joint version (SP) and a multi-prompt staged version (MP).","core_discovery":"The central claim is that multi-table text-to-SQL becomes more accurate when the task is split so that schema-element retrieval and SQL logic construction no longer compete in one step. UNJOIN's schema simplification merges all columns of a database into one flat schema with table-name prefixes; the LLM then generates an intermediate single-table query, and a later translation step maps it back to the original schema by reintroducing joins and set operations. On the multi-table subset of SPIDER, the multi-prompt variant UNJOIN MP reports exact-match accuracy 77.13 and execution accuracy 99.9 with Llama 3.3, against 75.28 and 98.87 for the strongest standard-prompt baseline; on BIRD it reports 56.35 exact match against 53.55 for chain-of-thought prompting. The paper argues that schema simplification alone improves retrieval precision and recall, but that the translation step is essential, since generating final SQL directly on the simplified schema without translation collapses exact-match accuracy.","pith_inferences":["The method's success likely depends on LLMs having memorized common join keys such as customer_id and account_id from training; on schemas with opaque or randomized column names the translation stage would have no names to lean on, which is a testable failure mode the paper does not address.","The framework could be extended to include explicit foreign-key metadata in the translation prompt without sacrificing the flat retrieval stage, which would isolate whether name-based inference is what actually carries the second step.","Because the simplification is deterministic and prompt-only, the same two-stage idea may transfer to other structured output tasks where retrieval and composition compete, such as multi-document extraction or graph query generation.","The paper's own failure analysis suggests column-name ambiguity in the question-to-schema mapping is a residual error source; future work could add a disambiguation step before translation rather than after generation."],"forward_implications":["On the paper's evidence, a frozen general-purpose LLM can reach or exceed fine-tuned SQL models on multi-table queries by changing only the prompt's schema representation.","Because the method uses only schema names, not row-level data, it can be applied to a new database without data access or training.","The translation step, not just flattened retrieval, is necessary; skipping it drastically lowers exact-match accuracy, so any follow-up should preserve that second stage.","In open-book settings, UNJOIN acts as a plug-in generator: over several retrievers it raises exact match by roughly 15 to 20 percentage points.","The reported gains concentrate in table and column selection, with column recall above 95 percent for GPT-4o on SPIDER under the single-prompt variant."],"supporting_citations":[{"why":"Supplies the SPIDER benchmark dataset and evaluation split used for the closed-book and open-book experiments.","marker":"Yu et al., 2018"},{"why":"Supplies the BIRD benchmark, the larger cross-domain dataset on which UNJOIN is evaluated for generalization.","marker":"Li et al., 2023b"},{"why":"Defines DIN-SQL, a decomposed in-context-learning baseline that UNJOIN compares against on both datasets.","marker":"Pourreza and Rafiei, 2023"},{"why":"Defines RSL-SQL, a schema-linking in-context baseline on SPIDER and BIRD used as a comparison point.","marker":"Cao et al., 2024"},{"why":"Defines JAR, a join-aware multi-table retriever used in the open-book setting where UNJOIN is applied after retrieval.","marker":"Chen et al., 2025b"},{"why":"Provides the few-shot chain-of-thought prompting baseline whose exact-match and execution scores anchor the closed-book comparison.","marker":"Wei et al., 2023"},{"why":"Supplies CodeS-7B, the supervised fine-tuning baseline that UNJOIN is compared against on both datasets.","marker":"Li et al., 2024"},{"why":"Supplies DTS-SQL, the decomposed small-LLM fine-tuning baseline used for the supervised fine-tuning comparison.","marker":"Pourreza and Rafiei, 2024"},{"why":"Provides the DeepSeek-R1 distilled reasoning models whose SPIDER and BIRD results are compared with UNJOIN.","marker":"DeepSeek-AI et al., 2025"},{"why":"Supplies the ARM retriever used in open-book experiments, where UNJOIN produces large exact-match gains.","marker":"Chen et al., 2025a"}],"fun_headline_variants":["UNJOIN flattens schemas to boost multi-table SQL","Simplify schema, then rebuild joins for better SQL","Two-step UNJOIN: flatten schema, then reconstruct joins","UNJOIN: separate retrieval from SQL logic for higher accuracy","UNJOIN: flattening schema improves multi-table text-to-SQL"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a language model can reconstruct the correct JOINs, UNIONs, and relational logic from table and column name prefixes alone, without explicit foreign-key constraints; if the schema names carry no join information, the translation step has nothing reliable to work with and the whole gain from simplification collapses.","fun_headline_variants_meta":{"raw":{"variants":["UNJOIN flattens schemas to boost multi-table SQL","Simplify schema, then rebuild joins for better SQL","Two-step UNJOIN: flatten schema, then reconstruct joins","UNJOIN: separate retrieval from SQL logic for higher accuracy","UNJOIN: flattening schema improves multi-table text-to-SQL"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000216,"raw_usage":{"total_tokens":1440,"prompt_tokens":963,"completion_tokens":477,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":579,"completion_tokens_details":{"reasoning_tokens":391}},"tokens_in":579,"tokens_out":477,"duration_ms":4741,"temperature":1.0,"reasoning_tokens":391,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:34:57.598186+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of SPIDER-style databases, replace table and column names with random tokens while preserving the relation graph, run UNJOIN's translation stage, and compare exact match to the named-schema version. If exact match falls to the level of direct prompting, then the claimed improvement comes from name-based join inference rather than from decoupling retrieval and logic.","supporting_citations":[],"review_version":1}