{"id":"1c92f2a1-66e4-4375-9c0f-2054219f2689","arxiv_id":"2411.18923","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A hand-designed SQL intermediate representation called EzSQL, combined with BART, improves SQL-to-text BLEU scores and boosts Text-to-SQL parser accuracy when used to synthesize pretraining data.","lead":"EzSQL rewrites SQL queries into a simpler, more natural-language-like form before feeding them into BART for text generation. The paper reports BLEU gains on WikiSQL and Spider and shows the same SQL-to-text model can generate pretraining data that improves Text-to-SQL parsers.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"EzSQL's set-operator rewrite in Algorithm 2 does not preserve query semantics when projections or aggregates differ, so the reported BLEU gains may reflect task simplification rather than improved faithful generation.","rationale":"The reader's weakest assumption is that the hand-crafted EzSQL rewrite rules preserve enough semantic content; my review sharpens this into a concrete, checkable failure mode in Algorithm 2. This is the most load-bearing concern because the entire method is the rewrite procedure: if the rewrite is frequently lossy, then the improved BLEU scores and the downstream parser gain do not establish that EzSQL helps generate faithful text. The concern is internal to the paper's stated goal of 'maintaining the context and semantic meaning' (Section 3.1), not a disagreement with external consensus. I do not think this warrants outright rejection, because many WikiSQL and Spider queries may involve only simple predicate-level set operations, and the empirical gains could still be genuine for those cases. However, the absence of an execution-equivalence check means the central claim remains conditional. The concrete test would settle the matter directly by measuring how often the transformation changes the query result. This supports the reader's CONDITIONAL verdict rather than altering it, so I recommend UNCHANGED.","tokens_in":10686,"tokens_out":5797,"duration_ms":53831,"concrete_test":"On the Spider development set, extract all queries containing UNION, INTERSECT, or EXCEPT. Implement Algorithms 1-5 exactly as specified, reconstruct an executable SQL statement corresponding to the EzSQL output of Algorithm 2 (using the first subquery's SELECT/FROM/GROUP BY structure and appending the OR/AND-joined predicate suffixes), and execute both the original and reconstructed queries against the corresponding database. Compute result-set equality with row multiplicity; if more than a small fraction (e.g., 5-10%) of transformed queries return different results, semantic preservation is disproven at the core.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim depends on EzSQL being a meaning-preserving simplification: Section 3.1 says the IR brings SQL 'as close as possible to natural language while maintaining the context and semantic meaning,' and Section 3.2 presents Algorithms 1-5 as the simplification procedure. The load-bearing step is Algorithm 2, which merges two SELECT subqueries after a UNION or INTERSECT by finding the first token where the subqueries differ, keeping the prefix of the first subquery, and inserting OR (for UNION) or AND (for INTERSECT) before the differing suffixes. That rewrite is semantically equivalent only when the subqueries are identical except for one predicate in the WHERE clause. It is not equivalent when the SELECT lists, aggregate functions (COUNT, SUM, AVG), GROUP BY/HAVING clauses, or duplicate semantics differ. For example, SELECT COUNT(*) FROM T WHERE A=1 UNION SELECT COUNT(*) FROM T WHERE B=2 returns two rows (two counts), whereas a predicate-level OR yields one count over the union of the two sets. EXCEPT is similarly not representable as a simple WHERE negation once aggregates or multiple result columns are present. The paper reports no execution check of EzSQL against the original SQL and no faithfulness evaluation of the generated text; BLEU rewards surface overlap, and the Sentence-BERT analysis in Section 4.1.2 is not a correctness oracle. If a substantial fraction of Spider/WikiSQL set-operation queries are rewritten lossily, the reported BLEU gains and the downstream RAT-SQL improvement could reflect easier, shorter inputs rather than better SQL-to-text translation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes EzSQL, a deterministic intermediate representation that rewrites SQL queries (removing set operators, JOINs, aliases, and replacing keywords with more natural-language-like phrasing) before feeding them to a BART sequence-to-sequence model for SQL-to-text generation. Experiments on WikiSQL and Spider report BLEU scores of 40.62 and 27.49, respectively, improving over previous BART-based baselines. The paper further uses EzSQL-based text generation to synthesize pretraining data for RAT-SQL text-to-SQL parsing and reports an improvement from 71.8 to 73.3 exact match on Spider (Table 2).","tokens_in":10972,"tokens_out":2813,"duration_ms":27139,"significance":"If the claims hold, EzSQL would be a simple, model-agnostic IR that improves SQL-to-text generation without introducing additional model parameters, and it would provide a useful data augmentation tool for text-to-SQL. The IR itself is a hand-designed rule set with no fitted numeric parameters, which is a strength: the reported gains are not artifacts of test-set parameter tuning. The paper also makes a falsifiable prediction about the benefit of the IR for pretraining data. However, the central validity depends on whether EzSQL preserves SQL semantics while simplifying, and the current evaluation does not establish this: there is no execution check of rewritten queries, no faithfulness evaluation of generated text, and no statistical significance testing. The load-bearing set-operator rewrite (Algorithm 2) is demonstrably non-equivalent for common query shapes, so the reported BLEU gains may partly reflect task simplification rather than improved faithful generation.","major_comments":[{"comment":"Algorithm 2 does not preserve query semantics in general. Merging two SELECT subqueries by finding the first differing token and inserting OR/AND is only equivalent when the subqueries differ in a single WHERE predicate. For example, SELECT COUNT(*) FROM T WHERE A=1 UNION SELECT COUNT(*) FROM T WHERE B=2 yields two rows (one count per condition), whereas the EzSQL merge produces a single COUNT over rows satisfying A=1 OR B=2. The same issue arises with INTERSECT when projections or aggregates differ, and EXCEPT is not representable as a simple WHERE negation in such cases. Since Section 3.1 explicitly states that EzSQL maintains the 'semantic meaning,' the paper must either restrict Algorithm 2 to provably equivalent cases, provide an execution-based validation of the rewrite on the actual benchmarks, or revise the claim that EzSQL is meaning-preserving. Without this, the BLEU improvements on set-operation queries are confounded.","section":"Section 3.2, Algorithm 2"},{"comment":"The evaluation reports single BLEU scores with no variance, confidence intervals, or significance tests. On WikiSQL, the gain over the BART baseline is 1.35 BLEU (39.27 vs. 40.62), which may be within run-to-run noise for a fine-tuned BART-large model. On Spider, the gain over the Shi et al. (2021) baseline is large in absolute terms, but that baseline is cited from prior work and not reproduced under identical conditions. The authors should run multiple seeds and report means and standard deviations, and they should reproduce the BART baseline in their own pipeline to ensure that the only difference is the input representation.","section":"Section 4.1.1, Table 1"},{"comment":"The error analysis with Sentence-BERT is anecdotal and does not constitute a faithfulness evaluation. BLEU measures surface overlap, and the Sentence-BERT comparison in Figure 7 only reports similarity scores between gold, generated, and corrected sentences for a handful of examples; it does not measure whether the generated text preserves the meaning of the SQL. The central claim that EzSQL 'brings it as close as possible to natural language while maintaining the context and semantic meaning' requires a direct faithfulness check, such as human evaluation or an execution-based consistency test (e.g., parsing generated questions back to SQL and comparing execution results). Without this, the reported BLEU gains cannot be attributed to improved semantic understanding.","section":"Section 4.1.2"},{"comment":"The text-to-SQL augmentation result is potentially confounded. The baseline RAT-SQL + Pre-Train (71.8) is taken from Wang et al. (2021), which uses a different data-generation pipeline. The 73.3 result with EzSQL-generated data is obtained with the same pretraining configuration but with newly generated questions; the improvement could come from the particular sampled SQL queries, the random seed, or other implementation details rather than from EzSQL specifically. The authors should reproduce the baseline pretraining with their own data-generation pipeline (e.g., using the original Wang et al. generator) and report variance across runs. The BERT-base comparison (69.6 vs. 65.4 and 68.2) is also based on a single run and lacks error bars.","section":"Section 4.2, Table 2"}],"minor_comments":[{"comment":"The pseudocode uses 'while' loops where conditional operations are intended, and the control flow is underspecified: Algorithm 2's 'while Nested query flag is equal to 1' would loop indefinitely if not for an implicit break. The authors should replace these with 'if' statements and clarify the termination conditions.","section":"Section 3.2, Algorithms 1–5"},{"comment":"Algorithm 4 replaces aliases with original names, but the subsequent Algorithm 5 deletes the 'as' keyword and the word after it. The description in the text (Section 3.2) says the original table name occurs before and after the 'as' keyword after Algorithm 4, which is confusing because Algorithm 4 replaces only alias occurrences, not the token immediately after 'as'. Please clarify the exact token-level transformations.","section":"Section 3.2, Algorithm 4 and Section 3.3"},{"comment":"The model name 'RY ANSQL' contains a spacing typo and should be 'RYANSQL'. Also, the table would benefit from a column indicating the base encoder (BERT-base vs. Electra-base) consistently, as the footnotes are easy to miss.","section":"Table 2"},{"comment":"The section on intermediate representation mentions 'Natural SQL' (Gan et al., 2021) but does not discuss how EzSQL differs from it beyond 'dispenses nested subqueries and set operators.' A more detailed comparison, particularly of the simplification rules and coverage, would help position the contribution.","section":"Section 2, Related Work"},{"comment":"The ablation figures report bar values but no error bars or statistical significance. The caption for Figure 9 labels the model 'RAT-SQL + Pre-Train with EzSQL' while the text and Table 2 use 'our model data'; please unify the terminology.","section":"Section 4.3, Figures 8 and 9"},{"comment":"The GPT-4 example in the conclusion (ORDER BY count(*) with default ascending sort) is presented as motivation for future work, but it is not part of the formal evaluation. The authors should either integrate such correctness cases into the experiments or clearly separate this discussion from the results.","section":"Section 5, Future Work"}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea is simple and potentially useful, but the evaluation does not currently support the strong claim that EzSQL preserves semantic meaning. The set-operator rewrite is the most serious technical concern; if the authors can show empirically that the number of affected queries is negligible or that the rewrite does not hurt faithfulness, the paper could be viable. The lack of significance testing and the reliance on a non-reproduced baseline are fixable but require additional experiments. I would also encourage the authors to make the EzSQL rewriting code public, since the rules are deterministic and machine-checkable; that would strengthen the reproducibility of the work."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"EzSQL is a deterministic intermediate representation for SQL-to-text that strips JOINs, set operators, and aliases before feeding queries into BART. It improves BLEU on WikiSQL (40.62 vs 39.27 for plain BART) and Spider (27.49 vs 19.34 for Shi et al.'s BART), and using it to synthesize pre-training data lifts RAT-SQL from 71.8 to 73.3 on Spider. The IR is hand-designed, not fitted to the test set, so the core idea is sound.\n\nWhat is new here is that EzSQL is the first IR aimed specifically at SQL-to-text, adapting ideas from text-to-SQL IRs for generation. That is a legitimate contribution. The downstream augmentation result is a useful demonstration that a better SQL-to-text model can feed a better parser.\n\nThe main soft spot is Algorithm 2. The paper merges two subqueries after UNION or INTERSECT by finding the first token where they differ and inserting OR or AND. That is semantically correct only when the subqueries are identical except for one predicate in the WHERE clause. With different SELECT lists, aggregates, GROUP BY/HAVING, or duplicate semantics, the rewrite changes the query's meaning. For example, SELECT COUNT(*) FROM T WHERE A=1 UNION SELECT COUNT(*) FROM T WHERE B=2 produces two counts, but the EzSQL version produces a single count over an OR predicate. The paper claims in Section 3.1 that EzSQL maintains semantic meaning, yet it provides no execution check and no faithfulness evaluation. If a substantial fraction of set-operation queries in WikiSQL or Spider is rewritten lossily, the BLEU gains could reflect easier, shorter inputs rather than better translation. The authors need to quantify the prevalence of such cases and verify equivalence, or restrict the rewrite to cases where equivalence provably holds.\n\nOther gaps: no variance or significance tests, so the word \"significant\" is unsupported. The Spider baseline is cited from prior work without re-running under identical conditions. No code or generated data is released. The error analysis uses Sentence-BERT similarity, which is not a correctness oracle, and the ablation is a bar chart without numeric values.\n\nNone of this is fatal. The core idea is plausible, the method is simple, and the improvements are consistent with the hypothesis that simplifying SQL helps a pre-trained encoder. But the evaluation needs to be tightened before the claims fully hold.\n\nThis paper is for researchers working on SQL-to-text or data augmentation for semantic parsing. It deserves a serious referee, but the current version needs revision. I would send it to peer review with a request for added rigor rather than desk-reject it.","headline":"A simple, deterministic SQL-to-text IR that improves BLEU and helps RAT-SQL augmentation; the set-operator rewrite needs a semantic equivalence check before the claims fully hold.","tokens_in":11500,"tokens_out":2838,"would_cite":false,"duration_ms":27704,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"EzSQL, a deterministic SQL intermediate representation that removes JOINs and replaces set operators with AND/OR conjunctions before BART, achieves state-of-the-art BLEU on WikiSQL and Spider and generates synthetic questions that improve…","keywords":["SQL-to-text generation","EzSQL","intermediate representation","BART","Text-to-SQL","data augmentation","WikiSQL","Spider"],"falsifier":"Run Algorithm 2 on a UNION query whose two branches reference different tables, such as SELECT name FROM employees WHERE dept='sales' UNION SELECT name FROM contractors WHERE status='active'. The first-different-word merge rewrites this into a single SELECT over the first table with WHERE dept='sales' OR status='active', which drops the second table and the union semantics entirely. If human judges then mark BART's generated description of the original query as unfaithful on a set of such queries, or if the BLEU on that subset falls below the raw-SQL baseline, the semantic-preservation premise of EzSQL fails.","tokens_in":10507,"feed_emoji":"🗄️","tokens_out":11338,"duration_ms":89116,"temperature":0.7,"pith_summary":"This paper tries to establish that the surface syntax of SQL—JOIN clauses, set operators, aliases, and keyword order—is what keeps pretrained language models from generating good natural-language descriptions. The authors propose EzSQL, a deterministic intermediate representation that rewrites a query into a simplified, more text-like form before it is fed to BART. On WikiSQL, EzSQL + BART reaches BLEU 40.62 versus 39.27 for BART on raw SQL, and on Spider it reaches 27.49 versus 19.34. The same model can generate synthetic question/SQL pairs that raise a RAT-SQL parser's exact-match accuracy on Spider from 71.8 to 73.3, which matters because text-to-SQL training data is expensive to annotate.","feed_headline":"EzSQL rewrite lifts BART SQL-to-text BLEU to 40.6","feed_subtitle":"On Spider, BLEU jumps from 19.3 to 27.5; the same synthetic data lifts RAT-SQL accuracy to 73.3.","key_machinery":"EzSQL is the central object: a six-step simplification that flattens set operators into AND/OR conjunctions, eliminates JOIN and nested subqueries, replaces aliases with original descriptive names, rephrases FROM and AS, aligns DISTINCT and COUNT usage with natural language, and shortens long sequences. The load-bearing mechanism is Algorithm 2's merge-by-first-difference rule, which collapses the split branches of a UNION or INTERSECT into a single SELECT with one conjunction. This rule is what makes the rewritten input line up with how people phrase questions, and it is also the main semantic risk: the merge can change query meaning when the branches are not truly disjunctive. The function of EzSQL is distributional—it narrows the gap between the token sequences the encoder sees and the natural-language tokens the decoder must emit, so a pretrained model can generate without graph or tree embeddings.","core_discovery":"The paper's central claim is that SQL's surface form, not the capacity of the generator, is the main obstacle to SQL-to-text: JOINs and set operators are rarely named in target questions, so they are noise for a pretrained encoder-decoder. EzSQL is a deterministic rewrite of the query that removes JOIN conditions, replaces UNION/INTERSECT/EXCEPT with OR/AND conjunctions, expands aliases into descriptive schema names, and rephrases keywords such as FROM and AS into language-like connectives. Because the rewrite is produced by hand-written rules, no new parameters are added, and the EzSQL string can be tokenized directly by BART. The paper reports BLEU 40.62 on WikiSQL and 27.49 on Spider for EzSQL + BART-large, against 39.27 and 19.34 for BART on raw SQL; when the same model generates questions for PCFG-sampled SQL queries, pre-training RAT-SQL on that data raises Spider exact-match accuracy from 71.8 to 73.3.","pith_inferences":["An untested boundary: because EzSQL removes JOIN clauses entirely, its benefit should shrink on queries whose gold descriptions explicitly mention the join relationship; stratifying BLEU by whether the question names two tables would put this to the test.","A cleaner isolate of the IR effect would hold the PCFG-sampled SQL fixed and swap only the question generator (EzSQL + BART vs raw-SQL + BART), keeping parser and pre-training budget identical.","The paper's closing GPT-4 example points to a concrete extension: adding explicit natural-language markers for implicit SQL defaults, such as 'ascending' for ORDER BY or 'unique' for DISTINCT, could reduce the model's reliance on world knowledge.","Because short single-table queries are already close to natural language, EzSQL's gains may concentrate on long, join-heavy queries; a length-stratified evaluation would show where the representation actually earns its keep."],"forward_implications":["On WikiSQL, EzSQL + BART-large reaches BLEU 40.62 against 39.27 for BART on raw SQL; on Spider, 27.49 against 19.34.","No graph or tree embeddings are needed: a plain pretrained sequence-to-sequence model fed the EzSQL string outperforms specialized encoders.","Pre-training RAT-SQL on questions generated by EzSQL + BART lifts Spider exact-match accuracy from 71.8 to 73.3 under the same PCFG sampling and parser configuration.","Because the rewrite rules are hand-written and parameter-free, EzSQL can be dropped into any existing seq2seq SQL-to-text system.","By shortening long queries, EzSQL also reduces the input sequence length, which can lower the compute cost of the transformer encoder."],"supporting_citations":[{"why":"Supplies BART, the pretrained encoder-decoder that EzSQL is designed to help; the model's text-infilling capability turns the simplified IR into a full narration.","marker":"Lewis et al. (2020)"},{"why":"Provides the WikiSQL SQL-to-text baselines (template, Seq2Seq, Tree2Seq, Graph2Seq) that EzSQL + BART must outperform.","marker":"Xu et al. (2018)"},{"why":"Sets the Spider BART baseline (BLEU 19.34) and the generation-augmented pre-training context for the task.","marker":"Shi et al. (2021)"},{"why":"Supplies the PCFG-based SQL sampler and the RAT-SQL pre-training baseline (71.8) that the data-augmentation experiment extends.","marker":"Wang et al. (2021)"},{"why":"Defines the RAT-SQL parser whose exact-match accuracy is improved by pre-training on EzSQL-generated data.","marker":"Wang et al. (2020)"},{"why":"Provides the WikiSQL dataset used for the SQL-to-text evaluation.","marker":"Zhong et al. (2017)"},{"why":"Provides the Spider dataset used for both SQL-to-text and parser augmentation evaluations.","marker":"Yu et al. (2018c)"},{"why":"Contributes a Natural SQL IR that removes nested subqueries and set operators, a direct source for EzSQL's simplification rules.","marker":"Gan et al. (2021)"},{"why":"Contributes an IR that removes FROM, JOIN ON, and GROUP BY clauses, another source for EzSQL's rewrites.","marker":"Guo et al. (2019a)"},{"why":"Contributes an IR representing SQL without FROM and JOIN ON, an earlier antecedent of EzSQL's simplification choices.","marker":"Yu et al. (2018a)"}],"fun_headline_variants":["EzSQL simplifies SQL to boost text generation BLEU to 40.6","EzSQL rewrite lifts BART SQL-to-text, benefits Text-to-SQL too","EzSQL rewrites queries to plain language boosting SQL-to-text BLEU","EzSQL turns SQL into natural text and lifts BLEU to 40.6"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The hand-crafted EzSQL rewrite rules preserve enough of a query's meaning—after deleting JOINs and replacing set operators with AND/OR—that a language model can still produce faithful descriptions of the original query.","fun_headline_variants_meta":{"raw":{"variants":["EzSQL simplifies SQL to boost text generation BLEU to 40.6","EzSQL rewrite lifts BART SQL-to-text, benefits Text-to-SQL too","EzSQL rewrites queries to plain language boosting SQL-to-text BLEU","EzSQL turns SQL into natural text and lifts BLEU to 40.6"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00074,"raw_usage":{"total_tokens":3312,"prompt_tokens":962,"completion_tokens":2350,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":578,"completion_tokens_details":{"reasoning_tokens":2263}},"tokens_in":578,"tokens_out":2350,"duration_ms":15668,"temperature":1.0,"reasoning_tokens":2263,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T10:44:00.804775+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run Algorithm 2 on a UNION query whose two branches reference different tables, such as SELECT name FROM employees WHERE dept='sales' UNION SELECT name FROM contractors WHERE status='active'. The first-different-word merge rewrites this into a single SELECT over the first table with WHERE dept='sales' OR status='active', which drops the second table and the union semantics entirely. If human judges then mark BART's generated description of the original query as unfaithful on a set of such queries, or if the BLEU on that subset falls below the raw-SQL baseline, the semantic-preservation premise of EzSQL fails.","supporting_citations":[{"cited_title":", author Ng, P","cited_arxiv_id":null,"evidence_quote":"Sets the Spider BART baseline (BLEU 19.34) and the generation-augmented pre-training context for the task."},{"cited_title":", author Yin, W","cited_arxiv_id":null,"evidence_quote":"Supplies the PCFG-based SQL sampler and the RAT-SQL pre-training baseline (71.8) that the data-augmentation experiment extends."},{"cited_title":", author Chen, X","cited_arxiv_id":null,"evidence_quote":"Contributes a Natural SQL IR that removes nested subqueries and set operators, a direct source for EzSQL's simplification rules."}],"review_version":1}