{"id":"d70b55fe-7184-4b5e-aea0-a7283d3bfdca","arxiv_id":"2505.21860","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Adding relative indexes, reusable and optional rule parts, bidirectional search, and simplicity tie-breaking raises the coverage of the best discovered transformation by up to about 10% on the authors' benchmarks.","lead":"This paper presents GXJoin, a method that learns simple text-transformation rules for joining tables that describe the same entities with different formatting. It shows that four generalization tricks make the rules cover more rows and stay easier for people to understand.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Recurrence-generalization gains may be an artifact of set-valued coverage: unit removal/repetition makes transformations output multiple candidates, and coverage counts a row if any candidate matches, so the reported coverage and rule-count improvements may overstate generality.","rationale":"The paper's central contribution is the claim that four generalization operators (relative indices, recurrence, bidirectionality, simplicity) improve the generality and explainability of transformations. The reader's weakest assumption was about simplicity's correlation with future coverage; however, the paper does provide a direct, if small, empirical test of that assumption in Table 4a versus Table 4b, where simplicity-based tie-breaking improves unseen coverage for both baseline and GXJoin. The more serious difficulty is that the coverage metric used to support the recurrence-generalization contribution is not a faithful measure of generality: Definitions 1–2 in Section 4.2 make units set-valued, Section 3.2 counts a row as covered if the target appears anywhere in the output set, and the baseline CST is deterministic. This asymmetric leniency can inflate the coverage and rule-count gains in Tables 1–3 as well as the unseen-data results if recurrence is enabled there. The paper's own end-to-end experiment appears to acknowledge this by setting repetition degree to one in Section 5.3, but it does not explicitly disable unit removal, and no ablation separates the effect of output-set size from genuine pattern generalization. This is a concrete, testable weakness rather than a disagreement with the community's consensus. The incremental columns in Table 1 show relative indices carry most of the gain, so the core idea is not necessarily wrong. The recommended resolution is to keep the CONDITIONAL verdict but add a mandatory ablation with single-output semantics and output-set-size statistics.","tokens_in":11050,"tokens_out":10559,"duration_ms":112959,"concrete_test":"Run a deterministic single-output variant: set the repetition factor r=1 and expand every optional unit U? into two separate transformations (one containing U, one omitting it), then recompute Tables 1a/b, 2, 3, and 4. If the +Rem./+Rep. columns in Tables 1 and 3 and the 'General Trans.' column in Table 4 lose most of their advantage over the Rel.-only column or over Baseline, the recurrence-generalization contribution is an artifact of the set-valued coverage metric. Also compute the mean output-set size per transformation for GXJoin versus CST and report precision-at-1 on the end-to-end join with and without optional units active.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the four generalization operators make transformations genuinely more general, as shown by coverage, rule-count, and unseen-data metrics. The most load-bearing problem is that the coverage definition in Section 3.2 — 'when a transformation can produce more than one output, coverage is the fraction of pairs for which the output set includes the target' — makes the evaluation asymmetric. Recurrence generalization (Section 4.2, Definitions 1–2) deliberately creates set-valued units: split(c,i)^r = {split(c,i+j)} and split(c,i)? = split(c,i) | ∅. A transformation containing such units can cover more rows simply by emitting more candidates, with no increase in semantic specificity. This affects Tables 1 (columns +Rem./+Rep.), 2, and 3, and potentially the unseen-data experiment in Table 4 if recurrence is active there. Since the baseline CST is deterministic, the lenient metric can only favor GXJoin. The text says end-to-end join experiments set 'repetition degree to one, which ensures a single output' (Section 5.3), but it does not state explicitly that optional/removal units are disabled; if they remain, even the join F1 gains could be influenced by multi-candidate lookup. At minimum, the paper must quantify the output-set sizes and report coverage with a single-output restriction before the generalization claim can be attributed to the semantic regularity of the learned rules.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies the problem of discovering syntactic cell transformations that make two columns equi-joinable, and proposes four generalization techniques: relative indices, recurrence (unit repetition and unit removal), source-target direction selection, and simplicity-based tie-breaking. The approach is evaluated on the Web and Spreadsheet benchmarks against CST and Auto-join, with metrics including best transformation coverage, number of transformations required, coverage on held-out rows, and end-to-end join F1. The paper reports consistent improvements from relative indices and bidirectional direction selection, smaller gains from recurrence, and improved unseen-data coverage from simplicity tie-breaking.","tokens_in":11327,"tokens_out":4340,"duration_ms":44621,"significance":"If the reported results hold, the paper provides a practical extension of CST-style transformation discovery and is the first systematic study of generality principles for cell transformations. Strengths include a clear formalization of relative indices and recurrence, an explicit held-out row experiment, and public code and datasets. The main caveat is that the coverage metric in the recurrence experiments counts a row as covered when any output of a set-valued transformation matches, so the central generalization claim is only as strong as the single-output robustness analysis.","major_comments":[{"comment":"The coverage metric in Section 3.2 is set-inclusion: a pair is covered if the output set contains the target. Definitions 1 and 2 deliberately create set-valued units (split^r and unit?), and the baseline CST is deterministic. Under this asymmetric metric, the coverage improvements in Table 1a/b and Table 3, and the rule-count reductions in Table 2, can be inflated simply because a transformation emits more candidates. The paper must report the distribution of output-set sizes and repeat the coverage and rule-count experiments under a single-output restriction (for example, requiring that exactly one produced candidate equals the target, or disabling recurrence units entirely) before attributing these gains to the semantic generality of the learned rules.","section":"Section 3.2, Definitions 1-2, Tables 1-3"},{"comment":"The unseen-data experiment relies on the load-bearing premise that simpler transformations, when coverage on seen rows ties, generalize better to future rows. The paper offers no distributional argument or independent evidence for this premise beyond the same two benchmarks. Moreover, Table 4 averages over at least five random splits but reports no confidence intervals or significance tests, so the 0.108 gap at Web 10% could be within sampling variability. The authors should provide a comparison that removes simplicity tie-breaking while keeping the other generalization operators, and should report variance or paired tests for the 10/20/40 sampling conditions.","section":"Section 4.4 and Table 4"},{"comment":"The minimum support threshold of 0.05 is chosen after observing that it gives the best performance-recall trade-off in the end-to-end experiments. Since this threshold affects all Table 5 results, the paper should either specify a threshold-selection protocol that does not use the test data or show sensitivity of the F1 results across a range of support values. Otherwise the end-to-end improvements may partly reflect test-set-based tuning.","section":"Footnote 5 and Section 5.3"},{"comment":"The text states that setting the repetition degree to one 'ensures a single output for each transformation applied on an input,' but Definition 2 still allows unit removal (unit? = unit | ∅), which can produce two outputs for an input. If removal units are disabled in the end-to-end experiments, the paper must state this explicitly; if they remain active, the F1 gains in Table 5 may also be influenced by multi-candidate lookups. A strictly single-output configuration should be reported.","section":"Section 5.3"}],"minor_comments":[{"comment":"Definition 1 introduces the repetition factor r, but the relationship between r and the hyperparameter 'repetition degree' is not defined; the paper should specify whether the repetition degree is the maximum value of r or a bound on the total number of repetitions in a transformation.","section":"Section 4.2"},{"comment":"The denominator for 'Affected Trans.' is unclear: the percentages should state whether they are relative to all transformations, to transformations with coverage strictly between 0 and 1, or to the set of relative-indexed transformations only.","section":"Table 3"},{"comment":"The word 'joinablity' in the first sentence of the conclusion should be corrected to 'joinability'.","section":"Section 6"},{"comment":"The pipeline in Figure 2 shows 'Generalization' as a separate stage after transformation generation, but the text in Section 4 describes generalization applied both during and after generation; the figure and the surrounding explanation should be harmonized.","section":"Section 3.1 / Figure 2"},{"comment":"The sentence 'splitSubstr becomes substr when the splitting character does not exist in the input, hence substr is not considered' is confusing because substr appears in Definition 2; the paper should clarify the exact base unit set and the intended scope of the recurrence operations.","section":"Section 4.2"}],"recommendation":"major_revision","confidential_remarks":"The baseline CST is co-authored by two of the current authors, and the end-to-end experiments reuse CST's example-generation pipeline; this is not itself a validity problem, but the paper would benefit from a clearer statement of which components are inherited from the baseline and which are newly added. The threshold selection in footnote 5 is a more substantive concern and should be addressed in revision. I have no other concerns about novelty or scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a useful, systematic extension of CST for joinable transformations. The four generalization techniques — relative indices, unit repetition/removal, direction generalization, and simplicity-based tie-breaking — are clearly defined and individually evaluated. The headline claim that coverage improves and transformation counts drop is backed by consistent numbers across two benchmarks, and the unseen-data experiment is a genuine held-out evaluation, which is more than many papers in this area do. Credit where due: the authors are open about what each technique contributes and do not hide the fact that gains are modest on the cleaner spreadsheet dataset.\n\nThe main soft spot is the one the stress-test flags. Coverage is defined as the fraction of rows where the output set includes the target (Section 3.2). Repetition and removal (Definitions 1 and 2) deliberately make transformations set-valued: repetition emits a range of indexed outputs, and removal adds an empty alternative. So a transformation can cover more rows simply by emitting more candidates, with no increase in semantic specificity. The paper does not quantify output-set sizes or report coverage under a single-output restriction, and the claim that repetition degree one ensures a single output (Section 5.3) does not address removal, which still produces a set including the empty string. This means Tables 1–3 and possibly the unseen-data experiment may overstate the benefit of recurrence generalization. The authors need to either restrict coverage to single-output transformations or show that the gains persist when output-set size is held fixed.\n\nOther issues are more minor but worth fixing. The simplicity tie-breaker is a heuristic with no distributional argument; the unseen-data gains could be dataset-specific, though the comparison between random and simplicity tie-breaking is a good start. The 0.05 support threshold is reported as chosen based on observations; a sensitivity analysis would help. And the abstract says \"outperforms\" but Table 5a shows GXJoin's F1 on Spreadsheet (0.810) is slightly below CST (0.812) — that should be softened to \"on par.\"\n\nOverall, the ideas are clear, the evaluation is honest in most places, and the paper makes a real contribution to a niche but active subfield. The set-valued coverage issue is load-bearing for the recurrence claims but fixable with additional experiments. I would send this to a serious referee, with the request that they focus on the coverage definition and the recurrence-specific experiments.","headline":"A solid, honest extension of CST with four generalization techniques, but the recurrence gains may be inflated by set-valued coverage and need a tighter evaluation before the results can be trusted.","tokens_in":11843,"tokens_out":1843,"would_cite":true,"duration_ms":19907,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Four generalization principles make mismatched table columns joinable with fewer, simpler rules.","keywords":["explainable join","cell transformation","table join","program synthesis","generalization","data integration","string transformation","entity resolution"],"falsifier":"Take a held-out set of table pairs where the true formatting rule is deliberately the more complex one (for example, names with variable middle initials require an extra splitSubstr unit while a simpler first-initial rule coincidentally covers the training rows), run the method with simplicity-based tie-breaking, and compare coverage on unseen rows against a random tie-breaking variant; if the simpler rule does not systematically cover more held-out rows, the simplicity prior is falsified.","tokens_in":10834,"feed_emoji":"🔗","tokens_out":7067,"duration_ms":67177,"temperature":0.7,"pith_summary":"Two columns from different sources often describe the same entities in different formats, so an equality join fails even though the data are joinable after a syntactic rewrite. The paper tries to establish that such rewrites—cell transformations built from copy, substring, split, and literal units—can be made more general by four principles: relative indices that ignore input length, repetition and optional removal of units for recurring patterns, choosing the source and target direction that yields fewer rules, and preferring simpler rules when coverage ties. On two real-world benchmarks it reports that these generalizations raise best-transformation coverage from 0.577 to 0.679 on web tables and from 0.667 to 0.725 on spreadsheets, cut the number of transformations needed to cover all rows from 431 to 394 and from 970 to 876, and improve end-to-end join F1 from 0.713 to 0.777 on the web benchmark. If true, the practical payoff is that a data analyst can join and explain mismatched columns with fewer, more readable rules that also hold up on newly added rows.","feed_headline":"Generalized transformations raise join F1 from .713 to .777","feed_subtitle":"Length-invariant, repeatable, bidirectional rules cover more rows and unseen data, with fewer rules to audit.","key_machinery":"The machinery is a cell transformation: a sequence of string units—literal, substr, split, and splitSubstr—that maps a source cell to a target cell, with coverage defined as the fraction of source–target pairs the sequence maps correctly. Onto this the paper adds four generalization operators: relative indices anchored at the start or end of the input to make rules length-invariant; a repetition factor r that applies a split or splitSubstr unit to r consecutive tokens; an optional-removal marker ? that lets a unit produce no output; and a bidirectional search that chooses the mapping direction with fewer transformations. A simplicity score, defined as the number of units or parameters, serves as the final tie-breaker. These operators work on top of an existing search for covering transformations, generalizing each candidate rule before or during selection.","core_discovery":"The central claim is that generalization, not just coverage of the given examples, should drive the search for table-join transformations, and that four concrete generalizations achieve this. Relative indices replace absolute positions so a rule like \"take the first letter of the first token and the whole last token\" works whether or not a middle name sits between them. Unit repetition and unit removal let one pattern absorb variable-length sequences, such as any number of middle-name initials. Bidirectional search picks the mapping direction that yields fewer transformations on a sample, avoiding the failure mode where a longer but less informative column is wrongly treated as the source. Simplicity tie-breaking—preferring fewer units or parameters when coverage is equal—is offered as a way to avoid accidental patterns and to cover unseen rows. The reported experiments on web tables and spreadsheet tables show the generalized transformations matching or beating the state-of-the-art baseline on coverage, number of rules, held-out data, and end-to-end join quality.","pith_inferences":["The simplicity prior is untested against distributions where the true rule is the complex one; one could build a benchmark of formatting rules of deliberately varying complexity and measure held-out coverage to see when tie-breaking by simpler rules hurts.","The paper limits repetition to a factor of 2 and to single-unit or consecutive-unit sequences; extending recursion to nested delimiters or to repetition over arbitrary subsequences could cover hierarchical formats such as address-like or JSON-like cells.","Relative indices are anchored at the start and end of the whole cell; anchoring at token boundaries or punctuation could generalize further, especially for cells with mixed alphabets.","The two benchmarks differ sharply in noise level, so the method's advantage is clearest on messy web tables; a cross-domain stress test with more than two datasets would indicate how portable the generalizations are."],"forward_implications":["A single generalized rule can replace several single-row rules, so joins over heterogeneous rows (for example, names with and without middle initials) need fewer rules to audit.","Rules learned from as little as 10% of rows transfer to the remaining rows, and the reported held-out coverage gap over the baseline widens as the sample shrinks.","End-to-end unequal joins gain in recall and F1 on noisy web tables—reported F1 rises from 0.713 to 0.777—without changing the explainable unit-based format.","Simplicity tie-breaking cuts the total number of units by at least 5% and parameters by about 17%, which should make re-running transformations on dynamic tables cheaper.","Choosing mapping direction by which side yields fewer rules matters when the longer column is not the more informative one, a situation that is common in web data."],"supporting_citations":[{"why":"The baseline transformation method that GXJoin extends; supplies the search space, the unit set, and the automated example-generation pipeline used in the comparisons.","marker":"[18]"},{"why":"Auto-join, the earlier transformation-based join approach that provides the web dataset and one of the baselines, and a source of the unit definitions.","marker":"[30]"},{"why":"FlashFill, the example-driven string-transformation synthesizer whose benchmarks form part of the spreadsheet dataset and whose unit set motivates the machinery.","marker":"[8]"},{"why":"Follow-up FlashFill work that contributes additional spreadsheet transformation tasks used in the evaluation.","marker":"[9]"},{"why":"BlinkFill, which supplies the spreadsheet transformation benchmarks and a semi-supervised synthesis baseline for comparison.","marker":"[22]"},{"why":"The synthesis-competition results cited as the origin of the public spreadsheet transformation benchmark used in the experiments.","marker":"[1]"}],"fun_headline_variants":["Generalized join rules beat SOTA with fewer, simpler patterns","Explainable transformations boost join F1 and coverage","Length-invariant rules make table joins more general","Bidirectional search yields simpler, reusable join transforms","Generalization wins: fewer rules, higher join quality"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a simpler transformation—one with fewer units or parameters—is more likely to describe unseen rows when two transformations cover the same seen rows, and the paper offers no distributional argument or independent evidence for that link.","fun_headline_variants_meta":{"raw":{"variants":["Generalized join rules beat SOTA with fewer, simpler patterns","Explainable transformations boost join F1 and coverage","Length-invariant rules make table joins more general","Bidirectional search yields simpler, reusable join transforms","Generalization wins: fewer rules, higher join quality"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000137,"raw_usage":{"total_tokens":1121,"prompt_tokens":886,"completion_tokens":235,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":502,"completion_tokens_details":{"reasoning_tokens":160}},"tokens_in":502,"tokens_out":235,"duration_ms":3263,"temperature":1.0,"reasoning_tokens":160,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:20:39.340959+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a held-out set of table pairs where the true formatting rule is deliberately the more complex one (for example, names with variable middle initials require an extra splitSubstr unit while a simpler first-initial rule coincidentally covers the training rows), run the method with simplicity-based tie-breaking, and compare coverage on unseen rows against a random tie-breaking variant; if the simpler rule does not systematically cover more held-out rows, the simplicity prior is falsified.","supporting_citations":[{"cited_title":"In: 2022 IEEE 38th ICDE","cited_arxiv_id":null,"evidence_quote":"The baseline transformation method that GXJoin extends; supplies the search space, the unit set, and the automated example-generation pipeline used in the comparisons."},{"cited_title":"Proceedings of the VLDB Endowment10(10) (2017)","cited_arxiv_id":null,"evidence_quote":"Auto-join, the earlier transformation-based join approach that provides the web dataset and one of the baselines, and a source of the unit definitions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"FlashFill, the example-driven string-transformation synthesizer whose benchmarks form part of the spreadsheet dataset and whose unit set motivates the machinery."},{"cited_title":"Communications of the ACM55(8) (2012)","cited_arxiv_id":null,"evidence_quote":"Follow-up FlashFill work that contributes additional spreadsheet transformation tasks used in the evaluation."},{"cited_title":"Proceedings of the VLDB Endowment9(10) (2016)","cited_arxiv_id":null,"evidence_quote":"BlinkFill, which supplies the spreadsheet transformation benchmarks and a semi-supervised synthesis baseline for comparison."},{"cited_title":"SyGuS-Comp 2016: Results and Analysis","cited_arxiv_id":"1611.07627","evidence_quote":"The synthesis-competition results cited as the origin of the public spreadsheet transformation benchmark used in the experiments."}],"review_version":1}