{"id":"83e4f7cd-2476-4bdf-b059-e4311fa8e7e5","arxiv_id":"2606.07060","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Auto-Relate is a mine-then-verify system that discovers reliable functional relationships (arithmetic, string, and dependency) in tables via four reliability criteria and three statistical tests, reporting 0.87 average PR-AUC on a 58k-table benchmark.","lead":"Auto-Relate mines candidate functional relationships across table columns then verifies them with statistical tests for accuracy, atomicity, stability, and integrity. A smart generalist might read it to understand how to recover non-spurious column rules that aid data quality and table analysis.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Validity of the 6,414 ground-truth FRs extracted from 58,679 tables as correctly labeled and representative benchmark","rationale":"The reader's weakest assumption directly identifies the benchmark labeling step as load-bearing. The abstract-only review correctly flags unverifiability; even with full text the same extraction step remains the least secured link because performance numbers are downstream of it. No machine-checked proof or parameter-free derivation mitigates this. Verdict therefore stays UNVERDICTED.","tokens_in":1775,"tokens_out":343,"duration_ms":11495,"concrete_test":"Sample 100 tables from the 58,679; for each claimed ground-truth FR and a matched non-FR candidate, independently re-apply the four criteria definitions from §3 and the test procedures from §4; measure label agreement rate. If disagreement exceeds 15% on accuracy or stability, recompute PR-AUC on the disputed subset.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline performance claim (PR-AUC 0.87, 59% above baselines) rests on the benchmark being a reliable oracle for true vs. spurious FRs. The paper states it extracts 6,414 ground-truth FRs spanning arithmetic, string, and FD types, then evaluates via the three statistical tests. However, the extraction process itself must independently satisfy the four reliability criteria (accuracy, atomicity, stability, integrity) without circularity; any systematic bias in how ground truths were identified (e.g., via heuristics that overlap with the proposed tests) would inflate apparent gains. No external validation or inter-annotator agreement is referenced in the provided description.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper introduces functional relationships (FRs) as a unified concept subsuming arithmetic relationships, string transformations, and functional dependencies in tables. It defines four reliability criteria (accuracy, atomicity, stability, integrity) and presents Auto-Relate, a mine-then-verify framework that generates candidates and verifies them via Minimality, Perturbation, and Independence Tests, with optimizations for efficiency. A benchmark is constructed from 58,679 real-world tables yielding 6,414 ground-truth FRs; experiments against 18 baselines report average PR-AUC of 0.87 (59% higher than the best baseline).","tokens_in":1918,"tokens_out":470,"duration_ms":13806,"significance":"If the benchmark construction and labeling process can be shown to be independent and unbiased, the work would offer a practically useful advance in table understanding, data quality, and provenance by providing a statistically grounded method for reliable relationship discovery. The empirical scale of the benchmark and the three-test verification approach are positive elements.","major_comments":[{"comment":"Benchmark construction section: The central performance claim (PR-AUC 0.87, 59% above baselines) rests on the 6,414 ground-truth FRs being correctly labeled and representative. The manuscript provides no description of the extraction heuristics, independent validation, or inter-annotator agreement for these labels, nor any check that the labeling process does not overlap with the proposed statistical tests. This is load-bearing for the empirical results.","section":"Benchmark construction (likely §4 or §5)"},{"comment":"Evaluation section: No ablation study, error analysis, or per-test contribution breakdown is reported for the three statistical tests (Minimality, Perturbation, Independence). Without this, it is impossible to determine whether the reported gains are driven by the full framework or by one component, weakening the claim that the unified approach is necessary.","section":"Experiments / Evaluation"}],"minor_comments":[{"comment":"The abstract states results 'across all settings' but does not define the settings or reference the corresponding table/figure; this should be clarified for reproducibility.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for their constructive feedback and positive assessment of the work's significance. We address each major comment below and commit to revisions where appropriate.","responses":[{"response":"We agree that additional details on the benchmark construction are necessary to substantiate the empirical claims. In the revised manuscript, we will add a dedicated subsection detailing the extraction heuristics employed to generate the ground-truth FRs, the independent validation procedures used, any inter-annotator agreement statistics, and explicit verification that the labeling process does not rely on or overlap with the Minimality, Perturbation, or Independence Tests. This revision will address concerns about potential bias or circularity in the evaluation.","revision_made":"yes","referee_comment":"[Benchmark construction (likely §4 or §5)] Benchmark construction section: The central performance claim (PR-AUC 0.87, 59% above baselines) rests on the 6,414 ground-truth FRs being correctly labeled and representative. The manuscript provides no description of the extraction heuristics, independent validation, or inter-annotator agreement for these labels, nor any check that the labeling process does not overlap with the proposed statistical tests. This is load-bearing for the empirical results."},{"response":"We concur that an ablation analysis would strengthen the evaluation by isolating the contribution of each statistical test. We will revise the Experiments section to include: (i) an ablation study measuring PR-AUC when each test is removed individually, (ii) a breakdown of how many candidates are filtered by each test, and (iii) an error analysis on cases where the full framework succeeds or fails. These additions will clarify the necessity of the combined approach.","revision_made":"yes","referee_comment":"[Experiments / Evaluation] Evaluation section: No ablation study, error analysis, or per-test contribution breakdown is reported for the three statistical tests (Minimality, Perturbation, Independence). Without this, it is impossible to determine whether the reported gains are driven by the full framework or by one component, weakening the claim that the unified approach is necessary."}],"tokens_in":1489,"tokens_out":450,"duration_ms":26280,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The paper's actual contribution is treating arithmetic relations, string transformations, and functional dependencies as instances of one thing called functional relationships, then applying a mine-then-verify pipeline that checks accuracy first and then uses Minimality, Perturbation, and Independence tests for the other three criteria. That combination and the three named tests are not standard in prior work on any of the separate problems.\n\nIt does the practical work of adding three optimizations (group-by bound, closed-form arithmetic speedup, binomial early stop) and running the whole thing on a benchmark built from 58k real tables. The reported average PR-AUC of 0.87 against 18 baselines is the kind of number that gets attention in data-management venues.\n\nThe soft spot is exactly the one the stress-test note flags. The performance claim rests on those 6,414 extracted ground truths being correctly labeled and independent of the proposed tests. The abstract gives no extraction procedure, no inter-annotator numbers, and no check that the labeling heuristics do not overlap with the Minimality/Perturbation/Independence criteria. If the labels were produced by methods close to the tests, the 59% lift is inflated. The lack of ablations or error analysis in the provided description makes it harder to judge what actually drives the gains.\n\nThis is for people who build table-understanding or data-cleaning tools inside spreadsheets and databases. A reader already working on relationship discovery or data quality will find the unified framing and the scale useful even if they end up re-implementing parts.\n\nIt should go to peer review. The empirical scope is large enough and the unification is concrete enough that referees can give targeted feedback on the benchmark construction and the test details.","headline":"Auto-Relate unifies three relationship types under four reliability criteria and three tests, but the 6,414 ground-truth labels from 58k tables are the unverified load-bearing piece.","tokens_in":2405,"tokens_out":432,"would_cite":false,"duration_ms":12071,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Auto-Relate recovers reliable functional relationships in tables by mining candidates then verifying them with three statistical tests.","keywords":["functional relationships","table mining","statistical tests","functional dependencies","data discovery","benchmark construction","relationship reliability","mine-then-verify"],"falsifier":"A new collection of tables in which the functional relationships labeled as ground truth by the benchmark construction process are shown by manual inspection to be mostly spurious, or in which Auto-Relate's PR-AUC falls substantially below the reported 0.87 value.","tokens_in":2681,"feed_emoji":"📊","tokens_out":745,"duration_ms":19390,"temperature":0.7,"pith_summary":"The paper defines functional relationships as a single category that includes arithmetic operations, string transformations, and functional dependencies between table columns. It specifies four reliability criteria and builds Auto-Relate as a two-stage system that first produces accurate candidate relationships and then applies targeted statistical tests to remove those that fail atomicity, stability, or integrity checks. Recovering only the reliable ones matters because most apparent column links in real tables arise from coincidence or limited data variety and therefore mislead downstream analysis. The authors release a benchmark of 6,414 verified relationships drawn from 58,679 spreadsheets and database tables and show that the new method outperforms eighteen prior techniques by a substantial margin on precision-recall curves.","feed_headline":"Method recovers reliable table column links 59% better than baselines","feed_subtitle":"Auto-Relate mines candidates then applies three statistical tests to filter spurious relationships across 58k real tables.","key_machinery":"The mine-then-verify framework that first produces candidate functional relationships and then filters them with a Minimality Test for atomicity, a Perturbation Test for stability, and an Independence Test for integrity.","core_discovery":"Auto-Relate is a mine-then-verify framework that first generates candidate functional relationships across arithmetic, string, and dependency types and then verifies the remaining reliability criteria through a Minimality Test, a Perturbation Test, and an Independence Test. Three efficiency optimizations further prune the search space. When evaluated on a benchmark constructed from 58,679 real-world tables that contains 6,414 ground-truth functional relationships, the system records an average PR-AUC of 0.87 and exceeds the strongest competing baseline by 59 percent across all tested settings.","pith_inferences":["The same verification tests could be applied to relationships discovered across multiple related tables rather than within a single table.","Replacing the current candidate generator with a learned model might further raise recall while still relying on the statistical tests for precision.","The stability and independence tests might also flag relationships that break under distribution shift when tables are updated over time."],"forward_implications":["Recovered functional relationships can directly support table understanding, data-quality repair, and provenance tracking without manual inspection.","The four reliability criteria give a uniform way to judge any inter-column relationship regardless of whether it is arithmetic, string-based, or a dependency.","The three statistical tests and the associated early-rejection bounds allow the approach to scale to tables with tens of thousands of rows.","The released benchmark supplies a common testbed that future methods can use to measure progress on reliable relationship discovery."],"fun_headline_variants":["Auto-Relate mines then verifies reliable table column links","Statistical tests confirm functional relationships across 58k tables","Auto-Relate beats baselines with 0.87 PR-AUC on 6414 FRs","Mine-then-verify recovers reliable inter-column table relationships"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The 6,414 ground-truth functional relationships extracted from the 58,679 real-world tables are correctly labeled and representative enough that the statistical tests can separate them from spurious candidates.","fun_headline_variants_meta":{"raw":{"variants":["Auto-Relate mines then verifies reliable table column links","Statistical tests confirm functional relationships across 58k tables","Auto-Relate beats baselines with 0.87 PR-AUC on 6414 FRs","Mine-then-verify recovers reliable inter-column table relationships"]},"model":"grok-4.3","cost_usd":0.009062,"raw_usage":{"total_tokens":4113,"prompt_tokens":761,"num_sources_used":0,"completion_tokens":70,"cost_in_usd_ticks":90624500,"prompt_tokens_details":{"text_tokens":761,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":3282,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":761,"tokens_out":70,"duration_ms":24808,"temperature":1.0,"reasoning_tokens":3282,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-27T20:31:59.908997+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A new collection of tables in which the functional relationships labeled as ground truth by the benchmark construction process are shown by manual inspection to be mostly spurious, or in which Auto-Relate's PR-AUC falls substantially below the reported 0.87 value.","supporting_citations":[],"review_version":1}