{"id":"0ea24b3f-1b15-4f0e-a36b-014f5005eea7","arxiv_id":"2502.08160","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Using the WikiDBs corpus, the paper shows that most realistic vertical federated learning scenarios involve fuzzy or no key alignment, a setting current algorithms rarely support.","lead":"This survey measures how often real-world databases can be paired for vertical federated learning and finds that only 0.2% of potential pairs have perfectly aligned records, while 70.9% have no exact matches. It proposes a taxonomy that separates precise, semi-precise, fuzzy, and latent VFL, and argues that research has focused on the rarest setting.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 3's 'precise' share is computed from any shared column and exact row identity, not from key-based VFL alignment; this likely undercounts precise VFL.","rationale":"The paper's contribution is an empirical demonstration that real-world VFL data are mostly not precisely aligned; Table 3 is the load-bearing evidence for the conclusion in Section 5 that VFL is 'not yet ready for widespread deployment.' I read Section 3.2 carefully: potential VFL pairs are defined via graph connectivity over shared columns, and record matched ratio is defined by exact identity of records between tables. Neither step distinguishes keys from attributes. Because VFL alignment is fundamentally about matching the same entities via keys (or fuzzy key similarity), measuring 'identically appear' over all columns conflates record linkage with attribute disagreement. This is the weakest point: the quantitative distribution could be an artifact of the measurement, not of real VFL practice. The reader's concern is representativeness of WikiDBs; my concern is complementary and more fundamental—even within WikiDBs, the computed percentages do not necessarily reflect key-based alignment. I therefore partially agree with the reader's weakest assumption. I would keep the CONDITIONAL verdict but tie it to a key-column recomputation; if the test leaves the percentages essentially unchanged, the paper's central claim would be substantially strengthened. I do not question the taxonomy or the qualitative review, which have independent value.","tokens_in":12831,"tokens_out":4197,"duration_ms":36626,"concrete_test":"Download WikiDBs; sample the same 1,000 databases and 10 seeds. For every connected pair, label columns as candidate keys by name patterns ('id', 'key', 'code') or by uniqueness ratio greater than 0.99 and non-null rate. Recompute the four ratios of Table 3 using only shared candidate-key columns to define alignment, treating a pair as precise when key intersection covers the primary party's records. Also run the same metric on two real datasets with known unique IDs (e.g., e-commerce orders and customer profiles joined on customer_id). If precise plus semi-precise remains near 3.7%, the concern does not materially change the verdict; if it rises substantially, the central quantitative claim is an artifact of the column-level definition.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central empirical claim—only 0.2% of potential VFL pairs are precise and 70.9% fuzzy—rests on how 'potential VFL pair' and 'record matched ratio' are operationalized in Section 3.2. Potential pairs are defined by graph connectivity from tables sharing at least one column, with no check that the shared column is a key or entity identifier; any coincidental column such as 'country' or 'year' can connect databases into the same component. Record matched ratio is defined as the fraction of records that 'identically appear' in the other table. If this means exact equality on all shared columns (the only reading supported by the text), then two tables that share a unique customer ID but have different attributes—exactly the situation VFL is designed for—will be classified as non-precise because the non-key attributes do not match. This systematically depresses the precise and semi-precise categories and inflates fuzzy/latent categories, independent of whether WikiDBs is representative. The quantitative headline therefore does not measure 'can parties be aligned?' but 'do arbitrary Wikidata tables exactly duplicate rows on common columns?' The paper provides no preprocessing code or schema metadata to verify the computation. Prior work (Wu et al. 2022a; Nock et al. 2021) supports the qualitative existence of fuzzy VFL, so the survey remains useful, but the 0.2%/70.9%/25.4% figures that drive the 'not ready' conclusion are not yet established for key-based VFL alignment.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript is a survey of vertical federated learning (VFL) that attempts to ground the research-to-deployment gap in an empirical analysis of real-world data distributions. Using the WikiDBs corpus, the authors treat each database as a VFL party and define potential VFL pairs as databases connected by at least one shared column. They report that 25.4% of potential pairs have no feature overlap (latent VFL), 70.9% have non-zero feature overlap but no exact record matches (fuzzy VFL), 3.5% are partially matchable (semi-precise VFL), and only 0.2% are fully matchable (precise VFL). On this basis, they propose a taxonomy of VFL algorithms along key alignment, feature balance, communication, and trustworthiness, and conclude that current VFL algorithms are not yet ready for widespread deployment. The survey also reviews existing algorithms and identifies open directions for fuzzy, latent, imbalanced, and trustworthy VFL.","tokens_in":13131,"tokens_out":3500,"duration_ms":31654,"significance":"If the reported distributional findings were reliable, the paper would make an important contribution: it would quantify a widely suspected mismatch between the precise-alignment assumption of most VFL algorithms and the messy alignment structure of real-world data. The taxonomy is sensible and the qualitative message that fuzzy and latent alignment scenarios are under-served is credible, supported by prior work such as Wu et al. (2022a) and Nock et al. (2021). The paper also usefully catalogs communication and trustworthiness dimensions. However, the headline quantitative claims—especially the 0.2% precise and 70.9% fuzzy figures—depend on operationalizations in Section 3.2 that are not adequately specified and that appear to measure something different from key-based VFL alignment. The paper does not provide code or schema-level details for the analysis, which limits reproducibility. The central qualitative conclusion may survive a corrected analysis, but the current quantitative backbone needs substantial revision.","major_comments":[{"comment":"The definition of record matched ratio is not operationalized precisely enough, and the phrase \"records in each table that identically appear in the other\" suggests exact equality on all shared columns. Under that reading, two databases that share a unique customer or patient identifier but have disjoint attribute sets—exactly the canonical VFL scenario—would be classified as non-precise because their non-key attributes do not match. This would systematically depress the precise and semi-precise categories and inflate the fuzzy and latent categories. The authors should specify whether matching requires equality on all shared columns, on a designated key column, or on some other criterion, and should report the sensitivity of Table 3 to that choice.","section":"Section 3.2, Record Matched Ratio"},{"comment":"Potential VFL pairs are defined by connected components in a graph where an edge exists between any two tables sharing at least one column. This treats coincidental shared column names such as \"year\" or \"country\" as evidence of joinability, and it treats transitive connectivity through intermediate tables as direct VFL compatibility. For two-party VFL, a path through other databases does not make two databases directly alignable, and a shared column that is not an entity identifier does not support record linkage. As a result, the 70.9% fuzzy and 0.2% precise figures are not estimates of key-alignment feasibility; they are estimates of exact-row duplication on arbitrarily shared columns. The authors should restrict the analysis to key-like columns or otherwise justify the connectivity definition.","section":"Sections 3.1 and 3.2, Potential VFL Pairs"},{"comment":"The central claim that only 0.2% of potential VFL pairs are precisely matchable is load-bearing for the paper's conclusion, but the manuscript provides no code, no schema metadata, no description of column-name normalization or type matching, and no per-pair statistics. The threshold separating \"partial match\" from \"full match\" is also not defined. Without these details, the aggregate percentages in Table 3 cannot be verified or reproduced, and the claim that existing VFL algorithms are inadequate for 'most real-world applications' is not established by this analysis. The authors should release the analysis pipeline or provide a detailed, reproducible specification of the matching procedure.","section":"Section 3.3, Table 3 and Finding 3"},{"comment":"Latent VFL is defined as a potential VFL pair with zero overlapping features, yet the pair is included only because it lies in the same connected component via other tables. Two databases with no direct shared columns are not alignable by record linkage and are not obviously usable for two-party VFL; calling them 'latent VFL' presupposes a correlation that the connectivity-based definition does not establish. This conflation of indirect graph connectivity with VFL applicability inflates the 25.4% latent category and weakens the taxonomy's empirical grounding. The authors should either redefine latent VFL using direct, non-key but distributional alignability or present it as a hypothetical scenario rather than an empirically measured category.","section":"Section 3.3, Finding 2 and Latent VFL definition"}],"minor_comments":[{"comment":"The heading \"Imbalance VFL\" should be \"Imbalanced VFL\" to match the surrounding terminology.","section":"Section 4.2"},{"comment":"The sampling description says 1,000 databases generate 1,000,000 pairs from their Cartesian square. This includes ordered pairs and self-pairs; the authors should clarify whether self-pairs were excluded and whether pairs are unordered, since these choices affect the reported statistics.","section":"Section 3.1"},{"comment":"The row labeled \"Ratio of ✓\" would be clearer as \"Proportion of algorithms supporting each category,\" especially since the percentages are not defined in the table caption.","section":"Table 4"},{"comment":"The reference to \"V ogelet al. 2024\" contains a spacing artifact and should be typeset as \"Vogel et al.\" throughout.","section":"References"},{"comment":"The taxonomy tree is visually dense and the placement of \"One-shot\" under the Key Alignment branch is confusing; the figure would benefit from separate subtrees with clearer labels for the communication dimension.","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The paper is a survey with an empirical component, and the empirical component drives the headline conclusion. The authors' own prior work is cited in support of the fuzzy/precise gap, but the external WikiDBs analysis is presented as independent evidence; the major comments concern the validity of that evidence. If the authors can recalibrate the quantitative claims or clearly reframe them as 'exact-duplicate analysis on arbitrary shared columns,' the survey portion could still be a useful contribution. I would not recommend rejection because the qualitative direction is well supported by the literature, but the current version overstates what the data analysis establishes."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe thing to know: the empirical spine of this paper—that only 0.2% of real-world VFL pairs can be precisely matched, and 70.9% are fuzzy—is not established by the measurement. The \"record matched ratio\" is computed as the fraction of records that \"identically appear\" in the other table. That is full-row equality, not key equality. Two tables that share a unique customer ID but store different attributes—the textbook VFL setup—will be scored as non-precise. The paper also defines potential VFL pairs as any databases sharing at least one column, so a column like \"country\" connects things that no one would actually join on. Both choices bias the headline numbers downward for precise/semi-precise and upward for fuzzy/latent.\n\nThat said, the paper is not without merit. The idea of quantifying VFL data distributions from a large relational corpus like WikiDBs is genuinely new. The four-way taxonomy (precise, semi-precise, fuzzy, latent) is a useful organizing frame, and the accompanying table mapping algorithms to these scenarios is a practical resource. The qualitative conclusion—most VFL algorithms assume clean key alignment while real data is messy—is consistent with earlier work from Nock et al. and Wu et al., so the survey portion stands on its own.\n\nThe soft spots beyond the measurement: the paper never defines the feature overlap ratio, only the balance ratio, and the 0.5 threshold for balanced/imbalanced is arbitrary. No code or schema preprocessing details are released, so the central analysis is not reproducible. That is a fixable problem, but as it stands the 0.2%/70.9%/25.4% numbers are more like an artifact of a proxy measure than a finding about VFL practice.\n\nWho is this for? VFL researchers and anyone designing benchmarks. The taxonomy and research agenda are worth engaging even if the specific figures don't survive contact with better measurement. It deserves a serious referee, not a desk reject, because the direction is important and the authors can reasonably be asked to re-run the analysis on key-based alignment and publish the scripts.\n\nMy recommendation: send it to review, but the referee should focus on the operationalization of \"precise.\" If the authors fix or re-scope the measurement, this becomes a solid survey-plus-empirics paper.","headline":"The paper's headline numbers on fuzzy/precise VFL rest on a row-equality match metric rather than key alignment, so they undercount precise scenarios; the qualitative gap and taxonomy remain useful.","tokens_in":13656,"tokens_out":4490,"would_cite":true,"duration_ms":48446,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Most real-world vertical federated learning collaborations involve fuzzy or latent data that current algorithms cannot handle.","keywords":["vertical federated learning","federated learning","record linkage","data distribution","key alignment","fuzzy VFL","latent VFL","WikiDBs"],"falsifier":"Perform the same pair analysis on a corpus of production enterprise databases that participate in actual data-sharing agreements, counting how many pairs can be joined on a unique identifier with complete record overlap; if a substantial fraction, for example over 10%, are precisely aligned, then the claim that precise VFL is nearly nonexistent in practice is falsified.","tokens_in":12665,"feed_emoji":"🧩","tokens_out":7846,"duration_ms":51101,"temperature":0.7,"pith_summary":"This paper attempts to establish a mismatch between vertical federated learning (VFL) research and real-world data. By treating each of 100,000 databases in the WikiDBs corpus as a potential VFL party and sampling one million database pairs, the authors measure how often parties share features and how precisely their records can be matched. They report that 70.9% of potential pairs have overlapping features but zero exact record matches (fuzzy VFL), 25.4% have no shared features at all (latent VFL), and only 0.2% achieve the full exact alignment that most VFL algorithms assume (precise VFL). A sympathetic reader would take this as evidence that current VFL research targets a rare ideal case, and that practical deployment requires algorithms for fuzzy and latent settings.","feed_headline":"Only 0.2% of real VFL data aligns precisely","feed_subtitle":"96% of real vertical-FL pairs are fuzzy or unlinkable, yet algorithms assume exact matches.","key_machinery":"The load-bearing object is the record matched ratio: the fraction of each table's records that identically appear in the other table under a potential VFL pair. This single ratio partitions real-world pairs into the four key-alignment classes, precise (full match), semi-precise (partial), fuzzy (overlap but no exact match), and latent (no shared features), and the taxonomy built from it is what makes the paper's claim measurable. The WikiDBs database graph supplies the sample of potential pairs: nodes are databases and edges join tables that share at least one column.","core_discovery":"The paper's central claim is that existing VFL algorithms are not ready for widespread real-world deployment because their core assumption, that parties can be aligned on a precise, complete set of shared records, is almost never met in practice. The empirical basis is an analysis of WikiDBs: a graph of 1,000 sampled databases has only 3 to 4 connected components, indicating abundant potential collaborations, but among potential VFL pairs only 0.2% have full record overlap, 3.5% partial overlap, 70.9% nonzero feature overlap with zero exact matches, and 25.4% no overlapping features. The paper proposes a data-oriented taxonomy spanning key alignment (precise, semi-precise, fuzzy, latent), feature balance (balanced vs skewed), communication, and trustworthiness, then reviews algorithms against it; the result is that fuzzy VFL is supported by only about 9% of surveyed methods and latent VFL by none. It concludes that fuzzy and latent VFL, heterogeneous data handling, and trustworthiness are the gaps that block real-world adoption.","pith_inferences":["If exact-match ratio were replaced with fuzzy similarity-based matching, some of the 70.9% fuzzy pairs would likely become semi-precise, so the precise 0.2% figure is not a fixed upper bound on usable collaborations.","The WikiDBs sample may overrepresent loosely structured, publicly derived tables; in sectors with mandatory unique identifiers such as banking, healthcare, or telecom, the share of precise or semi-precise pairs could be substantially higher.","Latent VFL is conceptually close to unsupervised domain adaptation and disentanglement; techniques from those areas could seed the first latent VFL algorithms, though this is an extension the paper does not make.","A practical implication the authors leave implicit is that the 0.2% precise case may still be the right first target for commercial VFL products, since that is where data governance and compliance are strongest, while fuzzy and latent methods need further research before productization."],"forward_implications":["Benchmarks and evaluations should include fuzzy, semi-precise, and latent splits, since precise-alone testing measures performance on the 0.2% case.","New VFL algorithms must handle key similarity rather than exact equality, using fuzzy matching or learned representations of keys, because 70.9% of pairs fall into the fuzzy category.","Latent VFL requires alignment methods that use distributional or relational structure instead of shared keys, an area with no existing VFL algorithm.","Imbalanced feature counts across parties are the norm, with 66.49% of pairs below the 0.5 balance ratio, so methods should be tested under skewed feature splits.","Deployment expectations should be reset: precise VFL applies mainly to collaborations with strong unique identifiers such as user IDs, not to general cross-organizational data."],"supporting_citations":[{"why":"Supplies the WikiDBs corpus of 100,000 databases and 1.6 million tables that grounds all four empirical findings.","marker":"[Vogel et al., 2024]"},{"why":"Earlier work showing real-world VFL records rarely align exactly; the paper cites it to support the rarity of precise matching.","marker":"[Wu et al., 2022a]"},{"why":"Analyzes how record linkage quality affects learning on feature-partitioned data, underpinning the claim that alignment assumptions matter.","marker":"[Nock et al., 2021]"},{"why":"The existing VFL survey whose pipeline definition and algorithm-centric taxonomy this paper extends with a data-oriented view.","marker":"[Liu et al., 2024a]"},{"why":"Defines federated learning and provides the baseline framing the paper builds on.","marker":"[McMahan et al., 2017]"},{"why":"Establishes the horizontal versus vertical FL distinction the entire survey relies on.","marker":"[Yang et al., 2019]"},{"why":"VertiBench benchmark that documents imbalanced feature distributions and motivates the feature-balance dimension of the taxonomy.","marker":"[Wu et al., 2024b]"}],"fun_headline_variants":["Real VFL data rarely aligns: only 0.2% exact","96% of real VFL pairs are fuzzy or unlinkable","VFL's real-world gap: algorithms assume exact alignment","Only 0.2% of VFL pairs fully overlap","Why VFL fails in practice: unlinkable data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The result hinges on treating WikiDBs databases as stand-ins for real VFL parties and column-sharing as potential collaboration; if WikiDBs is not representative of actual cross-organization collaborations, the 0.2%, 3.5%, 70.9%, and 25.4% figures will not carry over.","fun_headline_variants_meta":{"raw":{"variants":["Real VFL data rarely aligns: only 0.2% exact","96% of real VFL pairs are fuzzy or unlinkable","VFL's real-world gap: algorithms assume exact alignment","Only 0.2% of VFL pairs fully overlap","Why VFL fails in practice: unlinkable data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000652,"raw_usage":{"total_tokens":2966,"prompt_tokens":899,"completion_tokens":2067,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":515,"completion_tokens_details":{"reasoning_tokens":1980}},"tokens_in":515,"tokens_out":2067,"duration_ms":14924,"temperature":1.0,"reasoning_tokens":1980,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T10:11:44.674667+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Perform the same pair analysis on a corpus of production enterprise databases that participate in actual data-sharing agreements, counting how many pairs can be joined on a unique identifier with complete record overlap; if a substantial fraction, for example over 10%, are precisely aligned, then the claim that precise VFL is nearly nonexistent in practice is falsified.","supporting_citations":[{"cited_title":"The impact of record linkage on learning from feature parti- tioned data","cited_arxiv_id":null,"evidence_quote":"Analyzes how record linkage quality affects learning on feature-partitioned data, underpinning the claim that alignment assumptions matter."},{"cited_title":"Communication-efficient learning of deep networks from decentralized data","cited_arxiv_id":null,"evidence_quote":"Defines federated learning and provides the baseline framing the paper builds on."},{"cited_title":"Federated machine learning: Concept and applications","cited_arxiv_id":null,"evidence_quote":"Establishes the horizontal versus vertical FL distinction the entire survey relies on."}],"review_version":1}