{"id":"0b1518d8-10e8-4d5b-842a-f367f9686ecb","arxiv_id":"1909.01120","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"EmbDI learns local embeddings for relational data from random walks on a tripartite graph, yielding better schema matching and entity resolution than pre-trained embeddings.","lead":"This paper presents EmbDI, a system that learns custom embeddings from relational data by converting tables into graphs and taking random walks through them. The embeddings improve schema matching and entity resolution compared with pre-trained and standard graph representations.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Universal 'local beats pre-trained' claim rests on unreplicated F-measure runs; on IA and FZ margins are 0.01 and 0.00, so seed noise could flip the ordering.","rationale":"I read the paper in good faith and find the core construction plausible: a tripartite graph with token/RID/CID nodes, random walks, and skip-gram training is a reasonable way to obtain local relational embeddings, and the experiments cover multiple tasks and datasets. The reader's weakest_assumption about co-occurrence in random walks being a faithful proxy is a valid concern, but the paper already provides some indirect support for it, e.g., token matching results where EmbDI beats Jaccard similarity. The more immediate threat to the central claim is evidential rather than conceptual: the claim that local embeddings outperform pre-trained ones 'in every case' is backed by single unreplicated F-measure values, some of which are separated by only one percentage point, and one of the reported tokenization variants is actually worse than fastText in two datasets. Because the paper does not report variance or repeated trials, we cannot distinguish a real universal advantage from seed-to-seed noise. This does not invalidate the paper; it means the strong universal formulation is not yet established. The proposed repeated-run check is feasible since the paper references an implementation, and it would settle whether the small margins on the closest datasets are robust. If the check confirms stable positive differences, the central claim stands; if not, the verdict should remain conditional on a more qualified claim such as 'usually better' or 'better for the EmbDI-O variant on most datasets'.","tokens_in":19312,"tokens_out":14418,"duration_ms":135899,"concrete_test":"Repeat the full unsupervised ER evaluation (Algorithm 6 with nt_op=10) on all eight datasets using 20 independent random seeds for word2vec skip-gram training and random walk generation, for both the EmbDI-O configuration and the fastText baseline, and report the mean F-measure with a 95% bootstrap confidence interval for the paired difference (EmbDI-O minus fastText). If the confidence interval for the difference includes zero on IA or FZ, then the paper's universal 'outperform pre-trained' claim is not supported by the current evidence.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central empirical claim is stated as a universal: 'In every case, local embeddings obtained from our graph outperform pre-trained ones' (Section 7.2). The evidence for this is a single run per configuration, with no variance or significance information. In Table 4 the decisive comparisons are extremely close: for IA, unsupervised ER F-measure is 0.10 for fastText and 0.11 for EmbDI-O; for FZ both are 0.99. Word2vec skip-gram training and random walk generation are stochastic, so a 0.01 F-measure gap is within the range of seed-to-seed variability. The claim is also sensitive to tokenization: EmbDI-S scores below fastText on AG (0.14 vs 0.18) and IA (0.09 vs 0.10), so the advantage exists only for the EmbDI-O variant. Without repeated trials, the headline result that local embeddings beat pre-trained ones in every case is not established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes EmbDI, a framework for learning local embeddings of relational data for data integration tasks. It represents two relational datasets as a heterogeneous tripartite graph with token, record-id, and column-id nodes, generates a sentence corpus via uniform random walks on this graph, and trains word2vec embeddings on the resulting corpus. The method is evaluated on eight benchmark datasets for schema matching, entity resolution, and token matching, in both unsupervised and supervised settings, and compared against pre-trained fastText embeddings, graph-embedding baselines (Node2Vec, Harp), and existing integration systems (SeepP, DeepER). The central claim is that local embeddings from EmbDI outperform pre-trained embeddings for data integration on enterprise-style data with custom vocabulary.","tokens_in":19523,"tokens_out":3492,"duration_ms":37431,"significance":"If the empirical claims hold, the paper makes a useful contribution: it offers a practical way to obtain integration-specific embeddings without relying on pre-trained text models, which is attractive for enterprise datasets with specialized vocabularies. The tripartite graph design that treats record IDs and column IDs as first-class nodes is a reasonable and novel mechanism for capturing row- and column-level semantics. The paper includes a public code repository, an ablation study, and a diverse set of datasets, which strengthen reproducibility and the generalizability of the framework. The central 'local beats pre-trained' claim is plausible and relevant, but, as detailed below, the current evidence does not fully support the universal phrasing used in the text.","major_comments":[{"comment":"The claim 'In every case, local embeddings obtained from our graph outperform pre-trained ones' is not supported by the reported data. In the unsupervised ER results, for IA fastText attains F-measure 0.10 while EmbDI-S and EmbDI-F attain 0.09; for FZ, fastText attains 0.99 while EmbDI-S attains 0.98. The claim is only defensible if restricted to the EmbDI-O variant, but the sentence as written refers to local embeddings generally. Furthermore, the tables report a single F-measure per configuration, and both random-walk generation and word2vec training are stochastic; the 0.01 margins on IA and FZ are within the range of seed-to-seed variability. The authors should report repeated runs with variance or significance tests, or explicitly and carefully qualify the claim.","section":"Section 7.2, Table 4"},{"comment":"The experimental protocol uses a hand-picked default configuration (walk length 60, window size 3, 300 dimensions, corpus size formula (#dist.values + #rows) * 1000, n_top = 10) and then reports in the ablation that task-specific changes such as CBOW instead of skip-gram, or a walk length of 5 for SM, improve F-measure by several absolute points. This makes it unclear whether the reported advantage of EmbDI over pre-trained baselines reflects the method's intrinsic quality or favorable hyperparameter choices. The paper should provide a sensitivity analysis over a small grid of these parameters for the main integration tasks, or at least state explicitly whether the comparative conclusions in Tables 3 and 4 hold across the grid.","section":"Section 7.3"},{"comment":"The unsupervised ER and SM algorithms rely on mutual nearest neighbors with a fixed n_top, and Table 5 shows that F-measure varies noticeably with n_top (for example, on IA F-measure ranges from 0.068 at n_top=1 to 0.100 at n_top=100). The default n_top=10 is used without a principled justification, and for datasets such as IA the reported advantage over the fastText baseline is smaller than the variation induced by n_top. The authors should justify the choice of n_top and show that the qualitative comparison to baselines is stable for other values.","section":"Section 6, Algorithm 6 and Table 5"},{"comment":"The summary statement that 'local embeddings generated by EmbDI are superior to all other methods' is an overstatement of Table 2. EmbDI is not the best method on several entries, for example BB MR (0.50 vs. Node2Vec 0.66) and AG MC (1.00, tied with Harp and Node2Vec), and some baselines could not complete large datasets due to memory or time limits. The sentence should be revised to say that EmbDI is superior on average, and the incomplete baseline results should be taken into account when interpreting the comparison.","section":"Section 7.1, Table 2"}],"minor_comments":[{"comment":"There is an apparent typo in line 3: 'let T = C1 ∪ C 1' should presumably be 'C1 ∪ C2'.","section":"Algorithm 5"},{"comment":"In Algorithm 4, the symbol A is used for both the anchor set (U1 ∩ U2) and the matrix of embeddings of R1, which is confusing. The variable names should be made consistent.","section":"Section 5.4, Algorithm 4"},{"comment":"The sentence 'Hence, the data repairing task could be used to address both missing and noisy values' is a non-sequitur in context; the preceding paragraph describes using placeholders for missing values, but does not explain how this addresses noise. The wording should be clarified.","section":"Section 5.3"},{"comment":"The column header 'nt op' should either be written as 'n_top' in text style and with a proper subscript, or spelled out; the current formatting is difficult to read.","section":"Table 5"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely problem and the graph-based embedding framework is a sensible contribution. However, the central empirical claim is stated more strongly than the evidence supports: several tokenization variants underperform the pre-trained baseline on specific datasets, and the reported results lack variance information. The authors should be asked to either substantially temper the universal claim or provide repeated-trial results with significance tests. The other major issues, such as the sensitivity to hyperparameter choices and the n_top selection, are fixable within the scope of a revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: EmbDI is a solid, useful systems paper. The genuinely new idea is the tripartite graph with token, record-id, and column-id nodes, plus random-walk sentence construction that makes row and column ids first-class citizens in the embedding space. That is a real contribution, and the empirical work is broad enough to take seriously: eight integration scenarios, schema matching and entity resolution, supervised and unsupervised settings, plus ablations. The code is linked, though without a commit hash or shipped data.\n\nThe main result holds in a qualified form. The overlap-aware tokenization (EmbDI-O) beats or ties fastText in every dataset reported, and the gains on DBLP-Scholar and IMDB-Movielens are large. Supervised DeepER also improves by about five points when given local embeddings, which is a practical win.\n\nThe paper overstates the result. Section 7.2 says 'In every case, local embeddings obtained from our graph outperform pre-trained ones.' That is not what Table 4 shows: EmbDI-S is below fastText on AG and IA, EmbDI-F is below on DS, and on FZ EmbDI-O ties at 0.99. The experiments are single runs with no variance or significance information, and the IA gap is 0.01 (0.10 vs 0.11). Seed noise can flip that. So the universal claim is not established. The sensible claim—EmbDI-O local embeddings are competitive-to-better than fastText on these tasks, with strong gains on some—is supported.\n\nMinor issues: Algorithm 4 has malformed pseudocode (line 4 overwrites the anchor set, and line 6 is not a valid optimization statement). Algorithm 5 has a typo. The ablation is honest that tuning helps, but it also means the default configuration is not always the best.\n\nThis is a good engineering contribution with a useful representation, not a breakthrough. The evaluation is decent but needs repeated runs and significance testing before the headline should be universal. I would send it to peer review—it deserves careful referee time—and I would cite it for the tripartite graph and RID/CID embeddings, while avoiding the 'outperforms in every case' claim.","headline":"Useful systems paper whose tripartite graph with RID/CID nodes is a real contribution; the universal 'beats pre-trained' claim is overstated but the qualified result holds up.","tokens_in":20002,"tokens_out":2730,"would_cite":true,"duration_ms":25370,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Local embeddings learned from a tripartite graph of values, rows, and columns outperform pre-trained text embeddings for data integration.","keywords":["data integration","entity resolution","schema matching","local embeddings","graph embeddings","tripartite graph","random walks","relational data"],"falsifier":"Run EmbDI on two tables about the same entities with zero shared tokens—for instance, one table in English and one in French, or one using country codes and the other full country names—without external dictionaries or alignment; if the nearest-neighbor matches among RID and CID vectors do not rise well above chance, the co-occurrence-as-semantics assumption fails.","tokens_in":19150,"feed_emoji":"🔗","tokens_out":9214,"duration_ms":87628,"temperature":0.7,"pith_summary":"The paper sets out to show that relational databases can supply their own task-specific word vectors. It represents the combined tables as a tripartite graph with three kinds of nodes—cell values, row identifiers, and column identifiers—and generates a training corpus by running random walks over that graph, so that a single walk moves through values, tuples, and attributes. The claim is that a standard skip-gram embedding trained on these walks makes rows and columns first-class citizens of the vector space, and that the resulting local embeddings beat pre-trained text embeddings on schema matching and entity resolution in both supervised and unsupervised settings. A sympathetic reader would care because enterprise data frequently uses custom vocabulary that large-corpus models have never seen, and the paper's method needs no external corpus to produce integration signals.","feed_headline":"Local graph embeddings beat pre-trained vectors for data integration","feed_subtitle":"Walks over token, row, and column nodes learn relational semantics from your tables alone, no external corpus required.","key_machinery":"The load-bearing object is the tripartite graph: three node types—token nodes for cell values, RID nodes for tuples, and CID nodes for attributes—with edges from every value to its tuple and its column. Random walks over this graph are materialized as sentences, and those sentences are the corpus for a skip-gram embedding learner. The graph does the conceptual work: it makes row and column identities explicit in the vector space, allows rare values to appear often enough by giving every node a walk budget, and gives a natural place to inject external knowledge by merging nodes or replacing one token with another during sentence construction. The same structure also supports the paper's optimizations for imbalanced relations, missing values, numeric attributes, and multi-word cells.","core_discovery":"The central claim is that co-occurrence in random walks over a compact tripartite graph is enough to learn embeddings that carry the semantics needed for data integration. Because every cell value is connected to the row that contains it and to the column it belongs to, a walk can juxtapose values from the same tuple, values from the same attribute, or values connected only through the row and column identifiers. When those walks are used as sentences for a skip-gram model, the learned vectors put matching rows near each other and matching columns near each other, and the paper's experiments show that these vectors improve entity resolution and schema matching relative to pre-trained vectors, both when used directly with nearest-neighbor matching and when fed into supervised deep-learning matchers.","pith_inferences":["Because the graph makes rows, columns, and values live in one vector space, the same embeddings should supply features for single-table tasks such as duplicate detection and error detection; the paper evaluates only cross-table integration, so that use is my extrapolation.","A hybrid that starts from pre-trained vectors for generic terms and then refines them with local graph walks could inherit both wide lexical coverage and enterprise-specific vocabulary; the paper lists this combination as future work, not as a tested result.","The paper's sensitivity results suggest that the random-walk policy, not just the embedding learner, is the main tuning lever; learning transition probabilities from a handful of labeled matches could be a direct extension of the method.","Because the graph is built incrementally from tables, the framework could in principle be applied to streaming data by extending the graph and refreshing walks; the paper does not address incremental updates."],"forward_implications":["Deep-learning matchers for entity resolution can swap pre-trained input vectors for these local vectors and gain on average about 5 absolute points of F-measure when only 5% of matches are labeled.","Schema matching can run without attribute labels by measuring cosine distance between column-ID vectors, and an existing embedding-based schema matcher improves by an average of about 6 absolute points when given these embeddings instead of pre-trained ones.","Rare and custom tokens are not discarded: the walk budget guarantees each node appears as a walk start, which directly addresses the vocabulary gap of pre-trained models.","Datasets with long multi-word values benefit most from the 'overlap' tokenization, which keeps values shared across datasets as single nodes and splits the rest word by word; this variant wins most entity-resolution scenarios in the paper.","The embedding construction is light enough to run on a laptop for a million-row table, in settings where graph baselines end in memory errors or timeout."],"supporting_citations":[{"why":"Supplies the random-walk node-embedding approach and a baseline that EmbDI is compared against on the same graph.","marker":"[20]"},{"why":"Supplies a supervised entity-resolution method that consumes tuple embeddings; EmbDI is evaluated as a replacement for its pre-trained embeddings.","marker":"[14]"},{"why":"Supplies another deep entity-matching method and a contrasting strategy that treats tuples as sentences, which EmbDI claims to improve on.","marker":"[30]"},{"why":"Supplies the schema-matching baseline that uses embeddings on attribute labels; EmbDI compares against it with both pre-trained and local embeddings.","marker":"[16]"},{"why":"Supplies a higher-order graph-embedding baseline used in the quality and runtime comparisons.","marker":"[9]"},{"why":"Supplies the pre-trained fastText vectors used as the main pre-trained baseline throughout the experiments.","marker":"[4]"},{"why":"Supplies the orthogonal-Procrustes alignment procedure that EmbDI adapts to align embeddings trained on two relations.","marker":"[11]"},{"why":"Supplies benchmark datasets used for the entity-resolution and schema-matching experiments.","marker":"[12]"}],"fun_headline_variants":["Graph walks learn relational embeddings without external corpus","Local embeddings from graph walks improve data integration","Graph-based local embeddings beat pretrained vectors for integration","No Wikipedia needed: graph walks learn relational semantics","Relational embeddings via graph walks outperform pretrained models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that co-occurrence in graph walks is a faithful proxy for the semantic similarity that data integration needs; if two values, rows, or columns never share a graph neighborhood, their embeddings will be unrelated no matter how conceptually close they are.","fun_headline_variants_meta":{"raw":{"variants":["Graph walks learn relational embeddings without external corpus","Local embeddings from graph walks improve data integration","Graph-based local embeddings beat pretrained vectors for integration","No Wikipedia needed: graph walks learn relational semantics","Relational embeddings via graph walks outperform pretrained models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000407,"raw_usage":{"total_tokens":2101,"prompt_tokens":921,"completion_tokens":1180,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":537,"completion_tokens_details":{"reasoning_tokens":1109}},"tokens_in":537,"tokens_out":1180,"duration_ms":8814,"temperature":1.0,"reasoning_tokens":1109,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T05:25:58.791339+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run EmbDI on two tables about the same entities with zero shared tokens—for instance, one table in English and one in French, or one using country codes and the other full country names—without external dictionaries or alignment; if the nearest-neighbor matches among RID and CID vectors do not rise well above chance, the co-occurrence-as-semantics assumption fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the random-walk node-embedding approach and a baseline that EmbDI is compared against on the same graph."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies a supervised entity-resolution method that consumes tuple embeddings; EmbDI is evaluated as a replacement for its pre-trained embeddings."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies another deep entity-matching method and a contrasting strategy that treats tuples as sentences, which EmbDI claims to improve on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the schema-matching baseline that uses embeddings on attribute labels; EmbDI compares against it with both pre-trained and local embeddings."},{"cited_title":"HARP: Hierarchical Representation Learning for Networks","cited_arxiv_id":"1706.07845","evidence_quote":"Supplies a higher-order graph-embedding baseline used in the quality and runtime comparisons."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the pre-trained fastText vectors used as the main pre-trained baseline throughout the experiments."},{"cited_title":"C., AnHai Doan, Jeffrey F","cited_arxiv_id":null,"evidence_quote":"Supplies benchmark datasets used for the entity-resolution and schema-matching experiments."}],"review_version":1}