{"id":"0c8bd0e6-83c5-482c-892f-64db164aa7d5","arxiv_id":"2501.00252","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Booster, a plug-in augmentation for temporal knowledge graph completion, adds pattern-scored missing facts as training data and fine-tunes on hard examples, yielding up to 8.7% MRR gains.","lead":"This paper introduces Booster, a data augmentation pipeline for temporal knowledge graphs that filters false negatives, scores missing facts by triangle patterns, and fine-tunes models on the hard examples it finds. It reports up to 8.7% MRR improvement on several TKGC benchmarks and reduced variance across timestamps and runs.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Booster's added 'false negative' positives are validated only by recall, never precision, and split leakage is not ruled out; the method may train on fabricated positives or test answers.","rationale":"The paper is a broad, well-intentioned empirical study with real supporting evidence: released code, five backbones, five datasets, ablations, and variance analyses. The concern is not that Booster is fake; it is that the empirical support for the key pseudo-positive mechanism is incomplete. The reader's weakest assumption identified the precision problem, and I agree: the recall experiment only checks that deleted observed facts are detected among candidates, not that the candidates chosen as pseudo-positives are actually true facts. The relative threshold in Section 4.3 makes this worse, because a candidate is labeled positive whenever it outscores its paired seed, regardless of absolute confidence. I also share the reader's leakage concern: the paper never states that filtering and scoring are restricted to the training split, and the method section repeatedly refers to the full graph G. If test or validation facts are visible during augmentation, the headline gains could be leakage artifacts. The smooth-label mechanism does not fix this; it merely downweights incorrect labels while still pushing their scores up. Because the reader already made the verdict conditional on addressing exactly these issues, my read does not change the verdict. The condition is a precision and split-isolation audit, not a rejection of the method's plausibility.","tokens_in":23150,"tokens_out":5705,"duration_ms":62350,"concrete_test":"Using the released code, recompute the pipeline on ICEWS14 (and ICEWS05-15) with all candidate generation and scoring in Sections 4.2-4.3 strictly restricted to the training split. Then compare every added pseudo-positive against (i) the validation/test facts and (ii) a held-out set of 20% randomly removed training facts, and report precision, not just recall. If precision is low, or if the MRR gain over the no-DA baseline disappears or shrinks materially when scoring is split-restricted, the leakage/noise concern lands.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central mechanism of Booster is converting potential false negatives into positive training samples (Section 4.4, Eq. 12). The only validation offered for this labeling is a recall experiment on randomly deleted facts (Section 4.2, Figure 4), and the text misreports even that figure. Recall measures how many deleted observed facts are among the candidates; it says nothing about precision of the final pseudo-positive set. Section 4.3 labels every candidate f' as real if its hierarchical score exceeds the score of its seed fact (Eq. 10 and surrounding text); with noisy local structures this relative threshold can admit near-arbitrary candidates, and the perturbation/smooth-label step in Eq. 12 only downweights, never removes, wrong labels. The paper also never states whether filtering and scoring are restricted to the training split; if the full graph G including validation/test is used to generate pseudo-positives, the reported 8.7% MRR gains could come from directly training on test answers. Without a precision measurement on naturally missing facts, neither the enrichment story nor the magnitude of the improvements is supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Booster, a pattern-aware data augmentation framework for temporal knowledge graph completion (TKGC). Booster first filters potential false negatives using relation-, entity-, and time-based frequency strategies (Eqs. 2-4), then scores the candidates with a hierarchical triangle-counting algorithm on a time-irrelevant unified graph (Eqs. 5-10), and finally trains a given TKGC model in two stages: pre-training on the filtered graph and fine-tuning on identified false negatives, hard negatives, and model-specific hard samples (Eqs. 11-12). Experiments on five datasets with five backbone models report MRR improvements up to 8.7%, variance reductions, ablations, and comparisons against several graph and knowledge-graph augmentation baselines.","tokens_in":23291,"tokens_out":5372,"duration_ms":55482,"significance":"If the claims hold, Booster would be a useful plug-and-play augmentation for TKGC, and the paper has genuine strengths: a broad experimental matrix (five backbones, five datasets), ablation and variant studies, p-values on the main comparisons, and a public code repository. The preliminary analysis of model preferences in Section 3.4 is also an interesting observation. However, the central mechanism—labeling candidate facts as positives and training on them—is validated only by recall on randomly deleted facts and by the same heuristic's own scores. No precision measure is given for the pseudo-positive set, and the possibility of split leakage is not addressed. These issues are load-bearing because they directly affect the magnitude and direction of the reported gains.","major_comments":[{"comment":"The text states that \"more than 90% of removed facts can be detected,\" but Figure 4(a) labels 28.1% as \"Not detected\" for ICEWS14, implying a detection rate of about 71.9% on that dataset. The reported recall is therefore not uniformly above 90%. More importantly, recall on randomly deleted facts does not measure the precision of the final pseudo-positive set used in Eq. (12): a candidate can be detected by the filter and still be a genuine negative after scoring. Please report precision (or an estimated false-positive rate) for the pseudo-positives on naturally missing facts, using a held-out temporal split that is not used to construct the filters or triangle counts.","section":"Section 4.2, Figure 4"},{"comment":"A candidate is labeled a real false negative whenever m_f' > m_f, where both scores are produced by the same triangle-counting heuristic on the same graph. This relative threshold can admit candidates whenever the seed fact's local structure is noisy, and the perturbation/smooth-label step downweights but never removes low-confidence candidates. The paper should report how many pseudo-positives survive the threshold, what fraction are correct according to an independent signal (e.g., human annotation or facts from a future time interval not used in scoring), and how performance changes when only the highest-confidence positives are retained.","section":"Section 4.3, Eq. (10)"},{"comment":"It is never stated whether Booster's filtering and scoring operate on the training split only or on the full graph G. Since Section 5 splits each dataset into train/validation/test at 8:1:1, if validation/test facts are included in G when constructing R(r), N(e), the triangle counts, and the candidate sets, then pseudo-positives can include test answers, which would directly inflate the reported MRR gains. Please clarify this point and rerun the main tables with all augmentation computed from training facts only.","section":"Section 4.4 and Section 5"},{"comment":"The hyperparameters top-m, the sparsity threshold k, and the perturbation count k (Section 4.3) are never given values or a selection procedure; only L_r, L_e, and L_t are reported. These parameters directly control the size and composition of the pseudo-positive set and the candidate pool, so without them the experiments are not reproducible and the sensitivity of the main results to these choices is unknown.","section":"Section 4.2 and Section 5.1"}],"minor_comments":[{"comment":"The softmax in Eq. (9) is not fully specified: it should state over which set the softmax is computed and how the time-aware weights are normalized across the relation layer.","section":"Section 4.3, Eq. (9)"},{"comment":"The pie-chart legend is hard to parse; please give exact detection rates in the caption or a table, and correct the text to match Figure 4(a).","section":"Figure 4"},{"comment":"The ablation study reports only Hits@1 and Hits@10, not MRR; please either include MRR or explain why it is omitted.","section":"Table 3"},{"comment":"P-values are reported but the statistical test and number of independent runs are not described; please specify both.","section":"Section 5.1"},{"comment":"There are minor spacing inconsistencies such as \"TNTBooster\" versus \"TNT Booster\" and repeated phrases like \"we can see that\"; a light copyedit would be helpful.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a data-management/analytics venue and the plug-and-play augmentation idea is attractive. The main barrier is not the breadth of the experiments but the validation of the pseudo-positive labeling and the unresolved split-leakage question. If the authors can supply a precision measurement on naturally missing facts and confirm that augmentation is computed from the training split only, the paper could become acceptable. The misreported recall in Section 4.2/Figure 4 also needs correction."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: Booster is a plausible and genuinely new plug-in augmentation for TKG completion, with a broad experimental sweep and a novel empirical observation about model preferences. But the validation of its pseudo-positive labels is recall-only, the reported recall figure is inconsistent with its own plot, and split leakage is not ruled out. I'd send it to review, but I would not yet trust the 8.7% headline.\n\nWhat's new: first TKG-specific augmentation that combines relation/entity/time frequency filtering, triadic-closure scoring, and two-stage training. The model-preference diagnosis in Section 3.4 (TNT favoring frequent entities, TEMP favoring recently active ones) is a real observation, and the two-stage design that counteracts it is a reasonable response. The experiments are broad: five backbones, five datasets, ablations, p-values, variance reduction, plus an artifact link. That's more than most augmentation papers ship.\n\nSoft spots, in rough order of severity. First, the pseudo-positive labels: candidates are added as training positives whenever their hierarchical score exceeds the seed fact's score (Eq. 10), and the only validation is the 20% deletion recall experiment. Recall on deleted observed facts is not precision on naturally missing facts; the smooth-label step in Eq. 12 only downweights, never removes, wrong labels. If precision is low, gains could come from injected noise. Second, the text says 'more than 90%' recall, but Figure 4 shows 71–79% depending on dataset. That's a concrete inconsistency and it matters because the recall claim is the paper's main evidence for the filtering strategy. Third, leakage: nothing in Section 4.2 states that filtering and scoring use only the training split. The split is introduced later; if the full G is used, test answers can leak into training pseudo-positives. This is likely a one-line fix, but it has to be stated. Fourth, top-m and sparsity threshold k are used in Eqs. 2–3 and Section 4.2 but never reported in implementation details; reproducibility gap for a plug-and-play claim.\n\nThe core direction is sound. The ablations (Table 3, variants in Table 4) are informative, and the efficiency and variance analyses are adequate. The issues are empirical validation gaps, not conceptual holes. For a serious referee: yes, this deserves full review. I'd ask for precision measurement on naturally missing facts, split-leakage clarification, corrected recall reporting, and hyperparameter disclosure before acceptance. If those check out, it's a solid contribution to a niche subfield.","headline":"Booster is a plausible and genuinely new plug-in augmentation for TKG completion with a broad experimental sweep and a novel model-preference observation, but the pseudo-positive validation is recall-only, the reported recall figure is inconsistent with its own plot, and split leakage is not ruled out.","tokens_in":23878,"tokens_out":3657,"would_cite":true,"duration_ms":34378,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A pattern-aware augmentation strategy lifts temporal knowledge graph completion by up to 8.7% and cuts performance variance by 22.8%.","keywords":["temporal knowledge graph completion","data augmentation","false negative filtering","model preference","triadic closure","hierarchical scoring","two-stage training","knowledge graph embedding"],"falsifier":"Take a TKG with known ground-truth missing facts (or randomly delete a held-out set of true facts), run Booster's frequency filtering and hierarchical scoring, and compute the precision of the flagged false negatives. If precision is low, the reported gains would come from denoising or regularization rather than from recovering true missing facts; a direct check is to compare Booster's recovered samples with a manually verified set of naturally absent facts.","tokens_in":22880,"feed_emoji":"🧩","tokens_out":7513,"duration_ms":65986,"temperature":0.7,"pith_summary":"The paper argues that poor completion performance and unstable training in temporal knowledge graph completion (TKGC) are driven by two overlooked problems: imbalanced fact distribution across entities and timestamps, and 'model preference,' where each architecture systematically favors entities with particular properties. To counter both, it proposes Booster, a plug-and-play data augmentation strategy that filters candidate false negatives by co-occurrence frequency, scores them with a hierarchical triangle-based algorithm, and fine-tunes any pre-trained TKGC model on the recovered positives, hard negatives, and model-specific hard samples. On five real-world TKGs and five backbone models, Booster reports relative MRR improvements up to 8.7%, outperforming existing graph and knowledge graph augmentation baselines by about 7.1% on average, while reducing rank variance across training runs by 22.8% on average. If these results hold, model-agnostic augmentation is a practical route to more accurate and more stable TKG completion without redesigning the underlying models.","feed_headline":"Pattern-aware augmentation lifts TKG completion by up to 8.7%","feed_subtitle":"Plug-in augmentation adds pattern-validated missing facts and cuts rank variance of TKGC models by 22.8%.","key_machinery":"The load-bearing mechanism is a hierarchical scoring algorithm built on triadic closures in a time-irrelevant unified graph, where relation types are first anonymized to count entity triangles and then entities are anonymized to count relation triangles. Entity triangles capture how strongly three entities prefer to be mutually connected; relation triangles capture how often three relations span the same entity triangle. A candidate missing fact $(s,r,o,t)$ receives a confidence score by aggregating these global triangle scores over its recent local structure, with softmax time-aware weights that emphasize recent interactions. If the mean score over perturbed local structures exceeds the score of the observed seed fact from which the candidate was generated, the candidate is labeled a real false negative and added to training with a smooth label; otherwise it is kept as a hard negative. This scoring procedure is what lets Booster generate new samples that fit both the global semantic patterns and the local temporal trends of a TKG.","core_discovery":"The central claim is that missing-but-valid facts in a temporal knowledge graph can be distinguished from true negatives by how well they complete entity and relation triangles, and that feeding these recovered facts back as training positives while flagging hard negatives improves existing TKGC models. The paper also claims to be the first to show that current TKGC methods exhibit model preferences—for example, tensor-factorization models favor frequently interacting entities while recurrent models favor recently active ones—and that such preferences cause self-training-style augmentation to reinforce a single pattern. Booster's two-stage training first pre-trains on filtered facts to avoid false negative contamination, then fine-tunes on pattern-validated false negatives, hard negatives, and low-ranked positives that expose non-preferred patterns. The reported result is consistent MRR gains across HyTE, TA, DE, TNT, and TEMP on ICEWS14, ICEWS05-15, YAGO11k, Wikidata12k, and GDELT.","pith_inferences":["Editorial inference: Booster's benefits likely depend on the true missingness rate; on graphs with few genuinely missing facts, adding pseudo-positives could degrade rather than help, so the method should be paired with a precision check.","Implicit extension: the same triangle-based scoring could be reused outside training augmentation, for example to rank candidate completions at inference time or to flag likely errors in TKG curation.","Testable extension: applying Booster to an out-of-distribution temporal split would show whether the pattern scores generalize to new relations and entity pairs, or only capture repeated historical motifs."],"forward_implications":["Existing TKGC models can be upgraded by wrapping them in Booster, with reported relative MRR gains of 2.5% to 8.7% across five backbones and five datasets.","Sparse entities and sparse timestamps gain the most, so long-tail completion performance becomes more balanced rather than uniformly shifted.","Because rank variance across independent training runs drops by 22.8% on average, the results of TKGC evaluations become more reproducible.","Static KG and temporal graph augmentation baselines, some of which can degrade performance on TKGs, should be compared against pattern-aware augmentation in future benchmarks."],"supporting_citations":[{"why":"Supplies TEMP, a recurrent message-passing backbone whose preference for recently active entities motivates the model-preference analysis.","marker":"[40]"},{"why":"Supplies TNT, the tensor-factorization backbone used in the main experiments and ablations.","marker":"[19]"},{"why":"Supplies DE, a dynamic-embedding backbone used in the preliminary study of unstable training and variance.","marker":"[12]"},{"why":"Supplies HyTE, the timestamp-embedding baseline that shows the largest reported improvement under Booster.","marker":"[7]"},{"why":"Supplies NSCaching, a negative-sampling baseline that Booster outperforms and whose filtering approach is contrasted.","marker":"[55]"},{"why":"Supplies MeTA, a temporal-graph augmentation baseline used as a comparison point.","marker":"[39]"},{"why":"Supplies DeMix, a denoising negative-sampling baseline for knowledge graphs used as a comparison.","marker":"[5]"},{"why":"Supplies KG-Mixup, a degree-bias augmentation baseline for knowledge graph embeddings used as a comparison.","marker":"[28]"},{"why":"Supplies the ICEWS datasets that carry the preliminary imbalance and preference analyses.","marker":"[2]"}],"fun_headline_variants":["First TKG augmentation uses triadic closures to boost completion by 8.7%","Pattern-aware augmentation for TKG completion improves MRR by 8.7%","Revealing model preferences, Booster augments TKG completion by 8.7%","Booster: pattern-validated negatives improve TKG completion by 8.7%","Plug-in pattern-aware augmentation cuts TKG rank variance by 22.8%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that a candidate fact whose perturbed triangle score beats the score of an observed seed fact is genuinely missing rather than false; the paper validates this by recall on randomly deleted facts and never measures precision on naturally missing facts.","fun_headline_variants_meta":{"raw":{"variants":["First TKG augmentation uses triadic closures to boost completion by 8.7%","Pattern-aware augmentation for TKG completion improves MRR by 8.7%","Revealing model preferences, Booster augments TKG completion by 8.7%","Booster: pattern-validated negatives improve TKG completion by 8.7%","Plug-in pattern-aware augmentation cuts TKG rank variance by 22.8%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001095,"raw_usage":{"total_tokens":4614,"prompt_tokens":1027,"completion_tokens":3587,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":643,"completion_tokens_details":{"reasoning_tokens":3475}},"tokens_in":643,"tokens_out":3587,"duration_ms":23886,"temperature":1.0,"reasoning_tokens":3475,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:55:23.960628+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a TKG with known ground-truth missing facts (or randomly delete a held-out set of true facts), run Booster's frequency filtering and hierarchical scoring, and compute the precision of the flagged false negatives. If precision is low, the reported gains would come from denoising or regularization rather than from recovering true missing facts; a direct check is to compare Booster's recovered samples with a manually verified set of naturally absent facts.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the ICEWS datasets that carry the preliminary imbalance and preference analyses."},{"cited_title":"Hamilton","cited_arxiv_id":null,"evidence_quote":"Supplies TEMP, a recurrent message-passing backbone whose preference for recently active entities motivates the model-preference analysis."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies TNT, the tensor-factorization backbone used in the main experiments and ablations."},{"cited_title":"Brubaker, and Pascal Poupart","cited_arxiv_id":null,"evidence_quote":"Supplies DE, a dynamic-embedding backbone used in the preliminary study of unstable training and variance."},{"cited_title":"Talukdar","cited_arxiv_id":null,"evidence_quote":"Supplies HyTE, the timestamp-embedding baseline that shows the largest reported improvement under Booster."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies NSCaching, a negative-sampling baseline that Booster outperforms and whose filtering approach is contrasted."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies MeTA, a temporal-graph augmentation baseline used as a comparison point."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies DeMix, a denoising negative-sampling baseline for knowledge graphs used as a comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies KG-Mixup, a degree-bias augmentation baseline for knowledge graph embeddings used as a comparison."}],"review_version":1}