{"id":"7660d122-ef10-43e9-a9b1-78e3a1e2b9f2","arxiv_id":"2412.00881","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"MetaEU uses meta-learning over subgraph tasks, with relation-aware and neighbor-enhanced generators, to unlearn knowledge graph embeddings while preserving performance on the remaining data.","lead":"This paper introduces MetaEU, a meta-learning-based framework that generates replacement embeddings to remove the influence of selected triples from knowledge graph embedding models. It claims to preserve accuracy on the remaining data and to adapt to unlearning tasks that involve never-seen entities.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed generalization to unseen entities is not actually tested: all experiments use subgraphs from the same FB15k-237 graph, and Table 1 evaluates on the same entity set, so the central novelty remains unsupported.","rationale":"The reader's weakest assumption identifies the distribution-shift risk: subgraphs from the training KG may not match future unlearning scenarios. My concern is closely related but more direct: the paper never actually evaluates on genuinely unseen entities, so the central generalization claim is untested rather than merely at risk of distribution shift. This is the most load-bearing issue because the whole novelty over prior work depends on generalization to unseen entities, and Table 1 could be reproduced by an in-distribution method that memorizes entity-specific patterns. I partially agree with the reader because the missing evaluation is the more concrete problem, while the distribution-shift concern would become relevant only if a proper unseen-entity test were added. Other defects (undefined L4/L5, missing inner-loop update, no code release) are real and hamper reproducibility, but they are secondary: even if fully specified, the paper would still lack evidence for its headline capability. The reader's CONDITIONAL verdict remains appropriate, so I recommend no change: the paper needs major revision and a proper unseen-entity experiment before the central claim can be accepted.","tokens_in":10977,"tokens_out":5122,"duration_ms":47880,"concrete_test":"Construct a held-out set of entities not present in any meta-training subgraph: e.g., partition FB15k-237 into disjoint entity clusters, or use a second knowledge graph such as WN18RR with overlapping relation types. Run MetaEU's unlearning on triples containing only held-out entities, and compare Forget-set and Test-set Hits@10/MRR against a Retrained baseline, with 10 seeds and error bars. If the Unlearned model does not achieve significantly lower Forget metrics than Retrained while preserving Test metrics, the claimed generalization to unseen entities fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (Conclusion) is that MetaEU can perform KGE unlearning in unfamiliar scenarios with unseen entities. The experiments do not test this. In Sec 4.1, meta-training extracts subgraphs from the training KG and 'treat[s] the entities within these subgraphs as unseen' (after Eq. 3), but this is a simulation: those entities are still drawn from the same distribution and their relation patterns overlap with the rest of the graph. The main results in Table 1 are reported on the same FB15k-237 graph with the original entity set; there is no held-out set of entities never observed during meta-training. The only comparison mentioning unseen entities is Fig. 3(b), but the caption and text do not specify how unseen entities were created, whether they appear in any meta-training subgraph, what the final metrics are, or the variance across runs. Consequently, the results are consistent with MetaEU merely fitting relation/neighborhood patterns of FB15k-237 entities, not with transferring to genuinely new entities. Because the stated advantage over prior work (FedLU) rests entirely on this generalization, the central claim is unsupported by the presented evidence.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MetaEU, a meta-learning-based framework for knowledge graph embedding (KGE) unlearning. MetaEU generates replacement embeddings for entities in a forgetting set, using a Relation-Aware Entity Embedding Generator (RAEEG) and a Neighbor-Enhanced Embedding Modulator (NEEM), combined with ensemble learning and ensemble unlearning. The method is evaluated on FB15k-237 across four KGE models (TransE, DistMult, ComplEx, RotatE). Table 1 reports that after unlearning, performance on the forgetting set drops below both the original and retrained models, while performance on the test set remains close to the original. The paper also compares MetaEU with FedLU on unlearning tasks and claims that MetaEU generalizes to unseen entities and unfamiliar scenarios, a capability that existing methods lack.","tokens_in":11236,"tokens_out":4109,"duration_ms":37160,"significance":"If the generalization claim is validated, MetaEU would be a novel and valuable contribution to machine unlearning for knowledge graphs: it would offer a training-based alternative to retraining that can adapt to new unlearning requests without full model retraining. The meta-learning perspective on KGE unlearning is timely and the proposed modules are reasonably motivated. However, the evidence presented does not yet establish the central claim of generalization to unseen entities, because all experiments use a single dataset and the 'unseen entities' are simulated from subgraphs of the same graph. The paper also does not provide code, error bars, or statistical tests, which limits the strength of the empirical claims. The idea is promising, but the current support is insufficient for acceptance.","major_comments":[{"comment":"The central claim that MetaEU generalizes to unseen entities is not directly tested. In §4.1, the authors 'treat the entities within these subgraphs as unseen' (around Eq. 3), but this is a simulation: all entities are drawn from FB15k-237 and their relation patterns overlap with the rest of the graph. The main results in Table 1 are evaluated on the original entity set, and Figure 3(b), which is the only evidence involving 'unseen entitys,' does not state how these entities were constructed, whether they appear in any meta-training subgraph, what the final metric values are, or the variance across runs. The Conclusion explicitly claims 'MetaEU can efficiently perform KGE unlearning in unfamiliar scenarios with unseen entities,' but no experiment with a held-out entity set (e.g., entities excluded from meta-training, or a different KG) is reported. This is a load-bearing gap because the stated advantage over prior work rests entirely on this generalization.","section":"§4.1, §5.1, §5.3, Conclusion"},{"comment":"The unlearning objective and the evaluation metric are confounded. The ensemble unlearning objective in Eq. (10) maximizes the KGE loss on the query set of the forgetting set, and the evaluation of unlearning success in Table 1 uses the same performance metric (Hits@n and MRR on the Forget set). Thus the observed drop in Forget-set scores is partly by construction, since the optimization directly targets that measure. The paper should provide an independent verification that the model has actually 'forgotten' the targeted knowledge, for example by measuring the rank of the forgotten triples relative to a model that never saw them, using membership inference, or evaluating on a distribution of similar unlearning requests. Without such evidence, the claimed unlearning effect is not independently established.","section":"§4.3 (Eq. 10) and §5.4 (Table 1)"},{"comment":"The experimental results lack error bars and statistical tests. Section 5.2 states that experiments were conducted 10 times and averages are reported, but Table 1 reports only point estimates. For instance, in the TransE results, the Unlearned Test MRR is 0.7153 versus RAW 0.7254, and in the DistMult results, the Unlearned Forget MRR is 0.1853 versus Retrained 0.2269. These differences could easily be within run-to-run noise, especially given the small gaps. Reporting standard deviations, confidence intervals, or paired significance tests is necessary to support the claims in §5.4 that 'Unlearned exhibited the lowest performance on the Forget set' and that test performance is 'comparable to RAW.'","section":"§5.2 and Table 1"},{"comment":"Several experimental details essential for reproduction are missing. The paper does not specify how the forgetting set Tf was constructed (e.g., its size, whether it was random or targeted, how many triples per entity), how the k subgraphs in Eq. (3) were sampled (e.g., random walks, connected components, size distribution), or the values of hyperparameters wa and wb in Eq. (11), the number of base models N, and the number of NEEM layers L (Section 5.2 only lists the number of tasks, epochs, learning rate, and batch size). Moreover, Figure 3(b) does not describe the protocol for generating 'unseen entities.' Without these details, the experiments cannot be reproduced or the results independently assessed.","section":"§5.2, §5.3, §4.3"}],"minor_comments":[{"comment":"There are frequent typos and grammatical errors, including 'Seciton' (Section 3), 'entitys' (Figure 3 caption), 'forumulation' (Section 3), and 'unlearing' (Section 3). These should be corrected with a careful proofread.","section":"Throughout"},{"comment":"The notation in Eq. (5) is confusing: φ′ is not defined before its use, and the expression L_{T_support_i}(f_φ′(xj), yj) mixes set indexes with data points. Please clarify the meta-learning loss notation.","section":"§4.1, Eq. (5)"},{"comment":"Reference [15] (Meta-SGD) is a preprint with a DOI, but other references are incomplete in formatting; please ensure the bibliography is consistent and complete.","section":"§2"},{"comment":"The abstract mentions 'benchmark datasets' (plural) and 'thorough experimental study,' but experiments are conducted only on FB15k-237. Please either add experiments on additional datasets or revise the wording to match the scope.","section":"Abstract and §5.1"},{"comment":"The comparison with FedLU is only qualitative (Figure 3), and the diffusion-based method [8] is mentioned but never quantitatively compared. A controlled comparison on the same setup, with the same forgetting sets, would strengthen the claims.","section":"§5.3"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant and timely topic, and the meta-learning idea is interesting. However, the central claim of generalization to unseen entities is currently unsupported because all experiments use a single KG and the unseen-entity evaluation is not properly specified. The confounding between the unlearning objective and the evaluation metric is also a concern. These issues are fixable with additional experiments (e.g., a held-out entity split or a second KG, independent forgetting verification, and error bars), so I recommend major revision rather than rejection. I would also suggest that the authors release code to facilitate reproduction. The manuscript is not yet at the standard of a top conference/journal publication, but with the missing experiments it could be."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick read: MetaEU is a plausible first application of meta-learning to KG embedding unlearning. It combines MAML-style task sampling, an R-GCN-like neighbor modulator, and an ensemble to generate replacement embeddings that push forgetting-set triples down while keeping test performance close to the original model. Table 1 shows the expected pattern across four KGE models: Unlearned drops Forget metrics well below Retrained and stays near RAW on Test. That is real evidence the mechanism does something, and the ablation suggests RAEEG/NEEM matter more than the ensemble size.\n\nThe trouble is the central selling point. The paper says MetaEU generalizes to unfamiliar scenarios with unseen entities, and that this is what existing methods lack. But the experiments never actually hold out entities. Meta-training samples subgraphs from FB15k-237 and 'treats' their entities as unseen; Table 1 evaluates on the same graph's entity set; and Figure 3(b), which claims to show unseen-entity performance, gives no details on how the unseen set was built, whether those entities appear in any training subgraph, or variance across runs. So we are left with a simulation of unseenness, not a test of it. That is the load-bearing flaw.\n\nThere are also completeness issues: L4 and L5 are referenced but never defined, the inner-loop update for phi' in Eq. 5 is not given, and the text says '10,200' tasks where it likely means 10,000 training and 200 validation. Small stuff, but it makes the method harder to reproduce.\n\nI don't think the paper is broken on its own terms. The unlearning objective (maximize loss on the forgetting set) shares a criterion with the evaluation, so the Forget-set drop is partly by construction, but the Test-set retention and the relative comparison to Retrained are not circular. The core idea is sensible and worth taking seriously.\n\nBottom line: if the authors rerun with a true held-out entity split, report standard deviations, and fill in the missing updates, this could be a solid workshop or short-conference paper. As it stands, it deserves a serious referee because the topic is growing and the combination is genuinely new, but it is not accept-ready. I'd send it to review with a request for major revision.","headline":"Novel meta-learning framework for KG embedding unlearning, but the headline claim about unseen entities is not tested; the paper needs revision before the claim can stand.","tokens_in":11743,"tokens_out":1777,"would_cite":false,"duration_ms":15843,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a meta-learning framework called MetaEU can unlearn specific triples from knowledge graph embeddings—even for entities never seen in training—by generating replacement embeddings, while preserving the model's…","keywords":["knowledge graph embedding","machine unlearning","meta-learning","unseen entities","link prediction","privacy","ensemble learning","relation-aware embedding"],"falsifier":"Take a trained MetaEU and ask it to unlearn triples on a second knowledge graph whose relation vocabulary or entity types do not appear in the training subgraphs (for example, a biomedical graph after training on FB15k-237). If the Forget-set Hits@10 does not fall below the retrained baseline while Test-set metrics remain comparable, the claim of generalizing to unseen entities is unsupported.","tokens_in":10793,"feed_emoji":"🧠","tokens_out":6320,"duration_ms":56353,"temperature":0.7,"pith_summary":"Knowledge graphs power link prediction and question answering, but once an embedding model has trained on a triple, deleting that triple from the database does not remove its influence from the model. The paper proposes MetaEU, a meta-learning framework whose goal is to unlearn specific triples by generating replacement embeddings that dilute the forgotten knowledge while keeping the model's performance on the remaining triples intact. Its distinctive claim is generalization: because training simulates unlearning tasks on subgraphs whose entities are treated as unseen, the trained framework can handle unlearning requests involving entities never seen during training, a case existing KGE unlearning methods do not cover. Across TransE, DistMult, ComplEx, and RotatE, the unlearned embeddings score lowest on the forget set while remaining comparable to the original on the test set.","feed_headline":"Meta-learning makes knowledge-graph unlearning work on unseen entities","feed_subtitle":"Trained on simulated forget-tasks, MetaEU scrubs specific triples without retraining and keeps remaining performance intact.","key_machinery":"The load-bearing object is the unlearning function $F_u(E, T_f, \\theta) \\to E'$ implemented by two entity-embedding generators. RAEEG (Relation-Aware Entity Embedding Generator) computes each entity's initial embedding as the mean of its incoming and outgoing relation embeddings, coding the entity's type without storing the entity's identity. NEEM (Neighbor-Enhanced Embedding Modulator) then runs $L$ layers of relation-aware graph convolution (following R-GCN) over the entity's neighborhood and combines all layers with a learned hierarchical integrator. Ensemble learning (minimizing loss $L_1$ on the query set) and ensemble unlearning (maximizing loss $L_2$ on the forgetting set) are combined into $L_3$, with $L_4$ constraining forgetting strength and $L_5$ fine-tuning per graph; the meta-loss makes this whole generator transfer across tasks.","core_discovery":"The central claim is that unlearning can be cast as a task-level learning problem rather than a per-model data-deletion problem. MetaEU is trained under a bi-level meta-learning loop: k subgraphs are extracted from the knowledge graph, the entities inside each subgraph are treated as unseen, and each subgraph is split into a support set and a query set. The support set is used to generate replacement entity embeddings through two modules—RAEEG, which averages the embeddings of an entity's ingoing and outgoing relations, and NEEM, which refines the result by aggregating multi-hop neighbor information in the style of relational graph convolution and then hierarchically integrating layers. An ensemble of such base generators is trained so that the query-set loss on retained triples is minimized (learning) while the query-set loss on forgotten triples is maximized (unlearning), with a balancing loss to prevent over-forgetting. The output $E'$ then satisfies the paper's criteria: on the remaining set its performance is close to the original embedding and above a retrained model, while on the forgetting set it falls below both.","pith_inferences":["A direct cross-graph test—training on FB15k-237 subgraphs and unlearning on a second knowledge graph with unseen relation types or entity types—would reveal whether the generalization claim comes from meta-learning itself or from the benchmark's homogeneity.","The paper does not report whether replacement embeddings are stable across repeated unlearning runs or whether a downstream attacker could reconstruct forgotten triples from the modified embedding matrix; measuring reconstruction success would test whether the unlearning is genuinely removing influence or only suppressing rank-based signals.","Because RAEEG averages relations and NEEM aggregates neighbors, entities with very sparse neighborhoods may receive generic replacement embeddings; a stress test on low-degree entities could expose the boundary of the method."],"forward_implications":["A deployed service can honor deletion requests without retraining from scratch: a trained MetaEU produces replacement embeddings for a new forget set quickly, shifting the cost to a one-time meta-training phase.","The unlearning mechanism is agnostic to the base KGE model, working for TransE, DistMult, ComplEx, and RotatE, so swapping the embedding model does not require rebuilding the unlearning module.","Entities that appear only after deployment are in scope, because the generator uses relation types and neighborhood structure rather than stored entity identities.","Forget-set performance drops below both the original and retrained embeddings, which is the paper's criterion for the influence of the targeted triples being eliminated."],"supporting_citations":[{"why":"Baseline FedLU, which performs KGE unlearning on a fixed predefined KG and serves as the comparison target for unseen-entity generalization.","marker":"[7]"},{"why":"Existing diffusion-based KGE unlearning whose slow, complex data generation motivates MetaEU's alternative.","marker":"[8]"},{"why":"TransE supplies the scoring function and margin loss used as the base KGE objective in Equation 2 and is one of four tested base models.","marker":"[9]"},{"why":"DistMult is one of the four KGE models used to show MetaEU works across embedding families.","marker":"[10]"},{"why":"ComplEx is one of the four KGE models used to show MetaEU works across embedding families.","marker":"[11]"},{"why":"RotatE is one of the four KGE models used to show MetaEU works across embedding families.","marker":"[12]"},{"why":"Gradient-based meta-learning provides the task-adaptation regime that MetaEU adapts to the unlearning setting.","marker":"[14]"},{"why":"R-GCN provides the relational graph convolutional update used by the NEEM module.","marker":"[22]"}],"fun_headline_variants":["MetaEU: meta-learned unlearning that generalizes across graph entities","Turn unlearning into a learnable task—MetaEU scrubs graph embeddings","Unlearning without retraining: meta-learning for knowledge-graph edits","Forget specific triples on demand with meta-learned graph unlearning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that subgraphs cut from the training knowledge graph are a faithful stand-in for every future unlearning request, so that entities labeled 'unseen' during meta-training share the same relation patterns and neighborhood structure as truly unseen entities in deployment.","fun_headline_variants_meta":{"raw":{"variants":["MetaEU: meta-learned unlearning that generalizes across graph entities","Turn unlearning into a learnable task—MetaEU scrubs graph embeddings","Unlearning without retraining: meta-learning for knowledge-graph edits","Forget specific triples on demand with meta-learned graph unlearning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000413,"raw_usage":{"total_tokens":2100,"prompt_tokens":876,"completion_tokens":1224,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":492,"completion_tokens_details":{"reasoning_tokens":1145}},"tokens_in":492,"tokens_out":1224,"duration_ms":9291,"temperature":1.0,"reasoning_tokens":1145,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T04:53:43.062141+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a trained MetaEU and ask it to unlearn triples on a second knowledge graph whose relation vocabulary or entity types do not appear in the training subgraphs (for example, a biomedical graph after training on FB15k-237). If the Forget-set Hits@10 does not fall below the retrained baseline while Test-set metrics remain comparable, the claim of generalizing to unseen entities is unsupported.","supporting_citations":[{"cited_title":"In: Web and Big Data","cited_arxiv_id":null,"evidence_quote":"Existing diffusion-based KGE unlearning whose slow, complex data generation motivates MetaEU's alternative."},{"cited_title":"In: Advances in Neural Informa- tion Processing Systems","cited_arxiv_id":null,"evidence_quote":"TransE supplies the scoring function and margin loss used as the base KGE objective in Equation 2 and is one of four tested base models."},{"cited_title":"In: International Conference on Learning Representations (2014)","cited_arxiv_id":null,"evidence_quote":"DistMult is one of the four KGE models used to show MetaEU works across embedding families."},{"cited_title":"In: Proceedings of The 33rd International Conference on Machine Learning","cited_arxiv_id":null,"evidence_quote":"ComplEx is one of the four KGE models used to show MetaEU works across embedding families."},{"cited_title":"In: International Conference on Learning Representations (2018)","cited_arxiv_id":null,"evidence_quote":"RotatE is one of the four KGE models used to show MetaEU works across embedding families."},{"cited_title":"In: Proceedings of the 34th International Conference on Machine Learning","cited_arxiv_id":null,"evidence_quote":"Gradient-based meta-learning provides the task-adaptation regime that MetaEU adapts to the unlearning setting."}],"review_version":1}