REVIEW 4 major objections 5 minor 1 cited by
Learn to Unlearn: Meta-Learning-Based Knowledge Graph Embedding Unlearning
T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read This paper claims that a meta-learning framework called MetaEU can unlearn specific triples from knowledge graph embeddings—even for entities never seen in training—by generating replacement embeddings, while preserving the model's…
desk verdict Novel meta-learning framework for KG embedding unlearning, but the headline claim about unseen entities is not tested; the paper needs revision before the claim can stand. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the unlearning function $F_u(E, T_f, \theta) \to E'$ implemented by two entity-embedding generators. RAEEG (Relation-Aware Entity Embedding Generator) computes each entity's initial embedding as the mean of its incoming and outgoing relation embeddings, coding the entity's type without storing the entity's identity. NEEM (Neighbor-Enhanced Embedding Modulator) then runs $L$ layers of relation-aware graph convolution (following R-GCN) over the entity's neighborhood and combines all layers with a learned hierarchical integrator. Ensemble learning (minimizing loss $L_1$ on the query set) and ensemble unlearning (maximizing loss $L_2$ on the forgetting set) are combined into $L_3$, with $L_4$ constraining forgetting strength and $L_5$ fine-tuning per graph; the meta-loss makes this whole generator transfer across tasks.
What would settle it
Take a trained MetaEU and ask it to unlearn triples on a second knowledge graph whose relation vocabulary or entity types do not appear in the training subgraphs (for example, a biomedical graph after training on FB15k-237). If the Forget-set Hits@10 does not fall below the retrained baseline while Test-set metrics remain comparable, the claim of generalizing to unseen entities is unsupported.
Extended reading notes
Core claim
The central claim is that unlearning can be cast as a task-level learning problem rather than a per-model data-deletion problem. MetaEU is trained under a bi-level meta-learning loop: k subgraphs are extracted from the knowledge graph, the entities inside each subgraph are treated as unseen, and each subgraph is split into a support set and a query set. The support set is used to generate replacement entity embeddings through two modules—RAEEG, which averages the embeddings of an entity's ingoing and outgoing relations, and NEEM, which refines the result by aggregating multi-hop neighbor information in the style of relational graph convolution and then hierarchically integrating layers. An ensemble of such base generators is trained so that the query-set loss on retained triples is minimized (learning) while the query-set loss on forgotten triples is maximized (unlearning), with a balancing loss to prevent over-forgetting. The output $E'$ then satisfies the paper's criteria: on the remaining set its performance is close to the original embedding and above a retrained model, while on the forgetting set it falls below both.
Load-bearing premise
The load-bearing premise is that subgraphs cut from the training knowledge graph are a faithful stand-in for every future unlearning request, so that entities labeled 'unseen' during meta-training share the same relation patterns and neighborhood structure as truly unseen entities in deployment.
Editorial extensions
If this is right
- A deployed service can honor deletion requests without retraining from scratch: a trained MetaEU produces replacement embeddings for a new forget set quickly, shifting the cost to a one-time meta-training phase.
- The unlearning mechanism is agnostic to the base KGE model, working for TransE, DistMult, ComplEx, and RotatE, so swapping the embedding model does not require rebuilding the unlearning module.
- Entities that appear only after deployment are in scope, because the generator uses relation types and neighborhood structure rather than stored entity identities.
- Forget-set performance drops below both the original and retrained embeddings, which is the paper's criterion for the influence of the targeted triples being eliminated.
Reading between the lines
- A direct cross-graph test—training on FB15k-237 subgraphs and unlearning on a second knowledge graph with unseen relation types or entity types—would reveal whether the generalization claim comes from meta-learning itself or from the benchmark's homogeneity.
- The paper does not report whether replacement embeddings are stable across repeated unlearning runs or whether a downstream attacker could reconstruct forgotten triples from the modified embedding matrix; measuring reconstruction success would test whether the unlearning is genuinely removing influence or only suppressing rank-based signals.
- Because RAEEG averages relations and NEEM aggregates neighbors, entities with very sparse neighborhoods may receive generic replacement embeddings; a stress test on low-degree entities could expose the boundary of the method.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes MetaEU, a meta-learning-based framework for knowledge graph embedding (KGE) unlearning. MetaEU generates replacement embeddings for entities in a forgetting set, using a Relation-Aware Entity Embedding Generator (RAEEG) and a Neighbor-Enhanced Embedding Modulator (NEEM), combined with ensemble learning and ensemble unlearning. The method is evaluated on FB15k-237 across four KGE models (TransE, DistMult, ComplEx, RotatE). Table 1 reports that after unlearning, performance on the forgetting set drops below both the original and retrained models, while performance on the test set remains close to the original. The paper also compares MetaEU with FedLU on unlearning tasks and claims that MetaEU generalizes to unseen entities and unfamiliar scenarios, a capability that existing methods lack.
Significance. If the generalization claim is validated, MetaEU would be a novel and valuable contribution to machine unlearning for knowledge graphs: it would offer a training-based alternative to retraining that can adapt to new unlearning requests without full model retraining. The meta-learning perspective on KGE unlearning is timely and the proposed modules are reasonably motivated. However, the evidence presented does not yet establish the central claim of generalization to unseen entities, because all experiments use a single dataset and the 'unseen entities' are simulated from subgraphs of the same graph. The paper also does not provide code, error bars, or statistical tests, which limits the strength of the empirical claims. The idea is promising, but the current support is insufficient for acceptance.
major comments (4)
- [§4.1, §5.1, §5.3, Conclusion] The central claim that MetaEU generalizes to unseen entities is not directly tested. In §4.1, the authors 'treat the entities within these subgraphs as unseen' (around Eq. 3), but this is a simulation: all entities are drawn from FB15k-237 and their relation patterns overlap with the rest of the graph. The main results in Table 1 are evaluated on the original entity set, and Figure 3(b), which is the only evidence involving 'unseen entitys,' does not state how these entities were constructed, whether they appear in any meta-training subgraph, what the final metric values are, or the variance across runs. The Conclusion explicitly claims 'MetaEU can efficiently perform KGE unlearning in unfamiliar scenarios with unseen entities,' but no experiment with a held-out entity set (e.g., entities excluded from meta-training, or a different KG) is reported. This is a load-bearing gap because the stated advantage over prior work rests entirely on this generalization.
- [§4.3 (Eq. 10) and §5.4 (Table 1)] The unlearning objective and the evaluation metric are confounded. The ensemble unlearning objective in Eq. (10) maximizes the KGE loss on the query set of the forgetting set, and the evaluation of unlearning success in Table 1 uses the same performance metric (Hits@n and MRR on the Forget set). Thus the observed drop in Forget-set scores is partly by construction, since the optimization directly targets that measure. The paper should provide an independent verification that the model has actually 'forgotten' the targeted knowledge, for example by measuring the rank of the forgotten triples relative to a model that never saw them, using membership inference, or evaluating on a distribution of similar unlearning requests. Without such evidence, the claimed unlearning effect is not independently established.
- [§5.2 and Table 1] The experimental results lack error bars and statistical tests. Section 5.2 states that experiments were conducted 10 times and averages are reported, but Table 1 reports only point estimates. For instance, in the TransE results, the Unlearned Test MRR is 0.7153 versus RAW 0.7254, and in the DistMult results, the Unlearned Forget MRR is 0.1853 versus Retrained 0.2269. These differences could easily be within run-to-run noise, especially given the small gaps. Reporting standard deviations, confidence intervals, or paired significance tests is necessary to support the claims in §5.4 that 'Unlearned exhibited the lowest performance on the Forget set' and that test performance is 'comparable to RAW.'
- [§5.2, §5.3, §4.3] Several experimental details essential for reproduction are missing. The paper does not specify how the forgetting set Tf was constructed (e.g., its size, whether it was random or targeted, how many triples per entity), how the k subgraphs in Eq. (3) were sampled (e.g., random walks, connected components, size distribution), or the values of hyperparameters wa and wb in Eq. (11), the number of base models N, and the number of NEEM layers L (Section 5.2 only lists the number of tasks, epochs, learning rate, and batch size). Moreover, Figure 3(b) does not describe the protocol for generating 'unseen entities.' Without these details, the experiments cannot be reproduced or the results independently assessed.
minor comments (5)
- [Throughout] There are frequent typos and grammatical errors, including 'Seciton' (Section 3), 'entitys' (Figure 3 caption), 'forumulation' (Section 3), and 'unlearing' (Section 3). These should be corrected with a careful proofread.
- [§4.1, Eq. (5)] The notation in Eq. (5) is confusing: φ′ is not defined before its use, and the expression L_{T_support_i}(f_φ′(xj), yj) mixes set indexes with data points. Please clarify the meta-learning loss notation.
- [§2] Reference [15] (Meta-SGD) is a preprint with a DOI, but other references are incomplete in formatting; please ensure the bibliography is consistent and complete.
- [Abstract and §5.1] The abstract mentions 'benchmark datasets' (plural) and 'thorough experimental study,' but experiments are conducted only on FB15k-237. Please either add experiments on additional datasets or revise the wording to match the scope.
- [§5.3] The comparison with FedLU is only qualitative (Figure 3), and the diffusion-based method [8] is mentioned but never quantitatively compared. A controlled comparison on the same setup, with the same forgetting sets, would strengthen the claims.
Circularity Check
Minor by-construction element in the Forget-set metric; core claims are not circular.
-
fitted input called prediction
[Section 4.3, Eq. 10 and Section 5.4, Table 1 discussion]
"arg max_{f base,w} LensembleU = Σ_{i=1}^{N} w_i L(f base_i, T_query) ... The objective of Equation 10 is to maximize the overall loss in the query set of the forgetting set ... Unlearned exhibited the lowest performance on the Forget set ... This indicates that, through unlearning, the model effectively reduces the impact of the forgetting set."
Eq. 10 directly optimizes (maximizes) the KGE loss on the forgetting-set query triples, so the subsequent finding that the Unlearned model has the lowest Forget-set metrics is the training objective itself, not an independent outcome. Reporting this drop as evidence that the model 'effectively reduces the impact of the forgetting set' evaluates the framework against the same function that was used to produce it. The Test-set preservation and unseen-entity claims are not circular, but the Forget-set evidence is forced by construction.
full rationale
The paper's main methodological derivation is self-contained: RAEEG and NEEM are defined by explicit, standard equations (Eqs. 6-8), the meta-learning objective follows the usual support/query formulation (Eq. 5), and no load-bearing result is imported from a self-citation or from an externally asserted uniqueness theorem. The one genuinely by-construction element is the Forget-set result: the unlearning module is trained by maximizing the loss on the forgetting set (Eq. 10), and the paper then cites the resulting low Forget-set scores as evidence of effective unlearning. That part is circular in a mild sense, but it is not the central novelty. The claims about preserving Test-set performance and generalizing to unseen entities are not guaranteed by that objective and are evaluated on held-out comparisons. Note, however, that the unseen-entity generalization claim is not strongly supported by the experiments: meta-training subgraphs are extracted from the same KG and 'treated as unseen' (Eq. 3), and Table 1 reports results on the original entity set rather than a genuinely held-out entity set. That is an evidence gap and a correctness risk, but it is not a circular derivation. Overall, the core framework does not reduce to its inputs, so the circularity score is low.
Assumptions & free parameters
free parameters (5)
- wa (L1 weight in Eq. 11)
- wb (L2 weight in Eq. 11)
- Number of base models N =
4 (inferred from ablation)
- NEEM layers L =
3
- Task subgraph sampling parameters
assumptions (4)
- domain assumption The task distribution of sampled subgraphs matches real unlearning scenarios.
- domain assumption R-GCN-style aggregation captures transferable neighborhood structure.
- domain assumption Relation-aware average of relation embeddings is a sufficient initial embedding generator.
- domain assumption Link prediction ranks on the forgetting set measure unlearning effectiveness.
Cite this review
Pith. "Pith review of Learn to Unlearn: Meta-Learning-Based Knowledge Graph Embedding Unlearning." pith.science (2026). https://pith.science/paper/SJBHAYI5
@misc{pith2026241200881,
author = {Pith},
title = {Pith review of: Learn to Unlearn: Meta-Learning-Based Knowledge Graph Embedding Unlearning},
year = {2026},
howpublished = {\url{https://pith.science/paper/SJBHAYI5}},
note = {Machine review of arXiv:2412.00881}
}
read the original abstract
Knowledge graph (KG) embedding methods map entities and relations into continuous vector spaces, improving performance in tasks like link prediction and question answering. With rising privacy concerns, machine unlearning (MU) has emerged as a critical AI technology, enabling models to eliminate the influence of specific data. Existing MU approaches often rely on data obfuscation and adjustments to training loss but lack generalization across unlearning tasks. This paper introduces MetaEU, a Meta-Learning-Based Knowledge Graph Embedding Unlearning framework. MetaEU leverages meta-learning to unlearn specific embeddings, mitigating their impact while preserving model performance on remaining data. Experiments on benchmark datasets demonstrate its effectiveness in KG embedding unlearning.
Figures
Figures from the paper (1 more)
Forward citations
Cited by 1 Pith paper
-
Unlearning of Knowledge Graph Embedding via Preference Optimization
GraphDPO applies direct preference optimization to knowledge graph embedding unlearning, using out-boundary sampling and boundary recall to forget triples while retaining neighbors.
Reference graph
Works this paper leans on
-
[1]
In: Advances in Knowledge Discovery and Data Mining
Ying, X., Luo, S., Yu, M., Zhao, M., Yu, J., Guo, J., Li, X.: Two-Stage Knowl- edge Graph Completion Based on Semantic Features and High-Order Structural Features. In: Advances in Knowledge Discovery and Data Mining. pp. 143–155. Springer Nature (2024).https://doi.org/10.1007/978-981-97-2242-6_12
-
[2]
In: Advances in Knowledge Discovery and DataMining.pp.542–554.SpringerInternationalPublishing(2020)
Zhao, M., Jia, W., Huang, Y.: Attention-Based Aggregation Graph Networks for Knowledge Graph Information Transfer. In: Advances in Knowledge Discovery and DataMining.pp.542–554.SpringerInternationalPublishing(2020). https://doi. org/10.1007/978-3-030-47436-2_41
-
[3]
In: Advances in Knowledge Discovery and Data Mining
Yuan, J., Gao, N., Xiang, J., Tu, C., Ge, J.: Knowledge Graph Embedding with Order Information of Triplets. In: Advances in Knowledge Discovery and Data Mining. pp. 476–488. Springer International Publishing (2019).https://doi.org/ 10.1007/978-3-030-16142-2_37
-
[4]
In: Advances in Knowledge Discovery and Data Mining.pp.3–14.SpringerNatureSwitzerland(2023)
Xie, F., Zeng, X., Zhou, B., Tan, Y.: Improving Knowledge Graph Entity Align- ment with Graph Augmentation. In: Advances in Knowledge Discovery and Data Mining.pp.3–14.SpringerNatureSwitzerland(2023). https://doi.org/10.1007/ 978-3-031-33377-4_1
work page 2023
-
[5]
Neurocomputing494, 203–223 (2022).https://doi.org/ 10.1016/j.neucom.2022.04.078
Tian, Y., Zhao, X., Huang, W.: Meta-learning approaches for learning-to-learn in deep learning: A survey. Neurocomputing494, 203–223 (2022).https://doi.org/ 10.1016/j.neucom.2022.04.078
-
[6]
In: Advances in Knowledge Discovery and Data Mining
Chen, J., Zhang, C., Hu, Z.: Meta-Reinforcement Learning Algorithm Based on Reward and Dynamic Inference. In: Advances in Knowledge Discovery and Data Mining. pp. 223–234. Springer Nature (2024).https://doi.org/10.1007/ 978-981-97-2259-4_17
work page 2024
-
[7]
In: Proceedings of the ACM Web Conference 2023
Zhu, X., Li, G., Hu, W.: Heterogeneous Federated Knowledge Graph Embedding Learning and Unlearning. In: Proceedings of the ACM Web Conference 2023. pp. 2444–2454. Association for Computing Machinery (2023).https://doi.org/10. 1145/3543507.3583305
arXiv 2023
-
[8]
Liu, B., Fang, Y., Wang, X., Li, X.: Federated Knowledge Graph Embedding Un- learning via Diffusion Model. In: Web and Big Data. pp. 272–286. Springer Nature (2024). https://doi.org/10.1007/978-981-97-7235-3_18
Show all 23 references
-
[9]
In: Advances in Neural Informa- tion Processing Systems
Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating Embeddings for Modeling Multi-relational Data. In: Advances in Neural Informa- tion Processing Systems. vol. 26. Curran Associates, Inc. (2013)
2013
-
[10]
In: International Conference on Learning Representations (2014)
Yang, B., Yih, W.t., He, X., Gao, J., Deng, L.: Embedding Entities and Relations for Learning and Inference in Knowledge Bases. In: International Conference on Learning Representations (2014)
2014
-
[11]
In: Proceedings of The 33rd International Conference on Machine Learning
Trouillon, T., Welbl, J., Riedel, S., Gaussier, E., Bouchard, G.: Complex Em- beddings for Simple Link Prediction. In: Proceedings of The 33rd International Conference on Machine Learning. pp. 2071–2080. PMLR (2016)
2016
-
[12]
In: International Conference on Learning Representations (2018)
Sun, Z., Deng, Z.H., Nie, J.Y., Tang, J.: RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space. In: International Conference on Learning Representations (2018)
2018
-
[13]
Proceedings of the AAAI Conference on Artificial Intelligence38(16), 17835–17843 (2024).https://doi.org/10.1609/ aaai.v38i16.29737
Cheng, S., Zhang, N., Tian, B., Chen, X., Liu, Q., Chen, H.: Editing Language Model-Based Knowledge Graph Embeddings. Proceedings of the AAAI Conference on Artificial Intelligence38(16), 17835–17843 (2024).https://doi.org/10.1609/ aaai.v38i16.29737
2024
-
[14]
In: Proceedings of the 34th International Conference on Machine Learning
Finn, C., Abbeel, P., Levine, S.: Model-Agnostic Meta-Learning for Fast Adapta- tion of Deep Networks. In: Proceedings of the 34th International Conference on Machine Learning. pp. 1126–1135. PMLR (2017) Learn to Unlearn: Meta-Learning-Based KGE Unlearning 15
2017
- [15]
-
[16]
In: Proceedings of the 36th International Conference on Machine Learning
Zintgraf, L., Shiarli, K., Kurin, V., Hofmann, K., Whiteson, S.: Fast Context Adap- tation via Meta-Learning. In: Proceedings of the 36th International Conference on Machine Learning. pp. 7693–7702. PMLR (2019)
2019
-
[17]
ACM Trans
Qiao, Z., Wang, P., Wang, P., Ning, Z., Fu, Y., Du, Y., Zhou, Y., Huang, J., Hua, X.S., Xiong, H.: A Dual-channel Semi-supervised Learning Framework on Graphs via Knowledge Transfer and Meta-learning. ACM Trans. Web18(2), 18:1–18:26 (2024). https://doi.org/10.1145/3577033
2024 doi
-
[18]
In: 2015 IEEE Symposium on Security and Privacy
Cao, Y., Yang, J.: Towards Making Systems Forget with Machine Unlearning. In: 2015 IEEE Symposium on Security and Privacy. pp. 463–480 (2015).https: //doi.org/10.1109/SP.2015.35
2015 doi
-
[19]
In: Advances in Neural Information Processing Systems
Ginart, A., Guan, M., Valiant, G., Zou, J.Y.: Making AI Forget You: Data Deletion in Machine Learning. In: Advances in Neural Information Processing Systems. vol. 32. Curran Associates, Inc. (2019)
2019
-
[20]
In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recog- nition Workshops (CVPRW)
Kim, J., Woo, S.S.: Efficient Two-stage Model Retraining for Machine Unlearn- ing. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recog- nition Workshops (CVPRW). pp. 4360–4368 (2022).https://doi.org/10.1109/ CVPRW56347.2022.00482
2022
-
[21]
In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Yao, J., Chien, E., Du, M., Niu, X., Wang, T., Cheng, Z., Yue, X.: Machine Unlearning of Pre-trained Large Language Models. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 8403–8419. Association for Compu...
2024 doi
-
[22]
In: The Semantic Web
Schlichtkrull, M., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., Welling, M.: Modeling Relational Data with Graph Convolutional Networks. In: The Semantic Web. pp. 593–607. Springer International Publishing (2018).https://doi.org/ 10.1007/978-3-319-93417-4_38
2018 doi
-
[23]
In: Proceedings of the 37th International Conference on Machine Learning
Teru, K., Denis, E., Hamilton, W.: Inductive Relation Prediction by Subgraph Rea- soning. In: Proceedings of the 37th International Conference on Machine Learning. pp. 9448–9457. PMLR (2020)
2020
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.