REVIEW 3 major objections 3 minor
Membership and Memorization in LLM Knowledge Distillation
T0 review · 3 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read Knowledge distillation passes a teacher LLM's privacy leaks to its students.
desk verdict Useful empirical sweep of KD privacy risks, but the abstract's universal claim overreaches; worth refereeing if the authors narrow the scope. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is the knowledge-distillation pipeline itself: a student model is trained on outputs generated by a teacher that was trained on private data. The teacher's outputs encode not only general knowledge but also traces of specific training examples, and the student absorbs those traces. The paper's analytic machinery decomposes risk by three components—KD objective function, student training data, and NLP task—and measures both membership inference (whether a data point was in the teacher's training set) and memorization (whether training text is recoverable from outputs).
What would settle it
A concrete counterexample would be a knowledge-distillation technique—such as one using differentially private teacher outputs or output aggregation—where a standard membership-inference attack trained on student outputs cannot distinguish between teacher members and non-members, and a memorization extraction attack fails to recover any training text.
Extended reading notes
Core claim
The paper demonstrates that all existing LLM knowledge-distillation approaches carry membership and memorization privacy risks from the teacher to its students. Using six KD techniques, teacher models from GPT-2, LLAMA-2, and OPT families, and students of various sizes, the authors show that a student trained on teacher outputs can be used to infer whether a given example was in the teacher's training set, and to extract verbatim training content. The extent of risk varies by KD objective, student training data, and NLP task, and there is a significant disagreement between memorization and membership privacy risks for the same KD technique. Per-block analysis further shows that privacy risk
Load-bearing premise
The evaluated set of six distillation techniques, three teacher families, and seven tasks is assumed to represent all existing LLM knowledge-distillation approaches.
Editorial extensions
If this is right
- If the claim holds, any organization using KD to compress a private teacher model must treat the student as carrying the same membership and memorization risks as the teacher.
- Distillation technique choice matters: the paper shows risk varies meaningfully across KD objectives, so some methods leak more than others, though none leaks zero.
- The observed disagreement between memorization and membership risk means a defense that mitigates one may not mitigate the other; both need separate evaluation.
- Per-block variation suggests that pruning or modifying high-risk transformer blocks could reduce leakage without retraining the whole student.
- Privacy-preserving distillation, such as adding noise or restricting teacher outputs, would need to be validated against both membership and memorization attacks, not just one.
Reading between the lines
- The paper's 'all existing' claim is a strong universal statement supported only by a sample of six techniques; a natural extension is a broader survey to test whether any KD method avoids risk entirely.
- The per-block risk variation hints that a fine-grained attribution of which teacher outputs cause leakage could enable targeted sanitization of only the riskiest blocks.
- A testable extension would be to see whether the same leakage appears when the teacher is trained with differential privacy; if so, KD may partially cancel those protections.
- The disagreement between memorization and membership risks suggests that current attack definitions capture different underlying phenomena, and a unified privacy metric for LLM distillation may need to combine both.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper reports an empirical study of membership-inference and memorization privacy risks in six large language model knowledge distillation (KD) techniques. Using three teacher families (GPT-2, LLAMA-2, OPT), seven instruction-tuning tasks, and students of various sizes, the authors claim that all existing LLM KD approaches carry such risks from teacher to student, with risk levels varying across KD objectives, training data, and tasks. They further report a disagreement between memorization and membership risk rankings and characterize per-block variation in privacy risk.
Significance. If the empirical findings hold, the paper makes a useful contribution by extending privacy-leakage analysis to LLM KD pipelines and by showing that risk is not uniform across KD components or transformer blocks. The breadth of the evaluation—multiple teachers, tasks, and student sizes—is a genuine strength, as is the attempt to disentangle membership and memorization risk. However, the central claim is stated as a universal law ('all existing LLM KD approaches'), which is not supported by the finite sample of six techniques and lacks a theoretical mechanism or coverage argument. The paper also does not specify the attack definitions in the abstract, leaving the validity of the privacy measures unverifiable from the presented text.
major comments (3)
- [Abstract] The sentence 'we demonstrate that all existing LLM KD approaches carry membership and memorization privacy risks from the teacher to its students' is a universal claim. The evidence, as described, is six KD techniques, three teacher families, and seven tasks. This finite sample cannot support a universal conclusion without either an exhaustive enumeration of existing methods or a theoretical argument that every KD method shares the mechanism that transfers these risks. I recommend restricting the claim to the evaluated set ('the six KD techniques studied here') or providing a coverage argument in the full text. Otherwise, the central assertion overreaches the evidence.
- [Abstract, methodology unspecified] The abstract does not define the membership inference and memorization attacks used. Different attack definitions can yield materially different conclusions; for example, a weak attack may report no risk while a stronger attack reports risk. The paper must state the threat model, the exact attack(s), and whether they use external ground truth or self-defined measures. Without this, the finding 'all KD approaches carry risks' is not reproducible or falsifiable. Please add this information to both the abstract and the experimental section.
- [Abstract, statistical reporting] The claim that 'the extent of privacy risks varies across different KD techniques' is made without any effect sizes, confidence intervals, or error bars in the abstract. Given the small number of techniques and tasks, sampling variability could be substantial. The paper should report variance across seeds or task splits and show where differences are significant, or soften the language to 'varies in our experiments'.
minor comments (3)
- [Abstract] The phrase 'carry membership and memorization privacy risks from the teacher to its students' is ambiguous: does it mean that privacy leakage is always present, or that a risk exists in principle? Clarify the strength of the claim.
- [Abstract] Please enumerate the six KD techniques explicitly in the abstract or state that they are listed in Section 2. Similarly, name the seven NLP tasks or refer to the table where they are defined.
- [General] The per-block privacy risk analysis is interesting but the abstract gives no detail. Please specify what 'block' means (e.g., transformer layer group) and how the per-block risk is computed.
Assumptions & free parameters
assumptions (2)
- domain assumption The six selected KD techniques and three teacher families are representative of all existing LLM KD approaches.
- domain assumption The membership and memorization attacks used accurately measure privacy risks.
Cite this review
Pith. "Pith review of Membership and Memorization in LLM Knowledge Distillation." pith.science (2026). https://pith.science/paper/2W56YCBI
@misc{pith2026250807054,
author = {Pith},
title = {Pith review of: Membership and Memorization in LLM Knowledge Distillation},
year = {2026},
howpublished = {\url{https://pith.science/paper/2W56YCBI}},
note = {Machine review of arXiv:2508.07054}
}
read the original abstract
Recent advances in Knowledge Distillation (KD) aim to mitigate the high computational demands of Large Language Models (LLMs) by transferring knowledge from a large ''teacher'' to a smaller ''student'' model. However, students may inherit the teacher's privacy when the teacher is trained on private data. In this work, we systematically characterize and investigate membership and memorization privacy risks inherent in six LLM KD techniques. Using instruction-tuning settings that span seven NLP tasks, together with three teacher model families (GPT-2, LLAMA-2, and OPT), and various size student models, we demonstrate that all existing LLM KD approaches carry membership and memorization privacy risks from the teacher to its students. However, the extent of privacy risks varies across different KD techniques. We systematically analyse how key LLM KD components (KD objective functions, student training data and NLP tasks) impact such privacy risks. We also demonstrate a significant disagreement between memorization and membership privacy risks of LLM KD techniques. Finally, we characterize per-block privacy risk and demonstrate that the privacy risk varies across different blocks by a large margin.
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.