Pith. sign in

REVIEW 4 major objections 5 minor 21 references

Mr. Snuffleupagus at SemEval-2025 Task 4: Unlearning Factual Knowledge from LLMs Using Adaptive RMU

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Applying Adaptive RMU to decoder layers 12–14 of the 16-layer OLMo-1B gives the best tested unlearning score, and the paper scales that recipe to layers 24–26 for the 7B model.

desk verdict A useful 1B layer-sweep result for unlearning, with an unsupported 7B extension and a table typo that need fixing. read the letter →

arxiv 2506.16548 v1 pith:TGQHHKUI submitted 2025-06-19 cs.LG

Arjun Dosajh , Mihika Sanghi This is my paper · ORCID
classification cs.LG
keywords machineunlearninglargelanguagemodelsAdaptiveRMUrepresentationmisdirectiondecoderlayerselectionprivacypersonallyidentifiableinformationSemEval-2025Task4
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that factual knowledge in LLMs—names, phone numbers, SSNs, addresses—is most cleanly unlearned from later decoder layers, and that the Adaptive RMU variant of representation-misdirection unlearning can do this without crippling retained knowledge. On the 16-layer OLMo-1B, it sweeps every consecutive three-layer block and finds layers 12, 13, 14 give the best final score, driven almost entirely by a jump in membership-inference robustness. It then transfers that three-quarter-depth recipe to the 32-layer OLMo-7B as layers 24, 25, 26 and reports 4th place on both task leaderboards. If right, the practical takeaway is a cheap rule of thumb: target roughly the last quarter of the decoder for factual unlearning, rather than the early layers earlier RMU work stressed.

What carries the argument

Adaptive RMU's forget loss replaces RMU's fixed steering target with an adaptively scaled random unit vector: the random direction is multiplied by the frozen model's activation norm at each token, so the push to misdirect forget-set representations adapts to how strongly the original model would have activated. The retain loss keeps unlearned activations close to the frozen model on retain data, and the combined loss is optimised only at the chosen layer triple. The layer sweep carries the argument: by testing every consecutive three-layer combination in the 16-layer model, the paper maps where in the decoder unlearning succeeds, and that map determines the recommended depth.

What would settle it

Run the same consecutive-three-layer sweep on the seven-billion-parameter model: if any triple other than 24, 25, 26 scores higher, the paper's central two-model layer claim fails.

Watch

Extended reading notes

Core claim

The central claim is that unlearning factual and PII content with Adaptive RMU is best done in the later-middle decoder layers—specifically, layers 12, 13, and 14 among all triples from (0,1,2) to (13,14,15) for OLMo-1B, with the matching depth-scaled triple 24, 25, 26 recommended for OLMo-7B. The deciding factor is privacy: the MIA score jumps from 0.489 at layers 11–13 to 0.835 at layers 12–14, while task aggregate and MMLU stay roughly flat, so the later location buys privacy without sacrificing retention or general ability. This layer choice produced a 4th-place finish on both official leaderboards, and the paper reads it as evidence that factual memorisation concentrates at different depths than the hazardous conceptual knowledge targeted by original RMU, which favoured earlier layers.

Load-bearing premise

The load-bearing assumption is that the ideal layers for the seven-billion-parameter model can be inferred from the one-billion-parameter sweep (24–26 by proportional depth), because the paper says its experiments ran exclusively on the one-billion-parameter model.

Editorial extensions

If this is right

  • For unlearning factual or PII content, later decoder layers near three-quarters of the model's depth should be the first intervention point, not the early layers favoured by the original RMU recipe.
  • The task-specific retain set can replace the external corpus used in original RMU, so the recipe needs only data the unlearning task already provides.
  • Layer choice can be pinned down with a small-model sweep and then scaled by depth to a larger model, assuming the 7B extrapolation holds.
  • The main benefit of moving unlearning to later layers is membership-inference privacy: the MIA score rises sharply while task aggregate and MMLU stay approximately flat.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the 7B layer triple were verified with its own sweep, the three-quarter-depth rule would amount to a scaling statement about where factual memorisation concentrates, worth testing on other model families.
  • The later-layer concentration may be tied to this task's fine-tuned memorisation: the same sweep on a model that was not fine-tuned on the forget documents could separate intrinsic factual storage from instruction-tuning artifacts.
  • The MIA jump at layers 12–14 suggests membership signals live in a low-rank subspace around that depth; probing activations there before and after unlearning could reveal where PII is encoded.
  • Because adaptive RMU scales its steering by activation norm, the apparent layer preference could partly reflect activation magnitudes growing with depth; re-running the sweep with per-layer norm normalisation would test whether content, not scale, drives the result.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper applies Adaptive Representation Misdirection Unlearning (Adaptive RMU) to the SemEval-2025 Task 4 unlearning benchmark, using the task-provided OLMo-1B and OLMo-7B models. On the 1B model, the authors sweep all 14 consecutive triples of decoder layers, report the task-aggregate, MIA, and MMLU components of the final score, and conclude that layers 12,13,14 are optimal. They also state without presenting measurements that layers 24,25,26 are ideal for the 7B model. The submission ranked fourth on both official leaderboards. The paper argues that later decoder layers are more effective targets for unlearning factual/PII knowledge, in contrast to earlier-layer targets in original RMU work, and it releases code on GitHub.

Significance. If the central claim were fully supported, this would be a useful empirical datapoint for layer-selective unlearning, with the 1B sweep being a genuinely systematic comparison and the fourth-place leaderboard finish providing external validation that the chosen configuration is competitive. The paper's release of code and its use of the task's official evaluation metric are strengths. However, the two-model claim in Section 6 is only directly supported for the 1B model; the 7B optimum is asserted rather than measured, and a displayed arithmetic inconsistency in Table 3 undermines confidence in the reported numbers. The layer conclusion is also a tuning result over the final metric, so its status as a mechanistic finding is weaker than the text suggests.

major comments (4)
  1. [Section 4.2 and Section 6] The paper states in Section 4.2 that experiments were conducted exclusively on the 1B parameter model, yet Section 6 asserts that the ideal layers for the 7B model are 24,25,26. No 7B layer sweep is reported anywhere. This is not a cosmetic omission: the strongest claim of the paper, as stated in Section 6, is a two-model conclusion, and the 7B half has no direct empirical support. The optimal 7B triple appears to be extrapolated from the 1B pattern (both around 75% of decoder depth), which is a reasonable hypothesis but not a measurement. I ask the authors to either report the 7B sweep (or at least a comparison of a few candidate triples on the 7B model) or to explicitly label the 7B recommendation as an extrapolation rather than an empirical result.
  2. [Table 3] The Adaptive RMU row in Table 3 reports a final score of 0.376, but the displayed components (task aggregate 0.387, MIA 0.872, MMLU 0.485) have arithmetic mean (0.387 + 0.872 + 0.485)/3 = 0.581, not 0.376. The other four rows in the table are consistent with the stated arithmetic-mean definition in Section 3.3. This inconsistency must be corrected, and the authors should double-check all reported final scores against their running code, since a single transcription error in a headline table raises doubts about other numbers.
  3. [Section 5.2 and Section 6] The headline claim that later layers are ideal is selected by sweeping all layer triples against the task's own final score. This makes the conclusion a post-hoc selection on the evaluation metric rather than a falsifiable prediction. The fourth-place leaderboard rank is an external check on the overall configuration, but it does not test whether the chosen layer triple is optimal. Please clarify in Section 6 that the 'ideal layers' conclusion is a tuning outcome on the development/evaluation split, and ideally report the ranking stability across the triples relative to the spread of final scores in Table 2.
  4. [Section 5.2 / Table 2] No variance, seeds, or repeated runs are reported. Each configuration appears to be evaluated once, and the paper describes randomized sampling during training. The layer sweep in Table 2 shows several close scores (e.g., 11,12,13 at 0.442 vs. 12,13,14 at 0.554 is less close, but the middle-layer cluster is dense), so without variance information the separation between neighboring triples may not be reliable. Please report run-to-run variability or at least state the number of runs and the seed handling.
minor comments (5)
  1. [Section 7] The conclusion expands RMU as 'Rank-One Model Update,' but the paper's own Section 4.1 defines RMU as Representation Misdirection Unlearning. This acronym expansion is inconsistent and should be corrected.
  2. [Section 4.1] Equation (3) is followed by 'Where α controls' with a capital W; please make the prose style consistent. Also, the sentence 'RMU updates model weights iteratively, focusing on layers ℓ−2, ℓ−1, and ℓ' is ambiguous about whether the loss is applied at one layer or three; please clarify.
  3. [Figure 2 caption / Appendix] The appendix text says 'middle layers achieve a balance ... but it remains more susceptible to MIA' while the figure caption says later layers are more robust to MIA. The narrative is clear, but the phrasing in the appendix could be sharpened to avoid the impression that middle layers are recommended.
  4. [References] Several references are incomplete or inconsistently formatted: the Robins entry is in all caps, the Huu-Tien et al. reference gives no venue, and the SemEval task description reference is listed as 'arXiv preprint' without an identifier. Please normalize all bibliography entries.
  5. [Table 2 formatting] A few rows in Table 2 have merged or missing spaces (e.g., '5,6,70.6370.357' in the extracted text, and '0.367 0.3620.2650.419' in row 8,9,10). Please ensure the table is typeset cleanly so each column is readable.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper's loss equations are quoted from prior work, and its layer conclusions are empirical sweep results checked against an external leaderboard.

full rationale

The paper applies Adaptive RMU (Huu-Tien et al., 2025) and RMU (Li et al., 2024); the loss functions in Eqs. (1)-(4) are explicitly attributed to those prior works, not derived here, so quoting them is not circular. The central empirical claim is the layer sweep in Section 5.2 and Table 2: 'we experiment with all possible combinations of three consecutive layers... This allows us to identify the most effective layer range.' Selecting 12,13,14 for the 1B model as the configuration with the highest final score is a tuning/selection result, not a prediction, and it is externally anchored by the team's 4th-place leaderboard finish on both model sizes. The unsupported assertion that the 7B ideal layers are 24,25,26, despite Section 4.2 stating experiments 'were conducted exclusively on the 1B parameter model,' is an evidentiary/rigor concern (missing 7B sweep), not a circularity: no equation or fitted parameter is being relabeled as a derivation. There is no self-citation chain, no uniqueness theorem imported from the authors, and no ansatz smuggled in via citation. Accordingly, no load-bearing step reduces to its own inputs.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The method is imported from prior work (RMU and Adaptive RMU), so the unlearning mechanism and its loss are axioms rather than derived. Layer selection is tuned against the task metric, making the headline finding metric-relative. The 7B layer recommendation is not backed by displayed experiments. No invented entities are introduced.

free parameters (5)
  • retain-loss weight alpha = not reported
    Equation (3) weights the retain loss; the paper gives no value or search range, yet the retention/unlearning trade-off depends on it.
  • adaptive scaling factor beta = not reported
    Equation (4) scales the random control vector by beta times the frozen activation norm; no value or selection method is given.
  • target layer triple (1B) = (12,13,14)
    Picked from a sweep of all consecutive triples (0,1,2) through (13,14,15) using the task's own final metric, so it is a fit to that metric.
  • target layer triple (7B) = (24,25,26)
    Stated in Section 6 but no 7B sweep is shown; if it comes from a hidden run it is a selected value, and if extrapolated it is an assumption.
  • optimization settings (learning rate, steps, batch size) = not reported
    No training schedule is disclosed, so the reproduced method would have to choose these; they materially affect the loss optimum.
assumptions (3)
  • domain assumption The Adaptive RMU loss, as defined in Equation (4), removes the target knowledge rather than merely making evaluation-based guesses fail.
    The paper relies on RMU and Adaptive RMU papers for this effect and does not independently verify that the forget-set activations are genuinely unavailable after unlearning.
  • domain assumption The SemEval Task 4 metric (harmonic mean of 12 regurgitation terms, MIA score, and MMLU) is a valid measure of unlearning quality.
    Layer selection and the 'ideal layers' conclusion are optimized against this metric, so the paper's central recommendation is conditioned on the metric being meaningful.
  • domain assumption The organizers' fine-tuned OLMo models and retain/forget splits capture the intended real-world unlearning scenario.
    The system takes the dataset and base models as given; any bias in those would propagate to the layer recommendation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Mr. Snuffleupagus at SemEval-2025 Task 4: Unlearning Factual Knowledge from LLMs Using Adaptive RMU." pith.science (2026). https://pith.science/paper/TGQHHKUI

@misc{pith2026250616548,
  author       = {Pith},
  title        = {Pith review of: Mr. Snuffleupagus at SemEval-2025 Task 4: Unlearning Factual Knowledge from LLMs Using Adaptive RMU},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TGQHHKUI}},
  note         = {Machine review of arXiv:2506.16548}
}
read the original abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language understanding and generation. However, their tendency to memorize training data raises concerns regarding privacy, copyright compliance, and security, particularly in cases involving Personally Identifiable Information (PII). Effective machine unlearning techniques are essential to mitigate these risks, yet existing methods remain underdeveloped for LLMs due to their open-ended output space. In this work, we apply the Adaptive Representation Misdirection Unlearning (RMU) technique to unlearn sensitive information from LLMs. Through extensive experiments, we analyze the effects of unlearning across different decoder layers to determine the most effective regions for sensitive information removal. Our technique ranked 4th on the official leaderboard of both 1B parameter and 7B parameter models.

Figures

Figures reproduced from arXiv: 2506.16548 by the authors.

Figure 1
Figure 1. RMU performs machine unlearning by optimizing a two-part loss: a forget term, which misdirects the [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Observed trends of the MMLU Score (a), Task Aggregate Score (b), MIA Score (c), and Final Score (d) [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 2 canonical work pages

  1. [1]

    Chundawat, Ayush K

    Vikram S. Chundawat, Ayush K. Tarun, Murari Mandal, and Mohan Kankanhalli. 2023. https://doi.org/10.1109/tifs.2023.3265506 Zero-shot machine unlearning . IEEE Transactions on Information Forensics and Security, 18:2345–2354

    arXiv 2023

  2. [2]

    Ronen Eldan and Mark Russinovich. 2023. http://arxiv.org/abs/2310.02238 Who's harry potter? approximate unlearning in llms

    arXiv 2023

  3. [3]

    Eoin Farrell, Yeu-Tong Lau, and Arthur Conmy. 2024. http://arxiv.org/abs/2410.19278 Applying sparse autoencoders to unlearn knowledge in language models

    arXiv 2024

  4. [4]

    Jonathan Frankle and Michael Carbin. 2019. https://openreview.net/forum?id=rJl-b3RcF7 The lottery ticket hypothesis: Finding sparse, trainable neural networks . In International Conference on Learning Representations

    2019

  5. [5]

    Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio

    Ian J. Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. 2015. http://arxiv.org/abs/1312.6211 An empirical investigation of catastrophic forgetting in gradient-based neural networks

    arXiv 2015

  6. [6]

    Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khyathi Raghavi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Morrison, Niklas Muennighoff, Aakan...

    arXiv 2024

  7. [7]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. http://dblp.uni-trier.de/db/conf/iclr/iclr2021.html#HendrycksBBZMSS21 Measuring massive multitask language understanding. In ICLR. OpenReview.net

    2021

  8. [8]

    Dang Huu-Tien, Trung-Tin Pham, Hoang Thanh-Tung, and Naoya Inoue. 2025. http://arxiv.org/abs/2408.06223 On effects of steering latent representation for large language model unlearning

    arXiv 2025

Show all 21 references
  1. [9]

    Ronald Kemker, Marc McClure, Angelina Abitino, Tyler Hayes, and Christopher Kanan. 2018. https://doi.org/10.1609/aaai.v32i1.11651 Measuring catastrophic forgetting in neural networks . Proceedings of the AAAI Conference on Artificial Intelligence, 32(1)

    2018 doi

  2. [10]

    Varshita Kolipaka, Akshit Sinha, Debangan Mishra, Sumit Kumar, Arvindh Arun, Shashwat Goel, and Ponnurangam Kumaraguru. 2024. http://arxiv.org/abs/2412.00789 A cognac shot to forget bad memories: Corrective unlearning in gnns

    2024 arXiv

  3. [11]

    Li, Ann-Kathrin Dombrowski, Shashwat Goel, Long Phan, Gabriel Mukobi, Nathan Helm-Burger, Rassin Lababidi, Lennart Justen, Andrew B

    Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D. Li, Ann-Kathrin Dombrowski, Shashwat Goel, Long Phan, Gabriel Mukobi, Nathan Helm-Burger, Rassin Lababidi, Lennart Justen, Andrew B. Liu, Michael Chen, Isabelle Barrass, Oliver Zhang,...

    2024 arXiv

  4. [12]

    Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. 2025. http://arxiv.org/abs/2308.08747 An empirical study of catastrophic forgetting in large language models during continual fine-tuning

    2025 arXiv

  5. [13]

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in gpt. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS '22, Red Hook, NY, USA. Curran Associates Inc

    2022

  6. [14]

    Anil Ramakrishna, Yixin Wan, Xiaomeng Jin, Kai-Wei Chang, Zhiqi Bu, Bhanukiran Vinzamuri, Volkan Cevher, Mingyi Hong, and Rahul Gupta. 2025 a . Lume: Llm unlearning with multitask evaluations. arXiv preprint arXiv:2502.15097

    2025 arXiv

  7. [15]

    Anil Ramakrishna, Yixin Wan, Xiaomeng Jin, Kai-Wei Chang, Zhiqi Bu, Bhanukiran Vinzamuri, Volkan Cevher, Mingyi Hong, and Rahul Gupta. 2025 b . Semeval-2025 task 4: Unlearning sensitive content from large language models. arXiv preprint

    2025

  8. [16]

    ANTHONY ROBINS. 1995. https://doi.org/10.1080/09540099550039318 Catastrophic forgetting, rehearsal and pseudorehearsal . Connection Science, 7(2):123--146

    1995 doi

  9. [17]

    Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. 2017. https://doi.org/10.1109/SP.2017.41 Membership Inference Attacks Against Machine Learning Models . In 2017 IEEE Symposium on Security and Privacy (SP), pages 3--18, Los Alamitos, CA, USA. IEEE Computer Society

    2017 doi

  10. [18]

    Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.632 Editing large language models: Problems, methods, and opportunities . In Proceedings of the 2023 Conference on Empiri...

    2023 doi

  11. [19]

    Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. 2024. http://arxiv.org/abs/2404.05868 Negative preference optimization: From catastrophic collapse to effective unlearning

    2024 arXiv

  12. [20]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...

  13. [21]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.