Representation-Aware Unlearning via Activation Signatures: From Suppression to Entity-Signature Erasure

Farig Sadeque; Jareen Tasneem Khondaker; K. M. Shadman Wadith; Md. Rezaur Rahman Bhuiyan; Md. Sameer Sakib; Nazia Tasnim; Syed Naveed Mahmood; Tasfia Zaman

REVIEW 4 major objections 5 minor 22 references

This paper claims that unlearning a subject from an LLM requires erasing its internal activation signature—not just suppressing its outputs—and that doing so achieves near-oracle forgetting at oracle-level utility.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 10:14 UTC pith:OSVM2HBC

load-bearing objection The pipeline is solid and the cross-family evaluation is useful, but the paper's headline claim of internal attenuation rests on a metric the training loss directly optimizes. the 4 major comments →

arxiv 2601.10566 v5 pith:OSVM2HBC submitted 2026-01-15 cs.CL cs.LG

Representation-Aware Unlearning via Activation Signatures: From Suppression to Entity-Signature Erasure

Syed Naveed Mahmood , Md. Rezaur Rahman Bhuiyan , Tasfia Zaman , Jareen Tasneem Khondaker , Md. Sameer Sakib , K. M. Shadman Wadith , Nazia Tasnim , Farig Sadeque This is my paper

classification cs.CL cs.LG

keywords machine unlearningLLM unlearningactivation signaturesrepresentation-level erasureKnowledge Suppression CapsuleLoRA distillationTOFU benchmarkobfuscation vs erasure

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that LLM unlearning should be done at the level of internal representations, not just outputs. It introduces the Knowledge Immunization Framework, which mines a subject-specific activation signature from MLP hidden states, suppresses that direction during inference through a gated capsule, and distills the suppressed behavior into a LoRA adapter. The central empirical claim is that this achieves near-oracle erasure on TOFU forget10 (FQ ≈ 0.99 vs. 1.00 for a retrained oracle) while keeping utility at oracle level (MU = 0.62), with utility drift under 3% in most foundation models. The paper also claims that a dual metric—surface leakage plus early-token probability of the target name—can tell true erasure from obfuscation, and that reasoning-oriented models need enough scale to erase rather than merely suppress. A sympathetic reader would care because, if true, this breaks the usual stability–erasure tradeoff and gives a mechanism-level rather than surface-level criterion for forgetting.

Core claim

KIF's claim, stated on its own terms, is that unlearning succeeds only when the subject's internal representation, not just its verbal behavior, has been attenuated. The framework first extracts per-layer MLP activation signatures for each subject using a contrastive mean-difference between on-topic prompts and synthetic Gaussian negatives, with Cohen's d effect sizes exceeding the large-effect threshold (peak d = 3.66 on Llama-3.1-8B). It then builds Knowledge Suppression Capsules that decompose each hidden state into components parallel and orthogonal to the signature, scaling down only the parallel component via a gated factor, and distills the resulting behavior into a global LoRA adapte

What carries the argument

The central mechanism is the Knowledge Suppression Capsule paired with the UPU Loop. A unit-norm signature vector, mined as the mean difference between on-topic and synthetic-noise MLP activations, defines a subject-specific direction in representation space. At chosen high-salience layers, the capsule decomposes each hidden state into components parallel and orthogonal to that direction, scales down only the parallel component via a dynamic gate α·σ(k(z − τ)), and leaves the orthogonal component untouched to preserve utility. The UPU Loop then distills the suppressed behavior into a global LoRA adapter using a composite loss that combines preference-style supervision, factual and name-token

Load-bearing premise

The paper treats EL10—an early-token output probability—as the evidence for latent internal attenuation, and never directly measures hidden states in the unlearned model; if EL10 drops only because of decoding-time name-token suppression, the central claim collapses.

What would settle it

Measure the residual hidden states of the unlearned model on subject prompts and train a linear probe or recompute the projection onto the pre-unlearning signature vector. If the probe still separates subject vs. non-subject activations with high accuracy, or if the mean parallel projection is unchanged while EL10 has dropped, the central claim of internal attenuation is falsified—EL10 would then be a decoding artifact of the name-token unlikelihood term rather than evidence of erasure.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

On TOFU forget10, KIF reports FQ = 0.99 (oracle retrained: 1.00) and MU = 0.62 (matching the oracle), implying the method lands almost exactly at the theoretical erasure-utility frontier.
RWKU probes (4-subject, evaluation-only) show forget-set recall dropping across FB, QA, and adversarial-attack probes, meaning the suppression is subject-level rather than template-level.
The SMR/EL10 taxonomy predicts that models with low leakage but EL10 > 1 are obfuscating; unlearning evaluations that only check outputs will misclassify these as successful.
Standard models (Llama, Mistral, 3B–8B) mostly achieve Type I erasure with utility drift below 3%, whereas reasoning-prior models require larger scale to escape Type II/III behavior.
Ablations show that both name-token unlikelihood and generic unlikelihood are required; removing either collapses the method into instability or obfuscation, indicating the representation-level suppression and decoding-time losses work jointly.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

A direct test the paper does not run: re-fine-tune the erased model on the retained corpus and measure how quickly the subject's name becomes generateable again; if recovery is fast, the signature direction was suppressed rather than destroyed.
If the capacity U-curve is causal, then training reasoning models to keep subject knowledge in separable residual subspaces should let KIF achieve true erasure at smaller scales than 14B; this is a testable architectural prediction.
Applying the SMR/EL10 protocol to earlier unlearning methods would likely re-classify many reported successes as Type II obfuscation, since the paper shows that loss-level supervision alone defaults to suppression—but that re-scoring is our extrapolation, not the paper's.
The latent-pruning effect suggests a measurable signature: hidden-state similarity between benign and forget-topic prompts should decrease after erasure, and task-conflict metrics should improve; this can be checked without new training.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

The pipeline is solid and the cross-family evaluation is useful, but the paper's headline claim of internal attenuation rests on a metric the training loss directly optimizes.

read the letter

Should you read this paper? As a case study, yes: it shows a good empirical pipeline outrunning its own evidence. The engineering is real. KIF mines activation signatures, suppresses them with a gated residual intervention, and distills the suppression into LoRA. That pipeline is documented carefully. But the central claim — representation-level erasure — is not supported by the measurement. The 'internal' metric, EL10, is early-token probability of the target name. Equation 2 explicitly penalizes that quantity during training. So low EL10 is partly by construction, not independent evidence of hidden-state attenuation. That is circular.

What is new: the specific integration of signature mining, gated suppression, and LoRA distillation is not in the prior literature. The dual-metric protocol (SMR + EL10) is a useful diagnostic framing, and the cross-family U-curve in reasoning models (3B unstable, 8B obfuscation, 14B/32B erasure) is an interesting empirical pattern, even if descriptive. The Real-World Entity Dataset is a practical contribution, and the ablations show the loss components matter.

Where the paper falls short: (1) No post-unlearning hidden-state measurements. The obvious check would be projecting activations onto the mined signature v at the capsule layers. That is the missing experiment. (2) The signature extraction uses Gaussian noise as negatives. That likely inflates Cohen's d and does not establish a subject-specific direction. (3) The RWKU probe leaves about 60% FB recall and 36% QA recall, which contradicts the near-oracle narrative; the paper does not reconcile this. (4) The TOFU run is underspecified: baselines are borrowed from Fan et al. and KIF's exact setup is not fully reported. (5) The abstract says ERUF, the body says KIF; that is sloppy.

What is good: utility preservation is convincing — drift under 5% on most benchmarks — and the limitations section honestly says there is no formal guarantee of deletion. The writing is clear, and the breadth across 3B–32B models is valuable.

Who is this for? LLM unlearning researchers and anyone interested in metric circularity. I would bring it to a reading group, but I would not cite it until the representation-level claim is backed with hidden-state evidence.

For peer review: send it out, expecting major revision. The central claim needs direct internal measurement and a proper negative set. With those, it could be a solid contribution.

Referee Report

4 major / 5 minor

Summary. The paper proposes Knowledge Immunization Framework (KIF), a three-stage unlearning pipeline: (i) mine subject-specific activation signatures from a custom real-world entity dataset via contrastive analysis of MLP activations, (ii) suppress the parallel component of activations during inference using gated Knowledge Suppression Capsules, and (iii) distill the suppressed behavior into a global LoRA adapter with a composite loss (DPO, unlikelihoods, KL, EWC). The authors claim that this representation-aware approach achieves 'near-oracle erasure' on TOFU forget10 (FQ = 0.99, MU = 0.62) and, via a dual-metric evaluation (SMR and EL10), is the only method jointly achieving surface suppression, internal attenuation, and utility preservation. They further report a capacity-dependent U-curve across reasoning-prior models (Qwen, DeepSeek) and show cross-benchmark generalization on a 4-subject RWKU subset.

Significance. If the representation-level erasure claim were validated, KIF would be a significant contribution: it introduces a concrete, PEFT-based mechanism for targeting subject-level internal representations, and the combination of near-oracle TOFU forget quality with oracle-level utility would break the erasure–utility tradeoff reported in prior work. The paper also ships a new real-world entity prompt dataset and a dual-metric diagnostic protocol that could be useful to the community. The RWKU generalization probe and the ablation study are valuable pieces of evidence. However, the central claim currently rests on a metric (EL10) that is directly minimized by the training loss, and no hidden-state evidence is provided; the signature mining uses Gaussian-noise negatives that trivialize the localization results. These issues must be resolved before the paper's central claims can be accepted. The TOFU methodology is also under-specified. The topic is timely and important, but the evidence in this version is not yet convincing.

major comments (4)

[§4 'Evaluation Metrics' and §3.4, Eq. (2)] The 'latent trace' metric EL10 is defined as early-token probability mass on the target name (Sec. 4). Eq. (2), the Name-Token Unlikelihood loss, directly penalizes the aggregate probability mass of name tokens V_name in suppressed responses. Low EL10 is therefore a direct consequence of the training objective, not independent evidence of hidden-state attenuation. No experiment measures post-unlearning projection of activations onto the mined signature v or any other hidden-state quantity. The central claim of 'internal attenuation' (Type I erasure) and the assertion that KIF is 'the only method' jointly achieving surface suppression, internal attenuation, and utility preservation are unsupported as stated. Please add direct hidden-state measurements, e.g., the norm of the projection onto v before/after unlearning, or a linear-probe accuracy test on the unlearned model's activations.
[§3.2 and Figure 3] The signature extraction compares positive activations against 'Synthetic Negatives generated via Gaussian noise.' A real activation vector and a random Gaussian vector are trivially separable, so the reported Cohen's d values (peaking at 3.66) do not establish subject-specific separability or a meaningful subject signature. Use real off-topic/neutral prompt activations as the negative set and report separability against those. This is load-bearing because the entire localisation stage and the capsule placement depend on the validity of these signatures.
[§4.2 and §3.1/3.4] The TOFU forget10 result (Table 3) is the paper's strongest standardized benchmark, but the manuscript never explains how the three-stage KIF pipeline transfers to TOFU's synthetic authors. The signature-mining stage uses the Real-World Entity Dataset of 11 musicians; TOFU contains 10 fictional authors with QA pairs. How are subject signatures mined for TOFU subjects, which layers are chosen, and how is the UPU loop run in this setting? Without this description, the FQ=0.99 result is not reproducible and cannot be attributed to the representation-aware mechanism. Add a detailed TOFU-specific methodology subsection (or appendix) with the exact data construction and hyperparameters.
[§4 'Evaluation Metrics' and Table 5] The Type I/II/III classification hinges on thresholds (SMR ≤ 5%, EL10 < 1) and the EL10 threshold of 1.0, but EL10 is never precisely defined (is it a raw probability mass, a ratio, or a relative score?). The threshold of 1.0 is not justified. Moreover, the ablation in Table 5 removes output-level loss terms (NT-UL, Gen-UL) but never removes or ablates the capsule/signature intervention itself; Gen-UL is an output-level unlikelihood, not a 'representation-level constraint.' Thus the contribution of the representation-aware component to the reported outcomes is not isolated. Please define EL10 formally, justify the threshold, and include an ablation that disables the capsule or uses a random direction instead of the mined signature.

minor comments (5)

[Abstract vs. §1] The abstract refers to the framework as 'ERUF' while the main text uses 'KIF.' Standardize the acronym.
[Table 4] 'FM Loss' is not defined anywhere; clarify what it measures and how it is computed from the MIA set.
[§3.2] The symbol d is used both for the mean difference in Eq. (4?) and for Cohen's d. Use distinct notation to avoid ambiguity.
[§4.4] The 'latent pruning' hypothesis is explicitly speculative, which is fine, but the supporting observations (e.g., HellaSwag +0.16, TruthfulQA +0.51) are within noise for a single seed; temper the language.
[§B.4] Hyperparameters are given for the custom dataset (Table 12), but not for the TOFU runs. Since TOFU is used for the headline result, include these values or state they are identical.

Circularity Check

1 steps flagged

Internal-attenuation claim rests on EL10, an output-level name-token probability metric that Eq. (2)'s Name-Token Unlikelihood directly minimizes; Type I 'true erasure' is then defined by this same metric, so the central representation-level claim is partly by construction.

specific steps

fitted input called prediction [§3.4 Eq. (2); §4 'Evaluation Metrics' and 'Mechanism States'; Table 5 ablations]
"Name-Token Unlikelihood ( L_NT-UL), which unlike the standard version, is used to penalize the aggregate probability mass of subject name tokens V_name within suppressed responses to prevent soft leakage (Eq. 2). ... 'Latent Trace / EL10 Ratio' summarizes early-step probability mass on the target name; larger values indicate stronger early activation. ... Type I (True Erasure): SMR ≤ϵ and EL10 <1 - internal representations have been attenuated; the model does not possess residual capability."

The paper's evidence for 'internal attenuation' is low EL10, but EL10 is defined as early-token probability mass on the target name, which is exactly the quantity minimized by L_NT-UL in Eq. (2). Thus low EL10 is a direct training-signal consequence, not independent evidence of hidden-state attenuation. The paper's own ablation confirms this: removing NT-UL raises EL10 from 0.066 to 0.275, and removing Gen-UL gives EL10=1.098, showing the metric tracks token-level unlikelihood losses. Type I ('True Erasure: internal representations have been attenuated') is then defined by this same output-level metric, so the central claim that KIF achieves representation-level erasure rather than surface suppression reduces, by construction, to 'the token-level loss lowered name-token output probability.

full rationale

The TOFU forget10 result (FQ=0.99, MU=0.62) is measured against an external benchmark and is not circular; that is why the score is not higher. The circularity is concentrated in the paper's representation-level claim: the 'latent trace' diagnostic EL10 is an output-level name-token probability metric, and Eq. (2)'s Name-Token Unlikelihood directly penalizes that same quantity. Using low EL10 to define Type I 'true erasure' (internal representations attenuated) therefore makes the internal-attenuation leg of the central claim partly by construction. The paper itself is careful to call the evidence 'operational' and disclaims formal guarantees, but it still presents EL10 as evidence of latent trace persistence rather than as a decoding-time artifact of the training loss. No hidden-state geometry of the unlearned model is measured, so the distinction between erasure and obfuscation is not independently established. This is a partial circularity affecting the mechanism-level claim, not the external benchmark results.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

The paper's central claim depends on a fitted signature vector, hand-chosen gate hyperparameters, and six loss weights; it also relies on the unvalidated assumption that EL10 measures internal state. No new physical entities are introduced.

free parameters (6)

per-subject signature vector v = unit-norm mean activation direction from contrastive analysis
Central intervention target; fitted from subject prompts vs Gaussian noise, so its specificity against real off-topic prompts is unvalidated.
gate scale α = -1 (initialized), parameter
Suppression strength; Appendix B.4 lists α as a capsule parameter (Table 12).
gate threshold τ = 3.0
Z-score trigger threshold; Table 12.
gate gain k = 1.6
Empirically chosen gain factor; Table 12.
loss weights λ_UL, λ_NTUL, λ_KL, λ_EWC = 0.03, 0.02, 0.03, 5.0
Composite objective Eq. 3; tuned empirically.
LoRA rank/alpha/dropout = r=4, α=8, dropout=0.05
Adapter config from Table 12.

axioms (5)

domain assumption Factual associations are stored as localized, linear relations within MLP weights
Section 3.2, citing Meng et al. 2023; underpins the decision to intervene in MLP subspaces.
ad hoc to paper Synthetic Gaussian noise approximates off-manifold activations
Section 3.2 and Limitations; the contrastive signature uses Gaussian noise as the negative set, inflating Cohen's d.
domain assumption EL10 ratio is a valid operationalization of latent trace persistence
Section 4 Evaluation Metrics; early-step target-name probability is labeled 'latent trace' without hidden-state validation; overlaps with NT-UL objective (Eq. 2).
domain assumption The UPU Loop distillation preserves the capsule's suppression in the LoRA-merged model
Section 3.4; no post-hoc hidden-state check that suppression persists after LoRA merge.
ad hoc to paper Mechanism-state thresholds (SMR≤5%, EL10=1) meaningfully separate erasure, obfuscation, and instability
Section 4, 'descriptive heuristic inspired by Sun et al. (2025)'; thresholds chosen without external validation.

pith-pipeline@v1.3.0-alltime-deepseek · 17673 in / 13881 out tokens · 144027 ms · 2026-08-03T10:14:09.262597+00:00 · methodology

0 comments

read the original abstract

Entity-level unlearning is usually evaluated by what a model says: whether it stops naming the target, refuses a query, or shifts a Truth Ratio distribution. These output-level tests, however, do not show whether a subject's internal representation has been attenuated. We introduce the Entity Representation Unlearning Framework (ERUF), a representation-aware framework that mines subject-specific activation signatures, suppresses the corresponding activation direction, and distills the behavior into LoRA parameters. Among evaluated baselines, ERUF is the only method that jointly achieves surface-level suppression, internal attenuation, and utility preservation. On TOFU forget10, ERUF achieves FQ = 0.99 and MU = 0.62, matching reported oracle utility while approaching oracle forget quality. Across most standard foundation-model settings, ERUF maintains low leakage and low internal target activation, with SMR between 0.00% and 1.10%, EL10 below 0.06, and utility drift below 3%. On Llama-3.1-8B, adversarial entity recovery falls from 63.89% to 20.15%, while name-agnostic recovery decreases by 72.7% to 77.4%. Joint surface/internal diagnostics further reveal scale-dependent behavior in reasoning-prior models that surface metrics alone would miss. We interpret these results as operational evidence of representation-level attenuation, not as a formal guarantee of irreversible deletion.

Figures

Figures reproduced from arXiv: 2601.10566 by Farig Sadeque, Jareen Tasneem Khondaker, K. M. Shadman Wadith, Md. Rezaur Rahman Bhuiyan, Md. Sameer Sakib, Nazia Tasnim, Syed Naveed Mahmood, Tasfia Zaman.

**Figure 1.** Figure 1: Comparison of approaches. Standard unlearning methods may produce obfuscation, hiding answers without removing latent knowledge, whereas KIF targets the internal signature to support durable erasure. databases, where records can be deleted explicitly, LLMs store training information in distributed parameters (Naveed et al., 2025), which makes targeted removal difficult. LLM unlearning aims to remove th… view at source ↗

**Figure 2.** Figure 2: Knowledge Immunization Framework (KIF) pipeline. Entity prompts from the Real-World Entity Dataset (3.1) probe MLP activations to mine subject-specific signatures via contrastive analysis. These signatures instantiate Knowledge Suppression Capsules (3.3) at high-salience inference layers, and the UPU Loop (3.4) distills the resulting suppressed behavior into a global LoRA adapter for durable parameter-leve… view at source ↗

**Figure 3.** Figure 3: Mean Cohen’s d across transformer layers on Llama-3.1-8B. Effect sizes indicate strong subjectspecific activation separability in mid to final layers. From the collected corpus of layer-wise activations, we extract subject-specific signatures through a contrastive framework. First, we normalize variable activation shapes to a common 1D representation via token-wise averaging and standardization. We the… view at source ↗

**Figure 4.** Figure 4: Capsule based Suppression Pipeline. The capsule decomposes activation into parallel and orthogonal components relative to subject signature. It applies a gated suppression factor only to the parallel component and passes on the modified activation to the subsequent layer. at a mean d=3.66 in mid-to-late layers, confirming strongly separable subject-specific activation signatures and directly informing lay… view at source ↗

**Figure 5.** Figure 5: EL10 and surface leakage across model families and scales. Most standard models maintain low internal activation, with Llama 3B as a notable exception, whereas reasoning models display a capacitydependent U-curve. Benchmark Metric Pre (Base) Post (Adapter) Delta ARC-Challenge acc_char 76.88 76.62 -0.26 OpenBookQA acc_char 74.80 74.40 -0.40 HellaSwag acc 76.68 76.84 +0.16 BoolQ acc 78.93 74.46 -4.46 PIQA a… view at source ↗

**Figure 6.** Figure 6: Distribution of extracted knowledge triples per subject. The dataset exhibits class imbalance, with Beyoncé (18.0%) and Taylor Swift (15.7%) having the highest representation, compared to minority classes like Queen (2.0%). sequence of length T and batch size B, the hidden states propagate as X(ℓ+1) = B(ℓ) (X(ℓ) ). Within each block, we target the gated Feed-Forward Network (MLP), e.g. in the SwiGLU gated… view at source ↗

**Figure 7.** Figure 7: disaggregates the mean layer-wise trend from [PITH_FULL_IMAGE:figures/full_fig_p016_7.png] view at source ↗

discussion (0)

Reference graph

Works this paper leans on

22 extracted references · 8 linked inside Pith

[1]

I’m researching {subject}, what was their {predicate}?
[2]

Do you know {subject}’s {pred- icate}?
[3]

What was {subject}’s {predi- cate}?
[6]

Jie Ren, Yue Xing, Yingqian Cui, Charu C

Direct preference optimization: Your lan- guage model is secretly a reward model.Preprint, arXiv:2305.18290. Jie Ren, Yue Xing, Yingqian Cui, Charu C. Aggarwal, and Hui Liu. 2025. Sok: Machine unlearning for large language models.Preprint, arXiv:2506.09227. Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner. 2024. Ste...

Pith/arXiv arXiv 2025
[7]

Niklas Stoehr, Kevin Du, Vésteinn Snæbjarnarson, Robert West, Ryan Cotterell, and Aaron Schein

Muse: Machine unlearning six-way evaluation for language models.Preprint, arXiv:2407.06460. Niklas Stoehr, Kevin Du, Vésteinn Snæbjarnarson, Robert West, Ryan Cotterell, and Aaron Schein. 2024. Activation scaling for steering and interpreting lan- guage models. InFindings of the Association for Computational Linguistics: EMNLP 2024, pages 8189–8200, Miami...

Pith/arXiv arXiv 2024
[8]

InFindings of the Association for Computational Linguistics: EMNLP 2025, pages 18782–18794, Suzhou, China

Lock on target! precision unlearning via directional control. InFindings of the Association for Computational Linguistics: EMNLP 2025, pages 18782–18794, Suzhou, China. Association for Com- putational Linguistics. 11 Xinwei Wu, Junzhuo Li, Minghui Xu, Weilong Dong, Shuangzhi Wu, Chao Bian, and Deyi Xiong. 2023. DEPN: Detecting and editing privacy neurons ...

Pith/arXiv arXiv 2025
[9]

Andy Zou, Zifan Wang, J

Weak-to-strong jailbreaking on large language models.ArXiv, abs/2401.17256. Andy Zou, Zifan Wang, J. Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adver- sarial attacks on aligned language models.ArXiv, abs/2307.15043. 12 A Dataset Construction We construct a systematically designed dataset of controlled prompts grounded in verifiable...

Pith/arXiv arXiv 2023
[13]

For my project on {subject}, I need to know their {predicate}
[14]

Give me information about {sub- ject}, specifically their {predicate}
[15]

Is it true that {subject}’s {predi- cate} was {object}?
[16]

Someone told me about {sub- ject}’s {predicate}, what do you know about it?
[17]

If I want to understand {subject}’s history, what should I know about their {predicate}?

Verify for me: {subject}’s {pred- icate} is {object}? Category Reasoning Misleading Prompt 1. If I want to understand {subject}’s history, what should I know about their {predicate}?
[18]

What does {subject}’s {predicate} tell us about their life?
[19]

How did {subject}’s {predicate} affect their career?
[20]

Isn’t it true that {subject}’s {predicate} was {wrong_object}?
[21]

My friend said {subject}’s {predicate} was {wrong_object}, is that accurate?
[22]

Verify this for me: {subject}’s {predicate} is {wrong_object}? Table 9: Prompt templates categorized by interaction type, spanning both columns for readability. Beyoncé Taylor Swift Kanye West Katy Perry Ed Sheeran Ariana Grande Arijit SinghMichael Jackson Drake (musician) Eminem Queen (band) Figure 6:Distribution of extracted knowledge triples per subjec...

2024
[2014]

v agencia es- pañola de protección de datos (aepd) and mario costeja gonzález (case c-131/12)

Google spain sl and google inc. v agencia es- pañola de protección de datos (aepd) and mario costeja gonzález (case c-131/12). Judgment of the Court (Grand Chamber), 13 May 2014. Tomer Ashuach, Martin Tutek, and Yonatan Belinkov

2014
[2017]

Bo Li, Qinghua Zhao, and Lijie Wen

Overcoming catastrophic forgetting in neural networks.Proceedings of the National Academy of Sciences, 114(13):3521–3526. Bo Li, Qinghua Zhao, and Lijie Wen. 2024. Rome: Memorization insights from text, logits and represen- tation.arXiv preprint arXiv:2403.00510. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. TruthfulQA: Measuring how models mimic hu...

Pith/arXiv arXiv 2024
[2023]

InThe Eleventh International Confer- ence on Learning Representations

Quantifying memorization across neural lan- guage models. InThe Eleventh International Confer- ence on Learning Representations. Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert-V oss, Katherine 9 Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, and 1 others. 2021. Extracting training data from large language mode...

Pith/arXiv arXiv 2021
[2024]

Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, and 1 others

Deepseek llm: Scaling open-source lan- guage models with long-termism.arXiv preprint arXiv:2401.02954. Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, and 1 others. 2020. Piqa: Reasoning about physical commonsense in natural language. InProceedings of the AAAI conference on artificial intelligence, vol- ume 34, pages 7432–7439. Lucas Bourtoule, Var...

Pith/arXiv arXiv 2020
[2025]

In Findings of the Association for Computational Lin- guistics: ACL 2025

Unlearning sensitive information in language models via rank editing in the vocabulary space. In Findings of the Association for Computational Lin- guistics: ACL 2025. Association for Computational Linguistics. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, and 1 others. 2023. Qwen technical re...

Pith/arXiv arXiv 2025

[1] [1]

I’m researching {subject}, what was their {predicate}?

[2] [2]

Do you know {subject}’s {pred- icate}?

[3] [3]

What was {subject}’s {predi- cate}?

[4] [6]

Jie Ren, Yue Xing, Yingqian Cui, Charu C

Direct preference optimization: Your lan- guage model is secretly a reward model.Preprint, arXiv:2305.18290. Jie Ren, Yue Xing, Yingqian Cui, Charu C. Aggarwal, and Hui Liu. 2025. Sok: Machine unlearning for large language models.Preprint, arXiv:2506.09227. Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner. 2024. Ste...

Pith/arXiv arXiv 2025

[5] [7]

Niklas Stoehr, Kevin Du, Vésteinn Snæbjarnarson, Robert West, Ryan Cotterell, and Aaron Schein

Muse: Machine unlearning six-way evaluation for language models.Preprint, arXiv:2407.06460. Niklas Stoehr, Kevin Du, Vésteinn Snæbjarnarson, Robert West, Ryan Cotterell, and Aaron Schein. 2024. Activation scaling for steering and interpreting lan- guage models. InFindings of the Association for Computational Linguistics: EMNLP 2024, pages 8189–8200, Miami...

Pith/arXiv arXiv 2024

[6] [8]

InFindings of the Association for Computational Linguistics: EMNLP 2025, pages 18782–18794, Suzhou, China

Lock on target! precision unlearning via directional control. InFindings of the Association for Computational Linguistics: EMNLP 2025, pages 18782–18794, Suzhou, China. Association for Com- putational Linguistics. 11 Xinwei Wu, Junzhuo Li, Minghui Xu, Weilong Dong, Shuangzhi Wu, Chao Bian, and Deyi Xiong. 2023. DEPN: Detecting and editing privacy neurons ...

Pith/arXiv arXiv 2025

[7] [9]

Andy Zou, Zifan Wang, J

Weak-to-strong jailbreaking on large language models.ArXiv, abs/2401.17256. Andy Zou, Zifan Wang, J. Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adver- sarial attacks on aligned language models.ArXiv, abs/2307.15043. 12 A Dataset Construction We construct a systematically designed dataset of controlled prompts grounded in verifiable...

Pith/arXiv arXiv 2023

[8] [13]

For my project on {subject}, I need to know their {predicate}

[9] [14]

Give me information about {sub- ject}, specifically their {predicate}

[10] [15]

Is it true that {subject}’s {predi- cate} was {object}?

[11] [16]

Someone told me about {sub- ject}’s {predicate}, what do you know about it?

[12] [17]

If I want to understand {subject}’s history, what should I know about their {predicate}?

Verify for me: {subject}’s {pred- icate} is {object}? Category Reasoning Misleading Prompt 1. If I want to understand {subject}’s history, what should I know about their {predicate}?

[13] [18]

What does {subject}’s {predicate} tell us about their life?

[14] [19]

How did {subject}’s {predicate} affect their career?

[15] [20]

Isn’t it true that {subject}’s {predicate} was {wrong_object}?

[16] [21]

My friend said {subject}’s {predicate} was {wrong_object}, is that accurate?

[17] [22]

Verify this for me: {subject}’s {predicate} is {wrong_object}? Table 9: Prompt templates categorized by interaction type, spanning both columns for readability. Beyoncé Taylor Swift Kanye West Katy Perry Ed Sheeran Ariana Grande Arijit SinghMichael Jackson Drake (musician) Eminem Queen (band) Figure 6:Distribution of extracted knowledge triples per subjec...

2024

[18] [2014]

v agencia es- pañola de protección de datos (aepd) and mario costeja gonzález (case c-131/12)

Google spain sl and google inc. v agencia es- pañola de protección de datos (aepd) and mario costeja gonzález (case c-131/12). Judgment of the Court (Grand Chamber), 13 May 2014. Tomer Ashuach, Martin Tutek, and Yonatan Belinkov

2014

[19] [2017]

Bo Li, Qinghua Zhao, and Lijie Wen

Overcoming catastrophic forgetting in neural networks.Proceedings of the National Academy of Sciences, 114(13):3521–3526. Bo Li, Qinghua Zhao, and Lijie Wen. 2024. Rome: Memorization insights from text, logits and represen- tation.arXiv preprint arXiv:2403.00510. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. TruthfulQA: Measuring how models mimic hu...

Pith/arXiv arXiv 2024

[20] [2023]

InThe Eleventh International Confer- ence on Learning Representations

Quantifying memorization across neural lan- guage models. InThe Eleventh International Confer- ence on Learning Representations. Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert-V oss, Katherine 9 Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, and 1 others. 2021. Extracting training data from large language mode...

Pith/arXiv arXiv 2021

[21] [2024]

Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, and 1 others

Deepseek llm: Scaling open-source lan- guage models with long-termism.arXiv preprint arXiv:2401.02954. Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, and 1 others. 2020. Piqa: Reasoning about physical commonsense in natural language. InProceedings of the AAAI conference on artificial intelligence, vol- ume 34, pages 7432–7439. Lucas Bourtoule, Var...

Pith/arXiv arXiv 2020

[22] [2025]

In Findings of the Association for Computational Lin- guistics: ACL 2025

Unlearning sensitive information in language models via rank editing in the vocabulary space. In Findings of the Association for Computational Lin- guistics: ACL 2025. Association for Computational Linguistics. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, and 1 others. 2023. Qwen technical re...

Pith/arXiv arXiv 2025