REVIEW 5 major objections 4 minor 1 cited by
Targeted Angular Reversal of Weights (TARS) for Knowledge Removal in Large Language Models
T0 review · 5 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read TARS erases a named concept from an LLM by reversing a single feedforward weight vector, dropping target-token probability to 0.00 while preserving general performance.
desk verdict TARS is a novel weight-editing method with a plausible mechanism, but the evidence only shows token suppression under tuned prompts, not the broad concept removal claimed. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the targeting vector $\mathbf{v}_{\mathrm{target}}$, a 4096-dimensional vector in the model's internal representation space that triggers the target token with probability at least 0.95. It is built by averaging noise-perturbed versions of a prompt-derived approximate vector that pass the 0.95 threshold; this is meant to isolate the concept-specific direction from prompt artifacts. The method then computes cosine similarity $\mathrm{SC}(\mathbf{w}_i^{(\ell)}, \mathbf{v}_{\mathrm{target}})$ between this vector and every weight vector in the up- and gate-projection layers of Llama 3.1 8B, and replaces every weight above a threshold $\theta$ with the reversed, 3-norm-normalized targeting vector. The mechanism exploits the assumption that residual connections preserve a shared representation space across layers, so a weight vector with high alignment to the concept vector is what propagates the concept; reversing it turns the element-wise product negative and suppresses activation.
What would settle it
Measure target-token probability for a concept after a single TARS edit across a broad set of paraphrased descriptions and languages where the model may not use an English latent space; if the probability returns above zero for any reasonable paraphrase, the claim that the targeting vector isolates the concept fails.
Extended reading notes
Core claim
The central claim is that a concept in an LLM can be 'repelled' by replacing a small number of feedforward weight vectors with the negative of a normalized concept-targeting vector. The targeting vector is built in two steps: a long descriptive prompt is passed through the model and the final hidden state becomes an approximate concept vector; Gaussian noise is added and only variants that still produce the concept token with probability at least 0.95 are averaged to give a refined targeting vector. Cosine similarity between this vector and all 917,504 weight vectors of the gate and up projections picks the candidates, and the most similar ones are overwritten with $-\mathbf{v}_{\mathrm{target}}/\|\mathbf{v}_{\mathrm{target}}\|_3$. With one edit for 'Sherlock', four for 'Saturn', and three for 'Voldemort', the target token probability falls to 0.00 and completions become generic or factually wrong, which the authors take as evidence that the representation itself, not a prompt-specific path, has been suppressed.
Load-bearing premise
The load-bearing premise is that the refined targeting vector produced by averaging high-probability noisy variants is a stable representation of the concept itself; if it only captures the specific prompt's phrasing, the edited weights will not suppress the concept under other prompts, languages, or contexts.
Editorial extensions
If this is right
- After a single TARS edit, the probability that Llama 3.1 8B completes a Sherlock Holmes description with 'Sherlock' drops from 0.41 to 0.00; the model then gives generic or false completions rather than random text.
- Because the edit targets the internal representation, the model also fails the reverse direction: given 'Sherlock Holmes is', it no longer produces an accurate description.
- An English-targeted removal of 'dog' also drives the probabilities of French 'chien' and German 'Hund' to 0.00, so cross-lingual retrieval of the removed concept is blocked.
- Concepts can be removed sequentially from the same model, and after five removals the median KL divergence on Wikitext-2 is 0.0015, while divergence on pages about the removed concepts is at least 60-fold higher.
- The method needs no fine-tuning or retraining; the practitioner only chooses the similarity threshold, and can lower it to edit more vectors if needed.
Reading between the lines
- The refinement step implicitly assumes the language-model head is a reliable probe of concept identity; the same technique could be adapted to audit which internal directions drive other risky behaviors such as sycophancy or refusal.
- Because the method only edits weights with high cosine similarity, it leaves attention and other pathways untouched; a natural test would measure whether adversarial or paraphrased prompts that bypass the affected weights can still elicit the concept.
- The multilingual result suggests TARS edits the shared latent direction rather than token-specific paths; if that is correct, the same edit should suppress related concepts that live near the target direction, so specificity could be probed by measuring proximity to near-neighbor concepts like 'detective' or 'planet'.
- The 3-norm normalization and the threshold $\theta$ are tunable; systematic sweeps could reveal whether one edit per concept always suffices or whether concepts with diffuse representations need more edits.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes TARS (Targeted Angular Reversal of Weights), a no-retraining method for removing a concept from an LLM by (i) extracting a hidden-state vector from a descriptive prompt, (ii) refining it into a 'targeting vector' by averaging noisy variants that trigger the concept token with probability at least tau, (iii) computing cosine similarities between this vector and all gate/up-projection weight vectors in Llama 3.1 8B, and (iv) replacing weight vectors above a threshold theta with a reversed, normalized version of the targeting vector. Experiments report that target-token probabilities drop to 0.00 for concepts such as Sherlock Holmes, Saturn, Voldemort, dog, and Minecraft with as few as 1-4 edits, that removal works both when the concept is the answer and when it is the subject (bi-directionality), that removing 'dog' in English also suppresses 'chien' and 'Hund', that removals can be stacked modularly, and that after five removals the median KL divergence from the base model on Wikitext-2 is 0.0015.
Significance. If the central claim holds, TARS would be a computationally attractive, modular knowledge-removal method that works across prompt directions and languages without retraining. The paper's strengths are its simple, interpretable procedure, its explicit cross-lingual test using distinct tokens ('dog'/'chien'/'Hund'), and its attempt to quantify specificity via KL divergence on a Wikipedia corpus. However, the evidence as presented is mostly token-level and based on a very small number of prompts per concept, with key hyperparameters (theta, sigma, tau) and their sensitivity unreported. The paper does not yet establish that the refined targeting vector captures the concept rather than a token-specific logit direction, so the significance of the contribution is presently conditional on substantially stronger evaluation.
major comments (5)
- [3.1, Eqs. (7)-(11)] The refinement procedure conditions on the probability of a single vocabulary token, p(t_C | v_noise) >= 0.95. Because the LM head is linear (softmax of a linear map), the accepted noise set is approximately a half-space whose boundary normal is the logit-difference direction between t_C and its nearest competitor; the average in Eq. (11) is therefore biased toward that token-discrimination direction rather than toward a concept-level semantic direction. The paper's main quantitative evidence (Figures 2 and 4) is exactly token-level: p('Sherlock'), p('Saturn'), p('dog') drop to 0.00. This does not rule out that TARS suppresses a token rather than a concept. A concrete test would be to evaluate on paraphrased prompts that do not contain the target token (e.g., 'the detective of Baker Street' without the word 'Sherlock') and on synonyms or translations that use different tokens; if those are suppressed as well, the concept-level claim is supported.
- [3.3, Eq. (13) and Remark 3] The edit threshold theta is never reported, and the claim 'as few as 1 TARS edit' is therefore not reproducible. The number of edits per concept is determined by theta (Eq. 13), yet the paper does not state the theta values used for Sherlock, Saturn, Voldemort, dog, or Minecraft, nor does it provide a sensitivity analysis showing how the target-token probability varies with theta and with the number of edits. Without this, the '0.00' results may simply reflect per-concept threshold tuning. Please report the theta values, or better, a sweep over theta with the resulting target probabilities and edit counts.
- [4, Figures 2-4] The reported '0.00' probabilities are almost certainly rounded; the paper does not report exact probabilities, sample sizes, or confidence intervals for any of the target-token measurements. For the bi-directionality claim, Figure 3 shows only three qualitative completions per concept. For the multilingual claim, Figure 4 uses a single concept ('dog') with one prompt per language. This is too thin to support the general claims of bi-directional and multilingual knowledge removal. Please provide exact probabilities (e.g., 1e-8 rather than 0.00), standard errors over multiple paraphrased prompts, and results for additional concepts and languages.
- [4.4, Table 1] The KL divergence on Wikitext-2 measures next-token distribution shift on a single genre (Wikipedia text). This is not sufficient to conclude 'minimal impact on general model capabilities'. The paper should report perplexity or task accuracy on standard benchmarks (e.g., MMLU, HellaSwag, or at least multiple corpora) and should compare against at least one baseline method (e.g., ROME or RMU) on the same metrics. Without such comparisons, the claimed advantage over existing methods and the claimed specificity are not quantitatively substantiated.
- [3.1, Eqs. (4)-(5)] The paper does not specify which hidden state is used as v_approx and fed to the LM head. In Llama 3.1, the LM head follows a final RMSNorm layer. If the 'final hidden state' in Figure 1 is the raw output of the last transformer block (pre-norm), then p(t_C | v_noise) in Eq. (8) is not the model's actual next-token probability, and the refinement procedure would be conditioning on an incorrect distribution. This is load-bearing for the construction of v_target. Please clarify whether the hidden state is taken before or after the final layer norm, and correct the procedure if necessary.
minor comments (4)
- [Eq. (5)] The notation p(v_approx | P_C) is confusing: v_approx is a hidden-state vector, not a vocabulary token, yet it is used as the argument of a probability distribution over the vocabulary. Please write p(v | P_C) or p(t_C | P_C) for the token distribution.
- [Eq. (13)] The reversed target vector is normalized with the 3-norm, which is unusual and unexplained. Please justify the choice or replace it with the standard 2-norm unless the 3-norm is intentional.
- [3.1, after Eq. (7)] The description 'a batch size of 450 was repeated 25,000 times' is ambiguous: does this mean 450 noisy vectors per iteration for 25,000 iterations, or 450 total? Please state the total number of noisy samples and the number of retained candidates N in Eq. (11).
- [Figure 5] Figure 5 shows bars for target-token probabilities after sequential removal, but the y-axis scale, exact values, and markers are not described. Please add axis labels, numeric values, and error bars or at least specify that the values are single-prompt probabilities.
Circularity Check
No significant circularity: TARS is a constructive weight-edit method whose reported probabilities are measured outcomes, and the cross-lingual and KL-divergence results go beyond the selection objective.
full rationale
The TARS pipeline is a constructive intervention rather than a derivation that reduces to its inputs. The approximate vector comes from a forward pass on a descriptive prompt (Eq. 4), and the targeting vector is selected by thresholding the LM head's probability for the target token (Eqs. 7–11). The edit step (Eq. 13) replaces high-cosine FFN weight vectors with the reversed targeting vector. The reported probability drops to 0.00 are then measured on full-model forward passes after the weight edit; they are not read off from the selection equations. The success metric does coincide with the objective used to build v_target, which makes the demonstration partly self-referential in spirit, but the paper does not present this as an independent prediction, and the bi-directional, cross-lingual, and KL-divergence results are not forced by the equations. The only prior work invoked (Geva et al. for FFN knowledge storage, Wendler et al. for latent language) is external to the authors, so there is no self-citation chain. The threshold θ is tuned to achieve the reported probabilities, which is a reporting weakness rather than circularity, because the paper discloses the tuning protocol. The central mechanism is self-contained and externally checkable, so no circularity score above 0 is warranted.
Assumptions & free parameters
free parameters (5)
- Edit similarity threshold θ =
not reported; tuned per concept
- Noise standard deviation σ =
10 (example)
- Retention threshold τ =
0.95
- Number of TARS edits per concept =
Sherlock: 1, Saturn: 4, Voldemort: 3, dog: 1, Minecraft: unspecified
- Refinement samples (batch size, iterations) =
450 x 25,000 for Sherlock
assumptions (4)
- domain assumption Knowledge is stored in the feedforward networks of LLMs.
- domain assumption The internal representation space is consistent throughout the model due to residual connections.
- domain assumption The approximate vector from the final hidden state of a descriptive prompt aggregates the concept's information.
- ad hoc to paper Averaging noisy vectors that trigger the target token yields a vector that exclusively captures the concept.
Cite this review
Pith. "Pith review of Targeted Angular Reversal of Weights (TARS) for Knowledge Removal in Large Language Models." pith.science (2026). https://pith.science/paper/DI4IMSRY
@misc{pith2026241210257,
author = {Pith},
title = {Pith review of: Targeted Angular Reversal of Weights (TARS) for Knowledge Removal in Large Language Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/DI4IMSRY}},
note = {Machine review of arXiv:2412.10257}
}
read the original abstract
The sheer scale of data required to train modern large language models (LLMs) poses significant risks, as models are likely to gain knowledge of sensitive topics such as bio-security, as well the ability to replicate copyrighted works. Methods designed to remove such knowledge must do so from all prompt directions, in a multi-lingual capacity and without degrading general model performance. To this end, we introduce the targeted angular reversal (TARS) method of knowledge removal from LLMs. The TARS method firstly leverages the LLM in combination with a detailed prompt to aggregate information about a selected concept in the internal representation space of the LLM. It then refines this approximate concept vector to trigger the concept token with high probability, by perturbing the approximate concept vector with noise and transforming it into token scores with the language model head. The feedforward weight vectors in the LLM which operate directly on the internal representation space, and have the highest cosine similarity with this targeting vector, are then replaced by a reversed targeting vector, thus limiting the ability of the concept to propagate through the model. The modularity of the TARS method allows for a sequential removal of concepts from Llama 3.1 8B, such as the famous literary detective Sherlock Holmes, and the planet Saturn. It is demonstrated that the probability of triggering target concepts can be reduced to 0.00 with as few as 1 TARS edit, whilst simultaneously removing the knowledge bi-directionally. Moreover, knowledge is shown to be removed across all languages despite only being targeted in English. Importantly, TARS has minimal impact on the general model capabilities, as after removing 5 diverse concepts in a modular fashion, there is minimal KL divergence in the next token probabilities of the LLM on large corpora of Wikipedia text (median of 0.0015).
Figures
Figures from the paper (2 more)
Forward citations
Cited by 1 Pith paper
-
Flash Interpretability: Decoding Specialised Feature Neurons in Large Language Models with the LM-Head
Decoding MLP up-projection neuron weights with the LM-head reveals specialized single-token feature neurons in Llama 3.1 8B, such as a 'dog' neuron, which can be confirmed by clamping its activation.
Reference graph
Works this paper leans on
- [1]
-
[3]
Who’s Harry Potter? Approximate Unlearning in LLMs
Ronen Eldan and Mark Russinovich. Who’s Harry Potter? Approximate Unlearning in LLMs. arXiV: 2310.02238,
-
[5]
Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary C. Lipton, and J. Zico Kolter. TOFU: A Task of Fictitious Unlearning for LLMs. arXiv: 2401.06121,
-
[6]
Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. RealToxici- tyPrompts: Evaluating neural toxic degeneration in language models. InFindings of the Association for Computational Linguistics: EMNLP 2020, pages 3356–3369, November
work page 2020
-
[8]
Erasing Conceptual Knowledge from Language Models
Rohit Gandikota, Sheridan Feucht, Samuel Marks, and David Bau. Erasing Conceptual Knowledge from Language Models. arXiv: 2410.02760,
-
[9]
11 Mor Geva et al. Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the V ocabulary Space.arXiv: 2203.14680,
-
[10]
Towards safer large language models through machine unlearning
Zheyuan Liu, Guangyao Dou, Zhaoxuan Tan, Yijun Tian, and Meng Jiang. Towards safer large language models through machine unlearning. In Findings of the Association for Computational Linguistics: ACL 2024, pages 1817–1829, August
work page 2024
-
[11]
The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning
Nathaniel Li et al. The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning. arXiv: 2403.03218,
Show all 19 references
-
[12]
RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models
Zhuoran Jin, Pengfei Cao, Chenhao Wang, Zhitao He, Hongbang Yuan, Jiachun Li, Yubo Chen, Kang Liu, and Jun Zhao. RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models. arXiv: 2406.10890,
-
[13]
Jiaming Ji, Donghai Hong, Borong Zhang, Boyuan Chen, Josef Dai, Boren Zheng, Tianyi Qiu, Boxun Li, and Yaodong Yang
Accessed: 2024-09-24. Jiaming Ji, Donghai Hong, Borong Zhang, Boyuan Chen, Josef Dai, Boren Zheng, Tianyi Qiu, Boxun Li, and Yaodong Yang. PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preference. arXiv: 2406.15513,
2024 arXiv
-
[14]
Position: LLM Unlearning Benchmarks are Weak Measures of Progress
Pratiksha Thaker, Shengyuan Hu, Neil Kale, Yash Maurya, Zhiwei Steven Wu, and Virginia Smith. Position: LLM Unlearning Benchmarks are Weak Measures of Progress. arXiv: 2410.02879,
-
[15]
Locating and Editing Factual Associations in GPT
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and Editing Factual Associations in GPT. arXiv: 2202.05262,
-
[17]
Do Llamas Work in English? On the Latent Language of Multilingual Transformers
Chris Wendler, Veniamin Veselovsky, Giovanni Monea, and Robert West. Do Llamas Work in English? On the Latent Language of Multilingual Transformers. arXiv: 2402.10588,
-
[18]
Pointer sentinel mixture models
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. arXiv: 1609.07843,
-
[2016]
A.1.5 Minecraft “The game we are talking about is a popular sandbox game developed by Mojang Studios and released in 2011 that was originally created by Markus
12 A Appendix A.1 Prompts to form the approximate vectors A.1.1 Sherlock Holmes “Imagine a brilliant detective with an unparalleled knack for observation and deduction, often seen donning a deerstalker hat and an Inverness cape, and possessing a razor-sharp intellect capable o...
1997
-
[2017]
GLU Variants Improve Transformer
Noam Shazeer. GLU Variants Improve Transformer. arXiv: 2002.05202,
2002 arXiv
-
[2020]
A Categorical Archive of ChatGPT Failures
Ali Borji. A Categorical Archive of ChatGPT Failures. arXiv: 2302.03494,
-
[2023]
Copyright violations and large language models
Antonia Karamolegkou, Jiaang Li, Li Zhou, and Anders Søgaard. Copyright violations and large language models. arXiv: 2310.13771,
-
[2024]
The Llama 3 Herd of Models
Abhimanyu Dubey, Abhinav Jauhri, et al. The Llama 3 Herd of Models. arXiv: 2407.21783,
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.