Pith. sign in

REVIEW 5 major objections 4 minor 48 references

Reviving Your MNEME: Predicting The Side Effects of LLM Unlearning and Fine-Tuning via Sparse Model Diffing

T0 review · 5 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Sparse model diffing predicts LLM fine-tuning side effects at 95 percent accuracy.

desk verdict Plausible pipeline, but the headline claim of predicting side effects is not supported by the current evaluation, which measures agreement with known degraded categories. read the letter →

arxiv 2507.21084 v1 pith:3UHOZD2T submitted 2025-06-19 cs.CL cs.LG

Aly M. Kassem , Zhuan Shi , Negar Rostamzadeh , Golnoosh Farnadi This is my paper · ORCID
classification cs.CLcs.LG
keywords LLMunlearningsideeffectssparsemodeldiffingcross-codersinterpretabilityemergentmisalignmentbenignfine-tuningsafetymachinelatentscaling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MNEME is a lightweight, automated framework for spotting unintended side effects of fine-tuning or unlearning a large language model. Its central claim is that comparing the base and edited models on ordinary, task-independent text, without any access to the fine-tuning data, can expose the behavioral shifts that benchmarks typically miss. The method decomposes the difference between the two models' internal activations into sparse, interpretable features and labels those features in natural language, predicting which capabilities will degrade or which harmful behaviors will emerge. Across five models and three scenarios, the predictions match known side effects with up to 95 percent accuracy, and the paper shows that retraining on high-activation samples can partially reverse the detected effects.

What carries the argument

The central mechanism is the BatchTopK Cross-Coder: a shared encoder that maps the concatenated activations of base and edited models to a sparse latent code, with two separate decoders reconstructing each model's activations. BatchTopK applies global competition across a batch rather than per input, which the paper finds yields more interpretable, monosemantic features. Feature attribution then uses closed-form latent-scaling coefficients to place each feature on an amplified-to-suppressed spectrum, and an instruction-tuned LLM generates descriptions and category labels for the top-activating contexts. The whole pipeline runs on roughly 200 million tokens from task-agnostic corpora, at layer 14 with an expansion factor of 32, so it needs only the two models and generic text, not the fine-tuning data.

What would settle it

Take a fine-tuned model whose only measurable side effect is confined to a layer other than layer 14, or only manifests on tokens absent from generic corpora like The Pile or LMSYS-Chat-1M; if MNEME's layer-14 generic-text features still predict the side effect at benchmark-level accuracy, the claim is supported, and if they miss it while a data-rich baseline catches it, the fixed-layer and generic-corpus assumption is falsified.

Watch

Extended reading notes

Core claim

The paper's central discovery is that sparse model diffing turns an opaque intervention, fine-tuning or unlearning, into a readable list of 'what changed' at the feature level. A BatchTopK Cross-Coder learns a shared sparse latent space from paired activations of the base and edited models at a fixed layer, then a latent-scaling regression measures whether each feature is amplified, suppressed, or left unchanged. Auto-interpretation converts these features into natural-language descriptions and semantic categories. On hazardous-knowledge unlearning, emergent misalignment, and benign or implicit fine-tuning, the resulting feature set predicts observed side effects, such as lost chemistry knowledge, emergent deception, or increased compliance with harmful instructions, with accuracy up to 95 percent, outperforming random and heuristic baselines and approaching an oracle. The authors also report that retraining on the samples that most activate the changed features partially reverses the side effects.

Load-bearing premise

The method assumes the side effects of an intervention show up as changes in one fixed layer's activations on ordinary, task-independent text, so a side effect that lives in a different layer or only appears on rare, domain-specific tokens could pass through undetected.

Editorial extensions

If this is right

  • Model auditors can flag side effects of unlearning or fine-tuning using only the base model, the edited model, and generic text, without needing proprietary training data.
  • Emergent misalignment can be detected even when standard jailbreak-style benchmarks show no degradation, because the amplified harmful features appear on ordinary chat text.
  • Benign fine-tuning that silently erodes safety, including from as few as ten crafted harmless-looking examples, becomes visible as amplified compliance-related features.
  • Retraining on the samples that most strongly activate changed features offers a partial remedy, giving practitioners a targeted repair path rather than full retraining.
  • The method scales across model sizes and families, indicating the feature-level signature of an intervention is a general phenomenon rather than an artifact of one architecture.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the fixed layer-14 choice means MNEME's recall is bounded by how much of an intervention's effect flows through that layer; a layer-adaptive version, selecting the layer with maximal cross-coder reconstruction shift, would likely close the gap on the lower emergent-misalignment accuracy.
  • Editorial inference: if amplified features are reliably detectable before they manifest behaviorally, MNEME could serve as a pre-deployment gate that triggers inspection or retraining before a harmful response is ever served.
  • Editorial inference: the same paired-activation diffing could be applied to multimodal or vision-language models, where side effects of unlearning might appear in non-text residual streams.
  • Editorial inference: the reported 40-percent semantic overlap between fine-tuning data and auto-interpreted features suggests a ceiling; combining MNEME with low-cost probing on a few labeled samples might push accuracy toward the oracle.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes MNEME, a framework that applies sparse Cross-Coder model diffing to activations of a base model and its fine-tuned or unlearned counterpart, using task-agnostic corpora (The Pile, LMSYS-Chat-1M) so that fine-tuning data are not required. The pipeline generates latent features, attributes them via latent scaling, produces natural-language descriptions with an LLM, and maps those descriptions to semantic categories. The authors claim that this enables automated detection of side effects of unlearning and fine-tuning, reporting accuracies of up to 95% on WMDP unlearning, 85% on benign fine-tuning, and 50% on emergent misalignment (abstract/introduction), or 96%/98%/92% on WMDP, 92.9% on benign fine-tuning, and 68.2% on emergent misalignment (body). The paper also claims that retraining on high-activation samples can partially reverse side effects.

Significance. If the predictive claim were established, MNEME would be a practically valuable contribution: an automated, data-agnostic audit tool for post-training interventions, with applications to unlearning safety and alignment. The paper has real strengths: it builds on established Cross-Coder and BatchTopK machinery, evaluates across five models and three scenarios, includes baseline comparisons, and attempts to avoid label leakage when mapping generated categories to MMLU. However, the central claim is currently supported only by a semantic label-agreement metric, not by evidence that MNEME anticipates behavioral side effects on held-out tasks. The reported numbers are also internally inconsistent. The methodology is plausible, but the evaluation must be substantially reworked before the headline claim can be accepted.

major comments (5)
  1. [Abstract, §1 vs §4.2/§5.2/§6.2] The accuracy numbers are inconsistent between the abstract/introduction and the body. The abstract and §1 state up to 95% on WMDP, 85% on benign fine-tuning, and 50% on emergent misalignment, while §4.2 reports 96%, 98%, and 92% on WMDP, §5.2 reports 92.2% on MMLU-Pro and 68.2% on EM, and §6.2 reports 82.2% and 92.9% on the two fine-tuning scenarios. The reader cannot tell which set of numbers is authoritative, and the headline claims should be corrected to match the actual results.
  2. [§4.2, §5.2, Appendix B] The reported accuracy does not establish predictive side-effect detection. In the WMDP study, accuracy is defined as alignment between MNEME's LLM-generated semantic categories and the MMLU categories that were already known to be degraded after unlearning. In the emergent misalignment study, accuracy is the overlap between amplified features and features that Gemini 2.5-Pro judged to correspond to misalignment instructions or responses. Both are measures of agreement among LLM-produced semantic labels, not evidence that the detected features anticipate behavioral failures on unseen prompts. The paper should add item-level held-out evaluations (e.g., do amplified features predict per-prompt degradation?) and a null-diff control in which the method is applied to a base model against itself or to a model pair with no intervention.
  3. [§3.4, §4.2] There is no control that isolates the contribution of sparse model diffing from the LLM semantic-mapping stage. The naive baseline in §6.2, which uses GPT-4o to infer side effects from domain knowledge, achieves 94.1% accuracy on benign fine-tuning, essentially tied with MNEME's 92.9%. This is consistent with the hypothesis that the semantic mapping stage, rather than the activation-diffing mechanism, drives the reported accuracy. The authors should compare MNEME against an activation-based null (e.g., diffing the base model against itself on different data splits, or against a random perturbation) and against a mapping-only baseline that is given the same feature descriptions but no diffing information.
  4. [Abstract, §7–§8] The abstract claims that 'retraining on high-activation samples can partially reverse these effects,' but no experiment or section in the body reports such a retraining result. Section 7 discusses whether fine-tuning data activate the expected latents, and Section 8 is a conclusion; neither contains a retraining or reversal experiment. Either add the missing experiment and its results, or remove the claim from the abstract.
  5. [§7] The validation in Section 7 reports that only 40% of latents had over 90% semantic overlap when fine-tuning data were passed through the Cross-Coder, which the authors themselves describe as 'moderate correspondence.' This weakens the link between the detected features and the actual intervention, and it should be reconciled with the claim that MNEME achieves near-oracle accuracy in predicting side effects. The authors should clarify whether this validation is meant to support the predictive claim and, if so, why a 40% overlap is sufficient.
minor comments (4)
  1. [§3.1] The fixed choices of layer 14, expansion factor 32, and top-k of 100 are justified only by reference to prior SAE work; a per-scenario sensitivity analysis would strengthen the paper, and if such an analysis is not feasible the limitation should be stated more explicitly.
  2. [§4.1, Figure 3] Model names are inconsistent: the text in §4.1 refers to 'LaMA 3.2-3B Instruct' while Figure 3 and §4.2 refer to 'LLaMA-3.3-3B-Instruct.' Please standardize the model names throughout.
  3. [§6.2, Figure 4] The AOA accuracy is reported as 82.2% in the text, while the figure caption and axis labels suggest a value around 0.82; the random baseline values are given as 48.7% and 49.9% in the text. Please verify all numbers in the figure against the text.
  4. [Limitations] The Limitations section is candid about correlational rather than causal insights and about dependence on LLM-generated descriptions, but it should also explicitly state that the reported accuracies are label-agreement measures, not out-of-sample behavioral predictions.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: MNEME's evaluation compares independent sparse-diff features against externally defined benchmark categories.

full rationale

MNEME's pipeline is not fit to the side-effect labels it is evaluated against. The Cross-Coder is trained on fixed task-agnostic corpora (The Pile, LMSYS-Chat-1M) with hyperparameters (k=100, expansion factor 32, layer 14) taken from prior sparse-autoencoder work, and latent attribution uses a closed-form regression on reconstruction error; none of these steps consume the MMLU/MMLU-Pro categories or emergent-misalignment instruction sets that serve as ground truth. The evaluation metric measures alignment between independently generated semantic labels and externally defined degraded-category or harmful-response sets, so the reported accuracy is an external benchmark rather than a constructed identity. The paper explicitly states that MMLU categories were not consulted before generating latent feature categories, and the emergent-misalignment evaluation independently computes amplified features and Gemini-mapped harmful features before taking their overlap. The only self-citation (Kassem et al., 2023) appears in a list of prior unlearning work and is not load-bearing. The Limitations section's admission that MNEME provides correlational rather than causal insight is a scope caveat, not evidence of circularity. Concerns that the LLM-based semantic mapping could inflate agreement are threats to construct validity, not circular reductions, and do not warrant a nonzero circularity score under the requirement to exhibit a specific equation or fitted-variable identity.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim depends on several domain assumptions from mechanistic interpretability and on three hand-selected hyperparameters. No parameters are fitted to the side-effect labels, and no new entities are introduced.

free parameters (3)
  • top-k sparsity k = 100
    Hand-selected as a balance between sparsity and reconstruction loss; cited to Karvonen et al. (2025), not fitted to side-effect labels.
  • expansion factor = 32
    Chosen for interpretability and monosemanticity (Section 3.1); smaller factors gave broader features; not fitted to the target tasks.
  • layer index = 14
    Selected for its mix of semantic and syntactic signals, following Minder et al. (2025); no per-task validation.
assumptions (5)
  • domain assumption Sparse cross-coder decomposition of residual stream activations yields interpretable features whose diffs correspond to semantic content changes.
    Section 3.1 adopts the Cross-Coder and BatchTopK design; the whole pipeline assumes that learned latents are semantically meaningful and that diffing them reveals behavioral shifts.
  • domain assumption A single fixed layer (layer 14) captures the semantic and syntactic signals needed to detect side effects across all tasks.
    Section 3.1 selects layer 14 for its mix of signals; there is no per-scenario check that side effects appear in this layer.
  • domain assumption Task-agnostic corpora (The Pile, LMSYS-Chat-1M) provide enough coverage of the fine-tuning or unlearning relevant concepts to trigger the latent features underlying the side effects.
    Section 3.1 and Limitations: the method relies on generic corpora; narrow or rare domains may be undercovered.
  • domain assumption LLM-generated feature descriptions and semantic category mapping faithfully represent the latent features without leaking the evaluation taxonomy.
    Section 3.3 and Appendix B use LLaMA-3.1-70B and Gemini 2.5-Pro; Limitations concedes hallucination or misalignment risk, and MMLU knowledge in the mapper could create leakage.
  • domain assumption Latent scaling coefficients and decoder-weight norms provide a valid measure of feature amplification or suppression between base and fine-tuned models.
    Section 3.2 motivates latent scaling with least squares; Sections 4 and 5 use it to attribute features, which presumes the scale reflects real behavioral change.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reviving Your MNEME: Predicting The Side Effects of LLM Unlearning and Fine-Tuning via Sparse Model Diffing." pith.science (2026). https://pith.science/paper/3UHOZD2T

@misc{pith2026250721084,
  author       = {Pith},
  title        = {Pith review of: Reviving Your MNEME: Predicting The Side Effects of LLM Unlearning and Fine-Tuning via Sparse Model Diffing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3UHOZD2T}},
  note         = {Machine review of arXiv:2507.21084}
}
read the original abstract

Large language models (LLMs) are frequently fine-tuned or unlearned to adapt to new tasks or eliminate undesirable behaviors. While existing evaluation methods assess performance after such interventions, there remains no general approach for detecting unintended side effects, such as unlearning biology content degrading performance on chemistry tasks, particularly when these effects are unpredictable or emergent. To address this issue, we introduce MNEME, Model diffiNg for Evaluating Mechanistic Effects, a lightweight framework for identifying these side effects using sparse model diffing. MNEME compares base and fine-tuned models on task-agnostic data (for example, The Pile, LMSYS-Chat-1M) without access to fine-tuning data to isolate behavioral shifts. Applied to five LLMs across three scenarios: WMDP knowledge unlearning, emergent misalignment, and benign fine-tuning, MNEME achieves up to 95 percent accuracy in predicting side effects, aligning with known benchmarks and requiring no custom heuristics. Furthermore, we show that retraining on high-activation samples can partially reverse these effects. Our results demonstrate that sparse probing and diffing offer a scalable and automated lens into fine-tuning-induced model changes, providing practical tools for understanding and managing LLM behavior.

Figures

Figures reproduced from arXiv: 2507.21084 by the authors.

Figure 1
Figure 1. Overview of the MNEME pipeline. (A) Given base and fine-tuned activations, it uses BatchTopK Cross-Coder to learn sparse latent features. (B) Latent scaling attributes each feature to the base, fine-tuned, or both models. (C) Latent features are described in natural language by an LLM using top-activating inputs. (D) Generated descriptions are mapped to high-level semantic categories for analysis. 3.1 Feature Genera… view at source ↗
Figure 2
Figure 2. Illustrative examples of feature descriptions associated with three tasks evaluated in this work: [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Accuracy comparison of MNEME against [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Comparison of model accuracy under two fine-tuning scenarios—AOA (absolute obedience) and benign instruction alignment. For each scenario, four methods are shown: Oracle (ideal upper bound), MNEME (our approach), Naive (semantic heuristic), and Random (Bernoulli baseli…

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 15 canonical work pages

  1. [1]

    Jan Betley, Daniel Tan, Niels Warncke, Anna Sztyber-Betley, Xuchan Bao, Mart \' n Soto, Nathan Labenz, and Owain Evans. 2025. Emergent misalignment: Narrow finetuning can produce broadly misaligned llms. arXiv preprint arXiv:2502.17424

    arXiv 2025

  2. [2]

    Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. 2023. Language models can explain neurons in language models. https://openaipublic.blob.core.windows.net/neuron-explainer/paper/index.html

    2023

  3. [3]

    Joseph Bloom. 2024. Open source sparse autoencoders for all residual stream layers of gpt2 small. In AI Alignment Forum, page 24

    work page 2024

  4. [4]

    Thomas Bricken, Nelson Conmy, Eric Lieberum, Nelson Elhage, Catherine Olsson, Neel Nanda, Nicholas Joseph, and 1 others. 2023. Towards monosemanticity: Decomposing language models with sparse autoencoders. Transformer Circuits Thread. https://transformer-circuits.pub/2023/monosemantic-features/index.html

    work page 2023

  5. [5]

    Trenton Bricken, Jonathan Marcus, Kelley Rivoire, and Thomas Henighan. 2024 a . https://transformer-circuits.pub/2024/september-update/index.html Oversampling a topic in the sae training set results in more detailed features related to that topic . Anthropic Interpretability Team blog post. Circuits Updates – September 2024. Retrieved May 19, 2025, from h...

    work page 2024

  6. [6]

    Trenton Bricken, Siddharth Mishra-Sharma, Jonathan Marcus, Adam Jermyn, Christopher Olah, Kelley Rivoire, and Thomas Henighan. 2024 b . Stage-wise model diffing. https://transformer-circuits.pub/2024/model-diffing/index.html. Transformer Circuits

    work page 2024

  7. [7]

    Bart Bussmann, Patrick Leask, and Neel Nanda. 2024. Batchtopk sparse autoencoders. arXiv preprint arXiv:2412.06410

    arXiv 2024

  8. [8]

    Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. 2023. Free dolly: Introducing the world’s first truly open instruction-tuned llm

    2023

Show all 48 references
  1. [9]

    Edward Cunningham, Thibault Sellam, Tal Linzen, and Yonatan Belinkov. 2023. Sparse autoencoders find highly interpretable directions in language models. In ICLR

    2023

  2. [10]

    European Data Protection Board . 2025. Ai privacy risks & mitigations – large language models (llms). https://www.edpb.europa.eu/system/files/2025-04/ai-privacy-risks-and-mitigations-in-llms.pdf

    2025

  3. [11]

    Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, and 1 others. 2020. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027

    2020 arXiv

  4. [12]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  5. [13]

    Jia-Chen Gu, Hao-Xiang Xu, Jun-Yu Ma, Pan Lu, Zhen-Hua Ling, Kai-Wei Chang, and Nanyun Peng. 2024. Model editing harms general abilities of large language models: Regularization to the rescue. arXiv preprint arXiv:2401.04700

    2024 arXiv

  6. [14]

    Yihuai Hong, Yuelin Zou, Lijie Hu, Ziqian Zeng, Di Wang, and Haiqin Yang. 2024. Dissecting fine-tuning unlearning in large language models. arXiv preprint arXiv:2410.06606

    2024 arXiv

  7. [15]

    Yue Huang and 1 others. 2023. Out-of-distribution unlearning: Comprehensive benchmark and analysis. arXiv preprint arXiv:2311.11316

    2023

  8. [16]

    Samyak Jain, Robert Kirk, Ekdeep Singh Lubana, Robert P Dick, Hidenori Tanaka, Edward Grefenstette, Tim Rockt \"a schel, and David Scott Krueger. 2023. Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks. arXiv preprint arXiv:2311.12786

    2023 arXiv

  9. [17]

    Zhiwei Jin and 1 others. 2024. Rwku: Real-world knowledge unlearning in large language models. arXiv preprint arXiv:2405.14710

    2024 arXiv

  10. [18]

    Adam Karvonen, Can Rager, Johnny Lin, Curt Tigges, Joseph Bloom, David Chanin, Yeu-Tong Lau, Eoin Farrell, Callum McDougall, Kola Ayonrinde, and 1 others. 2025. Saebench: A comprehensive benchmark for sparse autoencoders in language model interpretability. arXiv preprint arXiv...

    2025 arXiv

  11. [19]

    Aly Kassem, Omar Mahmoud, and Sherif Saad. 2023. Preserving privacy through dememorization: An unlearning technique for mitigating memorization risks in language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 4360--4379

    2023

  12. [20]

    Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D Li, Ann-Kathrin Dombrowski, Shashwat Goel, Long Phan, and 1 others. 2024. The wmdp benchmark: Measuring and reducing malicious use with unlearning. arXiv preprint arXiv:2403.03218

    2024 arXiv

  13. [21]

    Jack Lindsey, Adly Templeton, Jonathan Marcus, Thomas Conerly, Joshua Batson, and Christopher Olah. 2024. Sparse crosscoders for cross-layer features and model diffing. Transformer Circuits Thread

    2024

  14. [22]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023 a . Visual instruction tuning. Advances in neural information processing systems, 36:34892--34916

    2023

  15. [23]

    Xiao Liu and 1 others. 2023 b . Unlearning in large language models: A benchmark and empirical study. In arXiv preprint arXiv:2306.05653

    2023 arXiv

  16. [24]

    Wei Lu, Rachel K Luu, and Markus J Buehler. 2024. Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling, model merging and synergistic capabilities. arXiv preprint arXiv:2409.03444

    2024 arXiv

  17. [25]

    Aengus Lynch, Phillip Guo, Aidan Ewart, Stephen Casper, and Dylan Hadfield-Menell. 2024. Eight methods to evaluate robust unlearning in llms. arXiv preprint arXiv:2402.16835

    2024 arXiv

  18. [26]

    Kevin Meng and 1 others. 2022. Locating and editing factual associations in gpt. In NeurIPS

    2022

  19. [27]

    Julian Minder, Cl \'e ment Dumas, Caden Juang, Bilal Chugtai, and Neel Nanda. 2025. Robustly identifying concepts introduced during chat fine-tuning using crosscoders. arXiv preprint arXiv:2504.02922

    2025

  20. [28]

    Minghao Pan and 1 others. 2023. Unlearning with knowledge distillation in large language models. In arXiv preprint arXiv:2309.11795

    2023 arXiv

  21. [29]

    Gon c alo Paulo, Alex Mallen, Caden Juang, and Nora Belrose. 2024. Automatically interpreting millions of features in large language models. arXiv preprint arXiv:2410.13928

    2024 arXiv

  22. [30]

    Baolin Peng and 1 others. 2024. Efficient model editing at scale. arXiv preprint arXiv:2401.05911

    2024 arXiv

  23. [31]

    Nikhil Prakash, Tamar Rott Shaham, Tal Haklay, Yonatan Belinkov, and David Bau. 2024. Fine-tuning enhances existing mechanisms: A case study on entity tracking. arXiv preprint arXiv:2402.14811

    2024 arXiv

  24. [32]

    Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. 2023. Fine-tuning aligned language models compromises safety, even when users do not intend to! arXiv preprint arXiv:2310.03693

    2023 arXiv

  25. [33]

    Weijia Shi, Jaechan Lee, Yangsibo Huang, Sadhika Malladi, Jieyu Zhao, Ari Holtzman, Daogao Liu, Luke Zettlemoyer, Noah A Smith, and Chiyuan Zhang. 2024. Muse: Machine unlearning six-way evaluation for language models. arXiv preprint arXiv:2407.06460

    2024 arXiv

  26. [34]

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. 2023. Alpaca: A strong, replicable instruction-following model. Stanford Center for Research on Foundation Models. https://crfm. stanford. edu/2023/03...

    2023

  27. [35]

    Bozhong Tian, Xiaozhuan Liang, Siyuan Cheng, Qingbin Liu, Mengru Wang, Dianbo Sui, Xi Chen, Huajun Chen, and Ningyu Zhang. 2024. To forget or not? towards practical knowledge unlearning for large language models. In Findings of the Association for Computational Linguistics: EM...

    2024

  28. [36]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  29. [37]

    Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, and 1 others. 2024. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. In The Thirty-eight Conference on Neural I...

    2024

  30. [38]

    Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, and 1 others. 2022. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682

    2022 arXiv

  31. [39]

    Yujia Xu and 1 others. 2023. Forget-me-not: A machine unlearning benchmark for language models. In arXiv preprint arXiv:2305.06893

    2023 arXiv

  32. [40]

    Haoran Yang, Yumeng Zhang, Jiaqi Xu, Hongyuan Lu, Pheng-Ann Heng, and Wai Lam. 2024. Unveiling the generalization power of fine-tuned large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Hu...

    2024

  33. [41]

    Qinyuan Ye. 2024. Cross-task generalization abilities of large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 4: Student Research Workshop), pages 255--2...

    2024

  34. [42]

    Qiang Yu and 1 others. 2023. Rejection tuning: Safely unlearning unwanted behaviors in language models. arXiv preprint arXiv:2310.01878

    2023 arXiv

  35. [43]

    Weixiang Zhao, Yulin Hu, Zhuojun Li, Yang Deng, Jiahe Guo, Xingyu Sui, Yanyan Zhao, Bing Qin, Tat-Seng Chua, and Ting Liu. 2024. Towards comprehensive post safety alignment of large language models via safety patching. arXiv preprint arXiv:2405.13820

    2024 arXiv

  36. [44]

    P Xing, Joseph E

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Tianle Li, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zhuohan Li, Zi Lin, Eric. P Xing, Joseph E. Gonzalez, Ion Stoica, and Hao Zhang. 2023 a . https://arxiv.org/abs/2309.11998 Lmsys-chat-1m: A large-scale real-world llm conversation...

    2023 arXiv

  37. [45]

    Siyuan Zheng, Yifan Wang, Zhihao Yan, Yu Shi, Yuntao Chen, Zhihan Chiang, Canwen Xu, Yizhong Wang, Yiming Huang, Jialiang Li, and 1 others. 2023 b . https://arxiv.org/abs/2306.05685 Judging llm-as-a-judge with mt-bench and chatbot arena . Preprint, arXiv:2306.05685

    2023 arXiv

  38. [46]

    Yujie Zhu and 1 others. 2023. Machine unlearning: A survey. arXiv preprint arXiv:2302.09531

    2023 arXiv

  39. [47]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  40. [48]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.