Pith. sign in

REVIEW 3 major objections 5 minor 44 references

Vulnerability-Aware Alignment: Mitigating Uneven Forgetting in Harmful Fine-Tuning

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a consistent subset of alignment examples is vulnerable to harmful fine-tuning across tasks, and that training these fragile examples harder with Group DRO measurably reduces post-fine-tuning harm.

desk verdict A genuinely new data-centric defense against harmful fine-tuning, but the proxy-based vulnerability grouping is unverified and the results lack error bars. read the letter →

arxiv 2506.03850 v2 pith:N5LE64PF submitted 2025-06-04 cs.LG

Liang Chen , Xueting Han , Li Shen , Jing Bai , Kam-Fai Wong This is my paper
classification cs.LG
keywords harmfulfine-tuningsafetyalignmentdatavulnerabilitygroupdistributionallyrobustoptimizationadversarialsamplinglargelanguagemodelsunevenforgetting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that safety problems caused by harmful fine-tuning are not spread evenly across the alignment data: a consistent subset of safety examples is forgotten first and most often, regardless of the downstream task or the amount of harmful data mixed in. It proposes Vulnerability-Aware Alignment (VAA), which simulates a harmful fine-tuning pass to label each alignment example as vulnerable or invulnerable, then trains the model with a group distributionally robust optimization objective that forces balanced learning between the two groups. The payoff, if the claim is right, is an alignment-stage defense that lowers the harmful score after fine-tuning (average 34.51 to 24.82 on Llama2-7B across four tasks) while keeping downstream accuracy at roughly the same level. The paper also shows the vulnerable subset transfers across models, so the grouping can be computed once and reused.

What carries the argument

The load-bearing object is the vulnerability partition, defined by $ForgotNum_i = \sum_{t=1}^T \mathbb{I}(HS_t^i > HS_0^i)$, where $HS$ is the harmful score of an alignment example. $ForgotNum > 0$ marks the example as vulnerable; this binary split is computed on a proxy harmful fine-tuning run (Alpaca plus 10% harmful data) and then used as prior knowledge. The training machinery is Group DRO realized as a two-player game: an EXP3 adversarial sampler maintains a probability distribution $q$ over the two groups and at each step samples the harder group, while the LLM minimizes a composite objective $f_i(\theta) = (1-\lambda)\ell_i(\theta) + \lambda\ell_i(\theta + \epsilon_i)$, with $\lambda$ ramped from 0 to 1 by curriculum learning and $\epsilon_i$ approximated by $\alpha$ times the normalized gradient of the group loss. The sampler and the model are updated adversarially until the sampler would settle on a uniform distribution, which is the signature that both groups are learned equally well.

What would settle it

Measure the overlap between the set of alignment examples that are forgotten in the Alpaca proxy run and the sets forgotten in each real task (SST2, AG News, GSM8K, AlpacaEval). If the overlap is close to what random chance would predict, the vulnerability prior provides no task-relevant signal and VAA's advantage over uniform training should disappear; conversely, high overlap would confirm the transferability claim. An even sharper test is to run VAA with groups assigned from each task's own vulnerability labels and compare against the proxy-assigned VAA: equal performance would mean the proxy is unnecessary.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that forgetting during harmful fine-tuning is data-dependent and predictable: roughly the same alignment examples flip from safe to harmful across poison rates (0%, 10%, and 20%) and across different fine-tuning tasks, and these examples sit at sharper points of the loss landscape. VAA turns that observation into a training objective. It partitions alignment data into vulnerable and invulnerable groups using ForgotNum, the number of fine-tuning steps at which an example produces a harmful output, measured on a proxy fine-tuning run over Alpaca with 10% harmful data. Training then applies Group DRO with an adversarial sampler that up-weights the currently underperforming group and adds a group-specific worst-case parameter perturbation, approximating the perturbation with a normalized gradient step. The result is that the model learns the fragile examples at least as well as the stable ones, so a later harmful fine-tuning pass has fewer weak points to exploit.

Load-bearing premise

The method's gains rest on the assumption that the vulnerable group identified by a proxy harmful fine-tuning pass on Alpaca with 10% harmful data is the same group that would be vulnerable on the user's actual fine-tuning task, and the paper does not directly measure that overlap on the real tasks.

Editorial extensions

If this is right

  • Safety alignment can be made more resistant without knowing the downstream fine-tuning distribution, because vulnerability patterns transfer across tasks and, between Llama2-7B and Qwen2.5-7B, across model families.
  • Benign fine-tuning (harmful ratio $p=0$) also degrades safety, and VAA reduces this degradation too, not just attacks that deliberately include harmful data.
  • The defense's benefit holds up as fine-tuning intensity grows: at 5 fine-tuning epochs, VAA keeps the harmful score at 20.00 on SST2 while SFT reaches 32.87, and fine-tuning accuracy is unchanged.
  • The extra compute is modest: roughly 1.5 times the backpropagation steps of standard SFT, versus about 2 times for Vaccine and 3 times for Booster.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension, noted only as future work in the paper, is to replace the binary vulnerable/invulnerable split with a continuous vulnerability score; if vulnerability is graded rather than categorical, Group DRO over a continuous weighting could extract more signal from the same alignment data.
  • The stability of the vulnerable set across models suggests a practical stress-test suite: a small fixed set of fragile safety examples could serve as a cheap early-warning test for whether a fine-tuning run is about to break alignment.
  • Because VAA operates at the alignment stage and is orthogonal to embedding-perturbation and unlearnable-data defenses, stacking it with those methods could plausibly close the residual gap the paper acknowledges: it reduces but does not eliminate harmful behavior.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies harmful fine-tuning (HFT) from a data-centric perspective. It defines a per-example vulnerability score (ForgotNum, Eq. 1) and shows, via proxy fine-tuning on an Alpaca dataset mixed with harmful examples, that alignment data can be partitioned into vulnerable and invulnerable groups. The proposed method, Vulnerability-Aware Alignment (VAA), combines this grouping with Group DRO and group-dependent adversarial parameter perturbations to promote balanced learning across groups during alignment. Experiments on Llama2-7B and Qwen2.5-7B across SST2, AG News, GSM8K, and AlpacaEval report lower harmful scores after HFT while preserving or improving downstream accuracy, compared with SFT, RepNoise, Vaccine, and Booster.

Significance. If the central claim holds, the paper offers a genuinely new axis for alignment-stage defenses: instead of treating all alignment examples equally, identify and reinforce the subset that is most prone to forgetting under harmful fine-tuning. The paper ships a code repository, defines quantitative vulnerability and transferability metrics (Eqs. 1-3), and reports consistent trends across four tasks, two model families, several poison ratios, and multiple HFT epochs. The cross-model result (grouping estimated on Llama2 and applied to Qwen2.5 without re-clustering, Table 4) is a particularly informative positive result. However, the empirical significance is currently undermined by the absence of variance estimates and by the unverified transferability of the Alpaca-based proxy grouping to the actual downstream tasks; these issues are fixable but must be addressed before the claims can be accepted.

major comments (3)
  1. [Section 2.2, Figure 2] The load-bearing assumption that vulnerability groups found on the Alpaca+HFT proxy transfer to the actual downstream tasks is not empirically supported. Section 2.2 states that the grouping is justified by the empirical transferability shown in Figure 2, but Figure 2(b) only reports CommonForgot among SST2, GSM8K, and AG News; Alpaca, the exact proxy used for all reported VAA models, is not included in that comparison. The transferability that matters is from Alpaca to each downstream task, not among three of the downstream tasks. Please report direct overlap statistics (e.g., Jaccard index or CommonForgotRatio) between the Alpaca-proxy vulnerable set and the task-specific vulnerable sets for all four tasks, and include an ablation with random group assignment or with a much larger label-swap fraction (e.g., 50%). The current Table 5 only swaps 10% of group labels and does not model the full proxy-mismatch scenario; the Limitations section itself concedes reliance on a 'pseudo fine-tuning process,' underscoring that this assumption is not independently verified.
  2. [Section 4.2, Tables 1-4] All reported HS and FA values appear to come from a single run; no standard deviations, number of seeds, or significance tests are provided. The abstract and Section 4.2 repeatedly say the reductions are 'significant,' but this is not statistically supported. Please report mean and standard deviation over at least three independent runs for the main comparisons (or, if that is infeasible for 7B models, provide a clear statement of the cost and justify the single-run protocol). Additionally, the method introduces several free parameters—perturbation magnitude α, the curriculum schedule for λ, the EXP3 step size η_q, and the ForgotNum>0 vulnerability threshold—none of which is ablated. A sensitivity analysis for α and the vulnerability threshold is needed to confirm that the reported gains come from the proposed mechanism rather than from one particular hyperparameter configuration.
  3. [Section 3.1, Eq. (4), and Algorithm 1] The 'worst-case' group perturbation ϵ_i is implemented as α·∇ℓ_i(θ)/||∇ℓ_i(θ)||, which is only a first-order ascent direction and is not guaranteed to maximize ℓ_i(θ+ϵ)-ℓ_i(θ) at the chosen α, especially for large α. The paper should explicitly state that this is a Taylor approximation (as hinted by citing Foret et al.) and discuss the parameter regime in which this approximation is valid for 7B-scale models. Alternatively, the authors could compare with random-direction perturbations of the same norm and with several α values; this would show whether the adversarial direction, rather than the added regularization from the ℓ_i(θ+ϵ_i) term, is what drives the robustness gain.
minor comments (5)
  1. [Table 1 caption / Section 4.2] The reported 'harmful score reductions of 12.9%, 12.0%, 10.6%, and 3.4%' are absolute percentage-point differences (e.g., 32.87 to 20.00 is a 12.87-point drop), not relative percentage reductions; please clarify this to avoid misinterpretation.
  2. [Section 4.1] The paper samples 2,000 alignment instances but does not report how many fall into the vulnerable versus invulnerable groups under the ForgotNum>0 threshold. Group sizes are important for interpreting the Group DRO reweighting and for reproducing the experiments; please add them.
  3. [Algorithm 1] The algorithm line 'i ∼ q(t-1); (x, y) ∼ G_i' suggests a single example is drawn per iteration, while the text says the adversary samples a batch. Please clarify how batches are formed (e.g., sampling a group, then drawing a mini-batch from that group) and how the EXP3 reward is computed over a batch.
  4. [Figure 3] The loss-landscape comparison in Figure 3 is qualitative; please add a quantitative measure (e.g., maximum loss change over the displayed perturbation range or a sharpness statistic) to support the claim that vulnerable examples exhibit greater sensitivity to weight perturbations.
  5. [Section 5] The statements 'we are the first to approach the problem from a data perspective' and 'we are the first to apply DRO to defend against the harmful fine-tuning problem' are strong novelty claims; please soften or support them with a more explicit comparison to prior data-centric analyses of alignment breakdown.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the vulnerability grouping is an external proxy input, not a redefinition of the evaluation metric; the Alpaca transferability gap is an empirical assumption, not a circular reduction.

full rationale

VAA's derivation chain is not circular. The vulnerability partition (ForgotNum > 0) is computed by running a proxy harmful fine-tuning run on Alpaca mixed with 10% harmful data, and this same partition is used as an input to the Group DRO alignment objective. The partition is not constructed from the evaluation outcomes, which are held-out BeaverTails harmful scores after HFT on SST2, AG News, GSM8K, and AlpacaEval. The robust objective f_i(θ) = (1-λ)ℓ_i(θ) + λℓ_i(θ+ε_i) is a standard perturbation-robust loss defined on group losses, not a re-expression of the evaluation harmfulness metric. The evaluation HS is measured by a separate moderation classifier on unseen malicious instructions. The only load-bearing external premise is that Alpaca-proxy vulnerability groups transfer to downstream fine-tuning tasks. That premise is empirically supported only indirectly: Figure 2 compares overlap across SST2, GSM8K, and AG News, not Alpaca, and the Limitations section concedes that the method relies on a pseudo fine-tuning process. This is an empirical validity gap, not circularity, because the transfer relation is contingent and falsifiable rather than true by construction. Self-citations in the paper appear only in passing related-work or limitations contexts and are not load-bearing for the central claim. No step exhibits the specific reduction pattern required for a circularity finding, such as an equation equaling its input by definition or a fitted parameter being renamed as a prediction.

Assumptions & free parameters 6 free parameters · 3 assumptions · 0 invented entities

The method's central claim depends on the transferability of the Alpaca-proxy vulnerability grouping and on the reliability of the harmful score classifier. Several hyperparameters (alpha, lambda schedule, eta_q, ForgotNum threshold, proxy poison ratio) are chosen by hand and not all are reported, but none is fitted to the evaluation metric.

free parameters (6)
  • perturbation magnitude alpha
    Step size for group-dependent adversarial weight perturbations in the robust objective (Section 3.1, Eq. 4); value not reported in the text, tuning details omitted.
  • curriculum schedule for lambda = 0 to 1 over training
    Curriculum learning gradually increases lambda from 0 to 1 (Section 3.1); the schedule is chosen by hand and not specified numerically.
  • learning rates for alignment and HFT = 1e-4 (alignment), 3e-5 (HFT)
    AdamW learning rates chosen for training; reported in Section 4.1.
  • EXP3 step size eta_q
    Update step size for the adversarial sampler (Eq. 9, 12); value not reported.
  • vulnerability threshold = ForgotNum > 0
    Binary criterion to partition data into vulnerable and invulnerable groups (Section 2.2); a modeling choice, though motivated by the definition of forgetting.
  • proxy poison ratio = 10%
    Proportion of harmful examples mixed into the Alpaca proxy dataset for grouping (Section 2.2); chosen to match the default HFT setting.
assumptions (3)
  • standard math Group DRO converges to a solution with equal objective across groups under standard assumptions.
    The learning algorithm relies on Group DRO (Sagawa et al., 2020) and mirror ascent/EXP3 (Auer et al., 2002) for the adversarial sampler. These are standard results.
  • domain assumption Vulnerability patterns measured on an Alpaca-based proxy HFT transfer to the target downstream tasks.
    Section 2.2 states that the proxy is used to approximate forgetting behavior because the downstream distribution is unavailable. The paper only shows overlap across SST2, GSM8K, AGNews in Figure 2, not overlap between the Alpaca-derived groups and the actual task groups.
  • domain assumption The harmful score classifier reliably identifies harmful outputs.
    Data vulnerability and the evaluation metric HS both rely on the BeaverTails moderation model (Ji et al., 2023) to judge harmfulness; if the classifier is noisy, the grouping is noisy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Vulnerability-Aware Alignment: Mitigating Uneven Forgetting in Harmful Fine-Tuning." pith.science (2026). https://pith.science/paper/N5LE64PF

@misc{pith2026250603850,
  author       = {Pith},
  title        = {Pith review of: Vulnerability-Aware Alignment: Mitigating Uneven Forgetting in Harmful Fine-Tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N5LE64PF}},
  note         = {Machine review of arXiv:2506.03850}
}
read the original abstract

Harmful fine-tuning (HFT), performed directly on open-source LLMs or through Fine-tuning-as-a-Service, breaks safety alignment and poses significant threats. Existing methods aim to mitigate HFT risks by learning robust representation on alignment data or making harmful data unlearnable, but they treat each data sample equally, leaving data vulnerability patterns understudied. In this work, we reveal that certain subsets of alignment data are consistently more prone to forgetting during HFT across different fine-tuning tasks. Inspired by these findings, we propose Vulnerability-Aware Alignment (VAA), which estimates data vulnerability, partitions data into "vulnerable" and "invulnerable" groups, and encourages balanced learning using a group distributionally robust optimization (Group DRO) framework. Specifically, VAA learns an adversarial sampler that samples examples from the currently underperforming group and then applies group-dependent adversarial perturbations to the data during training, aiming to encourage a balanced learning process across groups. Experiments across four fine-tuning tasks demonstrate that VAA significantly reduces harmful scores while preserving downstream task performance, outperforming state-of-the-art baselines.

Figures

Figures reproduced from arXiv: 2506.03850 by the authors.

Figure 1
Figure 1. Overview of the Vulnerability-Aware Alignment. In the first stage, the vulnerability of the alignment dataset is [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Analysis of forgetting behavior: (a) Forgetting pat [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Analysis of robustness behavior. The left panel [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

44 extracted references · 14 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Auer, P., Cesa-Bianchi, N., Freund, Y., and Schapire, R. E. The nonstochastic multiarmed bandit problem. SIAM Journal on Computing, 32 0 (1): 0 48--77, 2002. doi:10.1137/S0097539701398375. URL https://doi.org/10.1137/S0097539701398375

    doi:10.1137/s0097539701398375 2002

  3. [3]

    D., Melenberg, B., and Rennen, G

    Ben-Tal, A., den Hertog, D., Waegenaere, A. D., Melenberg, B., and Rennen, G. Robust solutions of optimization problems affected by uncertain probabilities. Management Science, 59: 0 341--357, 2013

    work page 2013

  4. [4]

    Curriculum learning

    Bengio, Y., Louradour, J., Collobert, R., and Weston, J. Curriculum learning. In International Conference on Machine Learning (ICML), pp.\ 41--48, 2009

    work page 2009

  5. [5]

    Buda, M., Maki, A., and Mazurowski, M. A. A systematic study of the class imbalance problem in convolutional neural networks. Neural Networks, 106: 0 249–259, October 2018. ISSN 0893-6080. doi:10.1016/j.neunet.2018.07.011. URL http://dx.doi.org/10.1016/j.neunet.2018.07.011

    doi:10.1016/j.neunet.2018.07.011 2018

  6. [6]

    Beyond factuality: A comprehensive evaluation of large language models as knowledge generators

    Chen, L., Deng, Y., Bian, Y., Qin, Z., Wu, B., Chua, T.-S., and Wong, K.-F. Beyond factuality: A comprehensive evaluation of large language models as knowledge generators. In Bouamor, H., Pino, J., and Bali, K. (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp.\ 6325--6341, Singapore, December 2023. Associa...

    doi:10.18653/v1/2023.emnlp-main.390 2023

  7. [7]

    W at ME : Towards lossless watermarking through lexical redundancy

    Chen, L., Bian, Y., Deng, Y., Cai, D., Li, S., Zhao, P., and Wong, K.-F. W at ME : Towards lossless watermarking through lexical redundancy. In Ku, L.-W., Martins, A., and Srikumar, V. (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.\ 9166--9180, Bangkok, Thailand, August 2024 a ....

    doi:10.18653/v1/2024.acl-long.496 2024

  8. [8]

    Simple permutations can fool LL a MA : Permutation attack and defense for large language models

    Chen, L., Bian, Y., Shen, L., and Wong, K.-F. Simple permutations can fool LL a MA : Permutation attack and defense for large language models. In ICLR 2024 Workshop on Secure and Trustworthy Large Language Models, 2024 b . URL https://openreview.net/forum?id=lwW7d8LMsf

    work page 2024

Show all 44 references
  1. [9]

    PEARL : Towards permutation-resilient LLM s

    Chen, L., Shen, L., Deng, Y., Zhao, X., Liang, B., and Wong, K.-F. PEARL : Towards permutation-resilient LLM s. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=txoJvjfI9w

    2025

  2. [10]

    Training verifiers to solve math word problems

    Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021

    2021 arXiv

  3. [11]

    Statistics of robust optimization: A generalized empirical likelihood approach

    Duchi, J., Glynn, P., and Namkoong, H. Statistics of robust optimization: A generalized empirical likelihood approach. arXiv, 2016

    2016

  4. [12]

    Sharpness-aware minimization for efficiently improving generalization

    Foret, P., Kleiner, A., Mobahi, H., and Neyshabur, B. Sharpness-aware minimization for efficiently improving generalization. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=6Tm1mposlrM

    2021

  5. [13]

    Gawlikowski, J., Tassi, C. R. N., Ali, M., Lee, J., Humt, M., Feng, J., Kruspe, A., Triebel, R., Jung, P., Roscher, R., Shahzad, M., Yang, W., Bamler, R., and Zhu, X. X. A survey of uncertainty in deep neural networks, 2022. URL https://arxiv.org/abs/2107.03342

    2022 arXiv

  6. [14]

    Safe lora: the silver lining of reducing safety risks when fine-tuning large language models, 2024

    Hsu, C.-Y., Tsai, Y.-L., Lin, C.-H., Chen, P.-Y., Yu, C.-M., and Huang, C.-Y. Safe lora: the silver lining of reducing safety risks when fine-tuning large language models, 2024

    2024

  7. [15]

    Does distributionally robust supervised learning give robust classifiers? In International Conference on Machine Learning (ICML), 2018

    Hu, W., Niu, G., Sato, I., and Sugiyama, M. Does distributionally robust supervised learning give robust classifiers? In International Conference on Machine Learning (ICML), 2018

    2018

  8. [16]

    Antidote: Post-fine-tuning safety alignment for large language models against harmful fine-tuning

    Huang, T., Bhattacharya, G., Joshi, P., Kimball, J., and Liu, L. Antidote: Post-fine-tuning safety alignment for large language models against harmful fine-tuning. arXiv preprint arXiv:2408.09600, 2024 a

    2024 arXiv

  9. [17]

    F., and Liu, L

    Huang, T., Hu, S., Ilhan, F., Tekin, S. F., and Liu, L. Booster: Tackling harmful fine-tuning for large language models via attenuating harmful perturbation, 2024 b . URL https://arxiv.org/abs/2409.01586

    2024 arXiv

  10. [18]

    F., and Liu, L

    Huang, T., Hu, S., Ilhan, F., Tekin, S. F., and Liu, L. Lazy safety alignment for large language models against harmful fine-tuning. arXiv preprint arXiv:2405.18641, 2024 c

    2024 arXiv

  11. [19]

    Vaccine: Perturbation-aware alignment for large language model

    Huang, T., Hu, S., and Liu, L. Vaccine: Perturbation-aware alignment for large language model. arXiv preprint arXiv:2402.01109, 2024 d

    2024 arXiv

  12. [20]

    Beavertails: Towards improved safety alignment of llm via a human-preference dataset

    Ji, J., Liu, M., Dai, J., Pan, X., Zhang, C., Bian, C., Sun, R., Wang, Y., and Yang, Y. Beavertails: Towards improved safety alignment of llm via a human-preference dataset. arXiv preprint arXiv:2307.04657, 2023

    2023

  13. [21]

    A watermark for large language models

    Kirchenbauer, J., Geiping, J., Wen, Y., Katz, J., Miers, I., and Goldstein, T. A watermark for large language models. International Conference on Machine Learning, 2023

    2023

  14. [22]

    and Zhou, E

    Lam, H. and Zhou, E. Quantifying input uncertainty in stochastic optimization. In 2015 Winter Simulation Conference, 2015

    2015

  15. [23]

    Li, X., Zhang, T., Dubois, Y., Taori, R., Gulrajani, I., Guestrin, C., Liang, P., and Hashimoto, T. B. Alpacaeval: An automatic evaluator of instruction-following models. https://github.com/tatsu-lab/alpaca_eval, 2023

    2023

  16. [24]

    Truthfulqa: Measuring how models mimic human falsehoods, 2022

    Lin, S., Hilton, J., and Evans, O. Truthfulqa: Measuring how models mimic human falsehoods, 2022

    2022

  17. [25]

    Fixing weight decay regularization in adam

    Loshchilov, I., Hutter, F., et al. Fixing weight decay regularization in adam. arXiv preprint arXiv:1711.05101, 5, 2017

    2017 arXiv

  18. [26]

    Keeping llms aligned after fine-tuning: The crucial role of prompt templates

    Lyu, K., Zhao, H., Gu, X., Yu, D., Goyal, A., and Arora, S. Keeping llms aligned after fine-tuning: The crucial role of prompt templates. arXiv preprint arXiv:2402.18540, 2024

    2024 arXiv

  19. [27]

    Virtual adversarial training: a regularization method for supervised and semi-supervised learning

    Miyato, T., Maeda, S., Ishii, S., and Koyama, M. Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018

    2018

  20. [28]

    H., and Dokania, P

    Mukhoti, J., Gal, Y., Torr, P. H., and Dokania, P. K. Fine-tuning can cripple your foundation model; preserving features may be the solution. arXiv preprint arXiv:2308.13320, 2023

    2023 arXiv

  21. [29]

    Robust stochastic approximation approach to stochastic programming

    Nemirovski, A., Juditsky, A., Lan, G., and Shapiro, A. Robust stochastic approximation approach to stochastic programming. SIAM J. on Optimization, 19 0 (4): 0 1574–1609, January 2009. ISSN 1052-6234. doi:10.1137/070704277. URL https://doi.org/10.1137/070704277

    2009 doi

  22. [30]

    Distributionally robust language modeling

    Oren, Y., Sagawa, S., Hashimoto, T., and Liang, P. Distributionally robust language modeling. In Empirical Methods in Natural Language Processing (EMNLP), 2019

    2019

  23. [31]

    Peng, S., Chen, P.-Y., Hull, M., and Chau, D. H. Navigating the safety landscape: Measuring risks in finetuning large language models. arXiv preprint arXiv:2405.17374, 2024

    2024 arXiv

  24. [32]

    Gradient starvation: A learning proclivity in neural networks

    Pezeshki, M., Kaba, S.-O., Bengio, Y., Courville, A., Precup, D., and Lajoie, G. Gradient starvation: A learning proclivity in neural networks. In Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Processing Systems, 2021. URL h...

    2021

  25. [33]

    Fine-tuning aligned language models compromises safety, even when users do not intend to! In The Twelfth International Conference on Learning Representations, 2024

    Qi, X., Zeng, Y., Xie, T., Chen, P.-Y., Jia, R., Mittal, P., and Henderson, P. Fine-tuning aligned language models compromises safety, even when users do not intend to! In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?...

    2024

  26. [34]

    Representation noising effectively prevents harmful fine-tuning on llms

    Rosati, D., Wehner, J., Williams, K., Bartoszcze, ., Atanasov, D., Gonzales, R., Majumdar, S., Maple, C., Sajjad, H., and Rudzicz, F. Representation noising effectively prevents harmful fine-tuning on llms. arXiv preprint arXiv:2405.14577, 2024 a

    2024 arXiv

  27. [35]

    Immunization against harmful fine-tuning attacks

    Rosati, D., Wehner, J., Williams, K., Bartoszcze, ., Batzner, J., Sajjad, H., and Rudzicz, F. Immunization against harmful fine-tuning attacks. arXiv preprint arXiv:2402.16382, 2024 b

    2024 arXiv

  28. [36]

    W., Hashimoto, T

    Sagawa, S., Koh, P. W., Hashimoto, T. B., and Liang, P. Distributionally robust neural networks. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=ryxGuJrFvS

    2020

  29. [37]

    D., Ng, A

    Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A. Y., and Potts, C. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, pp.\ 1631--1642, 2013

    2013

  30. [38]

    Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., and Hashimoto, T. B. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_alpaca, 2023

    2023

  31. [39]

    Llama: Open and efficient foundation language models

    Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozi \`e re, B., Goyal, N., Hambro, E., Azhar, F., et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  32. [40]

    Qwen2 technical report

    Yang, A., Yang, B., Hui, B., Zheng, B., Yu, B., Zhou, C., Li, C., Li, C., Liu, D., Huang, F., et al. Qwen2 technical report. arXiv preprint arXiv:2407.10671, 2024

    2024 arXiv

  33. [41]

    Y., Zhao, X., and Lin, D

    Yang, X., Wang, X., Zhang, Q., Petzold, L., Wang, W. Y., Zhao, X., and Lin, D. Shadow alignment: The ease of subverting safely-aligned language models. arXiv preprint arXiv:2310.02949, 2023

    2023 arXiv

  34. [42]

    A safety realignment framework via subspace-oriented model fusion for large language models

    Yi, X., Zheng, S., Wang, L., Wang, X., and He, L. A safety realignment framework via subspace-oriented model fusion for large language models. arXiv preprint arXiv:2405.09055, 2024

    2024 arXiv

  35. [43]

    Removing rlhf protections in gpt-4 via fine-tuning

    Zhan, Q., Fang, R., Bindu, R., Gupta, A., Hashimoto, T., and Kang, D. Removing rlhf protections in gpt-4 via fine-tuning. arXiv preprint arXiv:2311.05553, 2023

    2023 arXiv

  36. [44]

    Character-level convolutional networks for text classification

    Zhang, X., Zhao, J., and LeCun, Y. Character-level convolutional networks for text classification. Advances in neural information processing systems, 28, 2015

    2015

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.