Pith. sign in

REVIEW 6 cited by

SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selection

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.07471 v2 pith:BXY6Z5YI submitted 2024-10-09 cs.LG cs.AIcs.CL

Han Shen , Pin-Yu Chen , Payel Das , Tianyi Chen This is my paper · ORCID
classification cs.LGcs.AIcs.CL
keywords datafine-tuningsealmodelsbilevelgithubranksafety
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Fine-tuning on task-specific data to boost downstream performance is a crucial step for leveraging Large Language Models (LLMs). However, previous studies have demonstrated that fine-tuning the models on several adversarial samples or even benign data can greatly comprise the model's pre-equipped alignment and safety capabilities. In this work, we propose SEAL, a novel framework to enhance safety in LLM fine-tuning. SEAL learns a data ranker based on the bilevel optimization to up rank the safe and high-quality fine-tuning data and down rank the unsafe or low-quality ones. Models trained with SEAL demonstrate superior quality over multiple baselines, with 8.5% and 9.7% win rate increase compared to random selection respectively on Llama-3-8b-Instruct and Merlinite-7b models. Our code is available on github https://github.com/hanshen95/SEAL.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Activation Approximations Can Incur Safety Vulnerabilities Even in Aligned LLMs: Comprehensive Analysis and Defense

    cs.CR 2025-02 conditional novelty 7.0 of 10

    Activation approximations used for faster LLM inference can dramatically increase jailbreak success rates on aligned models, and the paper's QuadA training method restores robust refusal.

  2. Sharper Analysis of Single-Loop Methods for Bilevel Optimization

    cs.LG 2026-07 accept novelty 6.0 of 10

    Decoupled-norm analysis improves single-loop AID to O(κ⁵/K) and shows single-loop ITD's asymptotic error is exactly O(κ²), matching the known lower bound.

  3. DeFrame: Debiasing Large Language Models Against Framing Effects

    cs.CL 2026-02 conditional novelty 6.0 of 10

    LLM fairness scores shift substantially with positive vs negative framing of the same question, and DeFrame—a three-step self-revision prompt—reduces both average bias and this framing gap.

  4. BLISS: A Lightweight Bilevel Influence Scoring Method for Data Selection in Language Model Pretraining

    cs.LG 2025-10 conditional novelty 6.0 of 10

    A bilevel optimization method ranks pretraining data by training a small proxy model on weighted samples, yielding modest downstream-task gains without external pretrained models.

  5. A Survey on Open Dataset Search in the LLM Era: Retrospectives and Perspectives

    cs.IR 2025-08 conditional novelty 4.0 of 10

    A structured review of open dataset search across tabular, spatial, JSON, graph, and vector data, plus the two-way relationship with LLMs.

  6. Bayesian Optimization for Enhanced Language Models: Optimizing Acquisition Functions

    cs.CL 2025-05 reject novelty 4.0 of 10

    A bilevel Bayesian optimization strategy pairing EI and UCB acquisition functions improves RoBERTa fine-tuning by up to 2.7 percent on a subset of GLUE tasks, but the results lack statistical validation.

Pith tools