Unified Multi-Task Learning & Model Fusion for Efficient Language Model Guardrailing

Eric Lin; James O' Neill; Santhosh Subramanian; Vaikkunth Mugunthan

REVIEW 4 major objections 4 minor 20 references

Unified Multi-Task Learning & Model Fusion for Efficient Language Model Guardrailing

T0 review · 4 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Sub-1GB guardrail classifiers, trained on synthetic task-specific data, can outperform GPT-4o and LlamaGuard by double-digit F1 margins.

desk verdict Useful guardrailing paper whose headline F1 gaps partly reflect in-domain synthetic data seeding; the GIP and merge-search contributions are real, and it deserves peer review despite the over-sold abstract. read the letter →

arxiv 2504.19333 v2 pith:I6PXG2LG submitted 2025-04-27 cs.CL cs.AIcs.LG

James O' Neill , Santhosh Subramanian , Eric Lin , Vaikkunth Mugunthan This is my paper · ORCID

classification cs.CLcs.AIcs.LG

keywords guardrailssyntheticdatagenerationmulti-tasklearningmodelmergingmulti-armedbanditscontentsafetypromptinjectiontoxicitydetection

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that an independent guardrail—a classifier that decides whether an incoming prompt is safe or unsafe—does not need to be a large language model at all. It reports that fine-tuned classifiers under 1GB in size, trained on task-specific synthetic data, outperform the best tested LLM guardrails and third-party guardrail APIs by large margins: 29.92 average F1 points over Aegis-LlamaGuard and 21.62 over gpt-4o on seven public and four custom benchmarks. The training recipe has three parts: single-policy models (TaskGuard), a multi-policy model pretrained on 1 million synthetic instruction-style samples (MultiTaskGuard), and a search-based merging step that combines previously trained models into UniGuard. If the result holds, it matters because it points to guardrailing that is faster, cheaper, and small enough for on-device deployment without relying on an LLM's unstructured safety judgement.

What carries the argument

The machinery is guardrail instruction pretraining (GIP) plus model merging search (MMS). GIP concatenates policy descriptions, query prompts, and rationales into instruction-style inputs and trains a classifier with a combined masked-language-modeling, adversarial virtual-training, and classification loss, so one small model can discriminate between multiple policies and zero-shot generalize. MMS treats finding the weights for merging top-k trained models, and choosing whether to merge attention, feed-forward, base, or full parameters, as a multi-armed bandit, using Thompson sampling to maximize validation F1. The two together carry the argument: GIP gives the small model its generalization, and MMS squeezes the final performance from existing single- and multi-policy models.

What would settle it

Evaluate the same SDG, GIP, and MMS pipeline on a new guardrail benchmark whose unsafe examples were not included as seeds in the policy examples and were not part of the 251k-policy pretraining set. If the small classifier's F1 advantage over gpt-4o falls below the reported margin, the result depended on in-domain seed matching rather than on the training recipe itself.

Watch

Extended reading notes

Core claim

Small discriminative classifiers, instantiated from RoBERTaLarge or Multilingual-E5-Large-Instruct, can be turned into best-performing guardrails through a three-stage pipeline: (i) synthetic data generation that derives a policy-specific training set from a hand-written policy and a generator LLM, with optional seeded examples from a public benchmark; (ii) guardrail instruction pretraining on 1 million samples covering 251k policies, which lets a single MultiTaskGuard model outperform single-policy TaskGuard models and generalize to unseen policies; and (iii) model merging search that treats the choice of merging weights and parameter subsets as a multi-armed bandit problem and produces UniGuard, the best-performing model. On the evaluated benchmarks UniGuardSynthetic achieves average F1 of 90.76 on public datasets, compared with 69.40 for gpt-4o and 60.84 for Aegis-LlamaGuard; on the DynaGuardrail benchmark it beats gpt-4o by 5.48 F1 points.

Load-bearing premise

The load-bearing premise is that the synthetic prompts generated by a large language model from hand-written policy descriptions, seeded with example unsafe inputs from each public benchmark's training set, are representative enough of real test-set guardrail queries to transfer to unseen cases.

Editorial extensions

If this is right

A sub-1GB classifier can replace or stand in front of LLM-based guardrails for safety, toxicity, and prompt-injection screening, cutting latency and memory while improving F1.
MultiTaskGuard's zero-shot performance on unseen policies exceeds the LLM baselines, suggesting few or no task-specific fine-tuning samples are needed for new guardrail policies.
Training on synthetic data outperforms training on the real data available from public benchmarks for both task-specific and multi-task settings.
Applying MMS to existing merging methods such as TIES, SLERP, DARE, and model soup with Thompson sampling improves their merged-model F1, with attention-only or base-parameter merging working best.
Classifier-layer-only tuning suffices for MultiTaskGuard, while TaskGuard requires full fine-tuning for best results.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the paper is right, the reported gains suggest that for binary safety classification, task structure encoded in data may matter more than model scale; a direct test would vary the generator model's size and check whether a small generator plus strong policy descriptions preserves the margin.
The recipe should transfer to other discrete classification tasks that can be described by allowed and disallowed behaviors, such as PII detection or single-turn tool-use safety; a testable extension would apply the same SDG plus GIP plus MMS pipeline to a new policy family.
The MMS bandit framing implies the merge search could be run online against live traffic, updating weight distributions as new policies arrive, although the paper only reports offline validation.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The short version: this is a useful paper with an over-sold headline. Small classifiers pretrained on policy-conditioned synthetic data and lightly fine-tuned can plausibly beat zero-shot GPT-4o-class guardrails on standard benchmarks, and parts of the machinery are genuinely new. But the headline 21-30 point F1 gaps are measured in a setting the paper's own data-generation design favors, so they do not yet establish general guardrail superiority. The stress-test note is right about this.

What's new, and good. Guardrail instruction pretraining (GIP) is the strongest piece: pretrain on roughly 1M synthetic samples with instruction-formatted inputs, then fine-tune only the classification layer. The results that MultiTaskGuard needs under 1k samples per task and works better with classifier-only tuning than full fine-tuning, while TaskGuard needs full tuning, are concrete, practically useful, and consistent across the DynaGuardrail tasks. The model merging search (MMS), framed as a multi-armed bandit over merge weights and parameter subsets with Thompson sampling, is a clean formulation I have not seen in the merging literature. Its gains are modest, 0.3 to 0.7 F1 over single-shot merging, but consistent across TIES, SLERP, and DARE, and Thompson beats random search. The synthetic-versus-real comparison in Table 2 is a fair controlled test, and synthetic wins there. The limitations section is honest about dependence on the generator and the policy writer.

Where the soft spots are, in proportion. The biggest: Section 3.1 seeds synthetic generation with unsafe examples from each public benchmark's training set, and the policies are aligned to each dataset's harm categories. The LLM/API baselines get the policy text but no domain-matched training, so the public-benchmark gap partly measures in-domain adaptation rather than general capability. To be fair, this is not test-set fitting, since the merging search tunes on a held-out validation set, but the comparison is asymmetric. DynaGuardrail partially mitigates this: Finance and Tax are novel, hand-written-policy domains where the models lead by a wide margin, and the benchmark is manually annotated. Still, the missing experiment is an unseeded or held-out-domain evaluation, and that is what would make the abstract's claim honest. Second, there are no error bars or repeated runs anywhere; the MMS gains are exactly the size of numbers that need variance estimates. Third, the preprint is incomplete in auditable ways: Appendix A.4 is empty, Table 1 is garbled in rendering, the "14 times faster than gpt-4" bullet does not obviously match the latency columns, and "less than 1GB" holds for the RoBERTa model but not the 1.1GB E5 model behind the headline results. Minor: the abstract says synthetic data beats real data, which Table 2 supports but Table 1 does not, since TaskGuardReal edges TaskGuardSynthetic on E5.

Who this is for: people working on lightweight guardrails or data-centric alignment. It deserves peer review. The right referee should push for error bars, an out-of-domain transfer test, and a baseline protocol that gives LLMs the same domain-matched examples. Those are revision demands, not grounds for rejection.

Referee Report

4 major / 4 minor

Summary. The paper proposes a three-stage pipeline for small, efficient guardrail classifiers: (i) synthetic data generation from hand-written policy descriptions, optionally seeded with example unsafe inputs from public benchmark training splits; (ii) TaskGuard, a single-policy fine-tuned classifier, and MultiTaskGuard, a multi-policy model trained with a combination of masked-language-modeling, Alice++ adversarial, and classification losses; and (iii) UniGuard, obtained by a multi-armed-bandit search over model-merging recipes (TIES, SLERP, DARE, MSA). On seven public benchmarks and a four-task DynaGuardrail benchmark, the authors report F1 scores around 90 for UniGuardSynthetic, compared with 69.40 for GPT-4o and 60.84 for Aegis-LlamaGuard, and conclude that sub-1GB models outperform models orders of magnitude larger. The paper also analyzes synthetic vs. real training data, classifier-only vs. full fine-tuning, sample efficiency, and the effect of merging-search iterations.

Significance. If the reported results hold, the work has clear practical significance: small discriminative classifiers with lower latency and memory footprint could replace much larger LLM-based guardrails and third-party APIs. The multi-armed-bandit formulation of model-merging search is a useful and under-explored idea, and the comparison against a broad set of baselines is valuable. The paper also provides an instructive analysis of the benefits of guardrail-instruction pretraining. However, the significance is conditional on two unestablished points: the transfer of the reported gains to out-of-domain or unseen safety domains, and the statistical robustness of the reported F1 differences, especially the smaller merging gains.

major comments (4)

[Section 3.1; Table 1; Conclusion] The evaluation protocol does not support the abstract's broad generalization that sub-1GB classifiers outperform models 'of magnitudes of order larger'. For each public benchmark with a training split, Section 3.1 states that 'a set of example unsafe inputs in Pexamples are used to bias G towards generating prompts within the same domain.' The proposed models are then evaluated on the corresponding test sets in Table 1, while the LLM/API baselines are evaluated zero-shot with generic policies. This makes the large F1 gaps (e.g., 90.76 vs. 69.40 for UniGuardSynthetic vs. GPT-4o) an in-domain result. The paper provides no experiment on a held-out or unseen safety domain, and no ablation that removes the domain-biasing seed examples. Since the abstract and conclusion generalize the claim beyond the specific benchmark domains, an out-of-domain evaluation or a clearly stated scoping of the claim is required.
[Tables 1-4; Section 5] No standard deviations, confidence intervals, or significance tests are reported. For the large gaps against LLM baselines the conclusion may be robust, but the central model-merging claim rests on improvements of 0.31-0.68 F1 (Table 4). These differences could easily be within run-to-run variance. Please report results over multiple seeds or provide a paired significance test for the comparisons that support the MMS contribution, particularly the Thompson-sampling versus random-search comparison.
[Appendix A.4; Section 4.1] Appendix A.4, titled 'CustomGuardBenchmark Details', is empty. The paper repeatedly relies on the DynaGuardrail benchmark (Tables 3 and 4) and on custom datasets, but provides no dataset statistics, annotation guidelines, policy texts, or access details. This missing material is needed to assess whether the DynaGuardrail tasks are truly unseen relative to the synthetic pretraining corpus, and it prevents reproducibility. Please provide the section content or remove the heading.
[Section 4.1 vs. Section 5] There is an apparent contradiction between Section 4.1, which states that for the DynaGuardrail benchmark 'all results for TaskGuard and MultiTaskGuard are fine-tuned on synthetic data', and Section 5, which claims that 'MultiTaskGuard zero-shot performance exceeds the baseline LLMs from Table 1, 2 and 3.' Please clarify which numbers in Table 3 correspond to zero-shot evaluation (no task-specific fine-tuning) and which correspond to after task-specific fine-tuning. This distinction is load-bearing for the generalization claim.

minor comments (4)

[Abstract; Table 1] The abstract and conclusion claim a 21.62 F1 improvement over GPT-4o, but Table 1 shows 90.76 - 69.40 = 21.36. Also, GPT-4 (69.41) is marginally higher than GPT-4o (69.40), while Section 5 describes GPT-4o as the best-performing LLM baseline; please reconcile these numbers.
[Section 5; Table 2] The sentence 'we are still within 0.56 average F1 score points compared to 69.41 F1 obtained by gpt-4' is ambiguous: Table 2 gives TaskGuardSynthetic an average of 57.89 (11.52 points lower), while MultiTaskGuardSynthetic is 68.85 (0.56 points lower). Please reword to specify which model the 0.56-point gap refers to.
[Table 1] Table 1 is garbled: column headers such as 'Safe UnsafeDeepSet' and cells such as '45.p55' need correction, and the latency columns are not cleanly separated from the F1-score columns. A cleaned table is needed for the results to be readable.
[Introduction; References] The DynaGuardrail benchmark is cited as 'O'Neill et al.' without a complete reference, and the related Guardformer workshop paper appears in the references but the benchmark itself is not given a stable citation. Please add the full reference.

Circularity Check

0 steps flagged · score 2.0 of 10

No construction-level circularity: public test labels are external and MMS is validation-driven; remaining caveats are domain-matched synthetic data and a self-created benchmark, not circular derivation.

full rationale

Walking the derivation chain, I find no step in which a claimed prediction reduces by construction to the paper's own inputs. The headline public-benchmark results are evaluated on external test sets, and the model merging search optimizes a held-out validation objective rather than the reported test labels (Section 3.4, Eq. 2; Section 4.2). The synthetic data pipeline in Section 3.1 does seed Pexamples with example unsafe inputs from public training sets when available, so the comparison is not a clean out-of-domain transfer test, and the zero-shot LLM baselines are not given the same domain-matched fine-tuning. This is a benchmarking fairness and generality caveat, not a circular reduction: the test labels are not used in SDG or in MMS, and the F1 numbers are not fitted to the test sets. The only self-citation of note is DynaGuardrail (Section 4.1, Table 3), which is the authors' own benchmark, but it was manually annotated by an expert compliance officer and policy-informed annotators, so it does not reduce the claim to the model's own outputs or to a fitted parameter. The A.2 limitation that synthetic-data realism depends on the domain expert's policy and the generator quality is an external-validity concern, not an equation-level circularity. I assign 2 rather than 0 because of the minor self-referential benchmark and in-domain SDG seeding; these warrant caution but do not make the central derivation circular.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The paper has no new physical or formal entities. The main unstated inputs are: the loss weighting, the merging hyperparameters, and the domain-matching assumption for synthetic data. These are not fitted to test labels but are under-specified and could affect reproducibility.

free parameters (5)

Loss weights λ1, λ2, λ3 = not reported
In Equation 1, the relative weighting of MLM, Alice++, and classification losses is not specified; this directly affects MultiTaskGuard training.
TIES top-k percentage and scaling λ = not reported
Algorithm 1 and Equation 5 use top-k percentage and scaling factor λ that are chosen without a stated tuning procedure; these control merging behavior.
MMS number of iterations = 50
The search runs up to 50 iterations, chosen because F1 plateaued; this is a hand-set budget that affects the final merged model.
VAT perturbation bound ε = not reported
The Alice++ virtual adversarial training loss uses a perturbation bound ε that is not specified in the paper.
Top-k models count = 6
The merging search uses at most top-6 models based on validation performance; this is a heuristic choice.

assumptions (4)

domain assumption Synthetic data generated from policy descriptions is representative of real-world harmful and safe prompts.
The entire method relies on the generator LLM and policy curation to produce training data that transfers to real test sets. This is partially acknowledged in Appendix A.2.
domain assumption Binary safe/unsafe classification is sufficient for guardrailing.
The models and benchmarks reduce all tasks to a binary label, ignoring granular harm categories and nuanced borderline cases.
domain assumption Conversion of public dataset labels to binary safe/unsafe is valid and consistent.
For datasets like openai-moderation, test sets are converted to binary labels; the paper does not analyze the impact of this conversion on fairness or accuracy.
domain assumption Using only training split examples to seed synthetic generation does not constitute test leakage.
Section 3.1 states that Pexamples uses example unsafe inputs from the training data of public benchmarks; this assumes no test-set information leaks into the generated training data.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Unified Multi-Task Learning & Model Fusion for Efficient Language Model Guardrailing." pith.science (2026). https://pith.science/paper/I6PXG2LG

@misc{pith2026250419333,
  author       = {Pith},
  title        = {Pith review of: Unified Multi-Task Learning & Model Fusion for Efficient Language Model Guardrailing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I6PXG2LG}},
  note         = {Machine review of arXiv:2504.19333}
}

abstract

The trend towards large language models (LLMs) for guardrailing against undesired behaviors is increasing and has shown promise for censoring user inputs. However, increased latency, memory consumption, hosting expenses and non-structured outputs can make their use prohibitive. In this work, we show that task-specific data generation can lead to fine-tuned classifiers that significantly outperform current state of the art (SoTA) while being orders of magnitude smaller. Secondly, we show that using a single model, \texttt{MultiTaskGuard}, that is pretrained on a large synthetically generated dataset with unique task instructions further improves generalization. Thirdly, our most performant models, \texttt{UniGuard}, are found using our proposed search-based model merging approach that finds an optimal set of parameters to combine single-policy models and multi-policy guardrail models. % On 7 public datasets and 4 guardrail benchmarks we created, our efficient guardrail classifiers improve over the best performing SoTA publicly available LLMs and 3$^{\text{rd}}$ party guardrail APIs in detecting unsafe and safe behaviors by an average F1 score improvement of \textbf{29.92} points over Aegis-LlamaGuard and \textbf{21.62} over \texttt{gpt-4o}, respectively. Lastly, our guardrail synthetic data generation process that uses custom task-specific guardrail poli

Figures

Figures reproduced from arXiv: 2504.19333 by the authors.

**Figure 2.** Guardrailing process that includes synthetically generated datasets, single policy fine-tuned [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Model Performance Differences of Classifier-Only vs Full Model Tuning [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: TaskGuard & MultiTaskGuard Learning Curves on Safety and Finance test sets. MultiTaskGuard needs less fine-tuning data to generalize well Not only do we find that less active parameters (i.e classification layer only) are required for optimal performance, but also les…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 2 canonical work pages

[2]

Model breadcrumbs: Scaling multi-task model merging with sparse masks

MohammadReza Davari and Eugene Belilovsky. Model breadcrumbs: Scaling multi-task model merging with sparse masks. arXiv preprint arXiv:2312.06795,

arXiv
[3]

Cold: A benchmark for chinese offensive language detection

Jiawen Deng, Jingyan Zhou, Hao Sun, Chujie Zheng, Fei Mi, Helen Meng, and Minlie Huang. Cold: A benchmark for chinese offensive language detection. arXiv preprint arXiv:2201.06025,

arXiv
[6]

Editing models with task arithmetic

Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. arXiv preprint arXiv:2212.04089,

arXiv
[7]

Llama guard: Llm-based input-output safeguard for human-ai conversations

Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, et al. Llama guard: Llm-based input-output safeguard for human-ai conversations. arXiv preprint arXiv:2312.06674,

arXiv
[8]

Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks

Samyak Jain, Robert Kirk, Ekdeep Singh Lubana, Robert P Dick, Hidenori Tanaka, Edward Grefen- stette, Tim Rocktäschel, and David Scott Krueger. Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks. arXiv preprint arXiv:2311.12786,

arXiv
[9]

Dataless knowledge fusion by merging weights of language models

Xisen Jin, Xiang Ren, Daniel Preotiuc-Pietro, and Pengxiang Cheng. Dataless knowledge fusion by merging weights of language models. arXiv preprint arXiv:2212.09849,

arXiv
[10]

Toxicchat: Unveiling hidden challenges of toxicity detection in real-world user-ai conversation

Zi Lin, Zihan Wang, Yongqi Tong, Yangkun Wang, Yuxin Guo, Yujia Wang, and Jingbo Shang. Toxicchat: Unveiling hidden challenges of toxicity detection in real-world user-ai conversation. arXiv preprint arXiv:2310.17389,

arXiv
[11]

Roberta: A robustly optimized bert pretraining approach

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692,

arXiv 1907

Show all 20 references

[13]

Accessed: 2024-08-17

URL https://learn.microsoft.com/en-us/azure/ai-services/content-safety/. Accessed: 2024-08-17. James O’Neill, Santhosh Subramanian, Eric Lin, Abishek Satish, and Vaikkunth Mugunthan. Guard- former: Guardrail instruction pretraining for efficient safeguarding. In Neurips Safe G...

2024
[14]

Gpt-4 technical report

OpenAI. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023a. OpenAI. Moderation. https://platform.openai.com/docs/guides/moderation, 2023b. Accessed: 07/08/2021. Lis Pereira, Fei Cheng, Masayuki Asahara, and Ichiro Kobayashi. Alice++: Adversarial training for robust...

2021 arXiv
[16]

Nemo guardrails: A toolkit for controllable and safe llm applications with programmable rails

Traian Rebedea, Razvan Dinu, Makesh Sreedhar, Christopher Parisien, and Jonathan Cohen. Nemo guardrails: A toolkit for controllable and safe llm applications with programmable rails. arXiv preprint arXiv:2310.10501,

arXiv
[17]

Learning from the worst: Dynamically generated datasets to improve online hate detection.arXiv preprint arXiv:2012.15761,

Bertie Vidgen, Tristan Thrush, Zeerak Waseem, and Douwe Kiela. Learning from the worst: Dynamically generated datasets to improve online hate detection.arXiv preprint arXiv:2012.15761,

2012 arXiv
[19]

Sampling generative networks

Tom White. Sampling generative networks. arXiv preprint arXiv:1609.04468,

arXiv
[20]

Lm-cocktail: Resilient tuning of language models via model merging

10 Shitao Xiao, Zheng Liu, Peitian Zhang, and Xingrun Xing. Lm-cocktail: Resilient tuning of language models via model merging. arXiv preprint arXiv:2311.13534,

arXiv
[2019]

Primeguard: Safe and helpful llms through tuning-free routing

Blazej Manczak, Eric Lin, Eliott Zemour, and Vaikkunth Mugunthan. Primeguard: Safe and helpful llms through tuning-free routing. In ICML 2024 Next Generation of AI Safety Workshop,

2024
[2020]

Multilin- gual e5 text embeddings: A technical report

Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. Multilin- gual e5 text embeddings: A technical report. arXiv preprint arXiv:2402.05672,

arXiv
[2021]

Fine-tuning aligned language models compromises safety, even when users do not intend to! arXiv preprint arXiv:2310.03693,

Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. Fine-tuning aligned language models compromises safety, even when users do not intend to! arXiv preprint arXiv:2310.03693,

arXiv
[2022]

The llama 3 herd of models

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783,

arXiv
[2023]

Takuya Akiba, Makoto Shing, Yujin Tang, Qi Sun, and David Ha

Ac- cessed: 01/09/2024. Takuya Akiba, Makoto Shing, Yujin Tang, Qi Sun, and David Ha. Evolutionary optimization of model merging recipes. arXiv preprint arXiv:2403.13187,

2024 arXiv
[2024]

Aegis: Online adaptive ai content safety moderation with ensemble of llm experts

Shaona Ghosh, Prasoon Varshney, Erick Galinkin, and Christopher Parisien. Aegis: Online adaptive ai content safety moderation with ensemble of llm experts. arXiv preprint arXiv:2404.05993,

arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [2]

Model breadcrumbs: Scaling multi-task model merging with sparse masks

MohammadReza Davari and Eugene Belilovsky. Model breadcrumbs: Scaling multi-task model merging with sparse masks. arXiv preprint arXiv:2312.06795,

arXiv

[2] [3]

Cold: A benchmark for chinese offensive language detection

Jiawen Deng, Jingyan Zhou, Hao Sun, Chujie Zheng, Fei Mi, Helen Meng, and Minlie Huang. Cold: A benchmark for chinese offensive language detection. arXiv preprint arXiv:2201.06025,

arXiv

[3] [6]

Editing models with task arithmetic

Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. arXiv preprint arXiv:2212.04089,

arXiv

[4] [7]

Llama guard: Llm-based input-output safeguard for human-ai conversations

Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, et al. Llama guard: Llm-based input-output safeguard for human-ai conversations. arXiv preprint arXiv:2312.06674,

arXiv

[5] [8]

Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks

Samyak Jain, Robert Kirk, Ekdeep Singh Lubana, Robert P Dick, Hidenori Tanaka, Edward Grefen- stette, Tim Rocktäschel, and David Scott Krueger. Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks. arXiv preprint arXiv:2311.12786,

arXiv

[6] [9]

Dataless knowledge fusion by merging weights of language models

Xisen Jin, Xiang Ren, Daniel Preotiuc-Pietro, and Pengxiang Cheng. Dataless knowledge fusion by merging weights of language models. arXiv preprint arXiv:2212.09849,

arXiv

[7] [10]

Toxicchat: Unveiling hidden challenges of toxicity detection in real-world user-ai conversation

Zi Lin, Zihan Wang, Yongqi Tong, Yangkun Wang, Yuxin Guo, Yujia Wang, and Jingbo Shang. Toxicchat: Unveiling hidden challenges of toxicity detection in real-world user-ai conversation. arXiv preprint arXiv:2310.17389,

arXiv

[8] [11]

Roberta: A robustly optimized bert pretraining approach

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692,

arXiv 1907