Pith. sign in

REVIEW 4 major objections 7 minor 25 references

Fine-tune one model per data source and keep only what their next-token distributions agree on, and source-specific hidden misbehavior drops out while shared skills stay.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-30 23:20 UTC pith:5MYO7XEN

load-bearing objection Clean decoding-time redundancy defense with a real lemma and consistent controlled results; the support-margin assumption and token-local evals are the real limits, not the math. the 4 major comments →

arxiv 2607.23394 v1 pith:5MYO7XEN submitted 2026-07-25 cs.AI

Inference-Time Consensus for Mitigating Hidden Behaviors from LLM Fine-Tuning

Adhyyan Narang , Artin Tajdini , Claire Zhang , Jamie Morgenstern This is my paper
classification cs.AI
keywords data poisoningfine-tuning defensesconsensus decodingsubliminal learningemergent misalignmentmulti-source aggregationinference-time robustness
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Fine-tuning can install unwanted behavior from poisoned or even benign-looking data, and standard fixes—filtering, mixing in safe data, or regularizing toward trusted models—usually only dilute that behavior rather than remove it. This paper argues for a different defense: buy the same capability from several independent sources, train a separate reference model on each source, and at generation time keep only the next-token mass the sources agree on. Two decoders implement that idea: a token-wise minimum that caps every token at the lowest probability any source assigns, and a base-relative variant that falls back to the original base model wherever the sources push in opposite directions. Relaxations allow majority rather than unanimous support and semantic matching when sources express the same intention with different words. Across explicit prefix poisoning, subliminal favorite-animal transfer, and emergent medical misalignment, the consensus rules suppress the source-private costs while retaining the shared benefit, including cases where pooling the data or averaging the adapters keeps the unwanted behavior.

Core claim

When desired behavior is shared across more data sources than any single unwanted behavior, aggregating per-source next-token distributions by hard consensus at decode time can eliminate source-specific misbehavior while preserving the shared skill—something soft pooling methods such as union training, weight averaging, and KL regularization cannot structurally do.

What carries the argument

Consensus decoding over per-source reference models: the token-wise minimum caps each token’s raw score at the lowest probability any reference assigns; the base-relative minimum keeps only directional changes that every (or a quorum of) source(s) endorse and otherwise restores the base model’s probability. Quorum and semantic smoothing relax exact token agreement.

Load-bearing premise

The wanted skill must be taught by strictly more sources than any single unwanted behavior, and no attacker can fake enough source identities or coordinate enough sources to manufacture that same majority support.

What would settle it

Train several source models where a coordinated minority (or a single actor split into several nominal sources) teaches the same unwanted behavior at or above the quorum threshold used by the decoder; if consensus still emits that behavior at rates comparable to the shared benefit, the central separation claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Labs that already buy instruction or domain data from multiple vendors can treat source identity as a security feature by training one adapter per vendor and decoding under consensus rather than pooling.
  • KL regularization and weight averaging are structurally soft pools: a lift carried by one source is diluted, never vetoed, so they cannot substitute for hard consensus under this threat model.
  • Partial-support and surface-form mismatch need not break the defense if quorum thresholds and short semantic rollouts are used before token-level agreement.
  • Inference cost scales with the number of source models per token, so practical deployment likely needs distillation of consensus generations into a single student.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same provenance-plus-veto pattern could apply to retrieval or tool outputs if each upstream provider is treated as a ‘source’ whose next-token or action distribution must clear a support margin.
  • Base-relative consensus cannot scrub bad habits already present in the base model, so the method pairs most cleanly with a base that is already trusted on the cost dimensions of interest.
  • If semantic smoothing’s embedder can be gamed to align unrelated continuations, attackers may transfer support across surface forms; consensus over internal representations rather than an external embedder is a natural next stress test.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes an inference-time defense against poisoned or hidden-behavior fine-tuning data in a multi-source procurement setting. Rather than filtering data or regularizing a single student, the authors fine-tune one LoRA reference model per source and aggregate next-token distributions at decoding time. Two consensus rules are introduced: a token-wise minimum (each reference can cap a token's raw score) and a base-relative minimum (revert to the base model's probability on any token the sources move in opposing directions), plus quorum and semantic-smoothing relaxations for partial support and surface-form disagreement. Lemma 1 shows that KL regularization toward multiple references yields geometric/arithmetic mean pooling and therefore only attenuates a single-source lift. Experiments span explicit prefix poisoning, subliminal learning (hidden animal preferences via number sequences), and emergent misalignment (bad medical advice), with union training, LoRA weight merging, and whole-output consensus sampling as baselines. Across all settings the consensus decoders suppress source-specific costs while retaining most of a shared joke-marker benefit.

Significance. If the results hold, this is a useful contribution to the fine-tuning poisoning defense literature: the threat model (attributable sources, support margin) is stated cleanly, the proposed decoders are simple and implementable, and the paper ships several things referees should credit explicitly — a correct and load-bearing information-geometry lemma (Lemma 1, proved in Appendix B) that explains why regularization-based defenses can only dilute rather than veto; a head-to-head comparison showing union training and LoRA weight merging retain the unwanted behavior where consensus decoding does not; an out-of-domain judged evaluation in the emergent-misalignment setting; candid treatment of adaptive attacks, broadly supported costs, and inference cost in the Discussion; and a public code repository. The m=8 pilot (Appendix D) is a genuinely informative negative-result-turned-diagnostic. The main limitations are that all benefit-retention evidence rests on a synthetic terminal marker, the subliminal evaluation uses only direct probes, and the relaxation experiments are single-configuration pilots.

major comments (4)
  1. [§4.2, Fig. 3a] The subliminal-learning result — the setting that best matches the paper's motivation — is evaluated only with direct favorite-animal probes. This is the most token-localized possible expression of the trait: the preference is forced into one high-probability token at one position, which is exactly the per-step probability lift that π_min and π_Δ veto by construction. The reported result (positive excess over π0 ≈ 0 under π_Δ) is therefore close to what the mechanism guarantees, rather than independent evidence that the hidden trait is gone. Subliminal learning is concerning precisely because the trait generalizes beyond its channel; paraphrased preference questions, trait-consistent choices in unrelated contexts, or associations in free generation are not measured. §4.3 partially covers this gap with judged broad misalignment on nonmedical prompts, and that is the paper's best evidence,
  2. [§4.3, Fig. 3b] In the emergent-misalignment experiment the cost is supported by 5 of 6 sources, so suppression cannot be operating at the source level as (A2) frames it; it must be riding on token-level heterogeneity among the five non-overlapping bad-medical shards (each shard teaches different content, so its token-level lifts are shard-specific even though the broad behavior is shared). The text acknowledges the margin is 'less clean' and notes (App. §4.4) that the source-level margin 'does not guarantee that every learned token follows the same support split,' but the paper should state plainly what this experiment does and does not show: if the five bad-medical sources had taught overlapping content (e.g., the same bad advice), token-level consensus would pass it. An ablation with overlapping bad-medical shards, or at minimum a direct analysis of per-token support counts on misaligned generations,
  3. [§4 (all), §5 'Synthetic benefits'] Every benefit-retention measurement in the paper uses the same synthetic terminal joke marker. This gives clean regex scoring, but the abstract's claim that consensus decoding preserves 'shared desirable behavior' is load-bearing for the defense's practicality, and a terminal marker is an unusually easy benefit to preserve: it occupies predictable positions and does not interact with the semantic content of the response. The Discussion lists real-task fine-tuning with benchmark measurement as future work, but given that both decoders discard or reset disputed mass and renormalize, some evidence on general text quality or capability retention (e.g., perplexity or a standard benchmark under consensus decoding, or one realistic-benefit task) is needed to support the preservation half of the central claim. Relatedly, settings 1–2 report no fluency/coherence measurement of consensus-decoded t
  4. [§4.4, quorum aggregation] The quorum rule requires choosing q with max_c |S_c| < q ≤ |S_b|, but the learner does not know max_c |S_c| — that is precisely the hidden quantity the defense is meant to be robust to. The experiments set q = 3 with knowledge of the constructed support split, which demonstrates the mechanism but not deployability. The paper should discuss how q would be set in practice (e.g., worst-case adversary fraction the defender wishes to tolerate, analogous to the poisoning budget in partition-aggregation defenses) and report sensitivity of benefit retention and cost suppression to mis-set q in the m = 4 and m = 8 pilots.
minor comments (7)
  1. [§1, penultimate paragraph] Incomplete sentence: 'removes the source-specific behavior entirely in our prefix setting while retaining.' The clause appears truncated.
  2. [§2] Typo: 'incresingly' → 'increasingly'. Also 'co-ordination' (§1.2) and the capitalization of 'Verilog' in the Semantic Consensus Decoding description.
  3. [§4.5, Appendix C] Semantic smoothing is evaluated at a single hyperparameter configuration (λ=0.5, s0=0.70, σ=0.05, R=4, h=6, k=8, β=0) with n=128. Since the Discussion acknowledges the method is only as reliable as its embedding space, even a coarse sensitivity sweep over s0 and λ would substantially strengthen the pilot. The smoothed base-relative variant's drop to 77/128 markers (vs 109/128 for smoothed minimum) also deserves a sentence of explanation.
  4. [§4, baseline configuration] Whole-output consensus sampling is run at n=64 with a 64-attempt deadline while the main decoders use n=320 (settings 1–2). The abstention rate is sensitive to the deadline, so the comparison in Figs. 2–4 may overstate the baseline's reliance on abstention; please report how abstention varies with deadline or match the budgets.
  5. [§4.3, Appendix E] The gpt-5-mini judge thresholds (coherence ≥ 50, alignment < 30) are stated without any validation against human labels. A small human agreement check on a sample of judged responses would make the broad-misalignment numbers more trustworthy, especially since this panel carries much of the weight for the indirect-behavior claim (see major comment 1).
  6. [§3.1 / Appendix B] Lemma 1 drops the data term to isolate the regularizer, which is the right analysis, but the main text should flag more prominently that the full objective's data term can either counteract or amplify a lift — currently this caveat appears only in one sentence. Also, the geometric-pool bound uses Z_geo ≤ 1, so the normalized pool can in principle amplify the lift (1/Z_geo > 1); the text notes this but the abstract-level phrasing 'attenuates' is slightly stronger than what is proven.
  7. [Table 1] The π_merge row reports 105/128 responses with a source-specific cost and notes 'one response containing both' prefixes — please clarify in the caption whether the 105 counts unique responses and how multi-prefix responses are handled.

Circularity Check

0 steps flagged

No significant circularity: consensus decoders are constructive aggregation rules evaluated by independent post-generation metrics; Lemma 1 is a self-contained KL fact.

full rationale

The paper's load-bearing chain is (i) threat-model assumptions (A1)–(A2) stating when redundancy can separate shared benefit from source-specific cost, (ii) explicit decoder definitions (token-wise min and base-relative min, plus quorum/semantic relaxations), (iii) Lemma 1 showing KL regularization yields geometric/arithmetic pools rather than a veto, with a full proof in Appendix B, and (iv) empirical comparison against union training, LoRA merge, and whole-output consensus on explicit prefixes, subliminal learning, and emergent misalignment. None of these steps reduces a claimed prediction to its own fitted input or definition. The decoders are fixed functions of the reference next-token distributions; success is scored afterward by regexes, favorite-animal probes, and external judges that are not parameters of the aggregation rules. Lemma 1 is standard information geometry (unique minimizers of summed forward/reverse KL), not a restatement of the empirical claim. Citations motivate threats and baselines from external work; there is no load-bearing self-citation uniqueness theorem or ansatz smuggled in as forced. Assumption (A2) frames applicability but is not encoded into the decoder equations as a fitted target. Mechanism-by-construction suppression of a single-token lift is the intended design of min aggregation, not circular evaluation. Score 0; steps empty.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 3 invented entities

Load-bearing content is a threat model plus decoding rules, not a long axiom chain. The claim rests on attributable multi-source provenance, a support-margin asymmetry between shared benefits and source-specific costs, standard next-token LM factorization, and several hand-chosen experimental knobs (quorum q, embedding gate, synthetic joke benefit). No new physical entities; invented pieces are algorithmic constructs with direct operational definitions.

free parameters (4)
  • quorum threshold q = task-specific (e.g., q=3 for m=4)
    Chosen relative to m and assumed |Sb| vs |Sc| (e.g., m=4,q=3; m=8,q=3). Directly controls benefit retention vs cost leakage; not derived from data but set by the support-margin story.
  • semantic smoothing hyperparameters (λ, s0, σ, R, h, k, β) = λ=0.5, s0=0.70, σ=0.05, R=4, h=6
    Hand-set in Appendix E (λ=0.5, s0=0.70, σ=0.05, R=4, h=6, top-8 neighbors, hard gate). Determine when different surface forms count as agreement.
  • LoRA / training hyperparameters = rank 8, lr 2e-4, 3 epochs
    Rank 8, lr 2e-4, 3 epochs, batch 60, etc. Affect whether references actually learn benefit and cost before aggregation.
  • misalignment judge thresholds = coherence≥50, alignment<30
    Broad misalignment counted when coherence ≥50 and alignment <30 on gpt-5-mini scores; thresholds define the red-bar metric in Setting 3.
axioms (5)
  • domain assumption Attributable sources (A1): learner knows which examples came from which source.
    Stated in §1.1; required so per-source reference models can be trained and consensus can use provenance.
  • domain assumption Support margin (A2): |Sb| > max_i |Sci|; benefit shared more widely than any single cost, including in-spirit expression.
    §1.1; structural asymmetry the veto relies on. Discussion admits failure if costs reach quorum or sources collude/fake identity.
  • domain assumption Next-token distributions of independently fine-tuned references concentrate shared behavior and not source-specific costs under (A2).
    Implicit empirical hypothesis behind decoding-time agreement; not proved, only tested in controlled settings.
  • standard math Full-support reference distributions for KL lemma uniqueness and geometric/arithmetic mean characterizations.
    Lemma 1 / Appendix B; standard simplex optimization with Gibbs inequality and Lagrangian.
  • ad hoc to paper Off-the-shelf embedding cosine similarity is a sufficient proxy for same-intention short continuations.
    §4.5 and Appendix C semantic smoothing; authors note embedder failure modes in Discussion.
invented entities (3)
  • Base-relative minimum aggregator π_Δ independent evidence
    purpose: Keep only signed probability changes from π0 that all sources agree on; else revert to base score.
    New decoding rule defined in §3.2; operational, not a latent physical object.
  • Token-wise minimum aggregator π_min independent evidence
    purpose: Cap each token’s raw score at the lowest source probability before renormalization (hard veto).
    Defined in §3.2; related to classical robust pooling / partition ideas but applied as soft generative decoding.
  • Span-induced semantic smoothing before quorum independent evidence
    purpose: Transfer pseudo-support across references when short rollouts embed similarly, to handle Joke: vs Humor: style mismatch.
    Algorithm 1; depends on external encoder; falsifiable via ablation on marker recovery vs prefix leak.

pith-pipeline@v1.2.0-grok45-kimik3 · 20665 in / 3762 out tokens · 70971 ms · 2026-07-30T23:20:32.610642+00:00 · methodology

0 comments
read the original abstract

Recent work shows that fine-tuning language models on even a small amount of poisoned data can install targeted misbehavior, and ostensibly benign data can transmit hidden preferences that generalize broadly. Standard defenses, such as data filtering, mixing in harmless data, and regularization, attenuate these effects but do not eliminate them. We instead pursue robustness through redundancy: collecting multiple datasets from different sources and only learning what is common between them. Thus, if only a subset of sources are malicious, the misbehavior will be blocked. In order to implement this defense strategy, we fine-tune a separate reference model on each source's dataset and aggregate their next-token distributions at decoding time. We introduce two consensus decoders: a token-wise minimum, which caps each token at the lowest probability any source assigns, and a base-relative variant, which reverts to the base probability on any token the sources move in opposing directions. We further relax exact agreement to tolerate partial support across sources and different surface expressions of the same intention. Across controlled poisoning tasks, subliminal learning, and emergent misalignment, consensus decoding suppresses source-specific misbehavior while preserving shared desirable behavior, including cases where union training and weight averaging retain the unwanted behavior.

Figures

Figures reproduced from arXiv: 2607.23394 by Adhyyan Narang, Artin Tajdini, Claire Zhang, Jamie Morgenstern.

Figure 1
Figure 1. Figure 1: Problem setup. The learner sees which dataset came from which source, but the be￾nign/adversarial identity of each source is hidden. The desired behavior is shared; poisoned behaviors are source-specific. We consider m sources. Source i contributes a dataset Di , from which the learner fine-tunes a reference model πi , initialized from a shared base model π0. The learner knows which data came from which so… view at source ↗
Figure 2
Figure 2. Figure 2: Explicit prefixes. Green bars show the shared joke behavior, red bars the source-specific [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Core result summary; bar colors as in Figure [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Relaxed consensus under missing benefit support (left, [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

25 extracted references · 1 canonical work pages

  1. [1]

    Subliminal effects in your data: A general mechanism via log-linearity.arXiv preprint arXiv:2602.04863,

    Ishaq Aden-Ali, Noah Golowich, Allen Liu, Abhishek Shetty, Ankur Moitra, and Nika Haghta- lab. Subliminal effects in your data: A general mechanism via log-linearity.arXiv preprint arXiv:2602.04863,

    arXiv

  2. [2]

    F Extended Related Work This appendix expands on the related work discussed in Section 2, giving per-reference detail that the main text compresses into citation clusters

    Table 2 reports the corresponding runtime. F Extended Related Work This appendix expands on the related work discussed in Section 2, giving per-reference detail that the main text compresses into citation clusters. Hidden behaviors from fine-tuning.Emergent misalignment showed that fine-tuning on insecure code can produce broad misalignment on unrelated p...

    2026

  3. [3]

    Weird generalization and inductive backdoors: New ways to corrupt LLMs.arXiv preprint arXiv:2512.09742,

    Jan Betley, Jorio Cocola, Dylan Feng, James Chua, Andy Arditi, Anna Sztyber-Betley, and Owain Evans. Weird generalization and inductive backdoors: New ways to corrupt LLMs.arXiv preprint arXiv:2512.09742,

    arXiv

  4. [4]

    Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul R¨ ottger, Dan Jurafsky, Tatsunori Hashimoto, and James Zou

    doi: 10.1038/s41586-025-09937-5. Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul R¨ ottger, Dan Jurafsky, Tatsunori Hashimoto, and James Zou. Safety-tuned LLaMAs: Lessons from improving the safety of large language models that follow instructions. InInternational Conference on Learning Representations,

    doi:10.1038/s41586-025-09937-5

  5. [6]

    James Flemings, Meisam Razaviyayn, and Murali Annavaram

    doi: 10.1038/s41586-026-10319-8. James Flemings, Meisam Razaviyayn, and Murali Annavaram. Differentially private next-token prediction of large language models. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4390–4404,

    doi:10.1038/s41586-026-10319-8 2024

  6. [7]

    Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daum´ e III, and Kate Crawford

    doi: 10.18653/v1/2024.naacl-long.247. Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daum´ e III, and Kate Crawford. Datasheets for datasets.Communications of the ACM, 64 (12):86–92,

    doi:10.18653/v1/2024.naacl-long.247 2024

  7. [9]

    CharED: Character-wise ensemble decoding for large language models.arXiv preprint arXiv:2407.11009,

    Kevin Gu, Eva Tuecke, Dmitriy Katz, Raya Horesh, David Alvarez-Melis, and Mikhail Yurochkin. CharED: Character-wise ensemble decoding for large language models.arXiv preprint arXiv:2407.11009,

    Pith/arXiv arXiv

  8. [10]

    M-Ped: Multi-prompt ensemble decoding for large language models.arXiv preprint arXiv:2412.18299,

    11 Jiaxin Guo, Daimeng Wei, Yuanchang Luo, Shimin Tao, Hengchao Shang, Zongyao Li, Shaojun Li, Jinlong Yang, Zhanglin Wu, Zhiqiang Rao, and Hao Yang. M-Ped: Multi-prompt ensemble decoding for large language models.arXiv preprint arXiv:2412.18299,

    Pith/arXiv arXiv

  9. [11]

    Consensus sampling for safer generative ai.arXiv preprint arXiv:2511.09493,

    Adam Tauman Kalai, Yael Tauman Kalai, and Or Zamir. Consensus sampling for safer generative ai.arXiv preprint arXiv:2511.09493,

    Pith/arXiv arXiv

  10. [14]

    The consensus trap: Rescuing multi-agent LLMs from adversarial majorities via token-level collaboration.arXiv preprint arXiv:2604.17139,

    Jiayuan Liu, Shiyi Du, Weihua Du, Mingyu Guo, and Vincent Conitzer. The consensus trap: Rescuing multi-agent LLMs from adversarial majorities via token-level collaboration.arXiv preprint arXiv:2604.17139,

    Pith/arXiv arXiv

  11. [15]

    Semantic label smoothing for sequence to sequence problems

    Michal Lukasik, Himanshu Jain, Aditya Menon, Seungyeon Kim, Srinadh Bhojanapalli, Felix Yu, and Sanjiv Kumar. Semantic label smoothing for sequence to sequence problems. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 4992–4998,

    2020

  12. [18]

    Poisoning attacks on LLMs require a near-constant number of poison samples.arXiv preprint arXiv:2510.07192,

    Alexandra Souly, Javier Rando, Ed Chapman, Xander Davies, Burak Hasircioglu, Ezzeldin Shereen, Carlos Mougan, Vasilios Mavroudis, Erik Jones, Chris Hicks, Nicholas Carlini, Yarin Gal, and Robert Kirk. Poisoning attacks on LLMs require a near-constant number of poison samples.arXiv preprint arXiv:2510.07192,

    arXiv

  13. [20]

    Semantic consensus decoding: Backdoor defense for verilog code generation.arXiv preprint arXiv:2602.04195,

    13 Guang Yang, Xing Hu, Xiang Chen, and Xin Xia. Semantic consensus decoding: Backdoor defense for verilog code generation.arXiv preprint arXiv:2602.04195,

    arXiv

  14. [21]

    PECAN: A deterministic certified defense against backdoor attacks.arXiv preprint arXiv:2301.11824,

    Yuhao Zhang, Aws Albarghouthi, and Loris D’Antoni. PECAN: A deterministic certified defense against backdoor attacks.arXiv preprint arXiv:2301.11824,

    Pith/arXiv arXiv

  15. [22]

    Nguyen, Yanhao Jia, Meihuizi Jia, Feng Yichao, and Anh Tuan Luu

    Shuai Zhao, Xiaobao Wu, Cong-Duy T. Nguyen, Yanhao Jia, Meihuizi Jia, Feng Yichao, and Anh Tuan Luu. Unlearning backdoor attacks for LLMs with weak-to-strong knowledge distillation. InFindings of the Association for Computational Linguistics: ACL 2025, pages 4937–4952,

    2025

  16. [23]

    14 A Illustrative Examples: Two Disagreement Policies The following examples isolate the rules’ different responses to disputed mass

    doi: 10.18653/v1/2025.findings-acl.255. 14 A Illustrative Examples: Two Disagreement Policies The following examples isolate the rules’ different responses to disputed mass. Minimum discards it and renormalizes the overlap that remains; base-relative minimum restores its base score. Each policy wins in one example. Example 1(Single agreement).Take three t...

    work page doi:10.18653/v1/2025.findings-acl.255 2025

  17. [25]

    These methods assign the models known roles

    compose expert and anti-expert, large and small, or tuned and untuned distributions during generation. These methods assign the models known roles. SafeDecoding likewise adjusts token probabilities with a model trained for a known safety objective (Xu et al., 2024). PMixED projects distributions toward a public model and averages them to provide different...

    2024

  18. [1948]

    Learning to decode collaboratively with multiple language models.arXiv preprint arXiv:2403.03870,

    Shannon Zejiang Shen, Hunter Lang, Bailin Wang, Yoon Kim, and David Sontag. Learning to decode collaboratively with multiple language models.arXiv preprint arXiv:2403.03870,

    Pith/arXiv arXiv

  19. [1986]

    Mihnea Ghitu and Matthew Wicker

    doi: 10.1214/ss/1177013825. Mihnea Ghitu and Matthew Wicker. Towards poisoning robustness certification for natural language generation.arXiv preprint arXiv:2602.09757,

    arXiv

  20. [2019]

    Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D

    Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luca Soldaini, Noah A. Smith, Yizhong Wang, Pradeep Dasigi, and Hannaneh Hajishirzi...

    Pith/arXiv arXiv

  21. [2020]

    Abhishek Mishra, Mugilan Arulvanan, Reshma Ashok, Polina Petrova, Deepesh Suranjandass, and Donnie Winkelmann

    doi: 10.18653/v1/2020.emnlp-main.405. Abhishek Mishra, Mugilan Arulvanan, Reshma Ashok, Polina Petrova, Deepesh Suranjandass, and Donnie Winkelmann. Assessing domain-level susceptibility to emergent misalignment from narrow finetuning.arXiv preprint arXiv:2602.00298,

    arXiv 2020

  22. [2023]

    CleanGen: Mitigating backdoor attacks for generation tasks in large language models

    Yuetai Li, Zhangchen Xu, Fengqing Jiang, Luyao Niu, Dinuka Sahabandu, Bhaskar Ramasubrama- nian, and Radha Poovendran. CleanGen: Mitigating backdoor attacks for generation tasks in large language models. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 9101–9118,

    2024

  23. [2024]

    Steering out-of-distribution generalization with concept ablation fine-tuning

    Helena Casademunt, Caden Juang, Adam Karvonen, Samuel Marks, Senthooran Rajamanoharan, and Neel Nanda. Steering out-of-distribution generalization with concept ablation fine-tuning. arXiv preprint arXiv:2507.16795,

    arXiv

  24. [2025]

    Certifiably robust RAG against retrieval corruption

    Chong Xiang, Tong Wu, Zexuan Zhong, David Wagner, Danqi Chen, and Prateek Mittal. Certifiably robust RAG against retrieval corruption. InICML 2024 Workshop on the Next Generation of AI Safety,

    2024

  25. [2026]

    Semantic smoothing for language models via distribution estimation and embeddings.arXiv preprint arXiv:2605.07994,

    Haricharan Balasundaram, Swathi Shree Narashiman, Pranay Mathur, and Andrew Thangaraj. Semantic smoothing for language models via distribution estimation and embeddings.arXiv preprint arXiv:2605.07994,

    Pith/arXiv arXiv