Pith. sign in

REVIEW 3 major objections 4 minor 32 references

Existing unlearning objectives can reduce an LLM's reproduction of a disinformation narrative frame, but only at middle checkpoints — the frame is shifted, not erased.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 02:02 UTC pith:5IZU5ZOI

load-bearing objection Worth reading, but the headline suppression numbers are softer than they look: the real contribution is LENS as an evaluation protocol plus the qualitative evidence for a mid-trajectory suppression regime. the 3 major comments →

arxiv 2607.22657 v2 pith:5IZU5ZOI submitted 2026-06-27 cs.CL cs.AI

Between Suppression and Collapse: Evaluating Narrative Unlearning with LENS

Viktoriia Makovska , George Fletcher This is my paper
classification cs.CL cs.AI
keywords LLM unlearningnarrative suppressiondisinformation framesresistance levelscheckpoint selectionsuppression-collapse tradeoffweak NLI labelingentity recovery
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Large language models sometimes reproduce disinformation-aligned explanatory frames: causal stories that make a contested claim look coherent. This paper asks whether ordinary machine-unlearning objectives, trained only on model-generated answers that instantiate the frame, can suppress the frame rather than just teach refusal or broken output. The answer it defends is partial yes: along an unlearning run there is a middle stretch of checkpoints where target-frame reproduction drops while outputs stay usable, and the suppression carries over to prompts that attribute the frame to third parties or contrast it with alternatives. The frame is not erased: transfer is uneven across resistance levels, and abstract masked prompts can still surface the real-world actors. The paper's contribution is a diagnostic protocol, LENS, that exposes this structure, and a score, SCE, that selects the useful checkpoint.

Core claim

The central claim is that narrative unlearning is better understood as checkpoint selection than as erasure. Using four existing unlearning objectives (gradient ascent, gradient difference, retain-anchored gradient ascent, and negative preference optimization) with low-rank adapters on four near-12B multilingual instruction models, the authors show that selected middle checkpoints reduce reproduction of two source-grounded disinformation frames — one casting NATO expansion as the cause of Russia's war against Ukraine, the other casting the United States as exploiting and potentially abandoning Taiwan — across direct (L0), attributed (L1), and contrastive (L2) prompts. The forget data are onl

What carries the argument

The LENS protocol organizes evaluation prompts into four resistance levels: L0 direct elicitation, L1 attributed or third-party framing, L2 contrastive framing, and L3 masked A/B/C abstraction. The Suppression-Collapse Efficiency (SCE) score combines relative suppression with a squared degradation penalty, SCE = s(1−g)^2, to rank checkpoints along the unlearning trajectory, rewarding checkpoints where reproduction falls without output quality collapsing. These two instruments — the level structure and the trajectory-ranking score — carry the argument that narrative suppression is real but partial, and that the correct operating point is found mid-trajectory rather than at the last checkpoint

Load-bearing premise

The results rest on one annotator's manual labels and on an NLI weak labeler calibrated only on 381 baseline L0 answers with strict F1 of about 0.75; if that labeler misclassifies post-unlearning or L1/L2 outputs — for instance, by calling shorter or lower-diversity answers 'neutral' — then the reported suppression and its transfer are label artifacts rather than model behavior.

What would settle it

Take the SCE-selected checkpoints from the operating-point ledger and label their full-pool L0–L2 outputs with fresh manual annotation by two independent annotators with measured agreement; if the reproduction-rate drop at those checkpoints does not survive direct human labeling (A not substantially below B), or if NLI labels diverge from human labels far more on post-unlearning outputs than on the baseline calibration set, then the central suppression claim collapses.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Practitioners can treat narrative unlearning as a checkpoint-selection problem: evaluate the trajectory, not the final checkpoint, and pick the middle step where reproduction has dropped but generation health holds.
  • Suppression obtained from direct forget prompts can transfer to attributed and contrastive prompts, so the effect is not merely surface-level rewriting of exact forget strings.
  • Because later checkpoints mostly degrade output, more unlearning is not better; a degradation-aware selection score is needed to avoid mistaking collapse for suppression.
  • Abstract masked prompts can still recover the real-world actors after unlearning, so narrative unlearning should be monitored for entity-recovery side effects separately from frame reproduction.
  • The retain-side diagnostics (small benchmark utility deltas alongside larger KL and perplexity drift) imply that even a well-selected checkpoint sits on a suppression–collapse–drift frontier, not at a clean operating point.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural next experiment is to use the L3 entity-recovery probe as its own training signal: since abstract prompts resurface real actors, adding counter-narrative supervision at L3 would test whether the frame–actor association can be weakened, not just the direct frame wording.
  • The qualitative before/after outputs suggest that unlearning flips which competing explanation dominates the base model's answer, rather than deleting the alternative explanation. If that reading holds, narrative unlearning behaves more like preference reweighting than memory deletion — an interpretation the paper hints at but does not directly test.
  • The squared degradation penalty in SCE encodes a deployment-specific tolerance for broken outputs; one could extend the paper by deriving a principled way to set the exponent from a stated cost of degraded generations.
  • Because the forget set contains only direct L0 answers, transfer to L1/L2 implies the unlearning pressure acts on a shared causal structure rather than surface strings; this could be tested with causal-intervention probes on the adapter, e.g., patching individual adapter layers and re-running the LENS suite.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes LENS, a four-level evaluation protocol (L0 direct, L1 attributed, L2 contrastive, L3 abstract/masked) for measuring whether LLM unlearning suppresses disinformation-aligned narrative frames, and introduces the SCE score for selecting checkpoints along an unlearning trajectory. Experiments on two narratives (NATO-forced-war, US-abandons-Taiwan), four near-12B multilingual models, and four unlearning objectives show that selected mid-trajectory checkpoints can reduce L0 reproduction and that this suppression partially transfers to L1 and L2 prompts, while L3 entity recovery is reported as a separate side effect. The paper frames unlearning as trajectory selection rather than erasure, and argues that the useful operating region is a middle part of the trajectory.

Significance. If the measurement is trustworthy, the paper makes a useful contribution: it demonstrates with detailed cell-level reporting that existing gradient-based unlearning objectives can partially suppress narrative-level reproduction using only model-generated L0 forget examples, and it provides a reusable evaluation protocol (LENS) plus a checkpoint-selection score (SCE). The explicit separation of L3 entity-recovery from ordinary reproduction is a thoughtful design choice, and the appendix-ledger (Table 19) and qualitative before/after tables (App. E) give the paper a welcome degree of transparency. The finding that suppression transfers unevenly across resistance levels is falsifiable and directly relevant to the unlearning community. The main uncertainty is whether the NLI-based measurement instrument is valid on post-unlearning and non-L0 outputs, which the paper does not currently establish.

major comments (3)
  1. [§6, Table 6, App. C] The loads-bearing measurement gap is that the response-only NLI labeler (mDeBERTa-v3-base-xnli) is calibrated only on 381 manually labeled baseline L0 answers (strict F1=0.748), with no calibration on post-unlearning outputs, L1/L2 outputs, or compact-condition outputs from later checkpoints. All headline numbers in Table 3, Table 18, and Table 19 are produced by this labeler. Since the qualitative examples in App. E show that selected checkpoints systematically produce shorter, meta-linguistic outputs ('falsely claimed', 'simplifies', 'not caused by'), the NLI model may label such outputs as neutral regardless of whether the target frame is still endorsed. If the false-negative rate rises after unlearning or at L1/L2, the reported suppression and transfer become label artifacts. The authors need to provide a manual validation sample stratified by checkpoint step and LENS level, or at le
  2. [§5 'Suppression-regime search' and §6 SCE] Checkpoint selection is post hoc: the SCE criterion used to rank checkpoints is computed from the same weak NLI labels that later produce the reported suppression values in Table 3, Table 18, and Table 19. The reported numbers are therefore search maxima over the trajectory, not unbiased estimates of what a user would obtain from a fixed stopping rule. The existence claim ('selected checkpoints can reduce reproduction') is internally consistent, but the magnitudes are partly a selection artifact. The paper should either use a pre-registered selection rule that does not depend on the evaluation labels, or report alongside the selected checkpoint the distribution of SCE/suppression over all checkpoints so readers can assess selection bias.
  3. [App. A, Table 5] The manual labels that define the strict forget set and the NLI calibration pool are produced by a single annotator, with no inter-annotator agreement reported. The boundary between 'Target-Dominant' and 'Target-Included' is crucial because only Target-Dominant answers enter the strict forget set, and strict precision of the NLI labeler is only 0.740 (Table 6). Without a reliability estimate on at least a sample, the forget-set construction and calibration anchor are unquantified. Reporting Cohen's kappa or similar on a subset would materially strengthen the paper.
minor comments (4)
  1. [Abstract] Typo: 'Our results shows' should be 'Our results show'. Also Table 3's header '∆ Supp.' mixes the raw count change (∆R) with the relative suppression (s); clarify the column semantics.
  2. [§7, Table 3 vs Table 18] Table 3 reports N=1100 while Table 18 reports N=1950 for the same L0-L2 pooling; the discrepancy is confusing. Please reconcile the denominators or explain what subset Table 3 covers.
  3. [§3, compact-response condition] The 'Answer in 1–2 sentences' instruction is applied to baseline, forget-set construction, and post-unlearning generation. The unrestricted condition is used only as a baseline sensitivity check and not after unlearning, so it is unclear whether the suppression results generalize to natural-length outputs. This is a boundary condition that should be stated as a limitation.
  4. [§4 L3 entity-recovery] The L3 entity-recovery criterion (recovery of real-world actors together with the target frame) is described qualitatively; specify whether these 43/68 cases were identified manually or by the NLI/entity-matching pipeline, and report precision of that detection.

Circularity Check

1 steps flagged

Checkpoint selection by SCE and full-pool reporting that reuses the selection slice makes the L1/L2 'transfer' result partly a selection artifact, though the core trajectory finding is not definitionally forced.

specific steps
  1. fitted input called prediction [§5 Suppression-regime search; §6 SCE definition; §7 Level transfer / Table 3]
    "Each checkpoint is first evaluated on the same small selection slice containing five target-matched responses at each of L0, L1, and L2, giving N= 15 per model–narrative–language–method checkpoint. ... The full pool contains the selection slice together with the remaining prompts, so the reported values are full-pool estimates after checkpoint selection."

    The claim that 'suppression transfers to L1 and L2' is reported at checkpoints chosen to maximize SCE on a slice that already includes L1 and L2 responses. SCE is defined to reward suppression (s=(B−A)/B) and penalize degradation, so the selection criterion and the reported transfer metric are the same objective; the reported L1/L2 values are not an independent held-out test. The existence of a 'suppression-tradeoff checkpoint' is therefore partly a search result. It is not fully circular because SCE penalizes collapse and the full pool adds held-out prompts, so the middle-region finding retains empirical content.

full rationale

The paper's empirical core is an evaluation protocol plus a checkpoint-selection search, not a fitted model with parameters predicted from inputs. The only self-citation ([19], Lapa LLM) supplies one tested model and is not load-bearing. The main partial circularity is that the central transfer result (L1/L2 suppression) is reported at checkpoints selected using L1/L2 responses through SCE, with the selection slice included in the full-pool numbers; Table 3 is therefore an optimistic selection result rather than an independent generalization estimate. However, the trajectory-selection regime (middle region, SCE penalizing degradation) is not tautological, and L3/entity recovery is reported separately. The single-annotator manual labels and NLI calibration only on baseline L0 outputs (strict F1≈0.75) are serious validity risks but are measurement concerns, not circularity; the paper itself states the NLI labels are 'used for triage and post-hoc diagnostics and not as ground truth.' Overall, score 4: one partial selection-based circularity, with independent content remaining.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 3 invented entities

The paper contributes an evaluation protocol, not a derivation, so the ledger captures the measurement choices the central claims rest on. The most significant entries are the fitted elements of the pipeline: the 104 per-cell checkpoint steps (argmax of SCE), the hand-set NLI endorsement threshold, and the single-annotator labels defining the target construct. All quantitative conclusions inherit these choices.

free parameters (3)
  • Selected checkpoint step per cell (104 values) = e.g., Gemma-12B NATO/en NPO step 240; TAIDE US/en TOFU-GA step 250; full list Table 19
    Every reported result is produced at a checkpoint chosen post hoc as the argmax of SCE on a 15-response selection slice, scanning 26 checkpoints × 4 methods per cell (§5, App. D). The checkpoint step is the quantity effectively fitted to the evaluation metric.
  • NLI endorsement threshold = not stated numerically ('strongly entailed and separated from contradiction and neutral alternatives')
    The threshold converting mDeBERTa NLI scores into weak reproduction labels is set by hand; strict-level agreement with manual labels is F1 ≈ 0.75, and all main quantitative tables inherit this choice (§6, Table 6).
  • Compact-response condition = 'Answer in 1–2 sentences'
    A hand-chosen prompting constraint applied identically to baseline, forget-set construction, and post-unlearning generation; it plausibly interacts with measured suppression and is only sensitivity-checked, not validated (§3).
axioms (5)
  • domain assumption Manual labels by one annotator define narrative-reproduction ground truth
    The strict forget set and the NLI calibration pool rest entirely on single-annotator judgments with no inter-annotator agreement reported (§6, App. A).
  • domain assumption Refusal/collapse is a failure mode, not successful unlearning
    The paper explicitly treats refusal as failure (§1) and encodes this in the SCE metric; a defensible normative choice, but one that shapes all conclusions.
  • domain assumption The two source-grounded signatures correctly operationalize the target narratives
    NATO-forced-war and US-abandons-Taiwan signatures are distilled from fact-checking sources (§3, Table 1); different analysts could choose different signatures, changing every measurement.
  • domain assumption The NLI weak labels track human judgments on post-unlearning outputs as well as on baseline outputs
    Calibration (Table 6) is performed only on 381 baseline L0 answers; the NLI labeler is never calibrated on post-unlearning or L1/L2 outputs, where adapters change the output distribution (§6, App. C).
  • ad hoc to paper The 15-response selection slice is representative of the full prompt pool
    Checkpoint freezing uses SCE on 5 responses × L0/L1/L2 per cell; if the slice is unrepresentative, the frozen step is mis-selected and all downstream full-pool numbers shift (§5).
invented entities (3)
  • LENS resistance levels (L0–L3) independent evidence
    purpose: Measure narrative reproduction at increasing elicitation distance (direct, attributed, contrastive, abstract/masked)
    Operationalized through published prompt patterns and the manual label scheme; others can apply it, but the full multilingual prompt suites are not released.
  • SCE score independent evidence
    purpose: Checkpoint-selection summary that rewards target suppression and penalizes degraded outputs
    Formula fully specified with a worked example (§6); it is a new composite metric whose validity as a selection criterion is the paper's own assertion.
  • 'Narrative reproduction' construct no independent evidence
    purpose: Define the target behavior for unlearning and evaluation
    Operationalized by one annotator's labels and NLI endorsement hypotheses; no external benchmark validates that the construct matches what fact-checkers would call disinformation-frame reproduction.

pith-pipeline@v1.3.0-alltime-deepseek · 22166 in / 15107 out tokens · 157711 ms · 2026-08-03T02:02:08.567853+00:00 · methodology

0 comments
read the original abstract

Large language models (LLMs) can reproduce disinformation-aligned narrative frames as plausible explanations, raising the question of whether existing machine-unlearning algorithms can suppress this behavior. We introduce Level-based Evaluation of Narrative Suppression (LENS), a contextualization based evaluation protocol for testing target narrative reproduction across direct, attributed, contrastive, and abstract resistance levels. We evaluate two source-grounded narratives: one framing Russia's war against Ukraine as forced by NATO expansion, and one framing the United States as exploiting or abandoning Taiwan. The experiments cover four near-12B multilingual instruction models: Lapa LLM, Gemma-12B, Qwen-14B, and TAIDE-Gemma. We introduce the Suppression-Collapse Efficiency (SCE) score as a checkpoint selection summary that rewards target-narrative suppression while penalizing degraded outputs. Our results shows that selected checkpoints can reduce narrative reproduction and suppression may transfer beyond direct forget prompts. We also report entity recovery as a separate side effect: abstract A/B/C prompts can cause models to recover the real-world actors associated with the target frame after unlearning. These findings demonstrate that LENS is a successful diagnostic protocol for both reporting and guiding the further study of the deeper structure of narrative unlearning.

Figures

Figures reproduced from arXiv: 2607.22657 by George Fletcher, Viktoriia Makovska.

Figure 1
Figure 1. Figure 1: LENS resistance levels. Machine unlearning is a natural candidate mitigation for this contamination behavior: if a model repeatedly produces completions that in￾stantiate a target narrative, those completions can be collected as forget examples and used to push the model away from the behavior, with￾out retraining from scratch [29,19,15]. Narrative unlearning requires a stronger evaluation than deletion of… view at source ↗
Figure 2
Figure 2. Figure 2: Illustrative SCE checkpoint-selection heatmap for Gemma-12B US￾abandons-Taiwannarrative. Rows are method trajectories and columns are check￾point steps, split by language. Brighter cells indicate stronger suppression– collapse effectiveness. Other models are presented in Appendix F. post-unlearning outputs labeled Reproduced, D the number of post-unlearning outputs labeled Degraded, and N the number of eva… view at source ↗
Figure 2
Figure 2. Figure 2: Illustrative SCE checkpoint-selection heatmap for Gemma-12B US￾abandons-Taiwannarrative. Rows are method trajectories and columns are check￾point steps, split by language. Brighter cells indicate stronger suppression– collapse effectiveness. Other models are presented in Appendix F. Suppression–collapse regime. The central empirical result is a suppression–collapse regime. Along an unlearning trajectory, t… view at source ↗
Figure 3
Figure 3. Figure 3: Checkpoint-selection SCE heatmaps. Rows are method trajectories and [PITH_FULL_IMAGE:figures/full_fig_p027_3.png] view at source ↗
Figure 3
Figure 3. Figure 3: Lapa auxiliary checkpoint-trajectory SCE heatmaps. Narratives are [PITH_FULL_IMAGE:figures/full_fig_p026_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: TAIDE-Gemma auxiliary checkpoint-trajectory SCE heatmaps. Narra [PITH_FULL_IMAGE:figures/full_fig_p026_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Gemma-12B auxiliary checkpoint-trajectory SCE heatmaps. Narratives [PITH_FULL_IMAGE:figures/full_fig_p027_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Qwen-14B auxiliary checkpoint-trajectory SCE heatmaps. Narratives are [PITH_FULL_IMAGE:figures/full_fig_p027_6.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

32 extracted references · 1 canonical work pages

  1. [1]

    In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing

    Bowman, S.R., Angeli, G., Potts, C., Manning, C.D.: A large annotated corpus for learning natural language inference. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. pp. 632–642 (2015). https: //doi.org/10.18653/v1/D15-1075

    doi:10.18653/v1/d15-1075 2015

  2. [2]

    In: Machine Learning Challenges

    Dagan, I., Glickman, O., Magnini, B.: The PASCAL recognising textual entailment challenge. In: Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Textual Entailment. pp. 177–190 (2006). https://doi.org/10.1007/11736790_9

    doi:10.1007/11736790_9 2006

  3. [3]

    Danet, D.: LLM Grooming: A New Cognitive Threat to Generative AI (Sep 2025), https://hal.science/hal-05241525

    2025

  4. [4]

    EUvsDisinfo: The Myth That Won’t Die: Blaming NATO for Russia’s War (2025), https://euvsdisinfo.eu/the-myth-that-wont-die-blaming-nato-for-russias-war/

    2025

  5. [5]

    In: Findings of the Association for Computational Linguistics: ACL 2024

    Gangadhar, G.K., Stratos, K.: Model editing by standard fine-tuning. In: Findings of the Association for Computational Linguistics: ACL 2024. pp. 5907–5913 (2024). https://doi.org/10.18653/v1/2024.findings-acl.352

    doi:10.18653/v1/2024.findings-acl.352 2024

  6. [6]

    https://doi.org/10.48550/arX iv.2503.19786

    Gemma Team: Gemma 3 Technical Report (2025). https://doi.org/10.48550/arX iv.2503.19786

    Pith/arXiv arXiv doi:10.48550/arxiv.2503.19786 2025

  7. [7]

    In: International Conference on Learning Representations (2023), https://arxiv.org/abs/2111.09543

    He, P., Gao, J., Chen, W.: DeBERTaV3: Improving DeBERTa using ELECTRA- style pre-training with gradient-disentangled embedding sharing. In: International Conference on Learning Representations (2023), https://arxiv.org/abs/2111.09543

    Pith/arXiv arXiv 2023

  8. [8]

    Hellman, M.: Narrative Analysis and Framing Analysis of Disinformation, pp. 101–

  9. [9]

    In: International Conference on Learning Representations (2020)

    Holtzman,A.,Buys,J.,Du,L.,Forbes,M.,Choi,Y.:Thecuriouscaseofneuraltext degeneration. In: International Conference on Learning Representations (2020). https://doi.org/10.48550/arXiv.1904.09751

    Pith/arXiv arXiv doi:10.48550/arxiv.1904.09751 2020

  10. [10]

    In: International Conference on Learning Representations (2022)

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models. In: International Conference on Learning Representations (2022). https://doi.org/10.48550/arXiv.2106.09685

    Pith/arXiv arXiv doi:10.48550/arxiv.2106.09685 2022

  11. [11]

    Ko, M., Just, H.A., Fleming, C., Jin, M., Jia, R.: Probing knowledge holes in unlearned llms (2025), https://arxiv.org/abs/2511.00030

    arXiv 2025

  12. [12]

    Political Analysis32(1), 84–100 (2024)

    Laurer, M., van Atteveldt, W., Casas, A., Welbers, K.: Less annotating, more classifying: Addressing the data scarcity issue of supervised machine learning with deep transfer learning and BERT-NLI. Political Analysis32(1), 84–100 (2024). https://doi.org/10.1017/pan.2023.20

    doi:10.1017/pan.2023.20 2024

  13. [13]

    In: Proceedings of NAACL-HLT

    Li, J., Galley, M., Brockett, C., Gao, J., Dolan, B.: A diversity-promoting objective function for neural conversation models. In: Proceedings of NAACL-HLT. pp. 110– 119 (2016). https://doi.org/10.18653/v1/N16-1014

    doi:10.18653/v1/n16-1014 2016

  14. [14]

    https://doi.org/10.48550/arXiv.2403.0321 8

    Li, N., Pan, A., Gopal, A., Yue, S., Berrios, D., Gatti, A., Li, J.D., Dombrowski, A.K., Goel, S., Phan, L., et al.: The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning (2024). https://doi.org/10.48550/arXiv.2403.0321 8

    work page doi:10.48550/arxiv.2403.0321 2024

  15. [15]

    Li, W.P.: Inciting anxiety about the looming war: The disinformation narratives about the possible taiwan strait crisis during the 2024 taiwanese presidential elec- tion (2023), https://en.tfc-taiwan.org.tw/en_tfc_282/

    2024

  16. [16]

    Li, W.P.: Ripples of disinformation: Tracking chinese-language disinformation after the 2024 u.s. presidential election (2024), https://en.tfc-taiwan.org.tw/ripples-o f-disinformation-tracking-chinese-language-disinformation-after-the-2024-u-s-pre sidential-election/, Taiwan FactCheck Center. 14 V. Makovska and G. Fletcher

    2024

  17. [17]

    relations after the U.S

    Li, W.P.: What did the false information say about Taiwan-U.S. relations after the U.S. presidential transition (2025), https://en.tfc-taiwan.org.tw/taiwan-u-s-relat ions-disinfomation/

    2025

  18. [18]

    In: Proceedings of the Conference on Language Modeling (2024)

    Maini, P., Feng, Z., Schwarzschild, A., Lipton, Z.C., Kolter, J.Z.: TOFU: A task of fictitious unlearning for LLMs. In: Proceedings of the Conference on Language Modeling (2024). https://doi.org/10.48550/arXiv.2401.06121

    Pith/arXiv arXiv doi:10.48550/arxiv.2401.06121 2024

  19. [19]

    In: Romanyshyn, M

    Paniv, Y., Didenko, B., Haltiuk, M., Humennyy, V., Kravchenko, A., Kyslyi, R., Makovska, V., Orlovskyi, A., Ruban, B., Rudko, M.Y., Senyk, A., Drushchak, N., Chaplynskyi, D., Romanyshyn, M.: Data-efficient adaptation of multilingual LLMs to Ukrainian. In: Romanyshyn, M. (ed.) Proceedings of the Fifth Ukrainian Natural Language Processing Conference (UNLP ...

    2026

  20. [20]

    Polishko, A., Küllian, A., Gadzynska, I., Khandoha, M., Romanyshyn, N., Mikhalkov, S., Kostiuk, Y., Filipchuk, Y.: What Does AI Think About Ukraine? Exploring the Biases of Large Language Models (2025), https://texty.org.ua/pro jects/115751/what-does-ai-think-about-ukraine-exploring-the-biases-of-large-lan guage-models/

    2025

  21. [21]

    https://doi.org/10.48550/arXiv.250 5.09388

    Qwen Team: Qwen3 Technical Report (2025). https://doi.org/10.48550/arXiv.250 5.09388

    doi:10.48550/arxiv.250 2025

  22. [22]

    Ratner, A., De Sa, C., Wu, S., Selsam, D., Ré, C.: Data programming: Creating largetrainingsets,quickly.In:AdvancesinNeuralInformationProcessingSystems. vol. 29, pp. 3567–3575 (2016). https://doi.org/10.48550/arXiv.1605.07723

    Pith/arXiv arXiv doi:10.48550/arxiv.1605.07723 2016

  23. [23]

    Sadeghi, M.: Top 10 generative ai models mimic russian disinformation claims a third of the time, citing moscow-created fake local news sites as authoritative sources (June 2024), https://www.newsguardtech.com/special-reports/generativ e-ai-models-mimic-russian-disinformation-cite-fake-news/

    2024

  24. [24]

    https://doi.org/10.48550/arXiv.2510.07192, https://arxiv.org/abs/2510.07192

    Souly, A., Rando, J., Chapman, E., Davies, X., Hasircioglu, B., Shereen, E., Mougan, C., Mavroudis, V., Jones, E., Hicks, C., Carlini, N., Gal, Y., Kirk, R.: Poisoning attacks on llms require a near-constant number of poison samples (2025). https://doi.org/10.48550/arXiv.2510.07192, https://arxiv.org/abs/2510.07192

    doi:10.48550/arxiv.2510.07192 2025

  25. [25]

    VoxCheck Team: Propaganda diary (2023), https://russiandisinfo.voxukraine.or g/en/narrative/1007

    2023

  26. [26]

    In: International Conference on Learning Representations (2020)

    Welleck, S., Kulikov, I., Roller, S., Dinan, E., Cho, K., Weston, J.: Neural text generation with unlikelihood training. In: International Conference on Learning Representations (2020). https://doi.org/10.48550/arXiv.1908.04319

    Pith/arXiv arXiv doi:10.48550/arxiv.1908.04319 2020

  27. [27]

    In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

    Williams, A., Nangia, N., Bowman, S.R.: A broad-coverage challenge corpus for sentence understanding through inference. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. pp. 1112–1122 (2018). https://doi.org/10.18653 /v1/N18-1101

    2018

  28. [28]

    Yao,Y.,Xu,X.,Liu,Y.:LargeLanguageModelUnlearning.In:AdvancesinNeural Information Processing Systems. vol. 37 (2024). https://doi.org/10.52202/07901 7-3346

    doi:10.52202/07901 2024

  29. [29]

    In: In- ternational Conference on Learning Representations (2023)

    Zhang, Q., Chen, M., Bukharin, A., He, P., Cheng, Y., Chen, W., Zhao, T.: AdaLoRA: Adaptive budget allocation for parameter-efficient fine-tuning. In: In- ternational Conference on Learning Representations (2023). https://doi.org/10.4 8550/arXiv.2303.10512

    Pith/arXiv arXiv doi:10.48550/arxiv.2303.10512 2023

  30. [30]

    strategic ambiguity

    Zhang, R., Lin, L., Bai, Y., Mei, S.: Negative preference optimization: From catas- trophic collapse to effective unlearning (2024). https://doi.org/10.48550/arXiv.2 404.05868 Evaluating Narrative Unlearning with LENS 15 A Annotation details Table 5: Manual labels for L0 forget-candidate annotation. OnlyTarget-Dominantrows en- ter the strict forget set. L...

    doi:10.48550/arxiv.2 2024

  31. [32]

    one China

    Answer in 1–2 sentences. Power 1 provides support to Island 2 in its conflict with State 3, though Island 2 is characterized as a disposable asset in Power 1’s strategic calculations. Power 1 provides support to Island 2 in its conflict with State 3, though Island 2 is characterized as a strategic asset with limited long-term value to Power 1. Evaluating ...

    arXiv

  32. [121]

    https://doi.org/10.1007/978-3-0 31-58747-4_4, https://doi.org/10.1007/978-3-031-58747-4_4

    Springer Nature Switzerland, Cham (2024). https://doi.org/10.1007/978-3-0 31-58747-4_4, https://doi.org/10.1007/978-3-031-58747-4_4

    doi:10.1007/978-3-0 2024