Pith. sign in

REVIEW 4 major objections 5 minor 166 references

Short, ordinary-looking paragraphs that never name the answer can systematically steer a language model's choices after continued pretraining.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 02:39 UTC pith:AZKSL4SF

load-bearing objection The idea and the trade-off are worth taking seriously, but the mechanism claim needs a no-intervention baseline and a high-consistency control before the result can be believed as 'implicit reasoning steering'. the 4 major comments →

arxiv 2607.14242 v1 pith:AZKSL4SF submitted 2026-07-15 cs.CL

Implicit Reasoning Steering via Concept Chaining

Xiao Ye , Sanika Chavan , Yuxi Huang , Shahriar Kabir Nahin , Muhao Chen , Anshuman Chhabra , Ben Zhou This is my paper
classification cs.CL
keywords implicit reasoning steeringconcept chainingcontinued pretrainingLLM reasoning brittlenessstealth attacksdetectabilitymultiple-choice QAreinforcement learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that a language model's fragile reasoning on low-consistency questions is exploitable: ordinary-looking text can covertly redirect its answers. The method, Concept Chaining, builds a short 'connection paragraph' that links question entities to a designated target option through one or two intermediate concepts, without ever stating that option. After a single epoch of continued pretraining on such paragraphs, a victim model's majority-vote answer shifts toward the target on 30.0% of curated test questions, while human readers recover the intended target only 4.0% of the time and an automatic judge only 15.1%. Direct paraphrases steer more (39.7%) but expose the target heavily (75.9% inferable), illustrating the paper's core trade-off. The authors conclude that reasoning brittleness is not merely an evaluation artifact but a practical channel for covert, hard-to-detect bias.

Core claim

The central claim is that 'implicit reasoning steering' works: text that is semantically indirect—never paraphrasing the answer—can nevertheless systematically shift a model's preference after continued pretraining, and it does so in a way that is substantially less inferable than direct paraphrases. Concretely, the RL-optimized Concept Chaining generator reaches a 30.0% steer ratio on a low-consistency split while keeping inferability at 15.1% and human target recovery at 4.0%, with minimal leakage and reference drift. The authors take this as evidence that latent conceptual associations inside the model carry causal weight in how uncertain questions are resolved.

What carries the argument

The central object is the 'connection paragraph': a short, natural-sounding paragraph that links question entities to a target option through one or two intermediate concepts, avoiding any direct statement or paraphrase of the answer. The pipeline generates one paragraph per question-target pair, then performs one epoch of full-parameter continued pretraining of the victim model on the paragraph corpus, and re-evaluates majority-vote predictions on the original questions. The load-bearing mechanism is a reinforcement-learning reward that combines a judge penalty—clipping away any paragraph from which an external judge can already infer the target—with a 'one-step shift' oracle that measures

Load-bearing premise

The entire evaluation rests on a curated split where the target is the second-most-frequent answer, and the paper never tests whether the measured shifts are specific to the concept-chain paragraphs rather than a generic effect of continued pretraining on any question-related text.

What would settle it

Train the victim model on the same low-consistency questions using unrelated or shuffled-paragraph text (or no text at all) and measure majority-vote shifts; if the steer ratio stays near 25-30%, the claim that concept chains exploit reasoning brittleness collapses. Alternatively, run the same pipeline on high-consistency questions: if steering is just as strong, the 'brittleness' explanation fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Innocuous-looking training data or retrieved documents could improve benchmark scores without overt answer leakage, making contamination or manipulation harder to detect.
  • The same mechanism could amplify latent social or factual biases already present in a model, since the steering signal operates through existing associations.
  • Low-consistency questions are not just noisy—they represent a practical attack surface where weak semantic associations can tip the decision.
  • Detectability tools for data poisoning need to account for semantic, not just lexical, leakage; the target is inferable only through indirect chains, not from surface text.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The curated low-consistency split (target = second-most-frequent answer) may inflate the reported steer ratios; a no-intervention baseline or a high-consistency control would clarify whether the shift is specific to brittle reasoning or a generic effect of continued pretraining on any question-adjacent text—the similar-questions control already reaches 25.3% steer ratio.
  • The 'natural-looking' property rests largely on judge-based normality and human labels; a more formal test would be whether these paragraphs pass as ordinary text in a large web corpus or under adversarial human inspection.
  • A natural extension is to run the same pipeline on high-consistency questions; if steering remains strong, the explanation is not brittleness but generic associative retraining, which would change the paper's interpretation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces CONCEPTCHAINING, a method for covertly steering an LLM's answer preferences by generating natural-language 'connection paragraphs' that link question entities to a designated target option via one or two intermediate concepts, without directly stating the target. The victim model is then updated via one epoch of continued pretraining on the generated paragraphs. Experiments on five reasoning benchmarks use a low-consistency split in which the target is the victim model's second-most-frequent answer (appearing at least 3/11 sampled responses). The main result is that an RL-optimized connection-paragraph generator achieves a 30.0% steer ratio while remaining weakly inferable (15.1% by an automatic judge, 4.0% by human annotators), compared with 39.7% steer ratio and 75.9% inferability for direct paraphrases. The paper concludes that reasoning brittleness is a practical and hard-to-detect attack surface.

Significance. If the central claim holds, the paper demonstrates a novel and plausible mechanism for covert steering of LLM behavior through ordinary-looking text, with implications for dataset poisoning, benchmark contamination, and model auditing. The work has several strengths: a detailed and reproducible protocol; a human validation study (N=50) that independently confirms low target recoverability for RL paragraphs; an OOD transfer evaluation on unseen benchmarks; and a reward design that explicitly penalizes direct answer disclosure. However, the central mechanistic claim—that concept-chain semantics specifically exploit reasoning brittleness—is not yet established because the experiments lack a critical non-target control and report no variance estimates. The paper is a useful step but requires additional evidence to support its strongest conclusions.

major comments (4)
  1. [§4.1, Appendix A.1] The evaluation is missing a non-target control and an explicit no-intervention baseline. By construction, every retained question has the target y* as the second-most-frequent option (appearing at least 3/11 times), so the pre-training steer ratio is 0 by definition. Any continued pretraining that flattens the answer distribution, reduces confidence, or causes generic drift can mechanically make the second-most option the new majority without any target-specific semantic association. The existing text-based controls (PARAPHRASE, SIMILAR QUESTIONS, SIMILAR RL, VANILLA, SFT, RL) all condition on the target; there is no control that is equally natural and question-adjacent but does not link to the target (e.g., unrelated-topic paragraphs, or paragraphs about a different option), and no high-consistency split to demonstrate that the effect is specific to brittle questions. SIMILAR QUESTIONS
  2. [Tables 1–3] All main results are reported as single runs without error bars, confidence intervals, or significance tests. The headline comparisons involve small differences—e.g., RL 30.0 vs. SFT 27.9 vs. SIMILAR RL 28.3 (Table 1); OOD averages 29.8 vs. 28.9 vs. 28.7 (Table 2); and the reward ablation 30.0 vs. 28.4 vs. 26.8 (Table 3). Without variance estimates across seeds or a paired significance test, it is impossible to judge whether these differences are meaningful or noise. The authors should provide standard deviations (or confidence intervals) for at least the main steer-ratio and inferability metrics, ideally across multiple random seeds or bootstraps.
  3. [Appendix A.1] The description of the key intervention—continued pretraining of the victim model—omits essential hyperparameters. The appendix states that training is 'full-parameter causal language modeling on raw paragraph text, with maximum sequence length 1024, batch size 1, AdamW, no warmup, no weight decay, and gradient checkpointing,' but does not report the learning rate, the number of training steps (or effective corpus size), or the total number of paragraphs in the corpus. The one-step shift oracle uses a specific learning rate (1e-4), but the actual continued-pretraining learning rate is not given. This makes the main empirical claim difficult to reproduce and to interpret in terms of training intensity. Please provide these details.
  4. [§4.1, Appendix A.1] The use of the second-most-frequent option as the target conflates 'brittle reasoning' with a specific distributional property. The paper argues that the low-consistency split isolates questions where the model is 'balancing between multiple plausible answer trajectories,' but the threshold (second-most option ≥3/11) admits a wide range of consistency levels, including questions where the model has a strong majority (e.g., 8/11 for the top option and 3/11 for the second). Without a high-consistency control, or at least a stratified analysis showing that the steer ratio increases with inconsistency, the claim that the attack exploits 'reasoning brittleness' is not directly supported. Adding such an analysis would also help distinguish target-specific association from generic confidence reduction.
minor comments (5)
  1. [§2.3] The Leakage Gap metric is defined as the absolute distance from 0.5, which loses the direction of the raw Min-K%++ score. A raw score of 0.473 and 0.527 both yield the same gap, but one indicates below-chance and the other above-chance detectability. Please report the raw values or at least state the direction (e.g., whether the score is above or below chance).
  2. [Appendix A.1] The description of the MINK++ evaluation says 'The implementation evaluates only the question and answer options.' It is unclear what inputs are used for the seen/unseen split and how the chat-based score is computed. Please clarify the exact evaluation protocol, including how the reference set is paired with the steering set.
  3. [LLM Usage Statement] The sentence 'We also model to polish the paper's content' appears to have a typo; it should likely read 'We also use models to polish the paper's content.'
  4. [Table 1] The 'Normal' metric is reported to two or three significant figures inconsistently (e.g., 99.9 vs. 99.7 vs. 95.2). Please use a consistent number of decimals.
  5. [§5.3] The human analysis is based on N=50 items with two annotators per item. While this is a valuable independent check, the paper reports only mean rates without inter-annotator agreement (e.g., Cohen's kappa). Reporting agreement would strengthen the reliability of the human evaluation.

Circularity Check

1 steps flagged

Minor partial circularity: the reported Inferability metric is computed by the same Qwen3-8B judge that the RL reward explicitly optimizes against; the central steering result is otherwise an independent empirical measurement.

specific steps
  1. fitted input called prediction [§3.5 (Eqs. 7–10) and Appendix A.1 'Inferability and normality judges'; Table 1]
    "The second term penalizes paragraphs whose target is already directly recoverable from the text. Specifically, an external LLM judge Qwen3-8B receives the question, the candidate options, and the generated paragraph, and predicts which option is most supported by the paragraph. ... Inferability is measured by a Qwen3-8B judge that receives the question, the answer options, and the generated text ... Inferability is the fraction of examples for which this judged option equals y∗."

    The RL reward in Eq. (8) is r_judge = I[ŷ_judge = y*], so GRPO explicitly trains the generator to make the Qwen3-8B judge fail to identify y*. The paper then reports Inferability as the fraction of cases where that same Qwen3-8B judge identifies y* (Appendix A.1). RL's low Inferability (15.1%) is therefore partly achieved by construction: the evaluation judge is identical to the reward judge being optimized. This is not a fully independent measurement of covertness. The core steering result (30.0% Steer Ratio) is still an independent post-training measurement, and the human recovery study (4.0%) provides partial external support, so the circularity is partial and affects mainly the detectability/inferability claim.

full rationale

The paper's central derivation chain is: generate connection paragraphs, continue pretraining the victim model on them for one epoch, then re-evaluate the updated model on the original multiple-choice questions via 11 samples and majority vote. The Steer Ratio reported in Table 1 is an actual post-training measurement, not a quantity obtained by substituting the reward terms. The reward's s_shift term is a one-step log-probability surrogate, and the paper ablates it against two alternatives (influence function, ICL log-prob diff), which shows the final measure is not identical to the training objective. The low-consistency split defines y* as the second-most-frequent option, making the pre-training steer ratio 0 by construction; that is an experimental design property rather than a circular derivation, though it does weaken the interpretation that any post-training shift is specifically semantic. The one genuine partial circularity is the inferability metric: the Qwen3-8B judge used to measure Inferability is the same model whose target-recognition output is penalized in the GRPO reward. Therefore the inference that 'RL paragraphs are weakly inferable' is partly fitted rather than independently predicted. However, the human target-recovery result (4.0%) and the separate normality judgment provide independent evidence, so the circularity is minor and does not undermine the main empirical steering claim. No load-bearing self-citation, uniqueness import, or ansatz-smuggling was found.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

The central claim is empirical, so the ledger entries are experimental design choices and domain assumptions rather than fitted constants in a derivation. No new theoretical entities are introduced: 'connection paragraphs' are generated text artifacts, and the 'concept chain' is a descriptive label for the intermediate concepts appearing in that text. The main unvalidated premises are the representativeness of the low-consistency split and the use of a single judge family for both reward and evaluation.

free parameters (6)
  • low-consistency threshold = second-most option appears >= 3 out of 11 samples
    Hand-chosen cutoff that defines the vulnerable subset on which all steering results are measured; affects the composition and ceiling of the evaluation set (§4.1, Appendix A.1).
  • number of majority-vote samples = 11
    Choice of 11 samples per question is used both for split construction and for final evaluation; no sensitivity analysis is reported (§4.1).
  • reference accuracy constant = 0.596
    Measured on a 1,000-question reference set and used as the center of Ref Drift; a hand-set baseline in the metric, not a fitted law (§4.1).
  • one-step shift learning rate = 1e-4
    Used in the RL reward's one-step continued-pretraining oracle; chosen by hand and affects which paragraphs receive positive reward (§A.1).
  • warm-start size = 250 examples
    Synthetic warm-start set for the generator; a design choice with no reported sensitivity analysis (§3.4, A.1).
  • GRPO hyperparameters = batch 64, LR 1e-6, KL 0.001, 5 rollouts
    RL training configuration for the paragraph generator; no ablation over these settings is reported (§A.1).
axioms (5)
  • domain assumption The 11-sample majority-vote answer of Qwen2.5-1.5B-IT is a reliable proxy for the model's answer preference.
    Used throughout to define the low-consistency split and to measure steer ratio; no stability check across sampling configurations is reported (§4.1, A.1).
  • domain assumption Continued pretraining on a small corpus of generated paragraphs models a realistic channel by which ordinary text can alter model behavior.
    The threat model assumes an adversary can inject text into continued pretraining; this is a strong operational assumption for the 'practical channel' claim (§1, §6).
  • domain assumption The low-consistency split (second-most option >= 3/11) captures 'brittle reasoning' rather than a simple artifact of chance variation.
    Central to interpreting steering as exploiting reasoning fragility; no validation against a high-consistency or confident-question subset is provided (§4.1).
  • domain assumption Qwen3-8B's assessment that a target is not 'inferable' from a paragraph is a valid proxy for external detectability.
    Used both as a reward component and as the main inferability metric; only partially validated by a 50-item human study (§3.5, §5.3).
  • standard math Standard supervised and RL training mechanics (AdamW, causal LM loss, GRPO) behave as expected on small corpora.
    Implicitly relied upon for the continued-pretraining and generator-optimization steps; no failures or pathologies are discussed (§3.3, A.1).

pith-pipeline@v1.3.0-alltime-deepseek · 16814 in / 13933 out tokens · 145295 ms · 2026-08-02T02:39:50.668638+00:00 · methodology

0 comments
read the original abstract

Large language models often appear to reason reliably, yet on many questions repeated sampling yields both correct and incorrect answers, revealing an underlying fragility in how final decisions are formed. We study whether this fragility can be exploited through implicit reasoning steering: using natural-language text to bias a model toward a designated answer without explicit instructions, triggers, or direct answer cues. Our approach, Concept Chaining, generates a short connection paragraph that links question entities to a target option through one or two intermediate concepts. We then continue pretraining a victim model on these connection paragraphs and evaluate whether its answer preference shifts on the original multiple-choice questions. Our results show that indirect, natural-looking text can systematically steer model predictions while remaining substantially less inferable than direct paraphrases, which shows that reasoning brittleness is not merely an evaluation artifact: it creates a practical channel through which latent biases can be amplified by ordinary-looking text to covertly redirect model decisions.

Figures

Figures reproduced from arXiv: 2607.14242 by Anshuman Chhabra, Ben Zhou, Muhao Chen, Sanika Chavan, Shahriar Kabir Nahin, Xiao Ye, Yuxi Huang.

Figure 1
Figure 1. Figure 1: Illustration of CONCEPT CHAINING on a low-consistency question. Left: the base model assigns probability mass to multiple plausible options for the meaning of Dalai Lama, reflecting a fragile internal balance. Middle: a connection paragraph builds an indirect semantic bridge from the question entity to a designated target option (“Lover of Wisdom”) via intermediate concepts such as philosopher and “one who… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of our implicit reasoning steering pipeline. Starting from low [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

166 extracted references · 20 canonical work pages · 12 internal anchors

  1. [1]

    Advances in Neural Information Processing Systems , volume =

    Chain-of-Thought Prompting Elicits Reasoning in Large Language Models , author =. Advances in Neural Information Processing Systems , volume =. 2022 , url =

    2022

  2. [2]

    arXiv preprint arXiv:2205.11916 , year =

    Large Language Models are Zero-Shot Reasoners , author =. arXiv preprint arXiv:2205.11916 , year =

    Pith/arXiv arXiv

  3. [3]

    The Eleventh International Conference on Learning Representations , year =

    Self-Consistency Improves Chain of Thought Reasoning in Language Models , author =. The Eleventh International Conference on Learning Representations , year =

  4. [4]

    arXiv preprint arXiv:2205.10625 , year =

    Least-to-Most Prompting Enables Complex Reasoning in Large Language Models , author =. arXiv preprint arXiv:2205.10625 , year =

    Pith/arXiv arXiv

  5. [5]

    Advances in Neural Information Processing Systems , volume =

    Tree of Thoughts: Deliberate Problem Solving with Large Language Models , author =. Advances in Neural Information Processing Systems , volume =

  6. [6]

    arXiv preprint arXiv:2305.20050 , year =

    Let's Verify Step by Step , author =. arXiv preprint arXiv:2305.20050 , year =

    Pith/arXiv arXiv

  7. [7]

    Advances in Neural Information Processing Systems , volume =

    Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting , author =. Advances in Neural Information Processing Systems , volume =

  8. [8]

    Proceedings of the 38th International Conference on Machine Learning , series =

    Calibrate Before Use: Improving Few-Shot Performance of Language Models , author =. Proceedings of the 38th International Conference on Machine Learning , series =

  9. [9]

    Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =

    Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity , author =. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =. 2022 , address =. doi:10.18653/v1/2022.acl-long.556 , url =

    doi:10.18653/v1/2022.acl-long.556 2022

  10. [10]

    Findings of the Association for Computational Linguistics: NAACL 2024 , pages =

    Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions , author =. Findings of the Association for Computational Linguistics: NAACL 2024 , pages =. 2024 , address =. doi:10.18653/v1/2024.findings-naacl.130 , url =

    doi:10.18653/v1/2024.findings-naacl.130 2024

  11. [11]

    Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How

    Sclar, Melanie and Choi, Yejin and Tsvetkov, Yulia and Suhr, Alane , booktitle =. Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How

  12. [12]

    Findings of the Association for Computational Linguistics: ACL 2024 , pages =

    Unveiling Selection Biases: Exploring Order and Token Sensitivity in Large Language Models , author =. Findings of the Association for Computational Linguistics: ACL 2024 , pages =. 2024 , address =. doi:10.18653/v1/2024.findings-acl.333 , url =

    work page doi:10.18653/v1/2024.findings-acl.333 2024

  13. [13]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =

    When Benchmarks are Targets: Revealing the Sensitivity of Large Language Model Leaderboards , author =. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =. 2024 , address =. doi:10.18653/v1/2024.acl-long.744 , url =

    doi:10.18653/v1/2024.acl-long.744 2024

  14. [14]

    arXiv preprint arXiv:2207.05221 , year =

    Language Models (Mostly) Know What They Know , author =. arXiv preprint arXiv:2207.05221 , year =. doi:10.48550/arXiv.2207.05221 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2207.05221

  15. [15]

    Transactions on Machine Learning Research , year =

    Teaching Models to Express Their Uncertainty in Words , author =. Transactions on Machine Learning Research , year =

  16. [16]

    arXiv preprint arXiv:2306.13063 , year =

    Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs , author =. arXiv preprint arXiv:2306.13063 , year =

    Pith/arXiv arXiv

  17. [17]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =

    Quantifying Uncertainty in Answers from any Language Model and Enhancing their Trustworthiness , author =. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =. 2024 , address =. doi:10.18653/v1/2024.acl-long.283 , url =

    doi:10.18653/v1/2024.acl-long.283 2024

  18. [18]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages =

    Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words? , author =. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages =. 2024 , address =. doi:10.18653/v1/2024.emnlp-main.443 , url =

    doi:10.18653/v1/2024.emnlp-main.443 2024

  19. [19]

    Concealed Data Poisoning Attacks on

    Wallace, Eric and Zhao, Tony and Feng, Shi and Singh, Sameer , booktitle =. Concealed Data Poisoning Attacks on. 2021 , address =. doi:10.18653/v1/2021.naacl-main.13 , url =

    work page doi:10.18653/v1/2021.naacl-main.13 2021

  20. [21]

    Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages =

    Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer , author =. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages =. 2021 , address =. doi:10.18653/v1/2021.emnlp-main.374 , url =

    doi:10.18653/v1/2021.emnlp-main.374 2021

  21. [22]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages =

    Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Models , author =. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages =. 2023 , address =. doi:10.18653/v1/2023.emnlp-main.757 , url =

    doi:10.18653/v1/2023.emnlp-main.757 2023

  22. [23]

    Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , pages =

    Instructions as Backdoors: Backdoor Vulnerabilities of Instruction Tuning for Large Language Models , author =. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , pages =. 2024 , address =. doi:10.18653/v1/2024.naacl-long.171 , url =

    doi:10.18653/v1/2024.naacl-long.171 2024

  23. [24]

    Backdoor

    Du, Wei and Ju, Tianjie and Ren, Ge and Li, GaoLei and Liu, Gongshen , booktitle =. Backdoor. 2024 , address =

    2024

  24. [25]

    arXiv preprint arXiv:2004.00345 , year =

    Editable Neural Networks , author =. arXiv preprint arXiv:2004.00345 , year =

    Pith/arXiv arXiv 2004

  25. [26]

    Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages =

    Editing Factual Knowledge in Language Models , author =. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages =. 2021 , address =. doi:10.18653/v1/2021.emnlp-main.522 , url =

    doi:10.18653/v1/2021.emnlp-main.522 2021

  26. [28]

    Locating and Editing Factual Associations in

    Meng, Kevin and Bau, David and Andonian, Alex and Belinkov, Yonatan , journal =. Locating and Editing Factual Associations in

  27. [29]

    The Eleventh International Conference on Learning Representations , year =

    Mass-Editing Memory in a Transformer , author =. The Eleventh International Conference on Learning Representations , year =

  28. [30]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages =

    Editing Large Language Models: Problems, Methods, and Opportunities , author =. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages =. 2023 , address =. doi:10.18653/v1/2023.emnlp-main.632 , url =

    doi:10.18653/v1/2023.emnlp-main.632 2023

  29. [31]

    The Thirteenth International Conference on Learning Representations , year =

    Everything Is Editable: Extend Knowledge Editing to Unstructured Data in Large Language Models , author =. The Thirteenth International Conference on Learning Representations , year =

  30. [34]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =

    Probing Language Models for Pre-training Data Detection , author =. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =. 2024 , address =. doi:10.18653/v1/2024.acl-long.86 , url =

    work page doi:10.18653/v1/2024.acl-long.86 2024

  31. [43]

    arXiv preprint arXiv:2310.01558 , doi =

    Ori Yoran and Tomer Wolfson and Ori Ram and Jonathan Berant , year =. arXiv preprint arXiv:2310.01558 , doi =

    Pith/arXiv arXiv

  32. [44]

    arXiv preprint arXiv:2505.18761 , doi =

    Minglai Yang and Ethan Huang and Liang Zhang and Mihai Surdeanu and William Wang and Liangming Pan , year =. arXiv preprint arXiv:2505.18761 , doi =

    arXiv

  33. [45]

    arXiv preprint arXiv:2402.19255 , doi =

    Qintong Li and Leyang Cui and Xueliang Zhao and Lingpeng Kong and Wei Bi , year =. arXiv preprint arXiv:2402.19255 , doi =

    Pith/arXiv arXiv

  34. [46]

    arXiv preprint arXiv:2406.11020 , doi =

    Yuqing Wang and Yun Zhao , year =. arXiv preprint arXiv:2406.11020 , doi =

    Pith/arXiv arXiv

  35. [47]

    arXiv preprint arXiv:2410.05229 , doi =

    Iman Mirzadeh and Keivan Alizadeh and Hooman Shahrokhi and Oncel Tuzel and Samy Bengio and Mehrdad Farajtabar , year =. arXiv preprint arXiv:2410.05229 , doi =

    Pith/arXiv arXiv

  36. [48]

    arXiv preprint arXiv:2503.04550 , doi =

    Tong Yu and Yongcheng Jing and Xikun Zhang and Wentao Jiang and Wenjie Wu and Yingjie Wang and Wenbin Hu and Bo Du and Dacheng Tao , year =. arXiv preprint arXiv:2503.04550 , doi =

    Pith/arXiv arXiv

  37. [49]

    Su and Camillo J

    Bowen Jiang and Yangxinyu Xie and Zhuoqun Hao and Xiaomeng Wang and Tanwi Mallick and Weijie J. Su and Camillo J. Taylor and Dan Roth , year =. arXiv preprint arXiv:2406.11050 , doi =

    Pith/arXiv arXiv

  38. [50]

    arXiv preprint arXiv:2410.23856 , doi =

    Zhanke Zhou and Rong Tao and Jianing Zhu and Yiwen Luo and Zengmao Wang and Bo Han , year =. arXiv preprint arXiv:2410.23856 , doi =

    Pith/arXiv arXiv

  39. [51]

    arXiv preprint arXiv:2504.20769 , doi =

    Wenxiao Wang and Parsa Hosseini and Soheil Feizi , year =. arXiv preprint arXiv:2504.20769 , doi =

    Pith/arXiv arXiv

  40. [52]

    arXiv preprint arXiv:2310.16218 , doi =

    Song Wang and Yaochen Zhu and Haochen Liu and Zaiyi Zheng and Chen Chen and Jundong Li , year =. arXiv preprint arXiv:2310.16218 , doi =

    Pith/arXiv arXiv

  41. [53]

    arXiv preprint arXiv:2401.01286 , doi =

    Ningyu Zhang and Yunzhi Yao and Bozhong Tian and Peng Wang and Shumin Deng and Mengru Wang and Zekun Xi and Shengyu Mao and Jintian Zhang and Yuansheng Ni and Siyuan Cheng and Ziwen Xu and Xin Xu and Jia-Chen Gu and Yong Jiang and Pengjun Xie and Fei Huang and Lei Liang and Zhiqiang Zhang and Xiaowei Zhu and Jun Zhou and Huajun Chen , year =. arXiv prepri...

    Pith/arXiv arXiv

  42. [54]

    Manning , year =

    Eric Mitchell and Charles Lin and Antoine Bosselut and Chelsea Finn and Christopher D. Manning , year =. arXiv preprint arXiv:2110.11309 , doi =

    Pith/arXiv arXiv

  43. [55]

    Manning and Christopher Potts and Danqi Chen , year =

    Zexuan Zhong and Zhengxuan Wu and Christopher D. Manning and Christopher Potts and Danqi Chen , year =. arXiv preprint arXiv:2305.14795 , doi =

    Pith/arXiv arXiv

  44. [56]

    arXiv preprint arXiv:2403.17431 , doi =

    Yingfa Chen and Zhengyan Zhang and Xu Han and Chaojun Xiao and Zhiyuan Liu and Chen Chen and Kuai Li and Tao Yang and Maosong Sun , year =. arXiv preprint arXiv:2403.17431 , doi =

    Pith/arXiv arXiv

  45. [57]

    arXiv preprint arXiv:2405.14768 , doi =

    Peng Wang and Zexi Li and Ningyu Zhang and Ziwen Xu and Yunzhi Yao and Yong Jiang and Pengjun Xie and Fei Huang and Huajun Chen , year =. arXiv preprint arXiv:2405.14768 , doi =

    Pith/arXiv arXiv

  46. [58]

    arXiv preprint arXiv:2410.02355 , doi =

    Junfeng Fang and Houcheng Jiang and Kun Wang and Yunshan Ma and Xiang Wang and Xiangnan He and Tat-seng Chua , year =. arXiv preprint arXiv:2410.02355 , doi =

    Pith/arXiv arXiv

  47. [59]

    arXiv preprint arXiv:2402.18909 , doi =

    Xiaobao Wu and Liangming Pan and William Yang Wang and Anh Tuan Luu , year =. arXiv preprint arXiv:2402.18909 , doi =

    Pith/arXiv arXiv

  48. [60]

    arXiv preprint arXiv:2502.05628 , doi =

    Houcheng Jiang and Junfeng Fang and Ningyu Zhang and Guojun Ma and Mingyang Wan and Xiang Wang and Xiangnan He and Tat-seng Chua , year =. arXiv preprint arXiv:2502.05628 , doi =

    arXiv

  49. [61]

    arXiv preprint arXiv:2504.01196 , doi =

    Zian Su and Ziyang Huang and Kaiyuan Zhang and Xiangyu Zhang , year =. arXiv preprint arXiv:2504.01196 , doi =

    Pith/arXiv arXiv

  50. [62]

    arXiv preprint arXiv:2506.09672 , doi =

    Hao Xiong and Chuanyuan Tan and Wenliang Chen , year =. arXiv preprint arXiv:2506.09672 , doi =

    Pith/arXiv arXiv

  51. [63]

    arXiv preprint arXiv:2602.19043 , doi =

    Zisheng Zhou and Mengqi Zhang and Shiguang Wu and Xiaotian Ye and Chi Zhang and Zhumin Chen and Pengjie Ren , year =. arXiv preprint arXiv:2602.19043 , doi =

    arXiv

  52. [64]

    arXiv preprint arXiv:2511.20892 , doi =

    Xuyuan Liu and Zhengzhang Chen and Xinshuai Dong and Yanchi Liu and Xujiang Zhao and Shengyu Chen and Haoyu Wang and Yujun Yan and Haifeng Chen , year =. arXiv preprint arXiv:2511.20892 , doi =

    Pith/arXiv arXiv

  53. [66]

    arXiv preprint arXiv:2409.19663 , doi =

    Xiaopeng Li and Shangwen Wang and Shezheng Song and Bin Ji and Huijun Liu and Shasha Li and Jun Ma and Jie Yu , year =. arXiv preprint arXiv:2409.19663 , doi =

    Pith/arXiv arXiv

  54. [69]

    Sutton and Qinghua Zhou and Wei Wang and Desmond J

    Oliver J. Sutton and Qinghua Zhou and Wei Wang and Desmond J. Higham and Alexander N. Gorban and Alexander Bastounis and Ivan Y. Tyukin , year =. arXiv preprint arXiv:2406.12670 , doi =

    Pith/arXiv arXiv

  55. [70]

    2021 , pages =

    Nicholas Carlini and Florian Tram. 2021 , pages =

    2021

  56. [72]

    arXiv preprint arXiv:2405.05990 , doi =

    Yang Bai and Ge Pei and Jindong Gu and Yong Yang and Xingjun Ma , year =. arXiv preprint arXiv:2405.05990 , doi =

    Pith/arXiv arXiv

  57. [73]

    arXiv preprint arXiv:2506.19399 , doi =

    Ruihan Hu and Yu-Ming Shang and Jiankun Peng and Wei Luo and Yazhe Wang and Xi Zhang , year =. arXiv preprint arXiv:2506.19399 , doi =

    Pith/arXiv arXiv

  58. [74]

    arXiv preprint arXiv:2402.09363 , doi =

    Matthieu Meeus and Igor Shilov and Manuel Faysse and Yves-Alexandre de Montjoye , year =. arXiv preprint arXiv:2402.09363 , doi =

    Pith/arXiv arXiv

  59. [75]

    arXiv preprint arXiv:2402.07841 , doi =

    Michael Duan and Anshuman Suri and Niloofar Mireshghallah and Sewon Min and Weijia Shi and Luke Zettlemoyer and Yulia Tsvetkov and Yejin Choi and David Evans and Hannaneh Hajishirzi , year =. arXiv preprint arXiv:2402.07841 , doi =

    Pith/arXiv arXiv

  60. [76]

    arXiv preprint arXiv:2411.00154 , doi =

    Haritz Puerto and Martin Gubri and Sangdoo Yun and Seong Joon Oh , year =. arXiv preprint arXiv:2411.00154 , doi =

    Pith/arXiv arXiv

  61. [77]

    arXiv preprint arXiv:2412.13475 , doi =

    Bowen Chen and Namgi Han and Yusuke Miyao , year =. arXiv preprint arXiv:2412.13475 , doi =

    Pith/arXiv arXiv

  62. [78]

    Jamie Hayes and Ilia Shumailov and Christopher A. Choquette-Choo and Matthew Jagielski and George Kaissis and Milad Nasr and Sahra Ghalebikesabi and Meenatchi Sundaram Mutu Selva Annamalai and Niloofar Mireshghallah and Igor Shilov and Matthieu Meeus and Yves-Alexandre de Montjoye and Katherine Lee and Franziska Boenisch and Adam Dziedzic and A. Feder Coo...

    arXiv

  63. [79]

    arXiv preprint arXiv:2601.02751 , doi =

    Yuetian Chen and Yuntao Du and Kaiyuan Zhang and Ashish Kundu and Charles Fleming and Bruno Ribeiro and Ninghui Li , year =. arXiv preprint arXiv:2601.02751 , doi =

    arXiv

  64. [80]

    Edward Suh , year =

    Trishita Tiwari and G. Edward Suh , year =. arXiv preprint arXiv:2412.11302 , doi =

    Pith/arXiv arXiv

  65. [81]

    arXiv preprint arXiv:2410.02650 , doi =

    Ali Satvaty and Suzan Verberne and Fatih Turkmen , year =. arXiv preprint arXiv:2410.02650 , doi =

    arXiv

  66. [82]

    arXiv preprint arXiv:2505.01976 , doi =

    Kang Chen and Xiuze Zhou and Yuanguo Lin and Shibo Feng and Li Shen and Pengcheng Wu , year =. arXiv preprint arXiv:2505.01976 , doi =

    Pith/arXiv arXiv

  67. [83]

    Nature , volume =

    Detecting Hallucinations in Large Language Models Using Semantic Entropy , author =. Nature , volume =. 2024 , doi =

    2024

  68. [86]

    Proceedings of the 40th International Conference on Machine Learning , series =

    Large Language Models Can Be Easily Distracted by Irrelevant Context , author =. Proceedings of the 40th International Conference on Machine Learning , series =. 2023 , publisher =

    2023

  69. [94]

    2025 , howpublished =

    Exploiting Reasoning Patterns in Language Models for Indirect Targeted Poisoning , author =. 2025 , howpublished =

    2025

  70. [95]

    2024 , eprint=

    FamiCom: Further Demystifying Prompts for Language Models with Task-Agnostic Performance Estimation , author=. 2024 , eprint=

    2024

  71. [96]

    2025 , eprint=

    Qwen2.5 Technical Report , author=. 2025 , eprint=

    2025

  72. [97]

    2025 , eprint=

    Qwen3 Technical Report , author=. 2025 , eprint=

    2025

  73. [98]

    2025 , eprint=

    gpt-oss-120b and gpt-oss-20b Model Card , author=. 2025 , eprint=

    2025

  74. [99]

    2024 , eprint=

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models , author=. 2024 , eprint=

    2024

  75. [100]

    2025 , eprint=

    Min-K author=. 2025 , eprint=

    2025

  76. [101]

    2020 , eprint=

    Understanding Black-box Predictions via Influence Functions , author=. 2020 , eprint=

    2020

  77. [102]

    Bowman , title =

    Miles Turpin and Julian Michael and Ethan Perez and Samuel R. Bowman , title =. Advances in Neural Information Processing Systems , editor =

  78. [103]

    Bowman and Ethan Perez , title =

    Tamera Lanham and Anna Chen and Ansh Radhakrishnan and Benoit Steiner and Carson Denison and Danny Hernandez and Dustin Li and Esin Durmus and Evan Hubinger and Jackson Kernion and Kamile Lukosiute and Karina Nguyen and Newton Cheng and Nicholas Joseph and Nicholas Schiefer and Oliver Rausch and Robin Larson and Sam McCandlish and Sandipan Kundu and Saura...

    2023

  79. [106]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages =

    Yu Yuan and Lili Zhao and Kai Zhang and Guangting Zheng and Qi Liu , title =. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages =. doi:10.18653/v1/2024.emnlp-main.679 , year =

    doi:10.18653/v1/2024.emnlp-main.679 2024

  80. [107]

    Nature Computational Science , volume =

    Thilo Hagendorff and Sarah Fabi and Michal Kosinski , title =. Nature Computational Science , volume =. 2023 , doi =

    2023

Showing first 80 references.