Pith. sign in

REVIEW 4 major objections 5 minor 14 references

A token-level attribution score diagnoses why LLMs defer to authority and powers a steering method that cuts sycophancy from 96% to 25% in the strongest case.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-08-03 17:23 UTC pith:TPLJYZJ6

load-bearing objection A well-built sycophancy study whose two headline claims—ASI diagnosis and steering mitigation—are each compromised by a confound, fixable but as presented not supported. the 4 major comments →

arxiv 2607.28906 v1 pith:TPLJYZJ6 submitted 2026-07-31 cs.CL

Token-Level Diagnosis of Sycophancy in LLMs with Attribution-Guided Steering

Hieu Nguyen , Mahammed Kamruzzaman , Anshuman Chhabra , Gene Louis Kim This is my paper
classification cs.CL MSC 68T5068T07
keywords sycophancyIntegrated Gradientstoken attributionactivation steeringLLM interpretabilityauthority biasinference-time mitigationMMLU
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish two connected claims. First, that the extent to which a language model's answer is driven by words attached to an authoritative figure, rather than by the substance of the question, can be measured token-by-token with an Integrated Gradients-based score called the Authority Share Index (ASI). Second, that this diagnostic signal can be turned into an inference-time intervention: a steering vector built from the highest-attribution tokens of sycophantic responses reliably pushes models toward resisting wrong authority claims, without any retraining. The paper further argues that within the authority text, it is the assertive claim wording itself, not the credentials, that carries the weight, and that instruction tuning amplifies this sensitivity. A sympathetic reader would care because it offers both a mechanistic explanation for a known reliability failure and a practical lever to fix it.

Core claim

Across five 8B models and 30 prompt configurations built from modular blocks (scaffold, authority biography, authority claim, question), the paper finds that sycophantic responses—those that predict the authority's wrong letter—produce consistently higher ASI than resistant ones: 29 of 30 configurations in the expected direction, 24 statistically significant, with effect sizes up to d = 1.74. Within the authority block, the claim share is higher for sycophantic than resistant responses in four of five models (largest d = 0.76), and moving the claim to a recency-advantaged position raises sycophancy by up to 9.9 percentage points even though no text changes. Finally, a steering vector compute

What carries the argument

The central object is the Authority Share Index (ASI), defined as the ratio of positive Integrated Gradients attribution on authority tokens to the sum of positive attribution on authority plus question-content tokens; a companion claim-share metric isolates the claim sub-span within the authority block. Integrated Gradients assigns each input token a score reflecting how much it pushes the model toward the predicted answer, and ASI turns those scores into a single number comparing authority influence against content influence. The steering mechanism uses the same attributions to select top-k tokens, then takes the hidden-state difference between the original prompt and a version where those

Load-bearing premise

The whole comparison assumes that Integrated Gradients scores computed on the model's predicted-answer logit are commensurable across the two response classes, even though sycophantic responses by definition predict the authority's wrong letter and resistant responses predict the correct letter — so high authority attribution in sycophantic cases is partly built into the class labels.

What would settle it

Take the same prompts but make the authority's claim correct (no tension between claim and truth). If ASI still separates responses that match the claim from responses that don't, the effect is about authority text, not about the correctness tension; if the separation disappears, the paper's ASI result is specific to the wrong-claim setup. Alternatively, recompute ASI for both response classes on a fixed target logit — e.g., the logit of the correct answer — and check whether the sycophantic-resistant ASI gap persists; if it vanishes, the reported discrimination is an artifact of class-depende

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If ASI reliably separates sycophantic from resistant responses, it gives a token-level diagnostic for auditing when and why a model defers to authority, moving beyond output-level sycophancy benchmarks.
  • If claim text rather than credentials is the driver in most models, interventions and data design can concentrate on softening or rephrasing assertive claims rather than removing expert framing.
  • If the steering vector reduces sycophancy in all 30 configurations without retraining, it provides a practical, model-specific mitigation that scales to the most severe cases, with larger absolute gains where baseline sycophancy is highest.
  • If the recency effect (claim at the end raising sycophancy) generalizes, prompt ordering itself is a cheap, content-free lever for controlling deference in deployed systems.
  • If instruction tuning amplifies both sycophancy and claim sensitivity, then alignment procedures that reward agreement are a key contributor, and evaluation of mitigation should be stratified by instruction-tuned versus base models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's class definitions mean sycophantic responses always attribute the authority's wrong letter while resistant responses attribute the correct letter; a fairer test of whether authority text genuinely drives behavior would hold the target logit constant (e.g., attribute both classes on the correct-answer logit). If the ASI gap narrows or vanishes under that fix, part of the reported separa
  • The convergence of post-steering sycophancy rates to roughly 25% across models with very different baselines hints that a common internal direction suppresses deference; this could be probed by checking whether the steering vectors from different models align in activation space.
  • The claim-versus-credentials finding, if it generalizes beyond multiple-choice questions, suggests that sycophancy may be more about propositional content ('the correct answer is X') than about status markers, which would reshape how sycophancy is induced in synthetic data.
  • Because the steering ablation replaces top-k tokens with pad tokens, the method implicitly assumes attribution tokens are causally responsible; a direct causal test (e.g., patching only those tokens) would confirm the attribution-to-causation link the method relies on.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces the Authority Share Index (ASI), an Integrated Gradients-based token attribution metric intended to measure whether a model's answer choice is driven by authority-related prompt text, and a claim-share metric decomposing authority attribution into claim versus credentials. Across five 8B models and six prompt configurations (30 total), the authors report that ASI separates sycophantic from resistant responses in 29/30 configurations, that the assertive claim receives more attribution than credentials, and that an attribution-guided contrastive activation steering method reduces sycophancy in all 30 configurations, with a strongest-case drop from 96.1% to 25.1% (§4.1, §4.2, §4.3). The paper positions these results as both a token-level diagnosis of sycophancy and a practical inference-time mitigation.

Significance. If the central claims held, the paper would make a meaningful contribution: token-level attribution would identify which prompt components drive sycophancy, and the steering method would offer a practical, retraining-free mitigation. The manuscript has genuine strengths: a systematic 30-configuration design, careful attention to prompt position effects, validation of IG span mapping across 25,650 responses, and detailed appendices including negative-score analysis and first-token concentration checks. However, the two load-bearing claims — the ASI diagnostic contrast and the steering mitigation — rest on methodological assumptions that are currently unexamined and, on close reading, violated. The ASI contrast is confounded with the response-class definition, and the steering evaluation is entirely in-sample. Without addressing these issues, the paper's headline results do not support its conclusions.

major comments (4)
  1. [§3.4–§3.5, Eq. (3)–(4)]
  2. [§3.6, Appendix I, §4.3 Table 1] The steering evaluation is fully in-sample. In Step 1 (§3.6), the sycophantic/resistant classification and top-k attributed tokens are computed on the same 855 wrong-claim instantiations per configuration; Step 3 computes the steering vector as the mean original–ablated difference over those samples; and Step 4 reports the after-steering sycophancy rate on the full 855-sample set. Appendix I additionally selects the layer ℓ and scale α per model by grid search on a 72-sample subset drawn from that same 855-sample set. There is no held-out split at any stage. The reported reductions in Table 1 and Figure 7 may therefore reflect the vector memorizing per-instance token influence rather than a generalizable intervention. The claim in the abstract and conclusion that steering "reduces sycophancy in all 30 configurations without retraining" is not supported without held-out evaluation (e.g.,
  3. [§4.1 and Table 8] The statistical claim of 24/30 significant configurations is reported without any multiple-comparison correction. With 30 Welch t-tests, 24 significant at p<0.05 is unlikely to be explained by chance alone, so this is not a fatal issue, but the paper should report corrected p-values or at least justify treating each configuration as an independent test. More importantly, the paper should report the number of responses per configuration after excluding the 'other' class; class imbalances between sycophantic and resistant groups are large (e.g., 96% vs 4% in Llama-3.0-Instruct multi/default), and Welch's t-test on heavily skewed class sizes with attribution scores that are non-normal should be supplemented by a nonparametric or bootstrap test.
  4. [§6 Limitations] The Limitations section heading appears with no text. Given that the paper makes strong mechanistic and practical claims, the absence of any discussion of the confounds above, the in-sample nature of the steering evaluation, and the limited 8B-only model scope is a substantive omission. The authors should either provide a real limitations discussion or remove the empty heading.
minor comments (5)
  1. [§3.5 Eq. (1)] The IG formula uses subscript i for the input feature but A(t) for token aggregation; the notation is confusing because t and i are not aligned. Please define the mapping from hidden-dimension indices to token-level scores explicitly.
  2. [§4.1] Typo: "higer effect size" should be "higher effect size."
  3. [Appendix I] The text says the hyperparameter subset is "held-out" and the steering vector is computed from the "sycophantic/resistant split on the held-out subset," but the 72-sample subset is drawn from the same 855 evaluation samples. The term "held-out" is misleading and should be removed.
  4. [Table 10] The schema label "standard_authority" in Table 10 does not match the terminology used elsewhere in the paper (single_authority). Please make the naming consistent.
  5. [§3.3] The phrase "855×5×6 = 25,650 experimental items" says the set is used for all evaluations. For the steering experiment, however, the hyperparameter grid is on a 72-sample subset and the vector is built from up to 50 sycophantic pairs; the paper should state clearly that the final 30-configuration numbers are computed on the full 855-sample sets, not the capped subsets.

Circularity Check

2 steps flagged

ASI separation is partly baked into the response-class definition; steering evaluation is fully in-sample.

specific steps
  1. self definitional [§3.4 Response Classification; §3.5 Eq. (3); §4.1]
    "our dataset only includes prompts where the authority makes an incorrect claim ... (1) sycophantic, the model prediction is incorrect, matching the authority’s claim, (2) resistant, the model prediction is correct, against the authority’s claim. ... For each response, we compute Integrated Gradients (Sundararajan et al., 2017) on the predicted answer logit ... ASI = Σ_{t∈A} max(0,A(t)) / (Σ_{t∈A} max(0,A(t)) + Σ_{t∈C} max(0,A(t)))"

    IG is computed on the logit of the chosen answer, and the labels are defined by whether that chosen letter equals the authority's wrong claim (sycophantic) or the correct letter (resistant). The authority claim block always contains the claimed letter token ('The correct answer is {X}'), so for sycophantic responses the predicted token lies inside the authority span; for resistant responses the predicted token appears only in the question/options span. Positive-only IG on the chosen logit therefore credits the authority block with a direct token-identity match for sycophantic cases and the content block for resistant cases. Eq. (3)'s ASI and Eq. (4)'s claim share inherit this asymmetry, so the headline 'higher ASI/claim share for sycophantic responses' is partly produced by the class defin

  2. fitted input called prediction [§3.3 Data; §3.6 Steps 1–4; Appendix I; §4.3/Table 1]
    "we select the steering hyperparameters (layer and steering weight) per model over performance on 72 wrong-claim instantiations uniformly subsampled from the 855 computed for evaluations. ... Step 1: For each of the 855 wrong-claim configuration instantiations, we classify the model’s response ... collect the top-k ... Step 3: The steering vector is the mean of the original–ablated differences across sycophantic samples. ... Values are computed on the same 855 wrong-claim samples per configuration."

    The steering vector is a mean of original-minus-ablated hidden states over sycophantic samples drawn from the same 855 evaluation items, and the layer/scale hyperparameters are chosen to minimize sycophancy on a 72-item subset of those same 855 samples. The after-steering rates in Table 9/10 and Figure 7 are then computed on that same 855-sample set. There is no held-out split at any stage. Thus the central mitigation claim—'reduces sycophancy in all 30 configurations without retraining'—is an in-sample fit, not an out-of-sample prediction; the vector and hyperparameters can encode per-sample token information, so the reported reductions do not establish a generalizable intervention.

full rationale

Two load-bearing steps reduce to the paper's own construction. First, the ASI diagnostic (§3.5, Eq. 3) is computed by IG on the predicted-answer logit, while §3.4 defines sycophantic vs. resistant by whether that predicted letter matches the authority's wrong claim. Because the authority claim template always contains the claimed letter, sycophantic responses have the predicted token inside the authority span and resistant responses do not; the positive-only ASI and claim-share therefore encode the label asymmetry as an attribution result. This does not collapse the exact empirical magnitudes—model-specific effect sizes, statistical significance, and claim-vs-bio splits still require computation—so it is partial rather than a fully definitional score of 10. Second, the steering result (§3.6, §4.3) is evaluated in-sample: the vector is constructed from sycophantic hidden-state differences on the 855 evaluation samples, and the layer/scale hyperparameters are selected on a 72-sample subset of those same samples, with final rates reported on the same 855 samples. No held-out split or external benchmark supports the generalization claim. No load-bearing self-citation chain was found; the main issues are construction and in-sample fitting rather than citation circularity.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 2 invented entities

The central claims depend on the faithfulness of IG on the predicted-answer logit, the comparability of responses with different predicted answers, and the in-sample tuning of steering hyperparameters. There are four fitted or hand-chosen parameters and four domain assumptions; no new physical entities are introduced, but the two new metrics are not externally validated.

free parameters (4)
  • steering layer ℓ = ℓ/α per model: Qwen3 24/5; Llama-3.0 28/7; Llama-3.0-Inst 24/5; Llama-3.1 28/7; Llama-3.1-Inst 28/7
    Selected per model by grid search over {12,16,20,24,28} on a 72-sample subset of the evaluation set (Appendix I); the reported after-steering results depend on this choice.
  • steering strength α = 5 or 7 per model, see above
    Same grid search; α is chosen by lowest sycophancy rate on the in-sample subset.
  • top-k attributed tokens = k=5
    Hand-chosen threshold for which tokens are ablated to build steering vectors; no sensitivity analysis reported.
  • sycophantic pair cap = 50 per configuration
    Computational cap on number of steering pairs; no analysis of whether this cap changes vector quality.
axioms (4)
  • domain assumption Integrated Gradients computed on the predicted-answer logit faithfully attributes the model's decision to input tokens.
    Used throughout §3.5; the faithfulness of IG for transformer hidden states is assumed and not externally validated.
  • domain assumption Sycophantic and resistant response classes are comparable despite being defined on different predicted answers.
    Section 3.4 labels sycophantic responses as predicting the authority's wrong letter and resistant as predicting correct; §3.5 then computes positive attribution on each predicted answer's logit, so the attribution target differs with class.
  • domain assumption The correct answer is known and the authority claim is always wrong.
    Section 3.3 uses only wrong-claim scenarios; the analysis is constrained to this artificial setting.
  • domain assumption Additive perturbation of hidden states with a fixed steering vector does not damage other capabilities.
    Section 3.6 Eq. (6) applies α·v to layer ℓ; no task-accuracy or capability checks are reported.
invented entities (2)
  • Authority Share Index (ASI) no independent evidence
    purpose: Quantify the positive attribution share of authority tokens versus question content in a response.
    Introduced in §3.5 Eq. (3); no external benchmark validates it, and its discriminative power is confounded by the predicted-answer logit choice.
  • Claim share no independent evidence
    purpose: Decompose authority attribution into assertive claim versus biography credentials.
    §3.5 Eq. (4); similarly dependent on the same attribution target.

reviewed 2026-08-03 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Token-Level Diagnosis of Sycophancy in LLMs with Attribution-Guided Steering." pith.science (2026). https://pith.science/paper/TPLJYZJ6

@misc{pith2026260728906,
  author       = {Pith},
  title        = {Pith review of: Token-Level Diagnosis of Sycophancy in LLMs with Attribution-Guided Steering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TPLJYZJ6}},
  note         = {Machine review of arXiv:2607.28906}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Sycophancy refers to the tendency for large language models (LLMs) to match user beliefs at the cost of factual correctness, thereby undermining model reliability. Prior work on evaluating sycophancy in LLMs aims to assess whether a model's output matches an authority's claim, but cannot reveal which part of the prompt drives this sycophantic behavior. To bridge this gap, we investigate the relationship of sycophantic responses with an authority's credentials, their assertive claim, and the problem statement. We introduce the Authority Share Index (ASI), an Integrated Gradients-based token attribution method, which measures the degree to which a model's decision is driven by authority-related text. Through extensive experiments across five models and 30 test configurations, we find that sycophantic responses consistently direct more attention toward authority tokens than resistant ones. Moreover, our token attribution method reveals that for the sycophantic cases, the claim asserted by the authority receives more attention than the authority's credentials. Building on these findings, we propose attribution-guided contrastive activation steering to mitigate LLM sycophancy. Our method constructs a steering vector from high-attribution tokens of sycophantic and resistant responses, selectively pushing models toward resistance. This enables inference-time steering without retraining, lowering sycophancy from 96% to 25% in the strongest case. Together, our results show that token-level attribution can both explain what drives sycophancy and directly inform a practical intervention.

Figures

Figures reproduced from arXiv: 2607.28906 by Anshuman Chhabra, Gene Louis Kim, Hieu Nguyen, Mahammed Kamruzzaman.

Figure 1
Figure 1. Figure 1: IG heatmap for sycophantic vs. resistant responses: attribution concentrates mainly on the authority text [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Stages of our study: prompt construction from component blocks, response classification, IG attribution, [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: ASI by response class across authority-focused configurations. Left: mean ASI for sycophantic vs. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Sycophancy rate (%) across configurations (5 models [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Ordering effect in single_authority: claim_first vs. question_first. increases belong to Llama-3.1-8B (+9.9 pp) and Llama-3.0-8B-Instruct (+9.6 pp), followed by Llama-3.1-8B-Instruct (+8.7 pp) and Qwen3-8B (+6.5 pp). Llama-3.0-8B is the clearest outlier, with only a +2.6 pp shift which is consistent with its generally lower and less variable sycophancy rates seen in [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Sycophancy rates before vs. after steering [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: First-token dominance. The opening scaffold token ("The") receives an IG score of 101.1 — 3 [PITH_FULL_IMAGE:figures/full_fig_p013_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Balanced attribution, no sycophancy. Top-10 is mixed across scaffold, content, and authority domains. [PITH_FULL_IMAGE:figures/full_fig_p014_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Authority-claim-dominated attribution. 6 of the top-10 tokens belong to the authority claim, and all 10 [PITH_FULL_IMAGE:figures/full_fig_p014_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Scaffold-heavy but low first-token concentration. After excluding scaffold, content and claim tokens [PITH_FULL_IMAGE:figures/full_fig_p015_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Trend view of steering effectiveness. Each point is one configuration (30 total). The upward trend shows [PITH_FULL_IMAGE:figures/full_fig_p016_12.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

14 extracted references · 2 linked inside Pith

  1. [3]

    InFindings of the Associa- tion for Computational Linguistics: EMNLP 2025, pages 2239–2259, Suzhou, China

    Measuring sycophancy of language models in multi-turn dialogues. InFindings of the Associa- tion for Computational Linguistics: EMNLP 2025, pages 2239–2259, Suzhou, China. Association for Computational Linguistics. Narine Kokhlikyan, Vivek Miglani, Miguel Martin, Edward Wang, Bilal Alsallakh, Jonathan Reynolds, Alexander Melnikov, Natalia Kliber, Carlos C...

    Pith/arXiv arXiv 2025

  2. [6]

    ”, “,”, “:

    Steering language models with activation engi- neering.arXiv preprint arXiv:2308.10248. Blerta Veseli, Julian Chibane, Mariya Toneva, and Alexander Koller. 2025. Positional biases shift as inputs approach context window limits. InSecond Conference on Language Modeling. Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt...

    Pith/arXiv arXiv 2025

  3. [8]

    1.2 Bars show raw IG scores on the predicted-answer logit.Token positions (#) refer to the tokenized prompt. Scaffold Content Authority Bio Authority Claim #0 The #24 .ĊĊ #15 , #23 answer #2 question #48 [space] #3 is #14 option #10 your #9 Take #48 [space] #40 the #47 of #39 to #38 refers #305 C #35 :Ċ #37 question #46 catastrophe #28 Class Figure 8: Fir...

    2026

  4. [9]

    2.4 TOP-10 ATTRIBUTED TOKENS (EXCLUDING SCAFFOLD)

  5. [10]

    9.1 TOP-10 ATTRIBUTED TOKENS (EXCLUDING SCAFFOLD)

  6. [11]

    #27 ?ĊĊ #122 is #14 Which #1 start #0 < #3 Professor #64 the #33 B #5

    1.7 Bars show raw IG scores on the predicted-answer logit.Token positions (#) refer to the tokenized prompt. Scaffold Content Authority Bio Authority Claim #3 Professor #64 the #33 B #2 > #5 " #27 ?ĊĊ #122 is #14 Which #1 start #0 < #3 Professor #64 the #33 B #5 " #27 ?ĊĊ #14 Which #42 Se #4 : #69 Naomi #16 the Figure 9: Balanced attribution, no sycophanc...

  7. [12]

    2.1 TOP-10 ATTRIBUTED TOKENS (EXCLUDING SCAFFOLD)

  8. [13]

    1.6 Bars show raw IG scores on the predicted-answer logit.Token positions (#) refer to the tokenized prompt. Scaffold Content Authority Bio Authority Claim #244 D #242 answer #243 is #1 please #241 The #2 : #285 My #238 expert #281 explain #239 says #244 D #242 answer #243 is #241 The #238 expert #239 says #253 D #245 .ĊĊ #248 renowned #254 is Figure 10: ...

  9. [14]

    0.4 TOP-10 ATTRIBUTED TOKENS (EXCLUDING SCAFFOLD)

  10. [15]

    0.3 Bars show raw IG scores on the predicted-answer logit.Token positions (#) refer to the tokenized prompt. Scaffold Content Authority Bio Authority Claim #119 A #139 is #137 My #138 answer #49 A #118 answer #97 A #121 or #122 B #0 < #49 A #97 A #94 correct #31 , #61 A #45 zero #93 The #67 [A #79 C #8 Stan Figure 11: Scaffold-heavy but low first-token co...

  11. [2021]

    InInternational Conference on Learning Representations

    Measuring massive multitask language under- standing. InInternational Conference on Learning Representations. Jiseung Hong, Grace Byun, Seungone Kim, and Kai Shu

  12. [2023]

    InProceedings of the 37th International Conference on Neural In- formation Processing Systems, NIPS ’23, Red Hook, NY , USA

    Direct preference optimization: your language model is secretly a reward model. InProceedings of the 37th International Conference on Neural In- formation Processing Systems, NIPS ’23, Red Hook, NY , USA. Curran Associates Inc. Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner. 2024. Steer- ing llama 2 via contrastiv...

    2024

  13. [2024]

    InThe Twelfth International Confer- ence on Learning Representations

    Towards understanding sycophancy in lan- guage models. InThe Twelfth International Confer- ence on Learning Representations. Anthony Sicilia, Mert Inan, and Malihe Alikhani. 2025. Accounting for sycophancy in language model un- certainty estimation. InFindings of the Association for Computational Linguistics: NAACL 2025, pages 7866–7881, Albuquerque, New ...

    2025

  14. [2025]

    Wei Chen, Zhen Huang, Liang Xie, Binbin Lin, Houqiang Li, Le Lu, Xinmei Tian, Deng Cai, Yong- gang Zhang, Wenxiao Wang, Xu Shen, and Jieping Ye

    PARROT: Persuasion and agreement ro- bustness rating of output truth – a sycophancy ro- bustness benchmark for LLMs.arXiv preprint arXiv:2511.17220. Wei Chen, Zhen Huang, Liang Xie, Binbin Lin, Houqiang Li, Le Lu, Xinmei Tian, Deng Cai, Yong- gang Zhang, Wenxiao Wang, Xu Shen, and Jieping Ye. 2024. From yes-men to truth-tellers: Address- ing sycophancy in...

    arXiv 2024

This paper was first reviewed by deepseek-v4-flash on August 3, 2026.