From Context-Aware to Conflict-Aware: Generalizing Contrastive Decoding for Knowledge Conflict in LLMs

Bingyu Zhu; Longtao Huang; Runze Jiang; Taiqiang Wu; Yan Wang

arxiv: 2606.10298 · v1 · pith:H7JFNOF5 · submitted 2026-06-09 · cs.AI · cs.CL

From Context-Aware to Conflict-Aware: Generalizing Contrastive Decoding for Knowledge Conflict in LLMs

Runze Jiang , Taiqiang Wu , Yan Wang , Bingyu Zhu , Longtao Huang This is my paper

Reviewed by Pith2026-06-27 13:36 UTCgrok-4.3pith:H7JFNOF5 open to challenge →

classification cs.AI cs.CL

keywords knowledge conflictcontrastive decodinglarge language modelsadaptive routingcontext integrationparametric knowledge

0 comments

The pith

The affine combination of prior and context logits creates an asymmetry that static contrastive decoding cannot resolve in both conflict directions.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that contrastive decoding, which always boosts context over a model's prior knowledge, breaks down when the prior is correct because extrapolation amplifies mistakes without bound. It identifies that affine logit combinations form a power family where extrapolation and interpolation each fail in one conflict scenario, and no fixed setting works for both. By introducing a benchmark that tests correction, resistance, and agreement states calibrated to each model, and a method that routes between the two regimes step by step using conflict signals, the approach improves handling of cases where the model should resist erroneous context. This matters for making retrieved information reliable without losing accurate internal knowledge.

Core claim

The affine combination of prior and context logits yields a power family with an inherent regime asymmetry: extrapolation amplifies errors unboundedly when the prior is correct, interpolation under-corrects when the context is correct, and no static regime covers both. Adaptive Regime Routing resolves this by routing between regimes at each step based on conflict signals.

What carries the argument

Adaptive Regime Routing (ARR), which dynamically switches between extrapolative and interpolative regimes of the affine logit combination based on detected conflict at each generation step.

If this is right

Existing contrastive decoding methods are mostly extrapolative instances of the power family.
TriState-Bench measures performance in correction, resistance, and agreement by calibrating to per-model prior knowledge.
ARR raises resistance exact match scores from below 6 to between 16 and 33 while preserving correction and agreement performance.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Conflict signals might be useful for other generation techniques beyond contrastive decoding.
Similar regime asymmetries could appear in other methods that combine multiple logit sources.

Load-bearing premise

That accurate per-step signals for detecting whether prior or context is correct can be obtained without introducing additional errors.

What would settle it

A test where conflict signals are deliberately made unreliable or random, showing whether ARR still improves resistance performance or degrades it compared to static methods.

Figures

Figures reproduced from arXiv: 2606.10298 by Bingyu Zhu, Longtao Huang, Runze Jiang, Taiqiang Wu, Yan Wang.

**Figure 2.** Figure 2: Overview of TriState-Bench pipeline. Step 3: Benchmark Assembly (per-model). Given the prior labels from Step 2, we revisit the fact repository and assemble tri-state samples by pairing each fact with the corresponding pre-generated context. • Correction Scor (prior wrong, context right): fi ∈ F M wrong paired with c + i , targeting correction capability. • Resistance Sres (prior right, context wrong): fi … view at source ↗

**Figure 3.** Figure 3: Distribution of prior knowledge states across four LLMs. (a) Prior state distribution show [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

**Figure 5.** Figure 5: Blue/red shading separates the interpolation ( [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗

**Figure 4.** Figure 4: Gate accuracy of each candidate signal. Confidence-asymmetry signals (C, D) consistently outperform magnitude-only signals (A, B), confirming that directionality is necessary for regime separation. Magnitude-only signals fire whenever ppri,t and pctx,t diverge but cannot determine which side is more reliable; signal A, in particular, barely exceeds the chance rate of 0.5. Confidence-asymmetry signals ad… view at source ↗

**Figure 6.** Figure 6: Generation length of Llama3-8B across all methods in correction, resistance, and agreement [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗

**Figure 7.** Figure 7: Blue/red shading separates the interpolation ( [PITH_FULL_IMAGE:figures/full_fig_p026_7.png] view at source ↗

**Figure 8.** Figure 8: Detailed cases on Llama-3-8B, illustrating how varying [PITH_FULL_IMAGE:figures/full_fig_p026_8.png] view at source ↗

**Figure 9.** Figure 9: The detailed failure cases in TriState-Bench on Llama-3-8B, Qwen2.5-7B, and Mistral-7B. [PITH_FULL_IMAGE:figures/full_fig_p027_9.png] view at source ↗

read the original abstract

When large language models generate from retrieved or augmented contexts, conflicts between external context and parametric priors remain a central reliability bottleneck. Existing contrastive decoding methods follow a \emph{context-aware} paradigm that unilaterally amplifies context over parametric priors, overwriting correct priors when the context is erroneous. We generalize this to the \textbf{conflict-aware} paradigm that dynamically allocates authority between prior and context based on conflict signals, rather than presupposing context trustworthiness. We show that the affine combination of prior and context logits yields a \textbf{power family} with an inherent \textbf{regime asymmetry}: extrapolation amplifies errors unboundedly when the prior is correct, interpolation under-corrects when the context is correct, and no static regime covers both. Existing contrastive decoding methods are instances of this family, mostly extrapolative. To evaluate both conflict directions, we propose TriState-Bench, a model-aware evaluation protocol that calibrates per-model prior knowledge to measure three conflict states: correction, resistance, and agreement. To resolve the asymmetry, we propose Adaptive Regime Routing (ARR), which routes between regimes at each step, lifting resistance EM from below 6 to 16--33 without sacrificing correction or agreement. Our code is available at https://github.com/keith-Jiang/conflict-aware-decoding.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper derives a regime asymmetry in logit combinations for contrastive decoding and proposes dynamic routing to handle both directions of knowledge conflict.

read the letter

The core contribution is showing that affine combinations of prior and context logits create a power family where no single regime works for both correcting bad context and resisting bad priors. Extrapolation amplifies errors when the prior is correct; interpolation under-corrects when context is correct. Existing methods mostly sit in the extrapolative side.

They formalize this as the shift from context-aware to conflict-aware decoding, introduce TriState-Bench to calibrate per-model priors and score correction/resistance/agreement states, and give Adaptive Regime Routing (ARR) that switches regimes per step. The reported lift on resistance EM (under 6 to 16-33) without hurting the other two metrics is the main empirical claim.

The framing and the benchmark are useful additions; the asymmetry observation follows directly from the math and clarifies why static contrastive methods are limited. The routing idea is a reasonable response to the asymmetry.

The soft spot is the routing mechanism itself. ARR depends on per-step conflict signals derived from the TriState calibration. The abstract gives no error analysis or ablation on how noisy those signals are or how often misrouting occurs. If the signals are unreliable, the claimed gains could shrink or introduce new failures. The full paper needs to show that the signals are stable enough to carry the result.

This is for researchers working on decoding, RAG reliability, and alignment techniques. It is worth sending to peer review because the formalization is clean and the benchmark is new, even if the routing evidence needs more scrutiny on signal quality.

Referee Report

3 major / 2 minor

Summary. The paper claims that affine combinations of prior and context logits form a power family exhibiting regime asymmetry (extrapolation amplifies prior errors unboundedly when the prior is correct; interpolation under-corrects when the context is correct), that no static regime covers both directions, that existing contrastive decoding methods are mostly extrapolative instances of this family, and that Adaptive Regime Routing (ARR) using per-step conflict signals from the proposed TriState-Bench resolves the asymmetry to lift resistance EM from below 6 to 16-33 while preserving correction and agreement performance.

Significance. If the asymmetry derivation and ARR routing hold with reliable signals, the work would offer a principled way to handle bidirectional knowledge conflicts in context-augmented generation beyond unilateral context amplification. The public code release is a clear strength supporting reproducibility.

major comments (3)

[power family derivation section] The derivation of the power family and its regime asymmetry (abstract and the section introducing the affine logit combination): while the abstract framing is plausible, the claim that extrapolation amplifies errors unboundedly when the prior is correct (and symmetrically for interpolation) must be supported by explicit bounds or counter-example analysis rather than asserted from the affine form alone, as this is load-bearing for arguing that no static regime suffices.
[ARR and TriState-Bench sections] ARR routing logic and TriState-Bench calibration (the section describing Adaptive Regime Routing and the evaluation protocol): the central empirical claim of lifting resistance EM from below 6 to 16-33 rests on per-step conflict signals accurately distinguishing correction/resistance/agreement states without misrouting; no error analysis, robustness checks, or misrouting rate is provided, leaving open whether noisy signals could erase the reported gains.
[experimental evaluation] Experimental results (tables or figures reporting EM scores): the claimed improvements require visible comparison to static regimes, error bars or variance across seeds, and dataset/model details to confirm the lift is not fragile; absence of these details in the evaluation makes the practical advantage over existing contrastive methods hard to assess.

minor comments (2)

[abstract] Abstract: the range '16--33' should be clarified as mean, per-model, or per-dataset to avoid ambiguity.
[throughout] Notation: ensure 'power family' and 'regime' are defined once and used consistently to prevent reader confusion in later sections.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback. We address each major comment below and will revise the manuscript accordingly to strengthen the presentation of the power family, the reliability of ARR signals, and the experimental evaluation.

read point-by-point responses

Referee: [power family derivation section] The derivation of the power family and its regime asymmetry (abstract and the section introducing the affine logit combination): while the abstract framing is plausible, the claim that extrapolation amplifies errors unboundedly when the prior is correct (and symmetrically for interpolation) must be supported by explicit bounds or counter-example analysis rather than asserted from the affine form alone, as this is load-bearing for arguing that no static regime suffices.

Authors: We agree that the asymmetry claim requires more rigorous support beyond the affine form. We will revise the derivation section to include explicit mathematical bounds on error amplification (unbounded under extrapolation when the prior is correct) and under-correction (under interpolation when the context is correct), along with counter-example analysis illustrating why no single static regime suffices for bidirectional conflicts. revision: yes
Referee: [ARR and TriState-Bench sections] ARR routing logic and TriState-Bench calibration (the section describing Adaptive Regime Routing and the evaluation protocol): the central empirical claim of lifting resistance EM from below 6 to 16-33 rests on per-step conflict signals accurately distinguishing correction/resistance/agreement states without misrouting; no error analysis, robustness checks, or misrouting rate is provided, leaving open whether noisy signals could erase the reported gains.

Authors: We acknowledge that validating signal reliability is essential for the empirical claims. In revision we will add error analysis of the TriState-Bench per-step signals, robustness checks under injected noise, and explicit misrouting rates to confirm that the resistance EM gains remain stable and are not artifacts of perfect signal assumptions. revision: yes
Referee: [experimental evaluation] Experimental results (tables or figures reporting EM scores): the claimed improvements require visible comparison to static regimes, error bars or variance across seeds, and dataset/model details to confirm the lift is not fragile; absence of these details in the evaluation makes the practical advantage over existing contrastive methods hard to assess.

Authors: We will update the experimental section to include direct comparisons against static regime baselines, report error bars or variance across multiple seeds, and supply full dataset and model specifications. These additions will make the robustness of the ARR improvements over existing contrastive methods transparent. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation is mathematically independent

full rationale

The paper's core derivation states that the affine combination of prior and context logits produces a power family exhibiting regime asymmetry. This follows directly from the algebraic properties of logit interpolation/extrapolation, a standard operation independent of the paper's target claims about conflict resolution. ARR is introduced as a new dynamic routing algorithm using per-step signals from TriState-Bench; its logic does not reduce to any fitted parameter or self-citation by construction. No self-citations are invoked as load-bearing uniqueness theorems, no ansatzes are smuggled, and no known results are merely renamed. The evaluation protocol calibrates per-model priors externally to the central equations. The derivation chain remains self-contained against external mathematical benchmarks.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 2 invented entities

The central claims rest on the assumption that affine logit combinations form a useful family for decoding and that conflict signals are detectable; new entities include the benchmark and routing method, with limited free parameters visible in the abstract.

free parameters (1)

ARR routing parameters
Parameters controlling when and how to switch regimes are introduced by the method and likely require selection or fitting.

axioms (1)

domain assumption Affine combination of prior and context logits is a valid and complete way to model contrastive decoding behaviors
Invoked to derive the power family and regime asymmetry.

invented entities (2)

TriState-Bench no independent evidence
purpose: Model-aware evaluation protocol measuring correction, resistance, and agreement states
New benchmark introduced to evaluate both conflict directions.
Adaptive Regime Routing (ARR) no independent evidence
purpose: Dynamic routing between extrapolation and interpolation regimes
New algorithm proposed to resolve the identified asymmetry.

pith-pipeline@v0.9.1-grok · 5774 in / 1458 out tokens · 28781 ms · 2026-06-27T13:36:42.413339+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

21 extracted references · 8 canonical work pages · 4 internal anchors

[1]

The Llama 3 Herd of Models

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al- Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783,

work page internal anchor Pith review Pith/arXiv arXiv
[2]

Mistral 7B

URLhttps://arxiv.org/abs/2310.06825. Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. InProceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1601–1611,

work page internal anchor Pith review Pith/arXiv arXiv
[3]

Cocoa: Confidence-and context-aware adap- tive decoding for resolving knowledge conflicts in large language models

Anant Khandelwal, Manish Gupta, and Puneet Agrawal. Cocoa: Confidence-and context-aware adap- tive decoding for resolving knowledge conflicts in large language models. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 6846–6866,

2025
[4]

Large language models with controllable working memory

Daliang Li, Ankit Singh Rawat, Manzil Zaheer, Xin Wang, Michal Lukasik, Andreas Veit, Felix Yu, and Sanjiv Kumar. Large language models with controllable working memory. InFindings of the association for computational linguistics: ACL 2023, pp. 1774–1793, 2023a. Gaotang Li, Yuzhong Chen, and Hanghang Tong. Taming knowledge conflicts in language models. ar...

work page arXiv 2023
[5]

Contrastive decoding: Open-ended text generation as optimization

Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori B Hashimoto, Luke Zettlemoyer, and Mike Lewis. Contrastive decoding: Open-ended text generation as optimization. In Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: Long papers), pp. 12286–12312, 2023b. Shayne Longpre, Kartik P...

2021
[6]

Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning

Pan Lu, Liang Qiu, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, Tanmay Rajpurohit, Peter Clark, and Ashwin Kalyan. Dynamic prompt learning via policy gradient for semi-structured mathematical reasoning.arXiv preprint arXiv:2209.14610,

work page Pith review arXiv
[7]

WebGPT: Browser-assisted question-answering with human feedback

Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. Webgpt: Browser-assisted question- answering with human feedback.arXiv preprint arXiv:2112.09332,

work page internal anchor Pith review Pith/arXiv arXiv
[8]

Fabio Petroni, Tim Rockt ¨aschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. Language models as knowledge bases? InProceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural lan- guage processing (EMNLP-IJCNLP), pp. 2463–2473,

2019
[9]

Adam Roberts, Colin Raffel, and Noam Shazeer

URLhttps://arxiv.org/abs/2412 .15115. Adam Roberts, Colin Raffel, and Noam Shazeer. How much knowledge can you pack into the parame- ters of a language model? InProceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), pp. 5418–5426,

2020
[10]

Trusting your evidence: Hallucinate less with context-aware decoding

Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Wen-tau Yih. Trusting your evidence: Hallucinate less with context-aware decoding. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), pp. 783–791,

2024
[11]

Llama 2: Open Foundation and Fine-Tuned Chat Models

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288,

work page internal anchor Pith review Pith/arXiv arXiv
[12]

Adacad: Adaptively decoding to balance conflicts between contextual and parametric knowledge

Han Wang, Archiki Prasad, Elias Stengel-Eskin, and Mohit Bansal. Adacad: Adaptively decoding to balance conflicts between contextual and parametric knowledge. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 11636–11652,

2025
[13]

Knowl- edge conflicts for llms: A survey

Rongwu Xu, Zehan Qi, Zhijiang Guo, Cunxiang Wang, Hongru Wang, Yue Zhang, and Wei Xu. Knowl- edge conflicts for llms: A survey. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 8541–8565,

2024
[14]

Hotpotqa: A dataset for diverse, explainable multi-hop question answering

Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. InProceedings of the 2018 conference on empirical methods in natural language processing, pp. 2369–2380,

2018
[15]

Discerning and resolving knowledge conflicts through adaptive decoding with contextual information-entropy con- straint

Xiaowei Yuan, Zhao Yang, Yequan Wang, Shengping Liu, Jun Zhao, and Kang Liu. Discerning and resolving knowledge conflicts through adaptive decoding with contextual information-entropy con- straint. InFindings of the Association for Computational Linguistics: ACL 2024, pp. 3903–3922,

2024
[16]

Steering knowledge selection behaviours in llms via sae- based representation engineering

Yu Zhao, Alessio Devoto, Giwon Hong, Xiaotang Du, Aryo Pradipta Gema, Hongru Wang, Xuanli He, Kam-Fai Wong, and Pasquale Minervini. Steering knowledge selection behaviours in llms via sae- based representation engineering. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human La...

2025
[17]

Context-faithful prompting for large language models

Wenxuan Zhou, Sheng Zhang, Hoifung Poon, and Muhao Chen. Context-faithful prompting for large language models. InFindings of the Association for Computational Linguistics: EMNLP 2023, pp. 14544– 14556,

2023
[18]

The caseα=0 degenerates to standard decoding fromp ctx,t

14 A Background Methods: Detailed Formulations A.1 CAD The CAD decoding distribution can be derived from a point-wise mutual information style adjustment: qCAD t (y)∝p ctx,t(y) pctx,t(y) ppri,t(y) α =exp h (1+α)logp ctx,t(y)−αlogp pri,t(y) i , (19) followed by softmax normalization over the vocabularyV. The caseα=0 degenerates to standard decoding fromp c...

2024
[19]

20 NQ c: The Shivalik Hills to the northeast. x: Which geographical part of Haryana is Shivalik Hills situated? TabMWP c: Table: Name — Number of coins; Braden — 76; Camilla — 94; Rick — 86; Mary — 84; Hector — 80; Devin — 83; Emily — 82; Avery — 87 x: Some friends discussed the sizes of their coin collections. What is the mean of the numbers? TriviaQA c:...

1994
[20]

Cecilia Payne-Gaposchkin de- termined that stars are composed primarily of hydrogen and helium

was an American Founding Father, statesman, and politician who served as the first president of the United States from 1789 to 1797 . . . x: Who was the first president of the United States? Table 5: Examples of(c,x)pairs from each dataset. The top four blocks are standard QA benchmarks; the bottom three are the disjoint subsets of our TriState-Bench, def...

work page arXiv 2013
[21]

ARR consistently dominates the resistance subsetS res where all baselines collapse, while staying competitive onS cor andS agr

24 Model Method S cor S res S agr EM F1 EM F1 EM F1 Llama3-8B-Inst Greedy92.50 95.960.75 11.1089.75 93.82 CAD 60.75 75.38 0.25 9.55 53.25 65.57 COIECD 90.25 93.75 0.25 11.33 85.75 91.66 AdaCAD 91.75 95.13 0.25 10.88 88.50 93.22 CoCoA 87.25 92.06 0.25 10.81 83.25 90.70 ARR(Ours)86.75 91.7524.50 35.6889.00 93.77 Mistral-7B-Inst Greedy 81.00 87.37 2.50 13.93...

work page arXiv

[1] [1]

The Llama 3 Herd of Models

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al- Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783,

work page internal anchor Pith review Pith/arXiv arXiv

[2] [2]

Mistral 7B

URLhttps://arxiv.org/abs/2310.06825. Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. InProceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1601–1611,

work page internal anchor Pith review Pith/arXiv arXiv

[3] [3]

Cocoa: Confidence-and context-aware adap- tive decoding for resolving knowledge conflicts in large language models

Anant Khandelwal, Manish Gupta, and Puneet Agrawal. Cocoa: Confidence-and context-aware adap- tive decoding for resolving knowledge conflicts in large language models. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 6846–6866,

2025

[4] [4]

Large language models with controllable working memory

Daliang Li, Ankit Singh Rawat, Manzil Zaheer, Xin Wang, Michal Lukasik, Andreas Veit, Felix Yu, and Sanjiv Kumar. Large language models with controllable working memory. InFindings of the association for computational linguistics: ACL 2023, pp. 1774–1793, 2023a. Gaotang Li, Yuzhong Chen, and Hanghang Tong. Taming knowledge conflicts in language models. ar...

work page arXiv 2023

[5] [5]

Contrastive decoding: Open-ended text generation as optimization

Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori B Hashimoto, Luke Zettlemoyer, and Mike Lewis. Contrastive decoding: Open-ended text generation as optimization. In Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: Long papers), pp. 12286–12312, 2023b. Shayne Longpre, Kartik P...

2021

[6] [6]

Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning

Pan Lu, Liang Qiu, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, Tanmay Rajpurohit, Peter Clark, and Ashwin Kalyan. Dynamic prompt learning via policy gradient for semi-structured mathematical reasoning.arXiv preprint arXiv:2209.14610,

work page Pith review arXiv

[7] [7]

WebGPT: Browser-assisted question-answering with human feedback

Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. Webgpt: Browser-assisted question- answering with human feedback.arXiv preprint arXiv:2112.09332,

work page internal anchor Pith review Pith/arXiv arXiv

[8] [8]

Fabio Petroni, Tim Rockt ¨aschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. Language models as knowledge bases? InProceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural lan- guage processing (EMNLP-IJCNLP), pp. 2463–2473,

2019

[9] [9]

Adam Roberts, Colin Raffel, and Noam Shazeer

URLhttps://arxiv.org/abs/2412 .15115. Adam Roberts, Colin Raffel, and Noam Shazeer. How much knowledge can you pack into the parame- ters of a language model? InProceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), pp. 5418–5426,

2020

[10] [10]

Trusting your evidence: Hallucinate less with context-aware decoding

Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Wen-tau Yih. Trusting your evidence: Hallucinate less with context-aware decoding. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), pp. 783–791,

2024

[11] [11]

Llama 2: Open Foundation and Fine-Tuned Chat Models

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288,

work page internal anchor Pith review Pith/arXiv arXiv

[12] [12]

Adacad: Adaptively decoding to balance conflicts between contextual and parametric knowledge

Han Wang, Archiki Prasad, Elias Stengel-Eskin, and Mohit Bansal. Adacad: Adaptively decoding to balance conflicts between contextual and parametric knowledge. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 11636–11652,

2025

[13] [13]

Knowl- edge conflicts for llms: A survey

Rongwu Xu, Zehan Qi, Zhijiang Guo, Cunxiang Wang, Hongru Wang, Yue Zhang, and Wei Xu. Knowl- edge conflicts for llms: A survey. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 8541–8565,

2024

[14] [14]

Hotpotqa: A dataset for diverse, explainable multi-hop question answering

Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. InProceedings of the 2018 conference on empirical methods in natural language processing, pp. 2369–2380,

2018

[15] [15]

Discerning and resolving knowledge conflicts through adaptive decoding with contextual information-entropy con- straint

Xiaowei Yuan, Zhao Yang, Yequan Wang, Shengping Liu, Jun Zhao, and Kang Liu. Discerning and resolving knowledge conflicts through adaptive decoding with contextual information-entropy con- straint. InFindings of the Association for Computational Linguistics: ACL 2024, pp. 3903–3922,

2024

[16] [16]

Steering knowledge selection behaviours in llms via sae- based representation engineering

Yu Zhao, Alessio Devoto, Giwon Hong, Xiaotang Du, Aryo Pradipta Gema, Hongru Wang, Xuanli He, Kam-Fai Wong, and Pasquale Minervini. Steering knowledge selection behaviours in llms via sae- based representation engineering. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human La...

2025

[17] [17]

Context-faithful prompting for large language models

Wenxuan Zhou, Sheng Zhang, Hoifung Poon, and Muhao Chen. Context-faithful prompting for large language models. InFindings of the Association for Computational Linguistics: EMNLP 2023, pp. 14544– 14556,

2023

[18] [18]

The caseα=0 degenerates to standard decoding fromp ctx,t

14 A Background Methods: Detailed Formulations A.1 CAD The CAD decoding distribution can be derived from a point-wise mutual information style adjustment: qCAD t (y)∝p ctx,t(y) pctx,t(y) ppri,t(y) α =exp h (1+α)logp ctx,t(y)−αlogp pri,t(y) i , (19) followed by softmax normalization over the vocabularyV. The caseα=0 degenerates to standard decoding fromp c...

2024

[19] [19]

20 NQ c: The Shivalik Hills to the northeast. x: Which geographical part of Haryana is Shivalik Hills situated? TabMWP c: Table: Name — Number of coins; Braden — 76; Camilla — 94; Rick — 86; Mary — 84; Hector — 80; Devin — 83; Emily — 82; Avery — 87 x: Some friends discussed the sizes of their coin collections. What is the mean of the numbers? TriviaQA c:...

1994

[20] [20]

Cecilia Payne-Gaposchkin de- termined that stars are composed primarily of hydrogen and helium

was an American Founding Father, statesman, and politician who served as the first president of the United States from 1789 to 1797 . . . x: Who was the first president of the United States? Table 5: Examples of(c,x)pairs from each dataset. The top four blocks are standard QA benchmarks; the bottom three are the disjoint subsets of our TriState-Bench, def...

work page arXiv 2013

[21] [21]

ARR consistently dominates the resistance subsetS res where all baselines collapse, while staying competitive onS cor andS agr

24 Model Method S cor S res S agr EM F1 EM F1 EM F1 Llama3-8B-Inst Greedy92.50 95.960.75 11.1089.75 93.82 CAD 60.75 75.38 0.25 9.55 53.25 65.57 COIECD 90.25 93.75 0.25 11.33 85.75 91.66 AdaCAD 91.75 95.13 0.25 10.88 88.50 93.22 CoCoA 87.25 92.06 0.25 10.81 83.25 90.70 ARR(Ours)86.75 91.7524.50 35.6889.00 93.77 Mistral-7B-Inst Greedy 81.00 87.37 2.50 13.93...

work page arXiv