Pith. sign in

REVIEW 3 major objections 5 minor 50 references

Recompute or Reuse? Diagnosing and Mitigating Textual Shortcuts in VLM Self-Reflection

T0 review · 3 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A vision-language model that fails to recompute after an image changes is often reusing evidence-bearing reasoning from its prior chain of thought as a textual shortcut, and this shortcut can be suppressed by a training-free attention mask.

desk verdict Solid, careful VLM self-reflection diagnosis with a clean training-free fix; the 'evidence-bearing' label is broader than it looks, but the core shortcut and FSAF results hold. read the letter →

arxiv 2608.01930 v1 pith:MHLROCFY submitted 2026-08-03 cs.CV cs.AI

Wenxiao Fan , Jingling Fu , Fang Li , Luohang Liu , Yu He , Lichen Ma , Zhiyang Yu , Weishan Bi
show 4 more authors
Junshi Huang Yan Li Gu Simiu Kan Li
This is my paper
classification cs.CVcs.AI
keywords vision-languagemodelsself-reflectionchain-of-thoughttextualshortcutattentionmaskvisualrecomputationcounterfactualevaluationFSAF
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

When a vision-language model is asked to re-examine an image after the visual evidence has changed, it often fails to recompute the answer from the current image. This paper argues that the reason is a textual shortcut: the model's own prior chain of thought, written when the old image was present, remains in context and its evidence-bearing sentences (numbers, option letters, visual and reasoning terms) steer the new answer. Controlled removal shows this effect is content-specific: deleting those spans changes answer preference and free-generation outcomes more than deleting length-matched non-evidence text or the final-answer span, and the control weakens gradually as more stale evidence is removed. Correcting the answer does not retire the shortcut; a matched stress test shows the prior answer resurges when current support is withdrawn. The paper introduces Fresh-State Attention Firewall (FSAF), a training-free mask that prevents the fresh re-query from attending to the old chain of thought, raising the visual-update rate from 35.28% to 53.61% and lowering the prior-answer rate from 39.22% to 3.67% across five Qwen VLMs.

What carries the argument

The central machinery is a matched counterfactual comparison: a question paired with two visually similar images that imply different answers, with the model first producing a coherent prior CoT on the counterfactual image. A hand-written, outcome-independent taxonomy (Appendix C.3) marks sentences as evidence-bearing if they contain numbers, option letters, visual-lexicon terms, or mathematical/reasoning terms; removal and reordering of these spans, plus the answer-preference margin m(S) = log p(y|S) − log p(y⁻|S) and the VUR/PAR semantic outcomes, isolate what carries prior control. The mitigation is FSAF, an additive attention mask applied at every language layer that sets logits to −∞ fo

What would settle it

Run the same removal and reordering experiments with human-annotated evidence spans instead of the rule-based taxonomy: if length-matched random or non-evidence removal produces the same VUR/PAR shifts as evidence removal, the content-specificity claim fails. Alternatively, apply FSAF while replacing the prior CoT with a length-matched neutral history of the same role structure: if the VUR gain persists despite no stale evidence being present, the effect is not specifically about stale evidence access.

Watch

Extended reading notes

Core claim

Evidence-bearing reasoning in a prior chain of thought is not inert context; it is an organized, reusable path to an answer that competes behaviorally with recomputation from the current image. The paper establishes this by a matched counterfactual setup in which the model first answers a paired image correctly, then reflects under the changed image with its old chain of thought in context. Removing or reordering the evidence-bearing sentences of that chain shifts the model's preference and its generated answer back toward the current image, while length-matched non-evidence removal and final-answer removal do not. A correct reflected answer does not certify that the shortcut is gone: withdr

Load-bearing premise

The load-bearing premise is that the hand-written rule taxonomy for 'evidence-bearing' sentences (numbers, option letters, visual and reasoning terms) correctly identifies the spans whose presence causes prior-answer control; if those rules misclassify the causal spans, the content-specificity result could be an artifact of the rule scheme rather than a property of the models.

Editorial extensions

If this is right

  • If the claim is right, VLM self-reflection systems should treat the prior reasoning trace as a state to be quarantined, not merely as context to be prompted over.
  • A correct answer following reflection is not evidence that the model has revised its evidence state; evaluation protocols should stress-test recovered answers before trusting them.
  • The shortcut's strength depends on how much stale evidence remains and how it is organized, implying that summarising or compressing old reasoning may be less effective than blocking it.
  • Training-free intervention can yield large gains on updating behavior without retraining, suggesting that accessibility control is a practical lever for reliability.
  • Extensions of the same protocol could audit other forms of persistent context for shortcut behaviour.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the taxonomy of 'evidence-bearing' content is what carries the effect, a similar rule-based decomposition could be used to build automated detectors for when a model is relying on stale text rather than the current stimulus.
  • The finding that reordering evidence weakens control suggests the shortcut is not just a bag of cues but an ordered derivation; a testable extension is whether inserting contradictions or uncertainty markers in the prior CoT would disrupt it more than deletion.
  • The reported 26.42% disagreement between the rule evaluator and the 235B semantic judge implies the effect sizes may be judge-sensitive; an editorial extension would be to re-run the FSAF evaluation with a second independent judge to bound that sensitivity.
  • The idea that fresh computation must be protected from stale textual reuse could transfer to other domains, such as code editing or tool-use agents, where a prior reasoned plan lingers in context.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper investigates why vision-language models (VLMs) fail to revise their answers when the image changes, and what they reuse instead of recomputing. Using 800 matched image pairs from VS-Bench, the authors build a counterfactual protocol in which the model first produces a coherent incorrect prior CoT from a counterfactual image, then reflects under the current image. Across 16 VLMs they report that removing evidence-bearing spans (E-) from the prior CoT shifts answer preference more than length-matched non-evidence removal or final-answer removal, that reordering evidence weakens prior control, and that a correct reflected answer does not retire the shortcut: under support withdrawal, recovered responses relapse more than direct controls. They propose FSAF, a training-free attention-mask intervention that blocks fresh computation from attending to the prior CoT, and report improvements in visual update rate and reduction of prior-answer rate across five Qwen VLMs.

Significance. If the findings hold, the paper makes a solid empirical contribution: it identifies a specific textual carrier of prior-state influence, shows that answer correctness is not a certificate of evidence-state revision, and provides a simple, effective mitigation. Strengths include the matched-pair design, the breadth of models, the dose-response analysis, and the unusually complete protocol documentation with a claim–evidence index, exact prompts, and checkpoint IDs. The main risk is that the 'evidence-bearing' span definition (App. C.3) is a broad lexical proxy for substantive reasoning; without validation against human annotations or an alternative span operationalization, the content-specificity conclusion is not fully established. The paper's broader diagnostic results—prior-CoT influence, residual dependence after recovery, and the FSAF effect—do not rest on that taxonomy and are well supported.

major comments (3)
  1. [Appendix C.3, Tab. 2, Fig. 3] The operational definition of E- is a fixed lexical rule list (numbers/symbols, option letters, visual lexicon, reasoning connectives). On VS-Bench mathematical CoTs this is nearly coextensive with 'all substantive sentences'; the C control removes the leftover meta-commentary. The large E–C effects therefore demonstrate that removing task-relevant reasoning weakens prior-answer control, but not specifically that the 'evidence-bearing' category is the causal carrier. Since Tab. 2 and Fig. 3 are the main content-specificity evidence, please validate the E- decomposition against human causal annotations or a second operationalization (e.g., parse-based factual claims vs. discourse markers), and report the agreement. This is load-bearing for the paper's central characterization.
  2. [Tab. 3, App. C.3] The order-disruption result (D vs NE) also depends entirely on the same E- taxonomy. If the taxonomy over-captures reasoning content, the D–NE contrast is not a clean test of 'evidence organization' but rather of reordering meaningful vs. filler sentences. The authors should provide a human-annotated subset or a second independent span definition and check whether the D–NE effect is robust to the choice of decomposition.
  3. [Tab. 7, App. F.5] The headline FSAF improvements are reported without confidence intervals, and the rule-based evaluator and the 235B judge disagree on 26.42% of VUR/PAR labels in the FSAF evaluation. Please report paired bootstrap CIs for VUR/PAR/m, and ideally validate the semantic judge on a human-annotated sample. The magnitude of the reported effect makes it likely that the main conclusion stands, but the precision of the estimate and the judge's reliability are currently unquantified.
minor comments (5)
  1. [Abstract and App. B.4] The phrase 'weakening current-image support' suggests a visual manipulation, but the support-withdrawal condition deletes textual evidence-bearing spans while keeping the image. Please clarify in the abstract/main text.
  2. [App. C.3] The ranking rule for dose-response weights option cues by 2x; the rationale for this weighting is not given. Please justify or show that results are insensitive to the ranking.
  3. [Tab. 7] The 1,800-pair FSAF cohort is unbalanced across models (267–486 per model); consider reporting per-model CIs and the pooled estimate with model as a random effect.
  4. [App. H.1] The eligibility filter (direct correctness on both images) is necessary but reduces the pool to 4,159/12,800 model-pairs; a brief discussion of how this selection might shape the conclusions would be helpful.
  5. [A.5] The semantic judge is itself a Qwen-family VLM; a human-annotated sanity check on a random subset would strengthen confidence in the VUR/PAR labels.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; the evidence-span taxonomy is an operational assumption, not a definitional shortcut.

full rationale

The paper's central claim is that evidence-bearing prior CoT content, rather than generic context or the final answer span, carries textual-shortcut control. This is established by controlled ablations: E- spans are identified by fixed, outcome-independent lexical rules (Appendix C.3), and the comparison against a length-matched non-evidence control (C) is a direct ablation, not a fitted parameter. The E-C contrast is therefore an empirical result under the paper's taxonomy, not a tautology: the rules do not select spans by their measured effect, so it is possible in principle that removing E- would not shift preference more than removing C. The appendix explicitly disclaims uniqueness ('These rules deliberately provide a reproducible, high-recall operational decomposition; they do not claim that the selected sentences are unique human-annotated causal spans.'), which is a construct-validity limitation rather than circularity. The prior-only condition, support-withdrawal test, neutral-history control, and FSAF intervention are independent of the E- taxonomy and directly measure behavioral influence. No load-bearing self-citation is present: the main external resource, VS-Bench (Shi et al. 2026), has no author overlap with this paper, and no uniqueness theorem from the authors is invoked to force a choice. No fitted input is renamed as a prediction. The only mild self-reference is that the content-specificity conclusion is relative to the authors' own operational definition of 'evidence-bearing,' but the controlled length-matched comparison prevents this from reducing the claim to its definition. Overall, the derivation chain is self-contained against external benchmarks and interventions; the paper earns a low circularity score.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the validity of the VS-Bench labels, the reliability of the text-only semantic judge, the evidence-span taxonomy, and the attention-mask implementation. No numerical fitting is used, but the evidence-cue weights are hand-set and the E- definition is ad hoc.

free parameters (1)
  • Evidence-cue ranking weights = option-letter cues weighted 2x; numerical and lexical cues weighted 1x (Appendix C.3)
    Hand-set weights determine which evidence sentences are removed at each dose fraction in Fig. 3; the primary E vs C comparison is less sensitive to this choice.
assumptions (4)
  • domain assumption VS-Bench image pairs are human-checked and the two targets y and y- are correct and distinct for each model.
    All comparisons are built on these 800 pairs; if a label is wrong or a pair is not answer-changing for a given model, the matched contrasts misattribute influence. Appendix G describes the construction.
  • domain assumption The text-only semantic judge Qwen3-VL-235B-Instruct reliably maps free generations to current, prior, both, or other.
    VUR/PAR and eligibility all use this judge; the paper reports a 26.42% disagreement between the rule evaluator and the 235B judge (Appendix F.5), so the judge's judgments are a load-bearing measurement assumption.
  • ad hoc to paper Sentences marked by the fixed lexical rules in Appendix C.3 constitute the evidence-bearing content E- in the sense that matters for the shortcut.
    The E-C contrast and the dose-response analysis (Fig. 3) depend on this operationalization; the paper itself says the rules are a high-recall heuristic rather than unique causal spans.
  • domain assumption The Transformer attention mask M_FSAF in Eq. (8)-(9) removes access to the prior CoT without changing other computational paths in a way that produces the measured effect.
    FSAF is deployed via eager-attention hooks on five Qwen models; if the mask implementation accidentally scales or drops other tokens, the attribution of the improvement to blocking stale text would be confounded.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Recompute or Reuse? Diagnosing and Mitigating Textual Shortcuts in VLM Self-Reflection." pith.science (2026). https://pith.science/paper/MHLROCFY

@misc{pith2026260801930,
  author       = {Pith},
  title        = {Pith review of: Recompute or Reuse? Diagnosing and Mitigating Textual Shortcuts in VLM Self-Reflection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MHLROCFY}},
  note         = {Machine review of arXiv:2608.01930}
}
read the original abstract

Vision-language models (VLMs) are expected to revise their reasoning when visual evidence changes. Failures to do so are often attributed to insufficient visual attention or contextual inertia, leaving unclear what models reuse instead of recomputing from the current image. We show that evidence-bearing reasoning in a prior chain of thought (CoT) can form a textual shortcut that competes behaviorally with visual recomputation. Across 16 VLMs, a matched counterfactual analysis identifies evidence-bearing content as the most robust carrier of prior-CoT influence. Removing this evidence-bearing content shifts answer preference more than removing length-matched non-evidence context or the final-answer span, with prior control weakening progressively as more stale evidence is removed. Reordering this evidence also weakens prior control, showing that its organization modulates shortcut strength. Beyond the immediate answer, the shortcut can retain residual influence after answer correction: weakening current-image support shifts preference back toward the prior answer, while repeated prior answers and reused premises arise mainly when the shortcut remains active. To limit this influence, we introduce Fresh-State Attention Firewall (FSAF), a training-free intervention that isolates fresh computation from the prior CoT. Across five VLMs, FSAF raises visual update rate from 35.28% to 53.61% and reduces prior-answer rate from 39.22% to 3.67%. Reliable VLM self-reflection therefore requires more than looking again: fresh visual recomputation must be protected from stale textual reuse.

Figures

Figures reproduced from arXiv: 2608.01930 by the authors.

Figure 1
Figure 1. Textual shortcut during VLM self-reflection. Reusing this prior evidence path forms a textual shortcut that can steer the reflected answer without fresh recomputation from the current image. explains what is missing, but not what takes its place in the computation. We therefore ask: when a VLM fails to recom￾pute, what does it reuse instead? We argue that prior reasoning can become an alternative computation path. A… view at source ↗
Figure 2
Figure 2. Controlled evaluation framework. A matched counterfactual image elicits a coherent, model-authored incorrect prior CoT (red). Under the current image, we compare direct inference (blue) with prior-conditioned reflection (purple), including their initial responses and evidence-preserving continuations (gray), using behavioral and answer-preference metrics. equally from MathVista-MINI, MathVerse-MINI, MathVi￾sion, and… view at source ↗
Figure 3
Figure 3. Evidence removal progressively weakens prior￾answer control. The proportion of samples with m(Sα) > 0 increases monotonically at the model level in all 16 VLMs. and deductions supporting y −, as highlighted in [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: Extra Old Lift is positive across all 16 VLMs under matched support withdrawal. Tab. 4 and [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Prior-CoT length predicts vulnerability but does not explain the core intervention effects. (a) Model-equal PAR under image-present prior-conditioned reflection. (b) Answer-preference effects for evidence removal versus a length-matched non-evidence control (E–C), evid…
Figure 6
Figure 6. Figure 6: A prior CoT can override changed visual evidence. Top: Qwen2.5-VL-7B-Instruct answers y − = 4 from the prior image I − and records the supporting response R−. Middle: the image is replaced by the current image I, whose target is y = 7, while the prior response remains …
Figure 7
Figure 7. Figure 7: Current visual evidence can reopen recomputation. Top and middle: Gemma-4-12B first derives the prior relative risk y − = 3.0, after which the current table changes the target to y = 2.0. Bottom: a fresh turn without the current image remains stale, whereas restoring I…
Figure 8
Figure 8. Figure 8: Reasoning, not the answer token alone, carries prior-state control. The current geometry image I is fixed across all four conditions. Kimi-VL-A3B-Instruct remains stale when given either the full prior response or its reasoning body alone, but updates when only the pri…
Figure 9
Figure 9. Figure 9: Evidence-bearing content carries textual-shortcut control. Qwen3-VL-32B-Instruct initially favors the prior answer when the full prior response is retained. Removing all answer-supporting evidence switches the candidate margin to the current answer, whereas removing a …
Figure 10
Figure 10. Figure 10: Evidence removal progressively weakens prior-answer control. Qwen3-VL-8B-Instruct begins with a prior-favored margin under the full response R−. As 25%, 50%, 75%, and then all of the identified stale-evidence spans are removed, the margin progressively moves toward an…
Figure 11
Figure 11. Figure 11: Evidence-bearing content also controls free generation. With the current chart fixed, Kimi-VL-A3B-Thinking￾2506 repeats the prior answer under the full R− and after a length-matched random removal. Removing all stale evidence or replacing it with current-image evidenc…
Figure 12
Figure 12. Figure 12: A recovered response can carry current-answer information beyond its final span. Qwen3.5-9B remains current￾correct after the final answer span is masked, but no longer produces the current answer after all mentions of that answer within the recovered response are mas…
Figure 13
Figure 13. Figure 13: Disrupting evidence-chain organization weakens textual-shortcut control. Gemma-4-12B-it favors the prior answer under the natural response. Disrupting the order of evidence-bearing sentences weakens this preference more than either disrupting non-evidence sentences or…
Figure 14
Figure 14. Figure 14: Matched support withdrawal reveals prior-history dependence after answer recovery. Qwen3-VL-8B-Thinking has recovered the current answer before this test. When current supporting evidence is removed, the direct-current path remains current-favored, but the history-con…
Figure 15
Figure 15. Figure 15: An active textual shortcut recurs under equivalent re-query. After the image changes, InternVL2-8B receives the same concise future question under three histories. The direct-current and neutral-history paths favor the current answer, whereas the active stale-history …
Figure 16
Figure 16. Figure 16: An active textual shortcut transfers a stale premise into a new computation. Qwen3-VL-32B-Thinking is asked to double the answer after the image change. The direct-current and neutral-history paths produce the current-derived target, whereas the active stale-history p…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

50 extracted references · 4 canonical work pages

  1. [1]

    Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education

    Clancey, William J. Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education. Proceedings of the Eighth International Joint Conference on Artificial Intelligence (IJCAI-83)

  2. [2]

    Classification Problem Solving

    Clancey, William J. Classification Problem Solving. Proceedings of the Fourth National Conference on Artificial Intelligence

  3. [3]

    , title =

    Robinson, Arthur L. , title =. 1980 , doi =. https://science.sciencemag.org/content/208/4447/1019.full.pdf , journal =

    1980

  4. [4]

    New Ways to Make Microcircuits Smaller---Duplicate Entry

    Robinson, Arthur L. New Ways to Make Microcircuits Smaller---Duplicate Entry. Science

  5. [5]

    Clancey and Glenn Rennels , abstract =

    Diane Warner Hasling and William J. Clancey and Glenn Rennels , abstract =. Strategic explanations for a diagnostic consultation system , journal =. 1984 , issn =. doi:https://doi.org/10.1016/S0020-7373(84)80003-6 , url =

    doi:10.1016/s0020-7373(84)80003-6 1984

  6. [6]

    and Rennels, Glenn R

    Hasling, Diane Warner and Clancey, William J. and Rennels, Glenn R. and Test, Thomas. Strategic Explanations in Consultation---Duplicate. The International Journal of Man-Machine Studies

  7. [7]

    Poligon: A System for Parallel Problem Solving

    Rice, James. Poligon: A System for Parallel Problem Solving

  8. [8]

    Transfer of Rule-Based Expertise through a Tutorial Dialogue

    Clancey, William J. Transfer of Rule-Based Expertise through a Tutorial Dialogue

Show all 50 references
  1. [9]

    The Engineering of Qualitative Models

    Clancey, William J. The Engineering of Qualitative Models

  2. [10]

    2023 , eprint=

    Attention Is All You Need , author=. 2023 , eprint=

    2023

  3. [11]

    Pluto: The 'Other' Red Planet

    NASA. Pluto: The 'Other' Red Planet

  4. [12]

    Advances in Neural Information Processing Systems , volume =

    Aman Madaan and Niket Tandon and Prakhar Gupta and Skyler Hallinan and Luyu Gao and Sarah Wiegreffe and Uri Alon and Nouha Dziri and Shrimai Prabhumoye and Yiming Yang and Shashank Gupta and Bodhisattwa Prasad Majumder and Katherine Hermann and Sean Welleck and Amir Yazdanbakh...

    2023

  5. [13]

    Transactions of the Association for Computational Linguistics , volume =

    Ryo Kamoi and Yusen Zhang and Nan Zhang and Jiawei Han and Rui Zhang , title =. Transactions of the Association for Computational Linguistics , volume =. 2024 , publisher =. doi:10.1162/tacl_a_00713 , url =

    2024 doi

  6. [14]

    2026 , eprint =

    Chufan Shi and Cheng Yang and Yaokang Wu and Linghao Jin and Bo Shui and Taylor Berg-Kirkpatrick and Xuezhe Ma , title =. 2026 , eprint =. doi:10.48550/arXiv.2605.15864 , url =

    2026 doi arXiv

  7. [15]

    Zou and Xin Wang and Yuyin Zhou and Sheng Liu , title =

    Zhongxing Xu and Chengzhi Liu and Qingyue Wei and Juncheng Wu and James Y. Zou and Xin Wang and Yuyin Zhou and Sheng Liu , title =. Advances in Neural Information Processing Systems , volume =. 2025 , url =

    2025

  8. [16]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages =

    Pu Jian and Junhong Wu and Wei Sun and Chen Wang and Shuo Ren and Jiajun Zhang , title =. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages =. 2025 , address =. doi:10.18653/v1/2025.emnlp-main.470 , url =

    2025 doi

  9. [17]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume =

    Shuo Yang and Yuwei Niu and Yuyang Liu and Yang Ye and Bin Lin and Li Yuan , title =. Proceedings of the AAAI Conference on Artificial Intelligence , volume =. 2026 , doi =

    2026

  10. [18]

    The Twelfth International Conference on Learning Representations , year =

    Jie Huang and Xinyun Chen and Swaroop Mishra and Huaixiu Steven Zheng and Adams Wei Yu and Xinying Song and Denny Zhou , title =. The Twelfth International Conference on Learning Representations , year =

  11. [19]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages =

    Bryan Wilie and Samuel Cahyawijaya and Etsuko Ishii and Junxian He and Pascale Fung , title =. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages =. 2024 , address =. doi:10.18653/v1/2024.emnlp-main.586 , url =

    2024 doi

  12. [20]

    Advances in Neural Information Processing Systems , volume=

    Fine-Tuning Language Models with Just Forward Passes , author=. Advances in Neural Information Processing Systems , volume=

  13. [21]

    Liu and Balaji Lakshminarayanan , editor =

    Jie Ren and Yao Zhao and Tu Vu and Peter J. Liu and Balaji Lakshminarayanan , editor =. Self-Evaluation Improves Selective Generation in Large Language Models , booktitle =. 2023 , url =

    2023

  14. [22]

    Can Multiple-choice Questions Really Be Useful in Detecting the Abilities of

    Li, Wangyue and Li, Liangzhi and Xiang, Tong and Liu, Xiao and Deng, Wei and Garcia, Noa , booktitle=. Can Multiple-choice Questions Really Be Useful in Detecting the Abilities of

  15. [23]

    2025 , eprint =

    Xu Chu and Xinrong Chen and Guanyu Wang and Zhijie Tan and Kui Huang and Wenyu Lv and Tong Mo and Weiping Li , title =. 2025 , eprint =. doi:10.48550/arXiv.2505.23558 , url =

    2025 doi arXiv

  16. [24]

    Kanzhi Cheng and YanTao, Li and Fangzhi Xu and Jianbing Zhang and Hao Zhou and Yang Liu , title =. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , ...

    2025 doi

  17. [25]

    Findings of the Association for Computational Linguistics: EMNLP 2025 , pages =

    Jiazhou Ji and Xinru Lu , title =. Findings of the Association for Computational Linguistics: EMNLP 2025 , pages =. 2025 , publisher =. doi:10.18653/v1/2025.findings-emnlp.1384 , url =

    2025 doi

  18. [26]

    Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =

    Rheeya Uppaal and Phu Mon Htut and Min Bai and Nikolaos Pappas and Zheng Qi and Sandesh Swamy , title =. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =. 2026 , publisher =. doi:10.18...

    2026 doi

  19. [27]

    2026 , eprint =

    Liyan Tang and Fangcong Yin and Greg Durrett , title =. 2026 , eprint =. doi:10.48550/arXiv.2607.02490 , url =

    2026 doi arXiv

  20. [28]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

    Xiaoxiao Sun and Mingyang Li and Kun Yuan and Min Woo Sun and Mark Endo and Shengguang Wu and Changlin Li and Yuhui Zhang and Zeyu Wang and Serena Yeung-Levy , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =. 2026 , url =

    2026

  21. [29]

    Advances in Neural Information Processing Systems , volume =

    Noah Shinn and Federico Cassano and Ashwin Gopinath and Karthik Narasimhan and Shunyu Yao , title =. Advances in Neural Information Processing Systems , volume =. 2023 , url =

    2023

  22. [30]

    The Twelfth International Conference on Learning Representations , year =

    Zhibin Gou and Zhihong Shao and Yeyun Gong and Yelong Shen and Yujiu Yang and Nan Duan and Weizhu Chen , title =. The Twelfth International Conference on Learning Representations , year =

  23. [31]

    Transactions of the Association for Computational Linguistics , volume =

    Liangming Pan and Michael Saxon and Wenda Xu and Deepak Nathani and Xinyi Wang and William Yang Wang , title =. Transactions of the Association for Computational Linguistics , volume =. 2024 , doi =

    2024

  24. [32]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =

    Wenqi Zhang and Yongliang Shen and Linjuan Wu and Qiuying Peng and Jun Wang and Yueting Zhuang and Weiming Lu , title =. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =. 2024 , publisher =. doi:10.18653/...

    2024 doi

  25. [33]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =

    Zhe Yang and Yichang Zhang and Yudong Wang and Ziyao Xu and Junyang Lin and Zhifang Sui , title =. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =. 2025 , publisher =. doi:10.18653/v1/2025.acl-long.203 , url =

    2025 doi

  26. [34]

    Understanding the Dark Side of LLM s' Intrinsic Self-Correction

    Zhang, Qingjie and Wang, Di and Qian, Haoting and Li, Yiming and Zhang, Tianwei and Huang, Minlie and Xu, Ke and Li, Hewu and Yan, Liu and Qiu, Han. Understanding the Dark Side of LLM s' Intrinsic Self-Correction. Proceedings of the 63rd Annual Meeting of the Association for C...

    2025 doi

  27. [35]

    Cohen , title =

    Adi Simhi and Fazl Barez and Martin Tutek and Yonatan Belinkov and Shay B. Cohen , title =. 2026 , eprint =. doi:10.48550/arXiv.2603.03308 , url =

    2026 doi arXiv

  28. [36]

    2026 , eprint =

    Kuan-Yen Chen and Fang-Yi Su and Jung-Hsien Chiang , title =. 2026 , eprint =. doi:10.48550/arXiv.2606.05976 , url =

    2026 doi arXiv

  29. [37]

    Findings of the Association for Computational Linguistics: ACL 2026 , pages =

    Xu Pan and Jingxuan Fan and Zidi Xiong and Ely Hahami and Jorin Overwiening and Ziqian Xie , title =. Findings of the Association for Computational Linguistics: ACL 2026 , pages =. 2026 , publisher =. doi:10.18653/v1/2026.findings-acl.449 , url =

    2026 doi

  30. [38]

    Bowman , title =

    Miles Turpin and Julian Michael and Ethan Perez and Samuel R. Bowman , title =. Advances in Neural Information Processing Systems , volume =. 2023 , url =

    2023

  31. [39]

    Measuring Faithfulness in Chain-of-Thought Reasoning , journal =

    Tamera Lanham and Anna Chen and Ansh Radhakrishnan and Benoit Steiner and Carson Denison and Danny Hernandez and Dustin Li and Esin Durmus and Evan Hubinger and Jackson Kernion and Kamile Lukosiute and Karina Nguyen and Newton Cheng and Nicholas Joseph and Nicholas Schiefer an...

    2023 doi arXiv

  32. [40]

    Advances in Neural Information Processing Systems , volume =

    Kevin Meng and David Bau and Alex Andonian and Yonatan Belinkov , title =. Advances in Neural Information Processing Systems , volume =. 2022 , url =

    2022

  33. [41]

    The Twelfth International Conference on Learning Representations , year =

    Pan Lu and Hritik Bansal and Tony Xia and Jiacheng Liu and Chunyuan Li and Hannaneh Hajishirzi and Hao Cheng and Kai-Wei Chang and Michel Galley and Jianfeng Gao , title =. The Twelfth International Conference on Learning Representations , year =

  34. [42]

    Computer Vision -

    Renrui Zhang and Dongzhi Jiang and Yichi Zhang and Haokun Lin and Ziyu Guo and Pengshuo Qiu and Aojun Zhou and Pan Lu and Kai. Computer Vision -. 2024 , url =. doi:10.1007/978-3-031-73242-3\_10 , timestamp =

    2024 doi

  35. [43]

    Advances in Neural Information Processing Systems , volume =

    Ke Wang and Junting Pan and Weikang Shi and Zimu Lu and Houxing Ren and Aojun Zhou and Mingjie Zhan and Hongsheng Li , title =. Advances in Neural Information Processing Systems , volume =. 2024 , doi =

    2024

  36. [44]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =

    Xiang Yue and Tianyu Zheng and Yuansheng Ni and Yubo Wang and Kai Zhang and Shengbang Tong and Yuxuan Sun and Botao Yu and Ge Zhang and Huan Sun and Yu Su and Wenhu Chen and Graham Neubig , title =. Proceedings of the 63rd Annual Meeting of the Association for Computational Li...

    2025 doi

  37. [45]

    Fung and Heng Ji , title =

    Jiayi He and Hehai Lin and Qingyun Wang and Yi R. Fung and Heng Ji , title =. Findings of the Association for Computational Linguistics: ACL 2025 , pages =. 2025 , publisher =. doi:10.18653/v1/2025.findings-acl.331 , url =

    2025 doi

  38. [46]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

    Tianrui Guan and Fuxiao Liu and Xiyang Wu and Ruiqi Xian and Zongxia Li and Xiaoyu Liu and Xijun Wang and Lichang Chen and Furong Huang and Yaser Yacoob and Dinesh Manocha and Tianyi Zhou , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognit...

    2024

  39. [47]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

    Sicong Leng and Hang Zhang and Guanzheng Chen and Xin Li and Shijian Lu and Chunyan Miao and Lidong Bing , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =. 2024 , url =

    2024

  40. [48]

    Gomez and Lukasz Kaiser and Illia Polosukhin , editor =

    Ashish Vaswani and Noam Shazeer and Niki Parmar and Jakob Uszkoreit and Llion Jones and Aidan N. Gomez and Lukasz Kaiser and Illia Polosukhin , editor =. Attention is All you Need , booktitle =. 2017 , url =

    2017

  41. [49]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

    Junyang Ji and Qifan Liu and Wenming Yang and Zhihai He , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =. 2026 , url =

    2026

  42. [50]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

    Yongchang Zhang and Oliver Ma and Tianyi Liu and Guangquan Zhou and Yang Chen , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =. 2026 , url =

    2026

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.