Pith. sign in

REVIEW 5 cited by

Faithful Chain-of-Thought Reasoning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2301.13379 v3 pith:7TGCGI4Q submitted 2023-01-31 cs.CL

Qing Lyu , Shreya Havaldar , Adam Stein , Li Zhang , Delip Rao , Eric Wong , Marianna Apidianaki , Chris Callison-Burch This is my paper
classification cs.CL
keywords reasoningchainfaithfulaccuracyanswerperformancechain-of-thoughtfaithfulness
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
abstract

While Chain-of-Thought (CoT) prompting boosts Language Models' (LM) performance on a gamut of complex reasoning tasks, the generated reasoning chain does not necessarily reflect how the model arrives at the answer (aka. faithfulness). We propose Faithful CoT, a reasoning framework involving two stages: Translation (Natural Language query $\rightarrow$ symbolic reasoning chain) and Problem Solving (reasoning chain $\rightarrow$ answer), using an LM and a deterministic solver respectively. This guarantees that the reasoning chain provides a faithful explanation of the final answer. Aside from interpretability, Faithful CoT also improves empirical performance: it outperforms standard CoT on 9 of 10 benchmarks from 4 diverse domains, with a relative accuracy gain of 6.3% on Math Word Problems (MWP), 3.4% on Planning, 5.5% on Multi-hop Question Answering (QA), and 21.4% on Relational Inference. Furthermore, with GPT-4 and Codex, it sets the new state-of-the-art few-shot performance on 7 datasets (with 95.0+ accuracy on 6 of them), showing a strong synergy between faithfulness and accuracy.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 23 citations worldwide. Full citation record

  1. Constraint-Anchored Reasoning Traces

    cs.AI 2026-07 conditional novelty 6.0 of 10

    CART interleaves language-model reasoning with verifiable symbolic anchors and backtracking, reducing error snowballing in open-source MLLMs.

  2. Direct Behavior Optimization: Unlocking the Potential of Lightweight LLMs

    cs.CL 2025-06 conditional novelty 6.0 of 10

    A gradient-free Monte Carlo tree search over JSON key-step plans produces few-shot demonstrations that let LLaMA3-8B and LLaMA3.2-3B outperform GPT-3.5 on most of seven BIG-Bench Hard tasks.

  3. Mechanistic Evidence for Faithfulness Decay in Chain-of-Thought Reasoning

    cs.CL 2026-02 reject novelty 5.0 of 10

    Introduces NLDD, a step-level CoT faithfulness metric, and claims a consistent 'reasoning horizon' at 70-85% of chain length; the claim is unsupported by the paper's own reported data.

  4. Investigating the Effects of Cognitive Biases in Prompts on Large Language Model Outputs

    cs.CL 2025-06 conditional novelty 4.0 of 10

    Injecting explicit suggestions or biased recollections into prompts reduces LLM accuracy on multiple-choice QA tasks, and attention weights shift toward the suggested answer.

  5. Thinking About Thinking: SAGE-nano's Inverse Reasoning for Self-Aware Language Models

    cs.AI 2025-06 reject novelty 3.0 of 10

    A 4B-parameter model is claimed to explain its own reasoning through inverse attention analysis, but the paper offers no consistent evidence or artifacts.

Pith tools