Pith. sign in

REVIEW 9 cited by

Internal Consistency and Self-Feedback in Large Language Models: A Survey

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2407.14507 v3 pith:4Q7EDUHY submitted 2024-07-19 cs.CL

Xun Liang , Shichao Song , Zifan Zheng , Hanyu Wang , Qingchen Yu , Xunkai Li , Rong-Hua Li , Yi Wang
show 3 more authors
Zhonghao Wang Feiyu Xiong Zhiyu Li
This is my paper
classification cs.CL
keywords consistencyinternalframeworkllmsreasoningself-feedbackstudiesbeen
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large language models (LLMs) often exhibit deficient reasoning or generate hallucinations. To address these, studies prefixed with "Self-" such as Self-Consistency, Self-Improve, and Self-Refine have been initiated. They share a commonality: involving LLMs evaluating and updating themselves. Nonetheless, these efforts lack a unified perspective on summarization, as existing surveys predominantly focus on categorization. In this paper, we use a unified perspective of internal consistency, offering explanations for reasoning deficiencies and hallucinations. Internal consistency refers to the consistency in expressions among LLMs' latent, decoding, or response layers based on sampling methodologies. Then, we introduce an effective theoretical framework capable of mining internal consistency, named Self-Feedback. This framework consists of two modules: Self-Evaluation and Self-Update. The former captures internal consistency signals, while the latter leverages the signals to enhance either the model's response or the model itself. This framework has been employed in numerous studies. We systematically classify these studies by tasks and lines of work; summarize relevant evaluation methods and benchmarks; and delve into the concern, "Does Self-Feedback Really Work?" We also propose several critical viewpoints, including the "Hourglass Evolution of Internal Consistency", "Consistency Is (Almost) Correctness" hypothesis, and "The Paradox of Latent and Explicit Reasoning". The relevant resources are open-sourced at https://github.com/IAAR-Shanghai/ICSFSurvey.

Discussion (0). Sign in to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. FVRuleLearner: Operator-Level Reasoning Tree (Op-Tree)-Based Rules Learning for Formal Verification

    cs.AR 2026-03 unverdicted novelty 7.0 of 10

    FVRuleLearner retrieves learned operator-level reasoning rules to boost the functional correctness of LLM-generated SystemVerilog assertions by roughly 30 percentage points over simple prompting baselines.

  2. Unified Audio Intelligence Without Regressing on Text Intelligence

    cs.CL 2026-07 conditional novelty 6.0 of 10

    A unified 30B MoE audio-text LLM achieves state-of-the-art audio understanding, generation, and speech tasks while preserving text reasoning comparable to its text-only backbone.

  3. SAND: Boosting LLM Agents with Self-Taught Action Deliberation

    cs.CL 2025-07 conditional novelty 6.0 of 10

    A self-training framework that uses self-consistency action sampling and execution-guided critiques to teach LLM agents to explicitly deliberate over candidate actions, improving average rewards by about 20% over supe...

  4. Adaptive Termination for Multi-round Parallel Reasoning: An Universal Semantic Entropy-Guided Framework

    cs.CL 2025-07 conditional novelty 5.0 of 10

    A semantic entropy-guided stopping rule for multi-round parallel LLM reasoning improves accuracy while reducing inference steps on five benchmarks.

  5. A Survey of Automatic Evaluation Methods on Text, Visual and Speech Generations

    cs.CL 2025-06 conditional novelty 5.0 of 10

    A unified taxonomy and comparative meta-evaluation of automatic evaluation methods across text, vision, and speech generation, concluding that LLM-based evaluators dominate current practice.

  6. Customized Generative AI Agent for Transportation Engineering Practice: A Development and Continued Pre-training Guideline

    cs.AI 2026-06 unverdicted novelty 3.5 of 10

    LoRA continued pretraining on a small U.S. transportation corpus lifts BLEU-4 and ROUGE for Qwen2.5-7B and LLaMA-3.1-8B far above the other four models tested.

  7. Survey of GenAI for Automotive Software Development: From Requirements to Executable Code

    cs.SE 2025-07 conditional novelty 3.0 of 10

    A review of roughly 60 papers and 9 industry respondents finds GPT-family models dominate automotive code generation while requirements handling lags due to confidentiality constraints.

  8. Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Models

    cs.CL 2025-06 reject novelty 3.0 of 10

    A survey of LLM hallucination research that formalizes hallucination types and argues, via incompleteness and undecidability arguments, that hallucinations cannot be fully eliminated.

  9. Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges

    cs.AI 2025-07 reject novelty 1.0 of 10

    A broad survey of LLM alignment that catalogs objectives, benchmarks, SFT/RLHF/DPO methods, and safety challenges, without contributing new experimental or theoretical results.

Pith tools