Pith. sign in

REVIEW 18 cited by

Generating Sequences by Learning to Self-Correct

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2211.00053 v1 pith:3ZWTTWAR submitted 2022-10-31 cs.CL

Sean Welleck , Ximing Lu , Peter West , Faeze Brahman , Tianxiao Shen , Daniel Khashabi , Yejin Choi This is my paper · ORCID
classification cs.CL
keywords languagebasecorrectorgenerationgeneratorimperfectconstraintscorrect
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Sequence generation applications require satisfying semantic constraints, such as ensuring that programs are correct, using certain keywords, or avoiding undesirable content. Language models, whether fine-tuned or prompted with few-shot demonstrations, frequently violate these constraints, and lack a mechanism to iteratively revise their outputs. Moreover, some powerful language models are of extreme scale or inaccessible, making it inefficient, if not infeasible, to update their parameters for task-specific adaptation. We present Self-Correction, an approach that decouples an imperfect base generator (an off-the-shelf language model or supervised sequence-to-sequence model) from a separate corrector that learns to iteratively correct imperfect generations. To train the corrector, we propose an online training procedure that can use either scalar or natural language feedback on intermediate imperfect generations. We show that Self-Correction improves upon the base generator in three diverse generation tasks - mathematical program synthesis, lexically-constrained generation, and toxicity control - even when the corrector is much smaller than the base generator.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 18 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 30 citations worldwide. Full citation record

  1. Multi-Turn On-Policy Distillation with Prefix Replay

    cs.LG 2026-07 conditional novelty 6.0 of 10

    ReOPD offline-distills multi-turn agentic LLMs via teacher-prefix replay plus step-decay sampling, matching online OPD accuracy at ≥4× speed with zero tool calls.

  2. DenoiseRL: Bootstrapping Reasoning Models to Recover from Noisy Prefixes

    cs.AI 2026-05 unverdicted novelty 6.0 of 10

    DenoiseRL optimizes recovery from noisy prefixes in weak-model reasoning failures to improve performance and self-correction on math and general reasoning benchmarks without external supervision.

  3. SC-Captioner: Improving Image Captioning with Self-Correction by Reinforcement Learning

    cs.CV 2025-08 conditional novelty 6.0 of 10

    An RL framework that trains vision-language models to self-correct captions via a scene-graph-based reward outperforms SFT and DPO on caption quality.

  4. Stream Aligner: Efficient Sentence-Level Alignment via Distribution Induction

    cs.CL 2025-01 reject novelty 6.0 of 10

    A sentence-level iterative correction method is claimed to improve large language model alignment and math reasoning, but the evidence is compromised by GPT-4 serving as both trainer and judge.

  5. Teaching LLMs to Refine with Tools

    cs.CL 2024-12 conditional novelty 6.0 of 10

    CaP trains LLMs to fix chain-of-thought math solutions by producing program-of-thought code, and shows that DPO preference optimization is essential for the refinement to actually improve accuracy.

  6. Refining Over Resampling: Test-Time Self-Correction for LLM Reasoning

    cs.AI 2026-08 conditional novelty 5.0 of 10

    Sampling multiple reasoning paths, refining each with self-critique and self-correction, then majority voting improves math reasoning accuracy over width-only or verifier-based test-time scaling on several open-weight LLMs.

  7. ReflectRL: Learning from Golden Negative Trajectories via Reflective-to-Direct Reasoning

    cs.AI 2026-08 conditional novelty 5.0 of 10

    ReflectRL repurposes failed expert reasoning traces as reflective scaffolding during RL and distillation training, then transitions the policy to direct reasoning, improving math and science benchmark scores.

  8. I2CR: Intra- and Inter-modal Collaborative Reflections for Multimodal Entity Linking

    cs.CV 2025-08 conditional novelty 5.0 of 10

    A text-first, multi-round visual feedback framework reports state-of-the-art top-1 accuracy on WikiMEL, WikiDiverse, and RichMEL.

  9. R4ec: A Reasoning, Reflection, and Refinement Framework for Recommendation Systems

    cs.IR 2025-07 conditional novelty 5.0 of 10

    R4ec trains a small reflection model to critique and refine LLM-generated user and item knowledge, which then improves downstream recommendation accuracy.

  10. Hallucination Detection with Small Language Models

    cs.CL 2025-06 reject novelty 5.0 of 10

    A multi-small-model ensemble with sentence splitting, z-score normalization, and harmonic mean detects hallucinations in RAG answers with a reported 10% F1 gain over single-model baselines.

  11. Boosting LLM Reasoning via Spontaneous Self-Correction

    cs.AI 2025-06 conditional novelty 5.0 of 10

    SPOC trains LLMs to interleave self-verification and solution attempts in a single pass, reporting gains on math benchmarks, though most gains come from stronger first attempts.

  12. S$^2$-MAD: Breaking the Token Barrier to Enhance Multi-Agent Debate Efficiency

    cs.CL 2025-02 conditional novelty 5.0 of 10

    S2-MAD's decision mechanism filters redundant viewpoints and conditionally skips participation, cutting token costs by up to 94.5% versus standard multi-agent debate while keeping accuracy within about 2 points in the...

  13. Understanding the Dark Side of LLMs' Intrinsic Self-Correction

    cs.CL 2024-12 conditional novelty 5.0 of 10

    Intrinsic self-correction makes state-of-the-art LLMs overturn correct answers across four task types, and simple question repetition or tiny fine-tuning reduces this damage.

  14. ReflectEvo: Improving Meta Introspection of Small LLMs by Learning Self-Reflection

    cs.AI 2025-05 reject novelty 4.0 of 10

    ReflectEvo shows that small language models can improve their reasoning by fine-tuning on their own self-generated reflections, but the headline BIG-bench gains depend on oracle feedback and an unclear data split.

  15. CodeCoR: An LLM-Based Self-Reflective Multi-Agent Framework for Code Generation

    cs.SE 2025-01 conditional novelty 4.0 of 10

    CodeCoR improves LLM code generation by having four agents generate, test, prune, and repair candidate solutions, reporting a 77.8% average Pass@1 on four Python benchmarks.

  16. Recursive Decomposition of Logical Thoughts: Framework for Superior Reasoning and Knowledge Propagation in Large Language Models

    cs.CL 2025-01 reject novelty 4.0 of 10

    A prompting framework that recursively decomposes reasoning tasks and self-scores candidate thoughts is reported to improve LLM accuracy on math and letter-concatenation benchmarks, though the headline improvement is ...

  17. Zero-Shot Strategies for Length-Controllable Summarization

    cs.CL 2024-12 conditional novelty 4.0 of 10

    Simple zero-shot tricks, target remapping, best-of-N sampling, and iterative self-revision, substantially improve length compliance in LLaMA-3 summarization without fine-tuning.

  18. Towards Intrinsic Self-Correction Enhancement in Monte Carlo Tree Search Boosted Reasoning via Iterative Preference Learning

    cs.LG 2024-12 conditional novelty 4.0 of 10

    Adding a self-correction RL stage before MCTS-DPO improves arithmetic reasoning accuracy by 2.00 to 4.94 points on GSM8K and MATH.

Pith tools