Pith. sign in

REVIEW 4 major objections 5 minor 30 references

LLM Reasoning for Subjective Tasks: Failure Modes, Mitigation, and Dynamic Reasoning Routing

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that explicit reasoning often degrades subjective verification, that standard RLVR training triggers a collapse of deliberation, and that a conditional length-penalized reward plus persona routing recovers performance.

desk verdict The empirical core—reasoning degrades subjective verification and RLVR collapses the reasoning trace—is solid and useful; the persona-routing claim rests on an uncontrolled max-min statistic and should be reworked before publication. read the letter →

arxiv 2608.08889 v1 pith:FMCDIO2T submitted 2026-08-09 cs.AI

Juncheng Dong , Ding Tong , Ishan Gupta , Yuyan Wang This is my paper · ORCID
classification cs.AI
keywords subjectiveverificationLLMreasoningcollapsereinforcementlearningwithverifiablerewardsGRPOlength-penalizedrewardpersonaroutingLLM-as-a-Judge
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the reasoning techniques that work for math and code, such as chain-of-thought prompting and reinforcement learning with verifiable rewards, do not transfer to subjective verification tasks where correctness is a matter of human rubric rather than deterministic truth. Across four production-style verification rubrics, forcing explicit reasoning frequently made both proprietary and open models less accurate, and applying standard RLVR made things worse. The paper identifies a mechanism it calls reasoning collapse, in which the model abandons deliberation for rapid guessing once short answers earn equal reward at lower token cost. It then shows that a conditional length-penalized reward, which pays a reasoning bonus only when the final answer is correct and stops at a length cap, halts the collapse and exceeds baseline accuracy. The paper also reports that verification accuracy swings by nearly 0.38 macro-F1 depending on which of 1,500 reasoning personas the model impersonates, motivating a routing architecture that selects a persona per query.

What carries the argument

The load-bearing object is the conditional length-penalized reward of Eq. (2), where total reward is $r_{base}+\lambda L$ when the final verification is correct and the number of generated tokens $L$ is below a target $L_{target}$, and $r_{base}$ otherwise. Gating the length bonus on correctness is what distinguishes it from two failed formulations: an unconstrained length bonus that produced gibberish up to the context limit, and a target-constrained penalty that produced coherent but useless filler. The paper also names and characterizes reasoning collapse, the observed training dynamic in which mean terminated length plummets after roughly 70 steps as the policy discovers that guessing beats deliberation. The third piece of machinery is the persona-routing blueprint: synthesize diverse reasoning trajectories across many personas, supervise the base model on them, then use RLVR to teach the model which reasoning bias to apply rather than how to reason.

What would settle it

Re-run the persona synthesis on one sensitivity task with each persona producing many independent verification rounds, then compute macro-F1 after stratifying by each persona's label prior (base rate of 'Yes'). If the best-versus-worst spread collapses toward noise once label priors are matched, the persona-effect claim is not about reasoning style; if the spread persists, the routing blueprint gains direct support.

Watch

Extended reading notes

Core claim

The central claim is that explicit, math-flavored reasoning is frequently unhelpful and often actively harmful for subjective verification, and that standard RLVR post-training makes this worse by inducing reasoning collapse, a training dynamic in which the policy suppresses long deliberation once direct guessing earns the same reward at a fraction of the token cost. The proposed mitigation is a conditional length-penalized reward that gives a length bonus only for correct answers below a target length, which prevents both collapse and context-exploitation filler. On a conversational sensitivity task this lifts macro-F1 from 0.805 (reasoning baseline) and 0.749 (collapsed GRPO) to 0.851, and on the hardest rubric, where reasoning did not help at baseline, it raises F1 from 0.519 after collapse to 0.572, beating both baselines. The paper further claims that a large share of subjective-verification error is reasoning-style mismatch: across 1,500 synthesized personas, macro-F1 on a sensitivity task ranged from 0.416 to 0.792 solely from the adopted reasoning style.

Load-bearing premise

The persona analysis assumes that the 0.38 macro-F1 spread across 1,500 personas is caused by differences in reasoning style rather than by shifts in the label distribution the personas produce or by instability in the impersonating model; if that spread is an artifact, the claim that much subjective-verification error is reasoning-style mismatch loses its support.

Editorial extensions

If this is right

  • For subjective rubrics, a direct zero-shot verification prompt becomes a stronger and cheaper baseline than a prompted chain-of-thought; teams should not assume deliberation helps.
  • Reinforcement post-training on subjective tasks needs an explicit reasoning-budget term; outcome-only rewards silently erase reasoning and can lower accuracy below the pre-training baseline.
  • The conditional length reward is a drop-in shaping term for RLVR pipelines and recovers performance even on rubrics where reasoning was harmful at baseline.
  • Models with strong math-centric alignment are the most vulnerable to collapse, while a weaker-math model used the context window as a flexible scratchpad and improved consistently.
  • A library of vetted reasoning personas, selected per query by the policy, is a concrete alternative to a single monolithic chain-of-thought and makes the reason for a verdict auditable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the persona-driven spread survives controls, then much of what looks like irreducible label noise in subjective verification is actually a fixed reasoning prior; that would make persona routing a form of test-time adaptation rather than prompt decoration.
  • The same collapse mechanism should appear in any RLVR setting where the reward is noisy and the action space includes a cost dimension such as length: open-ended moderation or long-form answer generation, not just binary verification.
  • A testable extension is to vary $L_{target}$ from small to large on one rubric and plot macro-F1; the paper's account predicts an inverted-U with a peak near the reasoning budget that balances exploration against exploitation.
  • Controlling per-persona label priors would show whether the 0.38 spread is stylistic or distributional; that experiment separates the blueprint's core premise from the persona synthesis protocol.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper studies whether explicit reasoning and RLVR post-training benefit LLM verifiers on subjective, rubric-driven tasks. The authors evaluate four binary verification tasks from a production recommender platform, comparing direct, implicit, and explicit reasoning prompts on proprietary and open-source models; report that reasoning is frequently unhelpful or degrading; identify a 'reasoning collapse' under GRPO in which generation length and accuracy fall together; propose a conditional length-reward (Eq. 2) that preserves reasoning length and recovers or exceeds baseline macro-F1 on two tasks; and present a persona-conditioning experiment over 1,500 personas showing a ~0.38 macro-F1 spread, which they interpret as 'reasoning-style mismatch' and use to motivate a persona-routing mid-training architecture. The paper uses held-out human labels, multiple models and RLVR variants, and five-run averages with reported run-to-run standard deviation below 0.01.

Significance. If the core empirical patterns are robust, the paper is a valuable, cautionary contribution to LLM-as-a-judge deployment and to RLVR transfer beyond objective tasks. The strengths are real: production rubrics, held-out human labels, several models and RLVR algorithms, five-run averages with low reported standard deviation, and a detailed experimental protocol that supports replication. The reward-shaping remedy is simple and falsifiable. However, the persona-based 'reasoning-style mismatch' claim—which appears in the abstract and motivates the proposed architecture—is not yet supported by the evidence, and the paper's own limitations section concedes that the persona results are preliminary. The degradation and collapse findings are more credible, but they need tighter statistical reporting and one internal inconsistency in Table 3 must be resolved. The broader interpretive claim about persona routing should be treated as a speculative blueprint unless the persona analysis is substantially strengthened.

major comments (4)
  1. [Section 5.1 and Figure 3] The claim that verification accuracy swings by nearly 0.38 macro-F1 'depending solely on the adopted reasoning persona' is not supported by the reported evidence. The prompt instructs each persona to produce labels consistent with the assigned worldview, so the manipulation can shift the Yes/No label prior, and macro-F1 is highly sensitive to the base rate; the statement that persona is 'the only varied factor' in Section 5.1 is therefore false unless the label distribution is held fixed. The 0.38 figure is a max-min range over 1,500 draws, which grows with sample size and is not a robust dispersion measure, and the paper provides no null model, no inter-prompt stability check, and no measurement of trace fidelity. Because the abstract's 'reasoning-style mismatch' claim and the persona-routing blueprint rest on this result, the section should either include controls for label priors, a random-relabeling null, and per-persona trace metrics, or be explicitly presented as a preliminary observation that does not support the strong causal interpretation.
  2. [Section 2 and Tables 1-2] The claim that explicit reasoning 'frequently' and 'often actively degrades' verification needs stronger statistical backing. Many of the reported differences are small: for example, Frontier-LLM moves from 0.901 to 0.887 macro-F1 on Query Sensitivity and from 0.622 to 0.619 on Response Sensitivity with explicit reasoning, while Qwen improves on two tasks and degrades on two. The paper reports that run-to-run standard deviation was below 0.01, but it does not give per-cell standard errors or significance tests, so the reader cannot tell which of these differences are real. Please add per-cell errors, confidence intervals, or effect sizes, and calibrate the language of 'active degradation' to the actual magnitudes.
  3. [Section 3.1 and Table 3] The text says that for Qwen 'under RLVR or its sequence-level variants its performance collapses' and that 'all three algorithms yield counterintuitive outcomes,' but Table 3 shows Qwen improving on Text: Response Quality from 0.512 to 0.558, 0.561, and 0.565 under GRPO, Dr. GRPO, and GSPO, respectively. The broad collapse claim is thus contradicted by the paper's own table. Please reconcile the narrative with the full pattern of results, for example by specifying that collapse occurs on three of the four tasks, or by providing an analysis of why one task improves.
  4. [Section 3.2 and Figure 2] The reasoning-collapse phenomenon is documented with a single training curve of mean terminated length, with no error bars or multiple seeds shown, and no co-plotted accuracy trajectory over training steps. The claim that collapse consists of a simultaneous drop in length and accuracy is not directly evidenced: the figure shows only length, and Table 3 reports final-time F1 for different model-task pairs rather than the step-by-step accuracy curve. Please provide per-seed length curves with error bars and the corresponding accuracy curves to support the 'reasoning collapse' mechanism.
minor comments (5)
  1. [Section 4.1, Eq. (1)] The target-constrained penalty λ min(0, L - L_target) is zero whenever L ≥ L_target, so it does not penalize exceeding the target length; the text's description that this penalty 'smoothly guides' the length toward L_target is inconsistent with the formula. Please correct the formulation or the description.
  2. [Appendix C] The persona synthesis protocol says personas are drawn from a public roleplay dataset, but reference [21] (FSPO) appears to be a method paper rather than a roleplay dataset; please cite the correct data source or describe how the personas were derived.
  3. [Table 4] The caption and text do not state whether the length values for Approach I (8,192 tokens) and the F1 values are averaged over the five runs or are from a single run; please report means and standard deviations for both F1 and length columns.
  4. [Section 4.1] Equation (2) is better described as a conditional length bonus for correct answers below the target than as a 'length-penalized' reward; the terminology in the title, abstract, and Section 4 should be aligned.
  5. [Appendix A] The selection of the best validation checkpoint after training is reported, but the paper does not specify whether the hyperparameters λ and L_target were selected on the same validation set; please state the model-selection procedure to avoid optimism.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: degradation, collapse, and recovery are measured against held-out human labels; no prediction reduces by construction to a fitted input or self-citation.

full rationale

The paper's central empirical chain—explicit reasoning degrading subjective verification, GRPO causing reasoning collapse, and the conditional length-penalized reward of Eq. (2) recovering macro-F1—is evaluated against held-out, human-labeled test sets with reported protocols. Eq. (2) is a deliberately designed training reward (r_base gated by correctness and length), not a parameter fitted to the test outcome; the reported F1 values come from separate held-out evaluation, so the recovery claim is not forced by construction. The reasoning-collapse phenomenon is characterized from observed training dynamics (Figure 2) rather than assumed as an input. The persona analysis (Section 5.1) is explicitly preliminary and is used to motivate, not to derive, the routing blueprint; while the claim that persona was 'the only varied factor' is an overstatement given uncontrolled label-prior shifts and impersonation variability, that is a statistical-support weakness, not a circular definition. There are no load-bearing self-citations: the references to prior LLM-reasoning and RLVR work are external, and no uniqueness theorem or prior result by the present authors is invoked to forbid alternatives. Therefore no step in the derivation reduces to its own input.

Assumptions & free parameters 2 free parameters · 5 assumptions · 3 invented entities

The empirical core is an observational study on proprietary data: the claims rest on the validity of human labels, the representativeness of four tasks, and the stability of API-model behavior. The proposed reward adds two hand-set parameters, and the persona section adds a strong interpretive assumption about synthesized traces.

free parameters (2)
  • lambda (length-bonus scaling) = 2e-4
    Scaling constant for the length bonus in Eq. (2); hand-set and not swept in the paper (Appendix A). It controls how strongly correct answers are preferred when they stay under the length cap.
  • L_target (target reasoning length) = 1000 tokens
    Cap on the conditional length bonus in Eq. (2); chosen by hand and central to the reward shaping. All post-training results in Table 4 depend on this budget and on the best-validation checkpoint rule.
assumptions (5)
  • domain assumption Human annotation labels are ground truth for both evaluation and RLVR reward.
    Appendix B: annotators use the production rubric and 'the resulting labels serve as ground truth for both evaluation and post-training'. Subjective labels with known inter-annotator disagreement are treated as a fixed binary reward.
  • domain assumption The four Netflix internal tasks are representative of subjective verification.
    Generalization from four tasks to 'subjective industry rubrics' (Section 2.1); no external benchmark, task diversity analysis, or replications on other platforms.
  • domain assumption Binary outcome reward is a valid training signal for these subjective tasks.
    Section 4.1 defines r_base in {0,1} from human labels as the RLVR reward, even though the paper itself argues subjective rubrics lack a canonical correct trajectory (Section 1).
  • ad hoc to paper Frontier-LLM impersonating 1,500 personas yields a representative distribution of reasoning styles.
    Section 5.1 and Appendix C synthesize traces by prompting Frontier-LLM with persona system prompts; there is no validation that the traces are distinct, faithful, or causally tied to verified human reasoning preferences.
  • domain assumption Subjective domains lack a canonical correct reasoning trajectory, and this explains the failures.
    Section 1 and Section 3 use this premise to explain why reasoning degrades verification and why the policy collapses, but it is not tested against alternatives such as noisy labels or prompt-format effects.
invented entities (3)
  • reasoning collapse
    purpose: Name for the observed RLVR dynamic in which mean terminated length plummets as the policy switches to heuristic guessing.
    Defined and measured within the paper's own training runs (Figure 2); no external falsifiable handle or independent benchmark.
  • persona-routing mid-training architecture (contextual multi-armed bandit over personas)
    purpose: Blueprint for RLVR to select which reasoning persona to apply per subjective query.
    Not implemented or evaluated; proposed in Section 5.2 as future work, so it has no testable prediction yet.
  • reasoning-style mismatch
    purpose: Interpretive explanation that much verification error results from applying analytical math-style reasoning to human-centric rubrics.
    Inferred from the persona max-min spread; no direct measurement of reasoning style versus error attribution.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LLM Reasoning for Subjective Tasks: Failure Modes, Mitigation, and Dynamic Reasoning Routing." pith.science (2026). https://pith.science/paper/FMCDIO2T

@misc{pith2026260808889,
  author       = {Pith},
  title        = {Pith review of: LLM Reasoning for Subjective Tasks: Failure Modes, Mitigation, and Dynamic Reasoning Routing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FMCDIO2T}},
  note         = {Machine review of arXiv:2608.08889}
}
read the original abstract

Recommendation systems thrive on personalization, where ''correctness'' is rarely a binary truth but a matter of subjective human preference. As Large Language Models (LLMs) are deployed as autonomous verifiers of safety and quality guidelines, they face a distinctive challenge: context-aware preference alignment. Recent gains in Reinforcement Learning with Verifiable Rewards (RLVR) are indexed mostly on objective, mathematical tasks. Through a large-scale study spanning both proprietary and open-source models on four real-world verification tasks from a production recommender platform, we ask whether explicit reasoning generalizes to subjective, human-centric industry rubrics. We expose a fundamental vulnerability: rigid, math-centric reasoning traces actively degrade verification, and applying standard RLVR triggers a phenomenon we term reasoning collapse, in which the policy abandons deliberation in favor of rapid heuristic guessing. We introduce a conditional length-penalized post-training algorithm that intertwines verification accuracy with bounded reasoning length, halting collapse and recovering performance. Finally, we show that a reasoning trace's efficacy is tightly coupled with its socio-linguistic framing: across 1500 synthesized personas, verification accuracy swings by nearly 0.38 macro-F1 depending solely on the adopted reasoning persona---evidence that much subjective-verification error is really reasoning-style mismatch. This observation motivates a mid-training architecture that routes reasoning through contextually aligned personas. This work offers both a scalable algorithmic patch and a long-term architectural blueprint for aligning reasoning models with real-world subjective constraints.

Figures

Figures reproduced from arXiv: 2608.08889 by the authors.

Figure 1
Figure 1. (Left) RLVR enhances reasoning when pre-training contains relevant reasoning traces and verifiers are reliable. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Reasoning collapse during GRPO post-training: the [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Reasoning patterns of different personas lead to [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Persona-conditioned macro-F1 distribution on the [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 14 canonical work pages

  1. [1]

    Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. 2022. Constitutional AI: Harmlessness from AI Feedback.arXiv preprint arXiv:2212.08073(2022)

    arXiv 2022

  2. [2]

    Keqin Bao, Jizhi Zhang, Yang Zhang, Wenjie Wang, Fuli Feng, and Xiangnan He. 2023. TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation.ACM Conference on Recommender Systems(2023)

    work page 2023

  3. [3]

    Stephen Casper, Xander Davies, Claudia Shi, Thomas Krendl Gilbert, Jérémy Scheurer, Javier Rando, Rachel Freedman, Tomasz Korbak, et al . 2023. Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback.Transactions on Machine Learning Research(2023)

    work page 2023

  4. [4]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training Verifiers to Solve Math Word Problems.arXiv preprint arXiv:2110.14168(2021)

    arXiv 2021

  5. [5]

    DeepSeek-AI. 2025. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning.arXiv preprint arXiv:2501.12948(2025)

    arXiv 2025

  6. [6]

    Hashimoto

    Yann Dubois, Balázs Galambosi, Percy Liang, and Tatsunori B. Hashimoto. 2024. Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators. arXiv preprint arXiv:2404.04475(2024)

    arXiv 2024

  7. [7]

    Leo Gao, John Schulman, and Jacob Hilton. 2023. Scaling Laws for Reward Model Overoptimization.International Conference on Machine Learning(2023)

    work page 2023

  8. [8]

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring Mathematical Problem Solving With the MATH Dataset.NeurIPS Datasets and Benchmarks(2021)

    work page 2021

Show all 30 references
  1. [9]

    Yupeng Hou, Junjie Zhang, Zihan Lin, Hongyu Lu, Ruobing Xie, Julian McAuley, and Wayne Xin Zhao. 2023. Large Language Models are Zero-Shot Rankers for Recommender Systems.arXiv preprint arXiv:2305.08845(2023)

    2023 arXiv

  2. [10]

    Wang-Cheng Kang, Jianmo Ni, Nikhil Mehta, Maheswaran Sathiamoorthy, Lichan Hong, Ed Chi, and Derek Zhiyuan Cheng. 2023. Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Prediction.arXiv preprint arXiv:2305.06474(2023)

    2023 arXiv

  3. [11]

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large Language Models are Zero-Shot Reasoners.Advances in Neural Information Processing Systems(2022)

    2022

  4. [12]

    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let’s Verify Step by Step.arXiv preprint arXiv:2305.20050(2023)

    2023 arXiv

  5. [13]

    Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. 2025. Understanding R1-Zero-Like Training: A Critical Perspective. arXiv:2503.20783 [cs.LG] https://arxiv.org/abs/2503.20783

    2025 arXiv

  6. [14]

    OpenAI. 2024. OpenAI o1 System Card.Technical Report(2024)

    2024

  7. [15]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback.Advances in Neural Information Processing Systems(2022)

    2022

  8. [16]

    Joon Sung Park, Joseph C O’Brien, Carrie J Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative Agents: Interactive Simulacra of Human Behavior.ACM Symposium on User Interface Software and Technology (2023)

    2023

  9. [17]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D Manning, and Chelsea Finn. 2023. Direct Preference Optimization: Your Language Model is Secretly a Reward Model.Advances in Neural Information Processing Systems(2023)

    2023

  10. [18]

    Leonard Salewski, Stephan Alaniz, Isabel Rio-Torto, Eric Schulz, and Zeynep Akata. 2023. In-Context Impersonation Reveals Large Language Models’ Strengths and Biases.arXiv preprint arXiv:2305.14930(2023)

    2023 arXiv

  11. [19]

    Murray Shanahan, Kyle McDonell, and Laria Reynolds. 2023. Role Play with Large Language Models.Nature(2023)

    2023

  12. [20]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models.arXiv preprint arXiv:2402.03300(2024)

    2024 arXiv

  13. [21]

    Anikait Singh, Sheryl Hsu, Kyle Hsu, Eric Mitchell, Stefano Ermon, Tatsunori Hashimoto, Archit Sharma, and Chelsea Finn. 2025. FSPO: Few-Shot Optimization of Synthetic Preferences Personalizes to Real Users. arXiv:2502.19312 [cs.LG] https://arxiv.org/abs/2502.19312

    2025 arXiv

  14. [22]

    Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. 2022. Solving Math Word Problems With Process- and Outcome-Based Feedback.arXiv preprint arXiv:2211.14275(2022)

    2022 arXiv

  15. [23]

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-Consistency Improves Chain of Thought Reasoning in Language Models.International Conference on Learning Representations(2023)

    2023

  16. [24]

    Yufei Wang, Wanjun Zhong, Liangyou Li, Fei Mi, Xingshan Zeng, Wenyong Huang, Lifeng Shang, Xin Jiang, and Qun Liu. 2023. Aligning Large Language Models with Human: A Survey.arXiv preprint arXiv:2307.12966(2023)

    2023 arXiv

  17. [25]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models.Advances in Neural Information Processing Systems(2022)

    2022

  18. [26]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of Thoughts: Deliberate Problem Solving with Large Language Models.Advances in Neural Information Processing Systems (2023)

    2023

  19. [27]

    Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, Jingren Zhou, and Junyang Lin

  20. [28]

    Xing, et al

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, et al . 2023. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena.Advances in Neural Information Processing Systems(2023)

    2023

  21. [29]

    low” and “high

    Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. 2019. Fine-Tuning Language Models from Human Preferences.arXiv preprint arXiv:1909.08593(2019). A Detailed Experimental Setup Models.Proprietary models ...

    2019 arXiv

  22. [2025]

    arXiv:2507.18071 [cs.LG] https: //arxiv.org/abs/2507.18071

    Group Sequence Policy Optimization. arXiv:2507.18071 [cs.LG] https: //arxiv.org/abs/2507.18071

    arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.