Pith. sign in

REVIEW 4 major objections 6 minor 54 references

Exchange of Perspective Prompting Enhances Reasoning in Large Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that answering a question in two phrasings and swapping the answers between branches reliably improves LLM math reasoning, with GPT-4 on MATH jumping from 53.9% to 61.6%.

desk verdict Simple two-branch exchange prompting shows real gains on math benchmarks, but the main comparison with PHP is compute-confounded and needs a matched baseline. read the letter →

arxiv 2506.03573 v1 pith:GHTQSZF5 submitted 2025-06-04 cs.CL

Lin Sun , Can Zhang This is my paper
classification cs.CL
keywords exchangeofperspectivepromptingchain-of-thoughtself-correctionmathematicalreasoninglargelanguagemodelsprogressivehintquestionredefinition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that asking a language model to solve a question twice—once as originally worded and once reworded—and repeatedly feeding each branch the other's answer as a hint improves reasoning accuracy beyond either branch alone. On the AQuA benchmark, GPT-3.5-Turbo rises from 60.6% to 64.2% over the progressive-hint baseline, and GPT-4 on MATH rises from 53.9% to 61.6%. The authors argue the gain comes from external perspective that breaks a fixed mindset, not from rephrasing itself, since the reworded branch alone performs slightly worse than the original. If correct, the method offers a prompt-only route to better reasoning without fine-tuning or code execution.

What carries the argument

The machinery is the two-branch exchange loop. A redefinition function $f$ rewrites the original question $q_{\text{org}}$ into an augmented question $q_{\text{aug}}$, either by premise extraction and concatenation (PEC) or by direct question rephrasing (QR). In each iteration $j>1$, the model generates rationales and answers conditioned on the other branch's prior answers, so the history $A_{\text{aug}}^{(j-1)}$ informs the original branch and $A_{\text{org}}^{(j-1)}$ informs the augmented branch. The iteration stops when answers agree across branches or repeat within a branch. This structure converts a single-model self-correction loop into a two-model dialogue between differently phrased versions of the same problem.

What would settle it

Run EoP against a control that makes the same number of model calls but never exchanges answers—for example, two independent branches solved with plain progressive-hint prompting without cross-branch hints—and compare accuracies. If the control matches or beats EoP, the exchange of perspective is not the cause of the gain. A second check is to feed deliberately wrong hints into one branch and see whether the other branch is pulled off the correct answer; if it is, the mechanism is suggestion-following rather than error correction.

Watch

Extended reading notes

Core claim

The central discovery is that exchanging answers between an original question and a redefined version, using the prompt 'Hint: The answer is near to', yields consistent accuracy gains over progressive-hint prompting across eight math benchmarks and four models. The exchange is formalized as two sampling processes: the original branch and augmented branch generate initial answers, then each branch's next answer is conditioned on the other branch's prior answer set. Termination occurs on consensus across branches or stability within a branch. The ablation shows the augmented branch alone underperforms the original branch, while the combined exchange outperforms both, which the authors attribute to error correction and complementary information.

Load-bearing premise

The method assumes the hint pulled from the other branch is informative enough to nudge the model toward the correct answer, and that any measured gain over progressive-hint prompting comes from the exchange itself rather than from the roughly doubled number of model calls.

Editorial extensions

If this is right

  • EoP's gains concentrate on hard problems, with a 9.5% advantage over PHP at Level 5 on MATH.
  • The method works across model families and sizes, from 7B open models to GPT-4.
  • EoP beats code-based tools like PAL and ToRA on MATH without generating or executing code.
  • Because termination is based on consensus or stability, EoP adaptively spends more iterations on harder questions.
  • PEC redefinition outperforms QR, suggesting clarifying premises matters more than paraphrasing.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The doubled-call confound is testable: a control with two independent PHP runs would isolate whether cross-branch hints add signal beyond compute.
  • The framework could extend beyond definitions of the same question to different modalities, languages, or sampling temperatures, preserving the same exchange loop.
  • If consensus-based termination reliably predicts correctness, the consensus signal could be used as a cheap confidence measure for selective answer submission.
  • The hint phrase 'The answer is near to' can be harmful in Figure 2's example, transiently pulling the model to $109,000; a more calibrated hint mechanism might remove that failure mode.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Exchange-of-Perspective (EoP), a prompting framework in which an original math word problem is first rewritten into an augmented version (via Premise Extraction and Concatenation, PEC, or Question Rephrasing, QR) and then two branches — one using the original question and one using the augmented question — iteratively exchange their answers to each other as hints. The method is formalized in Section 2.1 (Eqs. 1–4), with termination conditions in Section 2.3. Experiments on eight arithmetic/math benchmarks with GPT-3.5-Turbo, GPT-4, and Qwen-2.5 models report consistent accuracy improvements over chain-of-thought, PHP, and other baselines, with the largest gains on MATH (GPT-4: 53.9% to 61.6%) and OlympiadBench (Qwen-2.5-72b: 43.5% to 47.0%). The paper also includes an ablation comparing individual branches to the combined exchange, a comparison of redefinition methods, and a difficulty-level analysis.

Significance. If the reported gains are real and specifically caused by the exchange of perspectives, EoP would be a simple, training-free way to boost LLM reasoning on mathematical problems, with a clean conceptual story about escaping a fixed formulation of a question. The method is easy to describe, the experiments cover a reasonable range of models and datasets, and the ablation in Table 4 at least shows that the augmented branch alone is not the source of the gain. However, the empirical evidence as presented does not yet isolate the claimed mechanism from two major confounds: EoP uses roughly twice as many LLM calls per iteration as PHP, and the redefinition strategy (PEC) is selected using the same test benchmarks that produce the headline numbers. Without a compute-matched control and a protected selection procedure, the central claim that 'the exchange of perspectives' is responsible for the improvements is not established. The paper therefore has a plausible and interesting core idea, but the current evaluation is insufficient to support the strength of the conclusion.

major comments (4)
  1. [Section 3.2, Table 3] The comparison with PHP is not compute-matched: EoP makes two LLM calls per interaction (one for each branch, per Eqs. 3 and 4), whereas PHP makes one. Table 3 reports N as the 'average interaction number' without converting to total calls; for Qwen2.5-72b on OlympiadBench, EoP has N=4.2 versus PHP's N=2.4, which corresponds to roughly 8.4 calls versus 2.4 calls per question. The +3.5 point gain could therefore be explained by the additional sampling budget rather than by the exchange of hints. A control that matches total LLM calls — for example, PHP run for more interactions, or self-consistency with a comparable number of samples — is needed to support the claim that the exchange itself causes the improvement. The ablation in Table 4 does not fix this, since the 'combined' condition uses twice the calls of either individual branch.
  2. [Section 3.3, Table 5] The redefinition strategy PEC is selected based on its performance on the same benchmarks that are used for the headline results. Table 5 compares PEC and QR on all six arithmetic datasets, and the paper then reports the main experiments using 'PEC redefinition' throughout, with no separate validation set or multiple-testing correction. Because the choice of PEC is data-dependent in this way, the reported accuracies are optimistically biased; the 64.2% on AQuA and the 85.3% average are conditional on having picked the better of two methods on the test set. The authors should either pre-register PEC, evaluate both variants as equally valid configurations, or use a validation split for the selection step.
  3. [Section 3.2, Tables 1–3] No measure of statistical uncertainty is reported for any accuracy figure. Many of the headline differences are small (e.g., AQuA 60.6% to 64.2%; GSM8K on GPT-3.5-Turbo is actually lower for EoP than PHP, 84.2% vs 85.1%), and datasets such as AQuA have only a few hundred test items. Without standard errors, confidence intervals, or repeated runs, the claim that EoP 'significantly improves' performance is not supported. The paper should report the number of test instances per dataset and provide error bars or at least a bootstrap estimate.
  4. [Section 3.3, Table 4] The ablation does not isolate the exchange mechanism. The 'Org' and 'Aug' rows are single-branch conditions, while the combined row uses two branches that exchange answers; this conflates the effect of the exchange with the effect of simply having a second chance and combining information. A more direct control would run two branches independently (no swapping of answers) with the same total number of LLM calls and then aggregate or select the final answer. Without such a control, the claim that 'the performance gain of EoP is not from rephrasing the question' but from error correction and complementary information (Section 3.3) is not demonstrated by the data shown.
minor comments (6)
  1. [Abstract and Section 1] The phrase 'non-commutative baseline PHP' appears to be a typo; PHP is 'Progressive-Hint Prompting' and 'non-commutative' does not seem to be a standard descriptor in this context. Please correct or clarify.
  2. [Section 2.2 and throughout] 'Redefination' is a recurring misspelling; it should be 'Redefinition' (e.g., Section 2.2 heading, Section 3.3 text 'PEC redefination method').
  3. [Table 5 caption and footer] The table contains unicode escape sequences such as '/uni0000002f/uni00000048/...' that appear to be a formatting artifact; these should be cleaned up so the table is readable.
  4. [Section 2.1, Eqs. (3)–(4)] The formal conditioning on the sets A_aug^(j-1) and A_org^(j-1) is not reflected in the prompt example in Figure 2, which shows only the latest or most recent answers being appended. Please specify explicitly whether all previous answers are concatenated in the hint or only a summary, and give the exact prompt template used.
  5. [Figure 2] The hint phrase 'Hint: The answer is near to' is used with multiple comma-separated numbers (e.g., '$109,000, $114,200'), which is not a precise 'near to' statement. Clarify how the hint string is constructed from the exchanged answers and whether the model ever receives contradictory hints.
  6. [Section 4, Related Work] Reference [10] (Exchange-of-Thought) is cited in the introduction as evidence that LLMs 'struggle to improve their responses without external feedback,' but that paper is about cross-model communication rather than about the inability to self-correct. Please ensure the citation supports the specific claim or rephrase the sentence.

Circularity Check

0 steps flagged · score 1.0 of 10

No derivation-level circularity: EoP is an empirical prompting study evaluated on external benchmarks; the PEC-vs-QR selection on the same test sets and the missing compute-matched control are validity caveats, not circular steps.

full rationale

EoP's claimed chain is empirical, not derivational. Equations (1)-(5) only describe the prompting procedure (initial sampling, cross-branch conditioning, and the PEC concatenation q_aug = [p1,...,pn] xor q_core); nothing in the paper derives an accuracy bound or a predicted improvement from these definitions. The headline results (AQuA 60.6->64.2, MATH 53.9->61.6, OlympiadBench 43.5->47.0) are measured against external benchmarks with external baselines, and the hint phrase 'Hint: The answer is near to' is explicitly attributed to PHP [5], an external prior work, not to a self-citation. Table 4 supplies independent empirical content: each branch alone (CoT: 83.3 and 82.1; Complex CoT: 83.1 and 81.7) performs near or below the PHP-like single-branch level, while the combined framework (84.9 and 85.3) exceeds both, so the gain is not manufactured by the individual branches. Two caveats exist, but both are correctness risks rather than circularity under the stated hard rules. First, PEC was adopted as the default redefinition method after measuring it against QR on the same Arithmetic test sets (Table 5); on AQuA with Complex CoT, the PEC-vs-QR gap is +7.1, matching the headline 64.2, so the main configuration was selected on the same data used to claim the gain. Second, each EoP iteration makes two LLM calls (Eqs. 3 and 4) versus PHP's one call, and Table 3 reports the interaction number N without a call-budget-matched control, so the +7.7 MATH gap may partly reflect additional compute rather than perspective exchange. Neither of these reduces a prediction to its inputs by construction, so they are not scored as circular.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The method introduces no new parameters that are fit to data in the classical sense, but three design choices (redefinition strategy, hint phrase, termination rule) were selected on the evaluation benchmarks and are load-bearing for the claimed gain. The paper assumes redefinition preserves correctness and that cross-branch hints are informative; both assumptions are untested and partially contradicted by the authors' own ablation and example.

free parameters (4)
  • Redefinition strategy = PEC (Premise Extraction and Concatenation)
    The paper compares PEC and QR on the same test sets (Table 5) and selects PEC because it yields higher average accuracy. This is a model choice fitted to the evaluation data.
  • Hint phrase = "Hint: The answer is near to"
    Borrowed unchanged from PHP; the exchange mechanism's only channel for cross-branch information is this phrase, so the result depends on it without independent tuning or validation.
  • Termination conditions = Consensus across branches or stability within a branch
    The stopping rule determines the number of LLM calls (reported as N). It is designed ad hoc and no comparison is made against fixed-iteration or budget-matched baselines.
  • Temperature = T=0 for EoP and most baselines; T=0.8 for self-consistency
    The choice of greedy decoding is reasonable, but self-consistency is run at higher temperature with more samples, making a fair comparison on cost unclear.
assumptions (3)
  • domain assumption LLMs are hindered by their fixed internal perspective and benefit from external feedback.
    Motivates the method (Section 1, citing refs 8, 9, 27). The paper does not test this assumption directly; it is the premise for why answer exchange should help.
  • domain assumption Redefining a question while preserving semantics does not change the final answer.
    Stated in Section 2.2 as a principle, but never verified. The ablation shows the augmented branch alone often performs worse than the original branch, indicating redefinitions sometimes introduce inconsistency.
  • domain assumption The model's own answers are reliable enough to serve as hints for the other branch.
    The hints are previous answers, which may be wrong. Figure 2 shows the model switching to a wrong answer after receiving a hint. The method's success depends on the model treating hints as noisy evidence rather than authoritative.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Exchange of Perspective Prompting Enhances Reasoning in Large Language Models." pith.science (2026). https://pith.science/paper/GHTQSZF5

@misc{pith2026250603573,
  author       = {Pith},
  title        = {Pith review of: Exchange of Perspective Prompting Enhances Reasoning in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GHTQSZF5}},
  note         = {Machine review of arXiv:2506.03573}
}
read the original abstract

Large language models (LLMs) have made significant advancements in addressing diverse natural language processing (NLP) tasks. However, their performance is often limited by inherent comprehension of problems. To address this limitation, we propose Exchange-of-Perspective (EoP), a novel framework designed to exchange perspectives across different definitions of problem, so that it can break the fixed mindset from any particular formulation of the question. We conducted extensive and comprehensive experiments on 8 benchmarks. The results show that EoP can significantly improve performance. For instance, compared to the non-commutative baseline PHP, with GPT-3.5-Turbo and EoP, we observe a 3.6% improvement on AQuA (60.6% to 64.2%), while GPT-4-powered EoP demonstrates a 7.7% overall accuracy enhancement on Math (53.9% to 61.6%) and a 3.5% improvement on OlympiadBench Maths (43.5% to 47.0%) when using Qwen-2.5-72b.

Figures

Figures reproduced from arXiv: 2506.03573 by the authors.

Figure 1
Figure 1. Comparison of CoT, PHP, and EoP. Both CoT and PHP rely on the model’s in￾ternal perspective to generate or refine out￾put, lacking external insights. LLMs have made significant progress in the field of NLP, but they often struggle to provide stable and accu￾rate answers when faced with highly complex tasks. This issue cannot be resolved by simply scaling up the model size [1, 2]. To address this limitation, chain￾of… view at source ↗
Figure 2
Figure 2. Our proposed EoP integrates the current question with answers from the alter￾native branch to facilitate perspective exchange. It consists of four stages: (1) We rede￾fine the given question into an augmented version, subsequently forming two branches: original branch and augmented branch. (2) Instruct the LLM to produce initial answers for both the original and augmented branches by providing it with a combination … view at source ↗
Figure 3
Figure 3. Performance comparison between CoT, PHP and EoP on math dataset with varying difficulty levels. It shows that EoP achieves the best performance across all difficulty levels. The enhancement in performance becomes more evident when tackling more challenging problems. The results are based on GPT-4 with greedy decoding and PEC redefination. demonstrations. For instance, PromptPG [11] employs policy gradients to choose… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

54 extracted references · 32 canonical work pages

  1. [1]

    W., Borgeaud, S., Cai, T., Millican, K., Hoffmann, J., Song, F., Aslanides, J., Henderson, S., Ring, R., Young, S., et al

    Rae, J. W., Borgeaud, S., Cai, T., Millican, K., Hoffmann, J., Song, F., Aslanides, J., Henderson, S., Ring, R., Young, S., et al. Scaling language models: Methods, analysis & insights from training gopher.arXiv preprint arXiv:2112.11446, 2021. Title Suppressed Due to Excessive Length 11

    arXiv 2021

  2. [2]

    V., Chi, E

    Wang, X., Wei, J., Schuurmans, D., Le, Q. V., Chi, E. H., Narang, S., Chowdhery, A., Zhou, D. Self-consistency improves chain of thought reasoning in language models. InThe Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1–5, 2023. OpenReview.net, 2023

    work page 2023

  3. [3]

    Let’s verify step by step

    Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., Cobbe, K. Let’s verify step by step. InThe Twelfth In- ternational Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7–11, 2024. OpenReview.net, 2024

    work page 2024

  4. [4]

    H., Le, Q

    Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E. H., Le, Q. V., Zhou, D. Chain of thought prompting elicits reasoning in large language models. InAdvances in Neural Information Processing Systems, ed. Alice H. Oh, Alekh Agarwal, Danielle Belgrave, Kyunghyun Cho. 2022

    work page 2022

  5. [5]

    Progressive-hint prompting improves reasoning in large language models.arXiv preprint arXiv:2304.09797, 2023

    Zheng, C., Liu, Z., Xie, E., Li, Z., Li, Y. Progressive-hint prompting improves reasoning in large language models.arXiv preprint arXiv:2304.09797, 2023

    arXiv 2023

  6. [6]

    Generating sequences by learning to self-correct

    Welleck, S., Lu, X., West, P., Brahman, F., Shen, T., Khashabi, D., Choi, Y. Generating sequences by learning to self-correct. InThe Eleventh International Conference on Learning Representations, ICLR 2023. 2023

    work page 2023

  7. [7]

    The capacity for moral self- correction in large language models.arXiv preprint arXiv:2302.07459, 2023

    Ganguli, D., Askell, A., Schiefer, N., Liao, T., Lukoši¯ ut˙ e, K., Chen, A., Goldie, A., Mirhoseini, A., Olsson, C., Hernandez, D., et al. The capacity for moral self- correction in large language models.arXiv preprint arXiv:2302.07459, 2023

    arXiv 2023

  8. [8]

    Can large language models really improve by self-critiquing their own plans?arXiv preprint arXiv:2310.08118, 2023

    Valmeekam, K., Marquez, M., Kambhampati, S. Can large language models really improve by self-critiquing their own plans?arXiv preprint arXiv:2310.08118, 2023

    arXiv 2023

Show all 54 references
  1. [9]

    GPT-4 doesn’t know it’s wrong: An analysis of iterative prompting for reasoning problems.arXiv preprint arXiv:2310.12397, 2023

    Stechly, K., Marquez, M., Kambhampati, S. GPT-4 doesn’t know it’s wrong: An analysis of iterative prompting for reasoning problems.arXiv preprint arXiv:2310.12397, 2023

    2023 arXiv

  2. [10]

    Exchange-of- thought: Enhancing large language model capabilities through cross-model commu- nication

    Yin, Z., Sun, Q., Chang, C., Guo, Q., Dai, J., Huang, X., Qiu, X. Exchange-of- thought: Enhancing large language model capabilities through cross-model commu- nication. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singap...

    2023

  3. [11]

    N., Zhu, S.-C., Rajpurohit, T., Clark, P., Kalyan, A

    Lu, P., Qiu, L., Chang, K.-W., Wu, Y. N., Zhu, S.-C., Rajpurohit, T., Clark, P., Kalyan, A. Dynamic prompt learning via policy gradient for semi-structured mathematical reasoning. InThe Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, M...

    2023

  4. [12]

    Learning to retrieve in-context examples for large language models

    Wang, L., Yang, N., Wei, F. Learning to retrieve in-context examples for large language models. InProceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2024 - Volume 1: Long Papers, St. Julian’s, Malta, March 17–22, 2...

    2024

  5. [13]

    V., Chi, E

    Zhou, D., Schärli, N., Hou, L., Wei, J., Scales, N., Wang, X., Schuurmans, D., Cui, C., Bousquet, O., Le, Q. V., Chi, E. H. Least-to-most prompting enables complex reasoning in large language models. InICLR, 2023

    2023

  6. [14]

    Decomposed prompting: A modular approach for solving complex tasks

    Khot, T., Trivedi, H., Finlayson, M., Fu, Y., Richardson, K., Clark, P., Sabharwal, A. Decomposed prompting: A modular approach for solving complex tasks. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1–5, 2023. OpenReview...

    2023

  7. [15]

    Chen, W., Ma, X., Wang, X., Cohen, W. W. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks.Trans. Mach. Learn. Res., vol. 2023, 2023. 12 Lin Sun

    2023

  8. [16]

    Pal: Program-aided language models

    Gao, L., Madaan, A., Zhou, S., Alon, U., Liu, P., Yang, Y., Callan, J., Neubig, G. Pal: Program-aided language models. InInternational Conference on Machine Learning, pp. 10764–10799. PMLR, 2023

    2023

  9. [17]

    I., Andreassen, A

    Nye, M. I., Andreassen, A. J., Gur-Ari, G., Michalewski, H., Austin, J., Bieber, D., Dohan, D., Lewkowycz, A., Bosma, M., Luan, D., Sutton, C., Odena, A. Show your work: Scratchpads for intermediate computation with language models.arXiv preprint arXiv:2112.00114, 2021

    2021 arXiv

  10. [18]

    M., Roller, S., Ung, M., Chen, M., Arora, K., Lane, J., Behrooz, M., Ngan, W., Poff, S., Goyal, N., Szlam, A., Boureau, Y.-L., Kambadur, M., Weston, J

    Shuster, K., Xu, J., Komeili, M., Ju, D., Smith, E. M., Roller, S., Ung, M., Chen, M., Arora, K., Lane, J., Behrooz, M., Ngan, W., Poff, S., Goyal, N., Szlam, A., Boureau, Y.-L., Kambadur, M., Weston, J. BlenderBot 3: a deployed con- versational agent that continually learns t...

    2022 arXiv

  11. [19]

    Toolformer: Language models can teach themselves to use tools

    Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Hambro, E., Zettle- moyer, L., Cancedda, N., Scialom, T. Toolformer: Language models can teach themselves to use tools. InAdvances in Neural Information Processing Systems 36: Annual Conference on Neural Informat...

    2023

  12. [20]

    Constitutional AI: Harmlessness from AI feedback.arXiv preprint arXiv:2212.08073, 2022

    Bai, Y., Kadavath, S., Kundu, S., et al. Constitutional AI: Harmlessness from AI feedback.arXiv preprint arXiv:2212.08073, 2022

    2022 arXiv

  13. [21]

    Faithful chain-of-thought reasoning

    Lyu, Q., Havaldar, S., Stein, A., Zhang, L., Rao, D., Wong, E., Apidianaki, M., Callison-Burch, C. Faithful chain-of-thought reasoning. InProceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Con- ference of the Asia-Pacific Chapter ...

    2023

  14. [22]

    M., Uribe, J

    McAleese, N., Pokorny, R. M., Uribe, J. F. C., Nitishinskaya, E., Trebacz, M., Leike, J. LLM critics help catch LLM bugs.arXiv preprint arXiv:2407.00215, 2024

    2024 arXiv

  15. [23]

    Learning from mistakes makes LLM better reasoner.arXiv preprint arXiv:2310.20689, 2023

    An, S., Ma, Z., Lin, Z., Zheng, N., Lou, J.-G., Chen, W. Learning from mistakes makes LLM better reasoner.arXiv preprint arXiv:2310.20689, 2023

    2023 arXiv

  16. [24]

    D., Singh, A., Baumli, K., Iqbal, S., Bishop, C., Roelofs, R., Zhang, L

    Kumar, A., Zhuang, V., Agarwal, R., Su, Y., Co-Reyes, J. D., Singh, A., Baumli, K., Iqbal, S., Bishop, C., Roelofs, R., Zhang, L. M., McKinney, K., Shrivastava, D., Paduraru, C., Tucker, G., Precup, D., Behbahani, F., Faust, A. Training language models to self-correct via rein...

    2024 arXiv

  17. [25]

    Language models can solve computer tasks

    Kim, G., Baldi, P., McAleer, S. Language models can solve computer tasks. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10–16, 2023, 2023

    2023

  18. [26]

    P., Hermann, K., Welleck, S., Yazdanbakhsh, A., Clark, P

    Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., Gupta, S., Majumder, B. P., Hermann, K., Welleck, S., Yazdanbakhsh, A., Clark, P. Self-refine: Iterative refinement with self-feedback. InAdvances in Neural ...

    2023

  19. [27]

    S., Yu, A

    Huang, J., Chen, X., Mishra, S., Zheng, H. S., Yu, A. W., Song, X., Zhou, D. Large language models cannot self-correct reasoning yet. InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7–11,

    2024

  20. [28]

    Recursiveintrospection:Teachinglanguage model agents how to self-improve.arXiv preprint arXiv:2407.18219, 2024

    Qu,Y.,Zhang,T.,Garg,N.,Kumar,A. Recursiveintrospection:Teachinglanguage model agents how to self-improve.arXiv preprint arXiv:2407.18219, 2024. Title Suppressed Due to Excessive Length 13

    2024 arXiv

  21. [29]

    Shepherd: A critic for language model generation.arXiv preprint arXiv:2308.04592, 2023

    Wang,T.,Yu,P.,Tan,X.E.,O’Brien,S.,Pasunuru,R.,Dwivedi-Yu,J.,Golovneva, O., Zettlemoyer, L., Fazel-Zarandi, M., Celikyilmaz, A. Shepherd: A critic for language model generation.arXiv preprint arXiv:2308.04592, 2023

    2023 arXiv

  22. [30]

    Refiner: Reasoning feedback on intermediate representations

    Paul,D.,Ismayilzada,M.,Peyrard,M.,Borges,B.,Bosselut,A.,West,R.,Faltings, B. Refiner: Reasoning feedback on intermediate representations. InProceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2024 - Volume 1: Long P...

    2024

  23. [31]

    Critic: Large language models can self-correct with tool-interactive critiquing

    Gou, Z., Shao, Z., Gong, Y., Shen, Y., Yang, Y., Duan, N., Chen, W. Critic: Large language models can self-correct with tool-interactive critiquing. InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Aus- tria, May 7–11, 2024. OpenReview.net, 2024

    2024

  24. [32]

    Teaching large language models to self- debug

    Chen, X., Lin, M., Schärli, N., Zhou, D. Teaching large language models to self- debug. InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7–11, 2024. OpenReview.net, 2024

    2024

  25. [33]

    T., Fan, Y., Zhao, V

    Gao, L., Dai, Z., Pasupat, P., Chen, A., Chaganty, A. T., Fan, Y., Zhao, V. Y., Lao, N., Lee, H., Juan, D.-C., Guu, K. Rarr: Researching and revising what language models say, using language models. InProceedings of the 61st Annual Meeting of the Association for Computational ...

    2023

  26. [34]

    J., Hajishirzi, H., Etzioni, O., Kushman, N

    Hosseini, M. J., Hajishirzi, H., Etzioni, O., Kushman, N. Learning to solve arith- metic word problems with verb categorization. InEMNLP, pp. 523–533, 2014

    2014

  27. [35]

    Solving general arithmetic word problems

    Roy, S., Roth, D. Solving general arithmetic word problems. InProceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1743–1752. Association for Computational Linguistics, Lisbon, Portugal, 2015

    2015

  28. [36]

    Koncel-Kedziorski, R., Hajishirzi, H., Sabharwal, A., Etzioni, O., Ang, S. D. Pars- ing algebraic word problems into equations.Transactions of the Association for Computational Linguistics, vol. 3, pp. 585–597. MIT Press, 2015

    2015

  29. [37]

    Patel, A., Bhattamishra, S., Goyal, N. Are NLP models really able to solve simple math word problems? InProceedings of the 2021 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2080–2094, 2021

    2021

  30. [38]

    Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., Schulman, J. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    arXiv

  31. [39]

    Program induction by rationale generation: Learning to solve and explain algebraic word problems

    Ling, W., Yogatama, D., Dyer, C., Blunsom, P. Program induction by rationale generation: Learning to solve and explain algebraic word problems. InProceed- ings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 158–167, 2017

    2017

  32. [40]

    Measuring mathematical problem solving with the MATH dataset

    Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D., Steinhardt, J. Measuring mathematical problem solving with the MATH dataset. InProceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, NeurIPS Datasets and B...

    2021

  33. [41]

    Complexity-based prompting for multi-step reasoning

    Fu, Y., Peng, H., Sabharwal, A., Clark, P., Khot, T. Complexity-based prompting for multi-step reasoning. InThe Eleventh International Conference on Learning Representations, 2023

    2023

  34. [42]

    Training language models to follow 14 Lin Sun instructions with human feedback.Advances in Neural Information Processing Systems, vol

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. Training language models to follow 14 Lin Sun instructions with human feedback.Advances in Neural Information Processing Systems, vol. 35, pp. 27730–27744, 2022

    2022

  35. [43]

    GPT-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    OpenAI. GPT-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  36. [44]

    S., Reid, M., Matsuo, Y., Iwasawa, Y

    Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., Iwasawa, Y. Large language models are zero-shot reasoners. InNeurIPS, 2022

    2022

  37. [45]

    K.-W., Lim, E.-P

    Wang, L., Xu, W., Lan, Y., Hu, Z., Lan, Y., Lee, R. K.-W., Lim, E.-P. Plan-and- solve prompting: Improving zero-shot chain-of-thought reasoning by large language models. InACL, July 2023

    2023

  38. [46]

    Re-reading improves reasoning in large language models.arXiv preprint arXiv:2309.06275, 2023

    Xu, X., Tao, C., Shen, T., Xu, C., Xu, H., Long, G., Lou, J.-G. Re-reading improves reasoning in large language models.arXiv preprint arXiv:2309.06275, 2023

    2023 arXiv

  39. [47]

    Zero-shot chain-of-thought reasoning guided by evolu- tionary algorithms in large language models.arXiv preprint arXiv:2402.05376, 2024

    Jin, F., Liu, Y., Tan, Y. Zero-shot chain-of-thought reasoning guided by evolu- tionary algorithms in large language models.arXiv preprint arXiv:2402.05376, 2024

    2024 arXiv

  40. [48]

    Large language models are contrastive reasoners.arXiv preprint arXiv:2403.08211, 2024

    Yao, L. Large language models are contrastive reasoners.arXiv preprint arXiv:2403.08211, 2024

    2024

  41. [49]

    Zhang, Y., Yang, J., Yuan, Y., Yao, A. C.-C. Cumulative reasoning with large language models.arXiv preprint arXiv:2308.04371, 2023

    2023 arXiv

  42. [50]

    ToRA: A tool-integrated reasoning agent for mathematical problem solving

    Gou, Z., Shao, Z., Gong, Y., Shen, Y., Yang, Y., Huang, M., Duan, N., Chen, W. ToRA: A tool-integrated reasoning agent for mathematical problem solving. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7–11, 2024. OpenReview....

    2024

  43. [51]

    Skills-in-context prompting: Unlocking compositionality in large language models.arXiv preprint arXiv:2308.00304, 2023

    Chen, J., Pan, X., Yu, D., Song, K., Wang, X., Yu, D., Chen, J. Skills-in-context prompting: Unlocking compositionality in large language models.arXiv preprint arXiv:2308.00304, 2023

    2023 arXiv

  44. [52]

    L., Shen, J., Hu, J., Han, X., Huang, Y., Zhang, Y., Liu, J., Qi, L., Liu, Z., Sun, M

    He, C., Luo, R., Bai, Y., Hu, S., Thai, Z. L., Shen, J., Hu, J., Han, X., Huang, Y., Zhang, Y., Liu, J., Qi, L., Liu, Z., Sun, M. Olympiadbench: A challenging benchmark for promoting AGI with olympiad-level bilingual multimodal scientific problems. InProceedings of the 62nd An...

    2024

  45. [53]

    premises

    Yang, A., Yang, B., Hui, B., Zheng, B., et al. Qwen2 technical report.arXiv preprint arXiv:2407.10671, 2024. Title Suppressed Due to Excessive Length 15 A Prompt PEC Prompt Extract premises and clear question from input, output a dictionary with ’premise’ and ’question’ as key...

    2024 arXiv

  46. [2024]

    OpenReview.net, 2024

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.