Pith. sign in

REVIEW 4 major objections 6 minor 37 references

As exemplars accumulate, chain-of-thought reasoning shifts from pretrained priors to in-context signals, so enough wrong examples can flip even an 8B model's answers.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

The paper finds that as exemplar count increases, LLMs shift from pretrained priors to in-context signals, and long chain-of-thought prompts induce longer reasoning chains that often improve accuracy.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A useful empirical paper whose central RQ2 label-flipping claim is under-evidenced; deserves a referee but needs per-label output analysis. the 4 major comments →

arxiv 2509.01236 v1 pith:5P3F4GPY submitted 2025-09-01 cs.CL cs.AI

Rethinking the Chain-of-Thought: The Roles of In-Context Learning and Pre-trained Priors

Hao Yang , Zhiyu Yang , Yunjie Zhang , Shanyi Zhu , Lin Yang This is my paper
classification cs.CL cs.AI
keywords Chain-of-Thought promptingIn-context learningPretrained priorsLarge language modelsNoisy exemplarsSlow thinkingReasoning mechanismsPrompt engineering
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Chain-of-thought prompting is standard practice, but why it works is unsettled. This paper argues that the answer lies in a shifting balance between two sources: pretrained knowledge the model already has and in-context signals from the exemplars in the prompt. With few examples, pretrained priors dominate and noise barely matters; as the number of exemplars grows, in-context signals strengthen and can override prior knowledge, so systematically wrong answers begin to drive the model's choices—even in an 8B model on a binary task. The same logic implies exemplar quality matters more as prompt size grows. Finally, the paper shows that prompts distilled from long-reasoning models induce longer chains and better downstream accuracy, with an optimal length.

Core claim

The paper's central discovery is a mechanism with a dial: the number of exemplars controls whether CoT reasoning follows pretrained priors or in-context signals. In controlled experiments, task-agnostic CoT prompts leave semantic reasoning intact while changing lexical form, and accuracy tracks the number of reasoning verbs with an optimum, indicating the model copies structure while relying on prior knowledge for content. Then, replacing answers with wrong answers in increasing numbers of exemplars shows little effect at 4–5 shots but progressive accuracy collapse at 40 shots; in the closed-domain binary Coin Flip task, an 8B model systematically flips labels. False rationales likewise degr

What carries the argument

The exemplar-count dial plus a lexical decomposition of rationales. Starting with a small number of CoT exemplars and incrementally adding corrupted ones turns the strength of in-context learning into a controllable scalar. A complementary lexical analysis splits model-generated rationales into structure words, feature words, verbs, and location/person entities to separate imitation of form from reliance on semantic priors; confidence is tracked through per-token generation probabilities.

Load-bearing premise

The load-bearing premise is that the accuracy collapse under many false-answer exemplars means the model actually learns to output the flipped labels, rather than merely copying final answers or getting confused; the paper's own limitation section adds that greedy decoding often produced endless repetitions, a separate risk for the slow-thinking result.

What would settle it

Record the actual final-answer tokens the model emits under 40 false-answer CoT exemplars on Coin Flip. If the model's output distribution still concentrates on correct labels, or collapses onto copied exemplar answers regardless of the input, then the claimed systematic input-label mapping and label flipping are not real, and the ICL-overrides-priors conclusion would need revision.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Few noisy exemplars are harmless, but at scale they can outweigh what the model knows, so prompt hygiene matters increasingly with prompt length.
  • Contrary to earlier claims that only very large models can learn flipped labels in context, CoT prompting lets an 8B model flip answers on a closed-domain binary task.
  • Misleading rationales make a model less confident as well as less accurate, so unreliable prompts are detectable in generation probabilities.
  • Long-CoT prompts distilled from stronger reasoners can transfer slow-thinking behavior to smaller models and improve accuracy, up to an optimal length.
  • Reasoning length is not 'more is better': beyond an optimum, longer chains hurt, and the optimum grows with model size and task difficulty.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The abrupt shift from priors-dominated to ICL-dominated behavior as exemplars accumulate suggests a phase-transition-like dynamic; a natural test is to fit accuracy as a function of exemplar count and look for a critical count that changes with model size.
  • If the mechanism is general, closed-domain tasks with small label spaces (multiple choice, yes/no, classification) are the most vulnerable to prompt poisoning, while open-ended generation is more robust.
  • The slow-thinking transfer result could support self-improvement loops: a model's own long outputs, once filtered for quality, could be reused as prompts or training data, provided the decoding repetition problem is solved.
  • The optimal-length finding implies adaptive control of reasoning length per question—stop early on easy items, extend on hard ones—could capture most gains at lower inference cost.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper studies Chain-of-Thought (CoT) prompting through the dual lens of in-context learning (ICL) and pretrained priors. It reports three experiments: (i) a lexical-level analysis of model-generated rationales comparing zero-shot CoT, manual CoT, and task-agnostic CoT; (ii) a noisy-exemplar scaling study that increments false-answer and false-rationale exemplars, claiming that sufficient exemplars shift the model from pretrained priors to ICL and that 8B models can flip labels in closed-domain tasks; and (iii) a prompt-engineering study that uses long CoT prompts distilled from R1/QwQ models to induce slow thinking. The paper claims three findings: models learn lexical and deeper reasoning structures while relying on pretrained priors; exemplar scaling shifts decision-making toward ICL while misleading prompts introduce instability; and long CoT prompting improves downstream performance.

Significance. If the label-flipping and prior-shift claims held, the paper would contribute to the debate on the nature of ICL and would challenge prior results (Wei et al., 2023) about model-size thresholds for learning flipped labels. The RQ3 results also have practical value for prompt design. The paper has notable strengths: it spans multiple model families and sizes (Gemma2-9B/27B, LLaMA3.1-8B, Qwen2.5-32B), the false-exemplar perturbation design is simple and clearly described, and the authors include a limitation section acknowledging dataset, noise-level, and decoding issues. However, the central mechanistic interpretation is not tightly supported by the reported measures, and several quantitative details are missing.

major comments (4)
  1. [Section 3.2 (Fig. 4)] The central inference of RQ2 is that 'the model tends to learn an input-label mapping that leads to systematic label flipping' and that this refutes Wei et al.'s claim that smaller models cannot learn flipped labels. The evidence is an aggregate accuracy decline in Coin Flip. A decline from high to near-chance accuracy is equally compatible with copying the (possibly balanced) false answers, with a global response bias, or with general confusion from contradictory evidence; only a decline to ~0% (or per-label flip rates close to 100%) would establish systematic flipping. The paper reports no exact Coin Flip accuracies, confusion matrices, or error bars. Please add output-level analysis (accuracy by true label, flip rate, and answer distribution at 4-shot and 40-shot) before the 'ICL overrides pretrained priors' claim is accepted.
  2. [Section 3.2 (Fig. 5)] The confidence analysis is one case study: a single trace of per-step probabilities for three prompts. The claim that CoT prompts maintain stable probabilities while false prompts fluctuate is not substantiated without aggregation over a sample of questions. Please provide mean and variance/autocorrelation statistics across the test set, or at least across multiple examples, with prompt type as the independent variable. As presented, Fig. 5 is anecdotal and cannot carry the 'misleading prompts introduce instability' takeaway.
  3. [Section 3.1 (Figs. 2 and 3)] The paper states that 'a positive correlation' exists between average verb count and accuracy, and that an 'optimal count' is evident, but no correlation coefficients, fitted curves, or confidence intervals are reported. The scatterplots combine multiple prompt types and model sizes in one panel, making within-task relationships difficult to assess. Please quantify these lexical-level claims (e.g., Spearman correlation per model/task, or a regression controlling for model size and task difficulty) and state the criteria for an 'optimal' verb count.
  4. [Table 1 (Section 3.3)] The 'improving performance' claim is based on point estimates without variance or significance tests. Some entries contradict the broad claim, e.g., LLaMA3.1-8B-Instruct on GSM8K with QwQ-32B-Long-CoT (0.7695 to 0.6202) and LLaMA3.1-8B-Instruct on DATE with DS-Qwen32B-Long-CoT (0.6504 to 0.6449). The 'Avg. tokens total' is reported but not analyzed as a continuous variable; the optimal-length conclusion would benefit from a systematic length-versus-performance analysis over a range of token budgets rather than selected model/task pairings.
minor comments (6)
  1. [Section 3.2] Typo: 'Our findings challenge previousous claims' should read 'previous claims'.
  2. [Table 1 caption] 'A vg. tokens total' should be 'Avg. tokens total'.
  3. [Fig. 3] The legend uses 'LLaMA3-8B' while the text uses 'LLaMA3.1-8B'; make consistent.
  4. [Introduction] The formalization of CoT prompting is incomplete: 'Input: X = {(q_i, r_i, a_i)}N_{i=1}, q' lacks clear set notation and the target question q is not incorporated into the conditioning set. Please rewrite the equation cleanly.
  5. [Section 3.2] The 'token generation probability at each time step' is used as confidence, but it is not specified whether this is the softmax probability over the full vocabulary, top-k logits, or a normalized logit transformation. Clarify how the 'Probability Value' in Fig. 5 is computed.
  6. [Section 6 (Limitations)] The authors acknowledge that greedy decoding 'often resulted in endless repetitions' for long CoT prompts, but the frequency of such repetitions is not reported. This could bias the RQ3 accuracy numbers; please state how many generations (per condition) were truncated or excluded.

Circularity Check

0 steps flagged

No significant circularity: the study reports empirical observations; its claims do not reduce to fitted parameters, self-citations, or definitional equivalences.

full rationale

This is an empirical study rather than a derivation, so the standard circularity patterns do not apply. RQ1 (Section 3.1) defines lexical categories (structure words, feature words, verbs, loc&per) and counts them in model-generated rationales; the conclusion that models 'rely on pretrained priors' is an interpretive claim about those counts, not a result that is equivalent to the counting procedure by construction. RQ2 (Section 3.2) constructs false-answer and false-rationale exemplars by explicit editing operations (inverting yes/no, swapping operators, shifting dates, replacing answers), then measures accuracy as a function of exemplar count. The observed accuracy decline is not a fitted parameter renamed as a prediction; the 'systematic label flipping' explanation is inferred from aggregate accuracy in a closed-domain task, but it is not built into the accuracy metric itself. Underdetermination of the mechanism by aggregate accuracy is an evidentiary limitation, not circularity. RQ3 (Section 3.3) is motivated by RQ1/RQ2 but independently tests whether long CoT prompts distilled from RLMs improve accuracy; the measured improvements on GSM8K, MATH-500, Date, and Last Letter are external outcomes, not consequences of the prompt construction. The paper uses standard prompts from Wei et al. [30] and Kojima et al. [9], and cites prior work for context and contrast, but no load-bearing claim depends solely on a self-citation by the present authors. There is no imported uniqueness theorem, no ansatz smuggled in via citation, and no renaming of a known result as a new framework. Accordingly, the circularity score is 0.

Axiom & Free-Parameter Ledger

1 free parameters · 5 axioms · 0 invented entities

No new entities (particles, forces, dimensions) are introduced; the paper is an empirical study of existing LLMs. The ledger captures the main measurement and interpretation assumptions on which the central claims rest.

free parameters (1)
  • Exemplar count range for scaling experiments = up to 40
    The shift from pretrained priors to ICL signals is demonstrated over the range 4 to 40 exemplars; the specific maximum of 40 is an arbitrary design choice, and the conclusion may not generalize beyond this range.
axioms (5)
  • domain assumption NLTK verb extraction identifies reasoning-relevant actions in model-generated rationales.
    Section 3.1 method: the paper counts verbs using NLTK and interprets verb count as 'reasoning actions' with an optimal level, but does not validate this mapping.
  • domain assumption The selected benchmarks (GSM8K, MATH-500, Date, Coin Flip, Last Letter) measure reasoning ability.
    Section 2.1: used as proxies for arithmetic, commonsense, and symbolic reasoning following prior work; whether they capture CoT's mechanism is assumed.
  • domain assumption Greedy decoding outputs are representative of model behavior under these prompts.
    Section 2.2: greedy decoding is used following Wei et al.; the paper notes in the Limitation that long CoT under greedy decoding caused endless repetitions, which may make the RQ3 results unrepresentative.
  • ad hoc to paper Task-agnostic CoT prompts isolate the contribution of in-context exemplars versus pretrained priors.
    Section 3.1: the paper infers pretrained priors from zero-shot CoT and ICL from task-specific/task-agnostic CoT, assuming the prompt type cleanly separates these factors.
  • domain assumption Distilled long CoT prompts from DeepSeek-R1 and QwQ are valid exemplars for teaching slow thinking.
    Section 3.3: the paper uses these as prompts without controlling for their content, length, or correctness, yet attributes downstream gains to induced slow thinking.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Rethinking the Chain-of-Thought: The Roles of In-Context Learning and Pre-trained Priors." pith.science (2026). https://pith.science/paper/5P3F4GPY

@misc{pith2026250901236,
  author       = {Pith},
  title        = {Pith review of: Rethinking the Chain-of-Thought: The Roles of In-Context Learning and Pre-trained Priors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5P3F4GPY}},
  note         = {Machine review of arXiv:2509.01236}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Chain-of-Thought reasoning has emerged as a pivotal methodology for enhancing model inference capabilities. Despite growing interest in Chain-of-Thought reasoning, its underlying mechanisms remain unclear. This paper explores the working mechanisms of Chain-of-Thought reasoning from the perspective of the dual relationship between in-context learning and pretrained priors. We first conduct a fine-grained lexical-level analysis of rationales to examine the model's reasoning behavior. Then, by incrementally introducing noisy exemplars, we examine how the model balances pretrained priors against erroneous in-context information. Finally, we investigate whether prompt engineering can induce slow thinking in large language models. Our extensive experiments reveal three key findings: (1) The model not only quickly learns the reasoning structure at the lexical level but also grasps deeper logical reasoning patterns, yet it heavily relies on pretrained priors. (2) Providing sufficient exemplars shifts the model's decision-making from pretrained priors to in-context signals, while misleading prompts introduce instability. (3) Long Chain-of-Thought prompting can induce the model to generate longer reasoning chains, thereby improving its performance on downstream tasks.

Figures

Figures reproduced from arXiv: 2509.01236 by Hao Yang, Lin Yang, Shanyi Zhu, Yunjie Zhang, Zhiyu Yang.

Figure 1
Figure 1. Figure 1: Framework highlighting the synergy between Pretrained Priors and In￾Context Learning in powering CoT reasoning. where (qi , ri , ai) represents the i-th exemplar consisting of a question qi , a rationale ri , and an answer ai ; N denotes the number of exemplars; and q is the target question. The model generates a response (r, a) by conditioning on the exemplars and the question: P(r, a | X, q) = P(r | X, q… view at source ↗
Figure 2
Figure 2. Figure 2: Proportion analysis of rationale components: Exemplars (left) compare in￾context CoT vs. task-agnostic CoT variants (Sports, Coin Flip, etc.); Model￾generated content (right) contrasts zero-shot CoT, CoT, and task-agnostic CoT frameworks. Method. From an ICL perspective, we conducted a fine-grained analysis of model-generated reasoning texts using the GSM8K test dataset. We decomposed exemplars into four k… view at source ↗
Figure 3
Figure 3. Figure 3: Scatter plot illustrating the relationship between the average number of reason￾ing verbs in model-generated content and accuracy under CoT and task-agnostic CoT settings. ity is strongly influenced by the number of reasoning steps, which are typically considered at the sentence level. In this work, we explored the lexical level, hy￾pothesizing that the number of reasoning actions also affects performance,… view at source ↗
Figure 4
Figure 4. Figure 4: Evolution of test accuracy as the number of noisy (false-answer, false-rationale) exemplars increases. Analysis. As shown in [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Probabilities evolution of model-generated outputs under greedy decod￾ing for three prompt types: CoT prompt, false-answer CoT prompt, and false￾rationale CoT prompt. Case Study. We further investigated the model’s internal mechanisms to un￾derstand how false-answer and false-rationale CoT prompts affect reasoning. Under greedy decoding, we recorded the token generation probabilities at each time step to r… view at source ↗
Figure 6
Figure 6. Figure 6: Framework for employing prompt engineering to encourage slow thinking in LLMs. Researchers enhance LLMs reasoning abilities by extending the length of CoT, leading to the development of Reasoning Language Models (RLMs). RLMs gen￾erate a large number of tokens during inference, a process known as slow thinking or test-time scaling. The results of two previous experiments indicate that LLMs effectively lever… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 24 canonical work pages

  1. [1]

    Transactions on Machine Learning Research (2023)

    BigBench: Beyond the imitation game: Quantifying and extrapolating the capabil- ities of language models. Transactions on Machine Learning Research (2023)

    work page 2023

  2. [2]

    In: Advances in Neural Information Processing Systems

    Brown, T., Mann, B., Ryder, N., et al.: Language models are few-shot learners. In: Advances in Neural Information Processing Systems. vol. 33, pp. 1877–1901. Curran Associates, Inc. (2020)

    work page 1901

  3. [3]

    Transactions on Machine Learning Research (2023)

    Chen, W., Ma, X., et al.: Program of thoughts prompting: Disentangling com- putation from reasoning for numerical reasoning tasks. Transactions on Machine Learning Research (2023)

    work page 2023

  4. [4]

    Chia, Y.K., Chen, G., et al.: Contrastive chain-of-thought prompting (2023)

    work page 2023

  5. [5]

    arXiv preprint arXiv:2110.14168 (2021) 12 Hao Yang, Zhiyu Yang, Yunjie Zhang3, Shanyi Zhu4, and Lin Yang

    Cobbe, K., Kosaraju, V., et al.: Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 (2021) 12 Hao Yang, Zhiyu Yang, Yunjie Zhang3, Shanyi Zhu4, and Lin Yang

    Pith/arXiv arXiv 2021

  6. [6]

    Psychological review101(3), 371 (1994)

    Graesser, A.C., Singer, M., Trabasso, T.: Constructing inferences during narrative text comprehension. Psychological review101(3), 371 (1994)

    work page 1994

  7. [7]

    language 56(2), 251–299 (1980)

    Hopper, P.J., Thompson, S.A.: Transitivity in grammar and discourse. language 56(2), 251–299 (1980)

    work page 1980

  8. [8]

    In: Findings of the Association for Computational Linguistics: ACL 2024

    Jin, M., Yu, Q., et al.: The impact of reasoning step length on large language models. In: Findings of the Association for Computational Linguistics: ACL 2024. pp. 1830–1842. Association for Computational Linguistics (2024)

    work page 2024

  9. [9]

    In: Advances in Neural Information Processing Systems

    Kojima, T., Gu, S.S., Reid, M., Matsuo, Y., Iwasawa, Y.: Large language models are zero-shot reasoners. In: Advances in Neural Information Processing Systems. vol. 35, pp. 22199–22213. Curran Associates, Inc. (2022)

    work page 2022

  10. [10]

    In: International Conference on Learning Repre- sentations (ICLR) (2024)

    Kossen, J., Gal, Y., Rainforth, T.: In-context learning learns label relationships but is not conventional learning. In: International Conference on Learning Repre- sentations (ICLR) (2024)

    work page 2024

  11. [11]

    Lee, A., Che, E., Peng, T.: How well do llms compress their own chain-of-thought? a token complexity approach (2025),https://arxiv.org/abs/2503.01141

    Pith/arXiv arXiv 2025

  12. [12]

    Li, D., Cao, S., et al.: Llms can easily learn to reason from demonstrations struc- ture, not content, is what matters! arXiv preprint arXiv:2502.07374 (2025)

    Pith/arXiv arXiv 2025

  13. [13]

    arXiv preprint arXiv:2305.20050 (2023)

    Lightman, H., Kosaraju, V., , et al.: Let’s verify step by step. arXiv preprint arXiv:2305.20050 (2023)

    Pith/arXiv arXiv 2023

  14. [14]

    In: Forty-first Inter- national Conference on Machine Learning (2024)

    Lin, Z., Lee, K.: Dual operating modes of in-context learning. In: Forty-first Inter- national Conference on Machine Learning (2024)

    work page 2024

  15. [15]

    Madaan, A., Yazdanbakhsh, A.: Text and patterns: For effective chain of thought, it takes two to tango (2022),https://arxiv.org/abs/2209.07686

    Pith/arXiv arXiv 2022

  16. [16]

    In: Findings of the Association for Computational Linguistics: EMNLP 2023

    Madaan, A., et al.: What makes chain-of-thought prompting effective? a coun- terfactual study. In: Findings of the Association for Computational Linguistics: EMNLP 2023. pp. 1448–1535. Association for Computational Linguistics (2023)

    work page 2023

  17. [17]

    In: The Twelfth International Conference on Learning Representations (2024)

    Merrill, W., Sabharwal, A.: The expressive power of transformers with chain of thought. In: The Twelfth International Conference on Learning Representations (2024)

    work page 2024

  18. [18]

    Min, S., et al.: Rethinking the role of demonstrations: What makes in-context learning work? In: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. pp. 11048–11064. Association for Computational Linguistics (2022)

    work page 2022

  19. [19]

    OpenAI: Gpt-4 technical report (2024),https://arxiv.org/abs/2303.08774

    Pith/arXiv arXiv 2024

  20. [20]

    In: Findings of the Association for Computational Linguistics: ACL 2023

    Pan, J., et al.: What in-context learning “learns” in-context: Disentangling task recognition and task learning. In: Findings of the Association for Computational Linguistics: ACL 2023. pp. 8298–8319. Association for Computational Linguistics (2023)

    work page 2023

  21. [21]

    In: The Eleventh International Conference on Learn- ing Representations (2023)

    Saparov, A., He, H.: Language models are greedy reasoners: A systematic formal analysis of chain-of-thought. In: The Eleventh International Conference on Learn- ing Representations (2023)

    work page 2023

  22. [22]

    arXiv preprint arXiv:2307.10573 (2023)

    Schaeffer, R., Pistunova, K., et al.: Invalid logic, equivalent gains: The bizarreness of reasoning in language model prompting. arXiv preprint arXiv:2307.10573 (2023)

    Pith/arXiv arXiv 2023

  23. [23]

    In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    Si, C., Friedman, D., et al.: Measuring inductive biases of in-context learning with underspecified demonstrations. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 11289– 11310. Association for Computational Linguistics (2023)

    work page 2023

  24. [24]

    Talmy, L.: Toward a cognitive semantics: Concept structuring systems, vol. 1. MIT press (2000)

    work page 2000

  25. [25]

    Team, G.: Gemma 2: Improving open language models at a practical size (2024), https://arxiv.org/abs/2408.00118 Title Suppressed Due to Excessive Length 13

    Pith/arXiv arXiv 2024

  26. [26]

    Team,L.:Thellama3herdofmodels(2024), https://arxiv.org/abs/2407.21783

    Pith/arXiv arXiv 2024

  27. [27]

    arXiv preprint arXiv:2412.15115 (2024)

    Team, Q.: Qwen2.5 technical report. arXiv preprint arXiv:2412.15115 (2024)

    Pith/arXiv arXiv 2024

  28. [28]

    In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    Wang, B., Min, S., et al.: Towards understanding chain-of-thought prompting: An empirical study of what matters. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 2717–

    work page

  29. [29]

    In: The Thirty-eighthAnnualConferenceonNeuralInformationProcessingSystems(2024)

    Wang, X., Zhou, D.: Chain-of-thought reasoning without prompting. In: The Thirty-eighthAnnualConferenceonNeuralInformationProcessingSystems(2024)

    work page 2024

  30. [30]

    In: Advances in Neural Information Processing Systems

    Wei, J., Wang, X., Schuurmans, et al.: Chain-of-thought prompting elicits rea- soning in large language models. In: Advances in Neural Information Processing Systems. vol. 35, pp. 24824–24837. Curran Associates, Inc. (2022)

    work page 2022

  31. [31]

    arXiv preprint arXiv:2303.03846 (2023)

    Wei, J., Wei, J., et al.: Larger language models do in-context learning differently. arXiv preprint arXiv:2303.03846 (2023)

    Pith/arXiv arXiv 2023

  32. [32]

    Wu, Y., Wang, Y., Du, T., Jegelka, S., Wang, Y.: When more is less: Understanding chain-of-thought length in llms (2025),https://arxiv.org/abs/2502.07266

    Pith/arXiv arXiv 2025

  33. [33]

    Yang, W., Ma, S., Lin, Y., Wei, F.: Towards thinking-optimal scaling of test-time compute for llm reasoning (2025),https://arxiv.org/abs/2502.18080

    arXiv 2025

  34. [34]

    In:FindingsoftheAssociationforComputationalLinguistics:ACL2023.pp.4469–

    Ye,X.,Iyer,S.,etal.:Complementaryexplanationsforeffectivein-contextlearning. In:FindingsoftheAssociationforComputationalLinguistics:ACL2023.pp.4469–

    work page

  35. [35]

    In: Proceedings of the 2022 Conference on Empirical Meth- odsinNaturalLanguageProcessing.pp.2422–2437.AssociationforComputational Linguistics (2022)

    Yoo, K.M., Kim, J., et al.: Ground-truth labels matter: A deeper look into input- label demonstrations. In: Proceedings of the 2022 Conference on Empirical Meth- odsinNaturalLanguageProcessing.pp.2422–2437.AssociationforComputational Linguistics (2022)

    work page 2022

  36. [36]

    In: The Eleventh International Conference on Learning Representations (2023)

    Zhou, D., Schärli, N., et al.: Least-to-most prompting enables complex reasoning in large language models. In: The Eleventh International Conference on Learning Representations (2023)

    work page 2023

  37. [4484]

    Association for Computational Linguistics (2023)

    work page 2023

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.