Pith. sign in

REVIEW 4 major objections 5 minor 51 references

Chain-of-Thought in Large Language Models: Decoding, Projection, and Activation

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that chain-of-thought prompting produces three measurable internal changes in large language models: imitation of exemplar format, concentration of final-answer logits, and broader final-layer neuron activation.

desk verdict A readable but statistically thin empirical battery; the activation claim is likely confounded by output length. read the letter →

arxiv 2412.03944 v1 pith:Z5UMLYKY submitted 2024-12-05 cs.AI

Hao Yang , Qianghua Zhao , Lei Li This is my paper · ORCID
classification cs.AI
keywords chain-of-thoughtlargelanguagemodelslogitsentropyneuronactivationfeed-forwardlayerspromptimitationreasoningtasks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that chain-of-thought prompting changes a language model in three observable ways: it imitates the exemplar format while drawing on its own understanding of the question, it produces a more concentrated probability distribution over the final answer, and it activates a wider set of neurons in the final feed-forward layers. These observations are offered as evidence about how chain-of-thought works internally, not just that it improves accuracy. The authors measure imitation with four word categories, logits with probability traces and entropy over candidate answers, and activation with range and intensity in feed-forward layers across four model sizes.

What carries the argument

The analysis rests on three measurement tools. Test points are four predefined word categories (time, action, location/person, number) matched by exact string against generated text to quantify exemplar imitation and transfer. Logits are normalized to probabilities and tracked horizontally across generated tokens and vertically as the entropy of the probability distribution over the answer space. Neuron activation is measured in the feed-forward layers by two metrics: activation range, the proportion of neurons with value greater than zero, and activation intensity, the average value of active neurons. These tools make the chain-of-thought versus standard difference legible at the decoding, projection, and activation levels.

What would settle it

Run a length-matched control: take standard-prompt answers, pad them to chain-of-thought length with neutral filler reasoning, and recompute final-layer activation range and intensity; if padded standard outputs show the same broad activation, the knowledge-retrieval interpretation fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that chain-of-thought prompting alters a model's internal and external behavior in three discernible ways. First, the model does not simply copy the exemplars: it adopts the sequential format markers (like 'first', 'so', 'then', arithmetic operators) while supplying content driven by its own understanding of the question, and this imitation-plus-understanding is what yields correct answers. Second, during generation the token-level logits fluctuate more than under standard prompting, yet at the final answer the probability distribution over candidate answers is more concentrated, as measured by lower entropy on tasks with finite answer spaces. Third, across the final twenty feed-forward layers, chain-of-thought expands the activation range (fraction of neurons with positive values) while lowering activation intensity, which the authors interpret as broader retrieval of pre-trained knowledge.

Load-bearing premise

The load-bearing assumption is that the wider neuron activation seen under chain-of-thought is not a side effect of chain-of-thought generating far longer outputs, since activation range and intensity are aggregated over generated tokens without a length-controlled baseline; the paper's own limitations section also restricts generalization to the tested datasets and prompt conditions.

Editorial extensions

If this is right

  • Chain-of-thought's benefit is not pure imitation: transfer tests show the model follows the exemplar format while using its own knowledge, and better imitation tracks better accuracy.
  • Logits behavior splits into two phases: turbulent intermediate reasoning, then a sharpened final answer distribution, suggesting chain-of-thought acts as a reasoning scratchpad that converges at the end.
  • The activation signature of broader range and lower intensity in final layers provides a quantitative marker for deeper knowledge retrieval that later work can use to compare prompt designs.
  • Entropy over the answer space offers a cheap confidence signal: chain-of-thought's more concentrated distribution at the answer token could be used to flag model certainty on finite-answer tasks.
  • The patterns reproduce across all four tested model sizes and across arithmetic, commonsense, and symbolic reasoning tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not control for output length in the activation comparison; a length-matched control would test whether the broader neuron range reflects wider knowledge retrieval or simply more generated tokens.
  • The imitation metric relies on predefined word lists, so the imitation finding is partly shaped by the chosen test points; an open-vocabulary measure of format adherence could confirm the claim.
  • If concentrated answer logits track correctness, chain-of-thought could double as a calibration signal; the paper's correct/wrong scatter plots suggest such a relationship but stop short of a calibration analysis.
  • The final-layer activation expansion could be probed by ablating the newly activated neurons; if answers degrade, the broadened retrieval is causally implicated, not merely correlated.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper reports an empirical study of how Chain-of-Thought (CoT) prompting changes the behavior of large language models. Using 50 randomly sampled questions from each of nine reasoning datasets and four models (Gemma2 2B/9B/27B and LLaMA2-13B), the authors study three aspects: (1) whether models imitate CoT exemplar formats, measured through predefined test-point word lists and a cross-task transfer test; (2) how token-level logits and the final answer distribution change, measured through token probability curves, kernel density estimates of the phrase "the answer is", and entropy of top-k answer probabilities; and (3) how FFN neuron activation changes, measured through activation range and intensity across the final 20 layers. The paper concludes that LLMs imitate exemplar formats while also integrating question understanding, that CoT produces a more concentrated final logits distribution, and that CoT activates a broader set of final-layer neurons, interpreted as more extensive knowledge retrieval. The paper closes with a Limitations section acknowledging limited task scope, fixed prompt types, FFN-only analysis, and the exploratory nature of the study.

Significance. If the findings were rigorously established, the paper would provide useful observational evidence about the internal and external effects of CoT prompting, complementing existing hypotheses from Saparov and He, Madaan et al., Geva et al., and Rai and Yao. The study has real strengths: it spans multiple model families and sizes, uses publicly available models, provides the full prompts in the appendix, and attempts to relate imitation, logits, and neuron activation to each other. However, the current evidence is largely descriptive and lacks the controls and statistical machinery needed to support the mechanistic claims. The most important strength is the breadth of datasets and models; the main weakness is that the headline conclusions depend on comparisons that are confounded by output length, on a partly circular test-point metric, and on visual inspection of scatter and density plots without confidence intervals or significance tests. As it stands, the paper is a useful exploratory report rather than a definitive mechanistic account.

major comments (4)
  1. [Section 3.5, Figure 7] The activation-range comparison is confounded by sequence length. The text defines Activation Range as "the proportion of activated neurons relative to the total number of neurons in the FFN layer" and Activation Intensity as "the averaged activation value of the activated neurons," but it does not state whether these metrics are computed per generated token and then averaged, or pooled over all tokens in a generation. CoT outputs are routinely much longer than Standard outputs; under a pooled reading, the union of active neurons will mechanically grow with output length even if per-token activation is unchanged. A length-matched control (e.g., per-token averaged activation, or comparing only the first N tokens) is necessary before interpreting the observed broader activation range as "more extensive knowledge retrieval." The Limitations section does not mention this confound.
  2. [Section 3.3 and Appendix B] The test-point measure of imitation is partly circular. The lists of "time," "action," and "loc&peo" words are manually derived from the exemplars (Table 18), and the "number" category is extracted from the questions; observing these words in CoT-generated text is therefore expected under the very definition of the metric. The imitation claim needs a control condition, such as a Standard-prompt baseline or a matched set of control words that are frequent in English but absent from the exemplars, together with effect sizes for the match proportions. Without such a baseline, the takeaway that LLMs "imitate exemplar formats" is not strongly supported.
  3. [Section 3.1 and Figures 5-7] All quantitative comparisons rest on only 50 examples per dataset, and no confidence intervals, error bars, or statistical tests are reported. For example, Figure 6 shows overlapping entropy scatter points between CoT and Standard conditions, and Figure 7 compares activation range and intensity with summary boxes only. The authors should add bootstrap confidence intervals, paired comparisons on the same questions, or per-dataset effect sizes so that the reader can assess whether the observed differences are larger than sampling noise.
  4. [Section 3.4 and Appendix H] The claim that "the final output shows a more concentrated logits distribution" relies on entropy values computed after selecting the top k probabilities and normalizing them, where k is the answer-space size. Because the top-k selection is informed by the observed outputs, and because the scatter plots show substantial overlap between conditions, the analysis should state the null hypothesis explicitly and report a quantitative comparison (e.g., paired differences between CoT and Standard for the same questions). The current presentation does not rule out the possibility that the difference is driven by a small number of outlier examples.
minor comments (5)
  1. [Abstract] There is a typo: "avialable" should be "available," and "Chainof-Thought" should be "Chain-of-Thought."
  2. [Appendix B, Table 18] The entry for "number" test points reads "it was obtained by regular expression," which appears to be a placeholder; please replace it with the actual list or the precise extraction rule.
  3. [Table 1] The header says "Top 10 tokens," but the table rows include subword-like items such as "pped" and "aten"; please clarify whether these are subword tokens and how spaces and punctuation were handled.
  4. [Appendix D] The imitation heuristic uses an ad hoc threshold of more than four occurrences of verbs like "flips, is, was, are, be, were" to decide whether the model generated new entities; the choice of threshold and the list of verbs should be justified or varied in a sensitivity analysis.
  5. [Appendix F] Figures 15-17 show probability traces for a single example per dataset per model; the paper should state how representative these examples are, since they are used to support the qualitative claim that CoT causes "sharp drops and oscillations" in token probabilities.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: all three central claims rest on independent empirical measurements with no fitted parameters and no load-bearing self-citation chain.

full rationale

This paper makes three empirical claims about CoT behavior: format imitation measured by predefined test-point word frequencies, logits concentration measured by probability densities and Shannon entropy, and broader FFN activation measured by activation range and intensity. None of these quantities is fitted to the outcome it is said to support. The test-point metric counts exact matches of predefined words, and the conclusion that the model imitates exemplar formats is an interpretive reading of those counts that could in principle have failed; the entropy values are computed from the model's own output distributions using the standard entropy formula; and the activation range and intensity are direct statistics over FFN activations. There are no parameters fitted to a subset of data and then presented as predictions on closely related quantities, no uniqueness theorem imported from the authors' prior work, and no self-citation chain that carries the argument. The activation comparison is potentially confounded by different generation lengths, and the imitation metric is operationally coarse, but these are validity and robustness concerns rather than circularity: the measurements are not equivalent to their inputs by construction. Accordingly, no circular step is identified.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No free parameters are fitted to data in the derivation sense; the listed thresholds are hand-set analysis choices. The central claims rest on two domain assumptions about the meaning of the test-point and activation metrics, plus standard statistical tools. No new entities are postulated.

free parameters (4)
  • top_1_3_difference_threshold = 1/3
    Section 3.4 selects the top 1/3 of tokens by difference from adjacent token probabilities to build Table 1. The choice is hand-set and affects which tokens are highlighted.
  • imitation_verb_count_threshold = 4
    Appendix D treats generated content as containing new entities when the count of verbs like 'flips, is, was, are, be, were' exceeds 4. This hand-set threshold defines the imitation metric.
  • final_layers_count = 20
    Section 3.5 reports activation only for the final 20 FFN layers for comparability across model sizes; results may differ for earlier layers.
  • answer_space_k = set by dataset
    Section 3.4 and Appendix H compute entropy over the top-k token probabilities at the answer step, with k equal to the size of the finite answer space (5 for AQuA, 2 for Sports and Coin Flip). This excludes open-ended datasets.
assumptions (4)
  • domain assumption Neurons with activation value greater than zero are 'active', and the mean value of active neurons is 'intensity'
    Section 3.5 adopts this from Geva et al. (2021b) and Rai and Yao (2024). The choice of threshold shapes the activation-range and intensity results.
  • domain assumption The frequency of predefined 'test point' words in generated text measures imitation of exemplar format
    Section 3.3 uses exact match over hand-built word lists to conclude that models imitate exemplar formats. The construct validity of this metric is assumed, not validated.
  • standard math Shannon entropy of the normalized top-k answer probabilities quantifies concentration of the model's choice
    Section 3.4 uses the standard entropy formula to compare standard and CoT answer distributions.
  • domain assumption Greedy decoding with a 300-token cap produces representative samples of model behavior
    Section 3.2 fixes greedy decoding and 300 tokens following Wei et al. (2022); sampling variability is not quantified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Chain-of-Thought in Large Language Models: Decoding, Projection, and Activation." pith.science (2026). https://pith.science/paper/Z5UMLYKY

@misc{pith2026241203944,
  author       = {Pith},
  title        = {Pith review of: Chain-of-Thought in Large Language Models: Decoding, Projection, and Activation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z5UMLYKY}},
  note         = {Machine review of arXiv:2412.03944}
}
read the original abstract

Chain-of-Thought prompting has significantly enhanced the reasoning capabilities of large language models, with numerous studies exploring factors influencing its performance. However, the underlying mechanisms remain poorly understood. To further demystify the operational principles, this work examines three key aspects: decoding, projection, and activation, aiming to elucidate the changes that occur within models when employing Chainof-Thought. Our findings reveal that LLMs effectively imitate exemplar formats while integrating them with their understanding of the question, exhibiting fluctuations in token logits during generation but ultimately producing a more concentrated logits distribution, and activating a broader set of neurons in the final layers, indicating more extensive knowledge retrieval compared to standard prompts. Our code and data will be publicly avialable when the paper is accepted.

Figures

Figures reproduced from arXiv: 2412.03944 by the authors.

Figure 1
Figure 1. Statistical analysis of test points matches in model-generated content when using CoT. (Press et al., 2023), StrategyQA (Geva et al., 2021a), Date, and Sports (bench authors, 2023) datasets. Bamboogle is a small, handcrafted dataset containing 2-hop questions designed to test com￾monsense reasoning abilities. StrategyQA is a ques￾tion answering benchmark where the required rea￾soning steps are implicit in the questi… view at source ↗
Figure 2
Figure 2. Results of transfer test for Gemma2-27b, comparing test point overlap between model-generated content and exemplars (upper) or input questions (lower). See Figures 8, 9, and 10 for complete results. GSM8K SVAMP AQuA Sports Bamboogle DateCoin Flip Last Letter Dataset Standard Last Letter Coin Flip Date Bamboogle Sports AQuA SVAMP GSM8K Prompt 0 0 0 0 0 0 0 0 49 34 42 4 21 17 49 50 34 6 41 5 6 14 50 50 50 44 49 3 30 3… view at source ↗
Figure 3
Figure 3. Number of samples imitating exemplars (left) and also answering correctly (right) for Gemma2-9b. See [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (21 more)
Figure 4
Figure 4. Figure 4: The normalized logits value of each generated token (Gemma2-9b is reported. See Figures 15, 16, and 17 for other models and other datasets) ioral shifts does a CoT-style prompt cause? To our knowledge, no prior work has explored the internal changes in LLMs when using …
Figure 5
Figure 5. Figure 5: Kernel density estimation of normalized logits for “ the answer is ...” generated by Gemma2-9b (see Figures 18-20 for more results). 0 10 20 30 40 50 Index of test samples 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 Logits entropy COT Correct COT Wrong Standard Correct Standar…
Figure 6
Figure 6. Figure 6: Entropy of normalized logits (Gemma2-27b; see Figures 21 to 23 for complete results). ically when outputting “the answer is...”. For the CoT prompt, we obtain a probability sequence PCoT = (pthe, panswer, pis, . . .), and similarly, for the standard prompt, we obtain a…
Figure 7
Figure 7. Figure 7: Neuron activation range and intensity across the final 20 layers of Gemma2-9b (See [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Results of transfer test for different tasks under other prompts, results of Gemma2-2b is reported [PITH_FULL_IMAGE:figures/full_fig_p023_8.png]
Figure 9
Figure 9. Figure 9: Results of transfer test for different tasks under other prompts, results of Gemma2-9b is reported [PITH_FULL_IMAGE:figures/full_fig_p024_9.png]
Figure 10
Figure 10. Figure 10: Results of transfer test for different tasks under other prompts, results of LLaMA2-13b is reported [PITH_FULL_IMAGE:figures/full_fig_p025_10.png]
Figure 11
Figure 11. Figure 11: Number of samples imitating exemplars (left) and also answering correctly (right) for Gemma2- 2b. [PITH_FULL_IMAGE:figures/full_fig_p027_11.png]
Figure 12
Figure 12. Figure 12: Number of samples imitating exemplars (left) and also answering correctly (right) for Gemma2- 9b. [PITH_FULL_IMAGE:figures/full_fig_p027_12.png]
Figure 13
Figure 13. Figure 13: Number of samples imitating exemplars (left) and also answering correctly (right) for LLaMA2-13b. [PITH_FULL_IMAGE:figures/full_fig_p028_13.png]
Figure 14
Figure 14. Figure 14: Number of samples imitating exemplars (left) and also answering correctly (right) for Gemma2- 27b. [PITH_FULL_IMAGE:figures/full_fig_p028_14.png]
Figure 15
Figure 15. Figure 15: Probability value of each generated token (results of Gemma2-2b is shown). [PITH_FULL_IMAGE:figures/full_fig_p029_15.png]
Figure 16
Figure 16. Figure 16: Probability value of each generated token (the results of LLaMA2-13b is shown). [PITH_FULL_IMAGE:figures/full_fig_p030_16.png]
Figure 17
Figure 17. Figure 17: Probability value of each generated token (the results of Gemma2-27b is shown). [PITH_FULL_IMAGE:figures/full_fig_p030_17.png]
Figure 18
Figure 18. Figure 18: Kernal density for probabilities of sequence “the answer is...” (the results of Gemma2-2b is shown). [PITH_FULL_IMAGE:figures/full_fig_p031_18.png]
Figure 19
Figure 19. Figure 19: Kernal density for probabilities of sequence “the answer is...” (the results of LLaMA2-13b is shown). [PITH_FULL_IMAGE:figures/full_fig_p031_19.png]
Figure 20
Figure 20. Figure 20: Kernal density for probabilities of sequence “the answer is...” (the results of Gemma2-27b is shown). [PITH_FULL_IMAGE:figures/full_fig_p032_20.png]
Figure 21
Figure 21. Figure 21: Probability distribution entropy of answer generation step (results of Gemma2-2b is shown). [PITH_FULL_IMAGE:figures/full_fig_p033_21.png]
Figure 22
Figure 22. Figure 22: Probability distribution entropy of answer generation step (the results of Gemma2-9b is shown). [PITH_FULL_IMAGE:figures/full_fig_p033_22.png]
Figure 23
Figure 23. Figure 23: Probability distribution entropy of answer generation step (the results of LLaMA2-13b is shown). [PITH_FULL_IMAGE:figures/full_fig_p033_23.png]
Figure 24
Figure 24. Figure 24: Neuron activation range and intensity across the final 20 layers of Gemma2-27b and Gemma2-2b. [PITH_FULL_IMAGE:figures/full_fig_p034_24.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 17 canonical work pages

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  2. [2]

    BIG bench authors. 2023. https://openreview.net/forum?id=uyTL5Bvosj Beyond the imitation game: Quantifying and extrapolating the capabilities of language models . Transactions on Machine Learning Research

    2023

  3. [3]

    Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, et al. 2024. Graph of thoughts: Solving elaborate problems with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 17682--17690

    2024

  4. [4]

    Zhen Bi, Ningyu Zhang, Yinuo Jiang, Shumin Deng, Guozhou Zheng, and Huajun Chen. 2024. When do program-of-thought works for reasoning? In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 17691--17699

    2024

  5. [5]

    Boosting of thoughts: Trial-and-error problem solving with large language models

    Sijia Chen, Baochun Li, and Di Niu. Boosting of thoughts: Trial-and-error problem solving with large language models. In The Twelfth International Conference on Learning Representations

    work page

  6. [6]

    Yew Ken Chia, Guizhen Chen, Luu Anh Tuan, Soujanya Poria, and Lidong Bing. 2023. Contrastive chain-of-thought prompting. arXiv preprint arXiv:2311.09277

    arXiv 2023

  7. [7]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

    arXiv 2021

  8. [8]

    Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022 a . https://doi.org/10.18653/v1/2022.acl-long.581 Knowledge neurons in pretrained transformers . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8493--8502, Dublin, Ireland. Association for Computational Linguistics

    doi:10.18653/v1/2022.acl-long.581 2022

Show all 51 references
  1. [9]

    Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022 b . Knowledge neurons in pretrained transformers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8493--8502

    2022

  2. [10]

    Richard A Davis, Keh-Shin Lii, and Dimitris N Politis. 2011. Remarks on some nonparametric estimates of a density function. Selected Works of Murray Rosenblatt, pages 95--100

    2011

  3. [11]

    Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. 2021 a . Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies . Transactions of the Association for Computational Linguistics (TACL)

    2021

  4. [12]

    Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021 b . Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5484--5495

    2021

  5. [13]

    Arnav Gudibande, Eric Wallace, Charlie Snell, Xinyang Geng, Hao Liu, Pieter Abbeel, Sergey Levine, and Dawn Song. 2023. The false promise of imitating proprietary llms. arXiv preprint arXiv:2305.15717

    2023 arXiv

  6. [14]

    Mingyu Jin, Qinkai Yu, Haiyan Zhao, Wenyue Hua, Yanda Meng, Yongfeng Zhang, Mengnan Du, et al. 2024. The impact of reasoning step length on large language models. arXiv preprint arXiv:2401.04925

    2024 arXiv

  7. [15]

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022 a . Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199--22213

    2022

  8. [16]

    Takeshi Kojima, Shixiang (Shane) Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022 b . Large language models are zero-shot reasoners. In Advances in Neural Information Processing Systems, volume 35, pages 22199--22213

    2022

  9. [17]

    Philip R Lee, Jonathan E Cohen, Dumitru A Iacobas, Sanda Iacobas, and R Douglas Fields. 2017. Gene networks activated by specific patterns of action potentials in dorsal root ganglia neurons. Scientific Reports, 7(1):43765

    2017

  10. [18]

    Xiaonan Li and Xipeng Qiu. 2023. Mot: Memory-of-thought enables chatgpt to self-improve. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6354--6374

    2023

  11. [19]

    Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. 2017. Program induction by rationale generation: Learning to solve and explain algebraic word problems. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p...

    2017

  12. [20]

    Fuxiao Liu, Paiheng Xu, Zongxia Li, Yue Feng, and Hyemi Song. 2023. Towards understanding in-context learning with contrastive demonstrations and saliency maps. arXiv preprint arXiv:2307.05052

    2023 arXiv

  13. [21]

    Aman Madaan, Katherine Hermann, and Amir Yazdanbakhsh. 2023. What makes chain-of-thought prompting effective? a counterfactual study. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1448--1535

    2023

  14. [22]

    Aman Madaan and Amir Yazdanbakhsh. 2022. Text and patterns: For effective chain of thought, it takes two to tango. arXiv preprint arXiv:2209.07686

    2022 arXiv

  15. [23]

    William Merrill and Ashish Sabharwal. 2024. The expressive power of transformers with chain of thought. In The Twelfth International Conference on Learning Representations

    2024

  16. [24]

    AI Meta. 2024. Introducing meta llama 3: The most capable openly available llm to date. Meta AI

    2024

  17. [25]

    Franz Nowak, Anej Svete, Alexandra Butoi, and Ryan Cotterell. 2024. On the representational capacity of neural language models with chain-of-thought reasoning. arXiv preprint arXiv:2406.14197

    2024 arXiv

  18. [26]

    Sinno Jialin Pan and Qiang Yang. 2009. A survey on transfer learning. IEEE Transactions on knowledge and data engineering, 22(10):1345--1359

    2009

  19. [27]

    Emanuel Parzen. 1962. On estimation of a probability density function and mode. The annals of mathematical statistics, 33(3):1065--1076

    1962

  20. [28]

    Arkil Patel, Satwik Bhattamishra, and Navin Goyal. 2021. https://doi.org/10.18653/v1/2021.naacl-main.168 Are NLP models really able to solve simple math word problems? In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Ling...

    2021 doi

  21. [29]

    Fernando Pereira, Naftali Tishby, and Lillian Lee. 1993. https://doi.org/10.3115/981574.981598 Distributional clustering of E nglish words . In 31st Annual Meeting of the Association for Computational Linguistics, pages 183--190, Columbus, Ohio, USA. Association for Computatio...

    1993

  22. [30]

    Akshara Prabhakar, Thomas L Griffiths, and R Thomas McCoy. 2024. Deciphering the factors influencing the efficacy of chain-of-thought: Probability, memorization, and noisy reasoning. arXiv preprint arXiv:2407.01687

    2024 arXiv

  23. [31]

    Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A Smith, and Mike Lewis. 2023. Measuring and narrowing the compositionality gap in language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5687--5711

    2023

  24. [32]

    Daking Rai and Ziyu Yao. 2024. An investigation of neuron activation as a unified lens to explain chain-of-thought eliciting arithmetic reasoning of llms. arXiv preprint arXiv:2406.12288

    2024 arXiv

  25. [33]

    Abulhair Saparov and He He. 2023. https://arxiv.org/abs/2210.01240 Language models are greedy reasoners: A systematic formal analysis of chain-of-thought . Preprint, arXiv:2210.01240

    2023 arXiv

  26. [34]

    Rylan Schaeffer, Kateryna Pistunova, Samar Khanna, Sarthak Consul, and Sanmi Koyejo. 2023. Invalid logic, equivalent gains: The bizarreness of reasoning in language model prompting. arXiv preprint arXiv:2307.10573

    2023 arXiv

  27. [35]

    Sree Harsha Tanneru, Dan Ley, Chirag Agarwal, and Himabindu Lakkaraju. 2024. On the hardness of faithful chain-of-thought reasoning in large language models. arXiv preprint arXiv:2406.10625

    2024 arXiv

  28. [36]

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan...

    2024 arXiv

  29. [37]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, W...

    2023 arXiv

  30. [38]

    Changsheng Wan, Li Wang, and Vir V Phoha. 2018. A survey on gait recognition. ACM Computing Surveys (CSUR), 51(5):1--35

    2018

  31. [39]

    Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, Luke Zettlemoyer, and Huan Sun. 2023. Towards understanding chain-of-thought prompting: An empirical study of what matters. In The 61st Annual Meeting Of The Association For Computational Linguistics

    2023

  32. [40]

    Self-consistency improves chain of thought reasoning in language models

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations

  33. [41]

    Xuezhi Wang and Denny Zhou. 2024. Chain-of-thought reasoning without prompting. arXiv preprint arXiv:2402.10200

    2024 arXiv

  34. [42]

    Albert Webson and Ellie Pavlick. 2021. Do prompt-based models really understand the meaning of their prompts? arXiv preprint arXiv:2109.01247

    2021 arXiv

  35. [43]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

    2022

  36. [44]

    Skyler Wu, Eric Meng Shen, Charumathi Badrinath, Jiaqi Ma, and Himabindu Lakkaraju. 2023. Analyzing chain-of-thought prompting in large language models via gradient-based feature attributions. arXiv preprint arXiv:2307.13339

    2023 arXiv

  37. [45]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2024. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36

    2024

  38. [46]

    Xi Ye, Srinivasan Iyer, Asli Celikyilmaz, Ves Stoyanov, Greg Durrett, and Ramakanth Pasunuru. 2023. Complementary explanations for effective in-context learning. Findings of the Association for Computational Linguistics: ACL 2023

    2023

  39. [47]

    Daoan Zhang, Junming Yang, Hanjia Lyu, Zijian Jin, Yuan Yao, Mingkai Chen, and Jiebo Luo. 2024. Cocot: Contrastive chain-of-thought prompting for large multimodal models with multiple image inputs. arXiv preprint arXiv:2401.02582

    2024 arXiv

  40. [48]

    Least-to-most prompting enables complex reasoning in large language models

    Denny Zhou, Nathanael Sch \"a rli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc V Le, et al. Least-to-most prompting enables complex reasoning in large language models. In The Eleventh International Conference on Learning R...

  41. [49]

    Shinan Zou, Chao Fan, Jianbo Xiong, Chuanfu Shen, Shiqi Yu, and Jin Tang. 2024. Cross-covariate gait recognition: A benchmark. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 7855--7863

    2024

  42. [50]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  43. [51]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.