Lost in Context: Addressing Context Anxiety in Large Language Models

Eric So; Etienne Ricardez; Ifueko Igbinedion; Jillian Ross; Sertac Karaman

REVIEW 4 major objections 5 minor 35 references

Frontier reasoning models sometimes have the ability to solve a long-horizon problem and still abandon it, because they overestimate how many tokens the answer will need; the paper calls this context anxiety and shows it can be measured and

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 12:42 UTC pith:Y2CNT477

load-bearing objection A well-designed measurement and SFT study whose causal story does not survive its own Appendix E budget arithmetic, but the anxiety-filtered SFT control is the real contribution. the 4 major comments →

arxiv 2607.21616 v1 pith:Y2CNT477 submitted 2026-05-29 cs.AI

Lost in Context: Addressing Context Anxiety in Large Language Models

Ifueko Igbinedion , Jillian Ross , Etienne Ricardez , Sertac Karaman , Eric So This is my paper

classification cs.AI

keywords context anxietytoken estimation calibrationlong-horizon reasoningTower of Hanoipremature disengagementsupervised fine-tuningself-assessmentLLM reasoning failure

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that a sizable share of LLM failures on long-horizon tasks are not capability failures but self-assessment failures: the model possesses the skill, yet prematurely decides the output will exceed its token budget. It introduces a detection protocol for this 'context anxiety' and a winsorized token ratio that measures how badly models misestimate their own output length. Across Tower of Hanoi and shortest-path tasks, anxious models overestimate token needs by 24%, and fixed-effects regressions link anxiety to 15.3% lower accuracy and roughly 54% more tokens on successful solutions. The paper then shows the failure mode is behaviorally mutable: supervised fine-tuning on anxiety-free reasoning traces cuts anxiety by more than half and improves or holds accuracy, including on held-out difficulties and a second task family. If right, this means reasoning performance can improve without scaling model size—by calibrating models' estimates of their own limits.

Core claim

The central claim is that context anxiety—premature abandonment of solvable tasks with explicit statements that the final answer will be too long or infeasible to output—is a systematic, measurable failure mode distinct from genuine capability limits. It is driven, in part, by miscalibrated token estimation: models that express anxiety overestimate their token usage by about 24%, while non-anxious models underestimate by about 19%. Anxiety predicts a 15.3-percentage-point accuracy drop and a 54% increase in tokens on correct answers, with model and difficulty fixed effects accounted for. The authors further claim that fine-tuning on correct, anxiety-free reasoning traces reduces detected anx

What carries the argument

The load-bearing instrument is the winsorized token ratio T, the model's self-reported estimate of tokens needed divided by actual tokens used, which quantifies perceived versus real output cost. The detection protocol C averages judge-model scores that flag phrases like 'too long to write out' or 'exceeds my context window,' separating anxiety-driven failures from capability failures. Tower of Hanoi supplies the controlled setting: optimal solution length is known exactly (2^n−1 moves), so the ground-truth output budget is computable. The mitigation machinery is supervised fine-tuning with loss masked to reasoning traces only, on a dataset filtered to exclude any anxious trace.

Load-bearing premise

The central claim depends on every evaluated problem genuinely fitting inside the model's output budget, so that abandoning it is never a rational response; the paper asserts this, but its own Appendix E arithmetic shows the hardest Hanoi instance may need more tokens than some 64k-budget models allow.

What would settle it

Compute the true output-token requirement for each Tower of Hanoi and shortest-path instance with each model's own tokenizer, then re-estimate the anxiety coefficients on the subset of instances whose requirement is strictly below the model's output budget. Appendix E's arithmetic suggests the n=12 Hanoi instance needs about 102k tokens on some tokenizers, above a 64k budget; if the accuracy and efficiency effects disappear when those instances are excluded, the 'anxiety' label loses its ground.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

Many long-horizon failures labeled as capability limits may actually be calibration failures, so accuracy benchmarks can misattribute cause.
The 24% overestimation versus 19% underestimation gap offers a quantitative target: improving token-count calibration should reduce premature abandonment.
The fixed-effects result implies anxiety has an efficiency cost beyond accuracy: successful anxious solutions use roughly 54% more tokens.
Supervised fine-tuning on anxiety-free traces reduces anxiety by over 50% and beats an all-correct SFT control, meaning the filter, not just correctness, drives the gain.
Transfer to held-out disk counts and shortest path suggests a domain-general reasoning policy, not memorized Hanoi traces.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If miscalibration is the mechanism, then prompting interventions that give models accurate token-budget information, such as explicit remaining-token readouts, might reduce anxiety without any training; this is a testable extension the paper does not run.
The same mechanism likely underlies refusals and truncations in code generation, tool use, and long-form writing, where effort estimation is harder; the paper notes this as future work.
The 54% token inflation on successful anxious solutions suggests anxiety degrades planning efficiency, not just engagement; reducing it could yield compute savings.
A cleaner test of the 'irrational abandonment' claim would compare models on instances whose true required tokens are below versus above their budget; the anxiety effect should vanish or invert above the budget.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

A well-designed measurement and SFT study whose causal story does not survive its own Appendix E budget arithmetic, but the anxiety-filtered SFT control is the real contribution.

read the letter

Worth reading for the SFT ablation alone: they filter reasoning traces for 'context anxiety' before fine-tuning and show it beats an all-correct control, with a held-out odd-disk split and a transfer to shortest-path. That is a real, reproducible-ish result and the most solid part of the paper. The LLM-judge detector with human validation (kappa ~0.42) and the winsorized token-ratio measure are also reasonable new instruments, even if the detector's false-positive rate of 21% is not negligible.

The central causal claim, however, does not hold up. They say miscalibrated token estimation causes context anxiety, but the token estimate is elicited after generation, not at the moment of abandonment. Section 3.2 promises a mediation analysis, and none appears. And the accuracy regression (Eq. 3) is partly circular: the anxiety indicator comes from the same trace whose success it predicts. Those are not minor nitpicks; they are load-bearing.

The bigger problem is the budget premise. The paper says all 270 Hanoi problems are 'well within' output limits, but Appendix E gives per-move token costs that imply n=12 needs ~102K tokens for Claude models with a 64K cap. For those models, saying 'too long' at n=12 is rational, not anxious. The stress-test note is right. The same appendix says shortest-path output scales as 4n^3 because each move 'outputs the full grid state' — but the prompt (Appendix D.2) asks for only a sequence of moves, not grid states. So that scaling is wrong too, which matters because they use it to argue the hardest grids are near the limit.

What survives? The descriptive pattern: models produce explicit 'cannot list all moves' statements more often on harder instances, and those traces correlate with lower accuracy and longer completions on successes. That is a real behavioral phenomenon, consistent with prior work on overthinking/underthinking and calibration. But the mechanism — miscalibrated token estimation causing premature abandonment — is unsupported. The authors needed to elicit token estimates at the point of abandonment, include a proper mediation test, and fix the budget arithmetic.

This deserves a serious referee but should not be accepted as-is. My recommendation: send it to peer review with a request for major revision. Reframe as a descriptive study of anxiety-like behavior and a useful SFT filtering intervention; drop or substantially weaken the causal language; fix the budget calculations and the shortest-path token model. If the authors do that, the SFT result could be a nice contribution on its own.

Referee Report

4 major / 5 minor

Summary. The paper introduces 'context anxiety' as a behavioral failure mode in frontier LLMs: models abandon long-horizon tasks despite, the authors claim, having sufficient output-token budget, often while expressing that the required output is too long. Using Tower of Hanoi (n=2–12 disks) and a shortest-path grid task, the paper reports that (i) models expressing context anxiety overestimate token requirements by ~24%, (ii) context anxiety predicts 15.3% lower accuracy and 54% more tokens on correct answers, and (iii) supervised fine-tuning on anxiety-free reasoning traces reduces anxiety and improves or maintains accuracy. The central causal claim is that miscalibrated token estimation causes premature abandonment.

Significance. If the causal claim were established, the paper would identify a novel, behaviorally mutable failure mode distinct from raw capability limits, with a concrete intervention (filtered reasoning-trace SFT). Strengths include the use of a deterministic task with known optimal output lengths, model and difficulty fixed effects, a judge-panel detector with human validation, and an SFT design that includes an all-correct control and held-out odd-disk and cross-task evaluations. The paper also provides detailed prompts and hyperparameters, facilitating replication. However, the main causal claim is currently not tested, and the feasibility premise underlying the interpretation of the headline numbers is contradicted by the paper's own appendix arithmetic.

major comments (4)

[§4, Appendix E] The paper's central premise — that all evaluated Tower of Hanoi problems are 'well within the maximum token limits' — is contradicted by the numbers in Appendix E. For n=12, the optimal solution has 4,095 moves. Appendix E gives ~13 output tokens/move for non-Claude tokenizers and ~25 for Claude, so the final answer alone requires ~53,235 or ~102,375 tokens, before any reasoning trace. Table 6 caps DeepSeek-R1/Kimi-K2 at 60K and Claude Sonnet 3.7/4.5 at 64K. Thus the hardest instances are not within budget, especially for Claude. Since anxiety rates rise sharply with disk count, the headline 24% overestimate and the 15% accuracy penalty may partly measure rational budget-aware refusal, not miscalibrated self-doubt. The authors should re-run all analyses on the subset of instances whose minimum required output (including reasoning overhead) is provably below the model's configured output
[§3.2, §4.2] The paper promises a mediation analysis but does not deliver one. §3.2 states 'We analyze the correlation between overestimation of required tokens and context anxiety and whether miscalibration mediates the relationship between anxiety and task accuracy,' yet no mediation results appear in §4 or the appendices. Moreover, the token estimate is elicited after generation is finished, whereas the proposed mechanism requires the model to overestimate token requirements before or during abandonment. A post-hoc estimate could be a rationalization of the abandonment rather than its cause. The authors should provide a genuine mediation analysis and, ideally, a prospective estimate elicited before the model begins solving, to support the causal arrow from miscalibration to anxiety.
[§3.1, Eq. (3), Table 2] The regression of accuracy on the context-anxiety indicator is substantially circular under the paper's own classification. Anxiety-driven failures are defined as traces in which 'the model abandons the task while claiming resource insufficiency,' meaning such traces are by construction incorrect. Entering that indicator into Eq. (3) with accuracy as the dependent variable therefore partly measures the annotation rule rather than a behavioral effect. The coefficient of -0.153 may mostly reflect that abandoned responses are scored as failures. The authors should report analyses that separate the act of abandonment from the expression of anxiety — for example, restricting the accuracy regression to completed responses, or using a continuous anxiety score from the judge panel and showing that it predicts accuracy among non-abandoned traces.
[Appendix D] The shortest-path token-scaling analysis is inconsistent with the task prompt. Appendix D.2 instructs the model to output only lines of the form 'Move {DIRECTION}' in the final answer. Yet Appendix D.3 estimates output length as 4n^3 tokens by assuming that 'each move step outputs the full n×n grid state.' Under the actual prompt, a 12×12 grid requires at most 24 moves, i.e., only on the order of 100 tokens, not ~7,000. The appendix's claim that the largest shortest-path instances are 'large enough for models to perceive proximity to their output limit' is therefore unsupported, and the generalization evidence in Appendix D does not measure output-length anxiety as described.

minor comments (5)

[Throughout] Typos and grammar: 'Adressing' in the running header; 'winsorization at the 1th percentile' should be '1st'; 'context anxiety operates hurts performance' should be 'operates to hurt performance' or similar; 'a the shortest-path task' in the Limitations paragraph.
[§5, Limitations] The Limitations paragraph says qualitatively similar patterns are observed 'as illustrated in Figure 2,' but Figure 2 is the SFT pipeline diagram. The relevant shortest-path results appear in Appendix D, Figure 6. The citation should be corrected.
[§4.3, Table 2] The 54% efficiency increase is stated as derived from the regression intercept (footnote 2), but the baseline mean completion-token count is not reported, so the reader cannot verify the percentage. Please report the implied baseline mean and the full regression table for completion tokens.
[Appendix F] The detector validation reports κhuman=0.48 and κhuman vs. LLM=0.42, with 21.2% false positives and 20.0% false negatives on a 50-trace sample. These are modest agreement levels for a measure that carries the paper's main conclusions. The authors should discuss the potential impact of this misclassification on the reported regression coefficients, or provide a larger validation set.
[Appendix D.3] Even aside from the grid-state issue, the derivation 'tokens ≈ (2n)^3' appears to have a units error: it multiplies 'chars per row × rows × moves' but each move would output one grid state, so the total tokens should be (moves) × (2n^2) = 4n^3 rather than (2n)^3. The table values match 4n^3, so the inline formula should be corrected to avoid confusion.

Circularity Check

4 steps flagged

The headline accuracy and calibration effects are substantially built into the paper's own labels and measures; Appendix E's token arithmetic also contradicts the 'sufficient tokens' premise that defines context anxiety.

specific steps

self definitional [§3.1 Detection Protocol; §4.3 Eq. (3) and Table 2]
"Anxiety-driven failures: The model abandons the task while claiming resource insufficiency ... 'cannot list all moves in my response' ... We construct the context anxiety indicator Cidm through semantic analysis ... context anxiety reduces accuracy by 15.3% (p < 0.01)"

The anxiety label and the accuracy outcome are read from the same response. A trace that says 'cannot list all moves in my response' and then stops is, by the protocol, both an anxiety-driven failure and an accuracy failure; the detection prompt counts exactly such statements. Regressing Y on C therefore partly restates the annotation rule (explicit admission of not completing ⇒ failure) rather than estimating an independent behavioral effect. The 15% penalty is not identified from a separate predictor; it is largely the classifier's label semantics.
self definitional [§1 definition; §4 first paragraph; Appendix E]
"context anxiety – when models worry about their ability to solve the task within their token limits, despite having sufficient tokens to complete them. ... All problems in our test set remain well within the maximum token limits of modern frontier models (typically 60K–128K tokens). ... Each move occupies approximately 13 output tokens for non-Claude tokenizers and approximately 25 output tokens for Claude tokenizers."

The definition of the phenomenon contains the key premise (sufficient tokens), but the detector never verifies it. For the n=12 Hanoi instance (4,095 moves), the paper's own per-move estimate implies 4,095×25 ≈ 102,375 output tokens for Claude models, above the 64,000 budget, and 4,095×13 ≈ 53,235 for non-Claude models before any reasoning trace, near or above the 60,000 budget. The hardest cells in the anxiety-rate, calibration, accuracy, and token-efficiency analyses therefore include rational budget-aware refusals that the definition excludes. The 'prediction' that anxiety causes failure despite sufficient budget is assumed by labeling those cells anxious rather than tested.
self definitional [§3.2 Calibration Measurement, Eq. (2)]
"T = Estimated tokens required / Actual tokens used ... Estimated token usage is elicited by prompting the same model to provide a numeric estimation of how many tokens it believes were required to complete the solution after generation is finished. Actual token counts are computed from the model's respective tokenizer."

For an anxious trace, 'actual tokens used' is the short, abandoned response, while the estimate is for the tokens 'required to complete the solution'—i.e., the full 2^n−1-move answer. The denominator is therefore small precisely when the numerator refers to the complete solution, making T > 1 (overestimation) near-mechanical for abandoned traces. The headline 'models with context anxiety overestimate by 24%' is thus partly an artifact of comparing full-solution estimates against partial outputs, not independent evidence that the model's token estimate is miscalibrated.
fitted input called prediction [§3.3 Behavioral Adaptation; §4.4]
"For each prompt, we filter responses to include only those that successfully complete the task and exhibit no detected context anxiety ... We then fine-tune the student model using standard supervised fine tuning on the filtered dataset. ... After fine tuning, the adapted model exhibits substantially less context anxiety compared to the baseline model."

The training set contains zero anxiety traces by construction, so the post-fine-tuning anxiety reduction is a direct outcome of next-token imitation of the filtered distribution, not an independent test of 'behavioral mutability'. The interesting parts are the held-out and all-correct controls; the headline 'reduces context anxiety by over 50%' is statistically forced by the filtering step itself.

full rationale

The paper does not rely on load-bearing self-citations or imported uniqueness theorems; its references are standard and external. The central problem is internal to how the constructs are operationalized. First, the 'context anxiety' label and the accuracy outcome are derived from the same trace: a trace that says it cannot list all moves and then stops is simultaneously classified as anxiety-driven and as a failure, so the regression coefficient of −0.153 partly encodes the annotation rule rather than a discovered behavioral effect. Second, the definition of anxiety includes 'despite having sufficient tokens to complete them,' but the detection protocol never checks budget sufficiency; Appendix E's own arithmetic shows the hardest Hanoi cells approach or exceed the stated output budgets, so those cells are mislabeled as anxiety under the paper's own definition. Third, the calibration ratio T compares estimated tokens 'required to complete the solution' with actual tokens used; for abandoned traces the denominator is a short partial output, mechanically inflating T and producing the reported 24% overestimation. Fourth, fine-tuning on filtered anxiety-free traces trivially reduces anxiety on the training distribution; only the held-out odd-disk and shortest-path transfer results provide independent evidence. Because the headline quantitative claims (24% overestimation, 15% accuracy penalty, 54% token inefficiency) are substantially entangled with these definitions and measurement choices, but the paper does contain genuinely independent transfer and ablation evidence, the appropriate finding is partial circularity rather than full reduction of the derivation to its inputs.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

No new physical or architectural entities are introduced; 'context anxiety' is a behavioral label for existing model outputs, operationalized by the detection prompt, so it has no independent falsifiable handle beyond the paper's own classifier. The free parameters and domain assumptions above are the real load-bearing choices the paper makes.

free parameters (5)

Context-anxiety detection threshold = 0.5 mean judge score
Appendix F.2 sets aggregation threshold at 0.5; changing aggregation changes labels, and human agreement is only κ=0.42.
Winsorization percentiles = 1st/99th
Eq (2), §3.2; arbitrary robust-statistics choice that affects the token ratio but not conclusions.
Per-move output token cost = 13 (non-Claude), 25 (Claude) tokens/move
Appendix E introduces these hand-estimated constants to argue all problems fit within budgets; for Claude n=12 they contradict that argument.
Shortest-path token scaling constant = 4n^3 tokens (≈ one token per character)
Appendix D.3 assumes each character is one token and each move prints a full grid, despite the D.2 prompt requiring only 'Move {DIRECTION}' lines; this multiplier drives the 'perceived proximity to limit' rationale.
SFT hyperparameters = 4 epochs, LR 1e-4, warmup 0.3
Appendix C; standard training choices, not fitted to the central claims.

axioms (5)

standard math Optimal Tower of Hanoi solution requires exactly 2^n - 1 moves
Used in §4 to define difficulty and compute required output lengths; true for 3 pegs under the stated rules.
domain assumption LLM judge panel labels are a valid operationalization of 'context anxiety'
§3.1 defines anxiety by judge-detected phrases; human agreement is modest (κ=0.42, 21.2% false positives), so the construct validity is weak but not absent.
ad hoc to paper A post-hoc self-estimate of tokens used measures the same quantity as the anticipated token requirement that drives abandonment
Eq (2) and §4.2; the prompt in Appendix A.2 asks for the cumulative token count of the provided (often truncated) final answer, not an a priori estimate of the full solution's length. The causal story depends on this equivalence, which is unverified.
domain assumption All evaluated problems are within every model's output budget
Stated in §4; contradicted by Appendix E's token-per-move estimates for Claude at n=12 (≈102k vs 64k). If false, apparent 'anxiety' at high difficulty is partly rational budget-awareness.
domain assumption Reasoning traces faithfully express the model's internal state
The detection protocol depends on explicit self-reports; the paper acknowledges this in Limitations, and cites Turpin et al. showing CoT can be unfaithful.

pith-pipeline@v1.3.0-alltime-deepseek · 16104 in / 17756 out tokens · 175526 ms · 2026-08-02T12:42:34.648583+00:00 · methodology

0 comments

read the original abstract

Conventional wisdom suggests that reasoning models fail when problems exceed their capabilities. However, we find that frontier reasoning models sometimes possess the necessary capabilities to solve problems but fail due to premature self-doubt -- a phenomenon informally known as context anxiety. We provide the first systematic study of context anxiety, demonstrating that it arises, in part, from a model's inability to accurately estimate the tokens required to complete a task. We also show that context anxiety leads to material efficiency losses when models operate under perceived constraints. Building on this analysis, we further show that models can learn alternative strategies for solving long-horizon problems without exhibiting context anxiety, suggesting that performance improvements may be achievable not through scaling model capabilities, but by improving models' ability to accurately assess and adapt to their own limitations.

Figures

Figures reproduced from arXiv: 2607.21616 by Eric So, Etienne Ricardez, Ifueko Igbinedion, Jillian Ross, Sertac Karaman.

**Figure 1.** Figure 1: Context anxiety stems from miscalibrated token estimation and creates a reduction in performance and efficiency (a), but context anxiety can be reduced through supervised fine tuning on reasoning traces without anxiety (b). because they believe they do. This raises a critical question: do models fail because problems are genuinely too complex, or because they believe they are? When faced with challenging … view at source ↗

**Figure 2.** Figure 2: Mitigating context anxiety via selective reasoning fine-tuning. Our pipeline consists of three steps. Step 1 (Data Generation): Frontier teacher models generate candidate solutions—comprising prompts, reasoning traces, and final solutions—for augmented Tower of Hanoi tasks. Step 2 (Data Filtering): A panel of LLMs evaluates the generated reasoning to detect context anxiety. Responses exhibiting anxiety are… view at source ↗

**Figure 3.** Figure 3: As task complexity increases, frontier models generally struggle to successfully complete the Tower of Hanoi. To distinguish between failures due to disengagement and failures due to genuine problem solving challenges, we further classify unsuccessful attempts into anxiety-driven and capability driven failures. This breakdown allows us to verify that many failures at higher disk counts are attributable t… view at source ↗

**Figure 4.** Figure 4: Impact of context anxiety on token usage. The results strongly support our hypothesis. As shown in Figure 4b, models that exhibit context anxiety overestimate their token requirements by approximately 24%. In contrast, models that attempt problems without expressing context anxiety underestimate their token requirements by 19%. The difference in estimation error between models with and without context anxi… view at source ↗

**Figure 5.** Figure 5: Selective reasoning fine-tuning mitigates context anxiety and expands reasoning limits. This analysis replaces subjective failure classifications with objective token and accuracy metrics. While baseline models frequently abandon long-horizon tasks, our selectively fine-tuned model demonstrates persistent reasoning chains prior to failure. Furthermore, the substantial accuracy drops observed when traini… view at source ↗

**Figure 6.** Figure 6: a shows per-model accuracy partitioned by whether context anxiety was detected in the trace; Figure 6b shows the analogous partition for completion tokens on correct solutions. The direction, magnitude, and per-model heterogeneity of the effects mirror those observed on Tower of Hanoi: traces exhibiting anxious self-reports are associated with markedly lower accuracy and with systematically shorter complet… view at source ↗

discussion (0)

Reference graph

Works this paper leans on

35 extracted references · 17 linked inside Pith

[1]

arXiv preprint arXiv:2506.06941 , year=

The illusion of thinking: Understanding the strengths and limitations of reasoning models via the lens of problem complexity , author=. arXiv preprint arXiv:2506.06941 , year=

Pith/arXiv arXiv
[2]

International Conference on Innovative Techniques and Applications of Artificial Intelligence , pages=

Rethinking the illusion of thinking , author=. International Conference on Innovative Techniques and Applications of Artificial Intelligence , pages=. 2025 , organization=

2025
[3]

Wang, Boshi and Min, Sewon and Deng, Xiang and Shen, Jiaming and Wu, You and Zettlemoyer, Luke and Sun, Huan , editor =. Towards. Proceedings of the 61st. 2023 , pages =. doi:10.18653/v1/2023.acl-long.153 , urldate =

doi:10.18653/v1/2023.acl-long.153 2023
[4]

Advances in Neural Information Processing Systems , author =

Chain-of-. Advances in Neural Information Processing Systems , author =. 2022 , pages =

2022
[5]

Complexity-

Fu, Yao and Peng, Hao and Sabharwal, Ashish and Clark, Peter and Khot, Tushar , month = jan, year =. Complexity-. doi:10.48550/arXiv.2210.00720 , urldate =

Pith/arXiv arXiv doi:10.48550/arxiv.2210.00720
[6]

Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: long papers) , author =

Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions , url =. Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: long papers) , author =. 2023 , pages =

2023
[7]

Ho, Namgyu and Schmid, Laura and Yun, Se-Young , editor =. Large. Proceedings of the 61st. 2023 , pages =. doi:10.18653/v1/2023.acl-long.830 , urldate =

doi:10.18653/v1/2023.acl-long.830 2023
[8]

Wang, Peiyi and Li, Lei and Chen, Liang and Song, Feifan and Lin, Binghuai and Cao, Yunbo and Liu, Tianyu and Sui, Zhifang , month = sep, year =. Making. doi:10.48550/arXiv.2309.02144 , urldate =

Pith/arXiv arXiv doi:10.48550/arxiv.2309.02144
[9]

Wang, Xuezhi and Wei, Jason and Schuurmans, Dale and Le, Quoc and Chi, Ed and Narang, Sharan and Chowdhery, Aakanksha and Zhou, Denny , month = mar, year =. Self-. doi:10.48550/arXiv.2203.11171 , urldate =

Pith/arXiv arXiv doi:10.48550/arxiv.2203.11171
[10]

Levy, Mosh and Jacoby, Alon and Goldberg, Yoav , month = jul, year =. Same. doi:10.48550/arXiv.2402.14848 , urldate =

Pith/arXiv arXiv doi:10.48550/arxiv.2402.14848
[11]

Sui, Yang and Chuang, Yu-Neng and Wang, Guanchu and Zhang, Jiamu and Zhang, Tianyi and Yuan, Jiayi and Liu, Hongyi and Wen, Andrew and Zhong, Shaochen and Zou, Na and Chen, Hanjie and Hu, Xia , month = aug, year =. Stop. doi:10.48550/arXiv.2503.16419 , urldate =

Pith/arXiv arXiv doi:10.48550/arxiv.2503.16419
[12]

Language

Kadavath, Saurav and Conerly, Tom and Askell, Amanda and Henighan, Tom and Drain, Dawn and Perez, Ethan and Schiefer, Nicholas and Hatfield-Dodds, Zac and DasSarma, Nova and Tran-Johnson, Eli and Johnston, Scott and El-Showk, Sheer and Jones, Andy and Elhage, Nelson and Hume, Tristan and Chen, Anna and Bai, Yuntao and Bowman, Sam and Fort, Stanislav and G...

Pith/arXiv arXiv doi:10.48550/arxiv.2207.05221
[13]

Yin, Zhangyue and Sun, Qiushi and Guo, Qipeng and Wu, Jiawen and Qiu, Xipeng and Huang, Xuanjing , month = may, year =. Do. doi:10.48550/arXiv.2305.18153 , urldate =

Pith/arXiv arXiv doi:10.48550/arxiv.2305.18153
[14]

Advances in Neural Information Processing Systems , author =

Large. Advances in Neural Information Processing Systems , author =. 2024 , pages =. doi:10.52202/079017-2729 , language =

doi:10.52202/079017-2729 2024
[15]

and Lin, Kevin and Hewitt, John and Paranjape, Ashwin and Bevilacqua, Michele and Petroni, Fabio and Liang, Percy , year =

Liu, Nelson F. and Lin, Kevin and Hewitt, John and Paranjape, Ashwin and Bevilacqua, Michele and Petroni, Fabio and Liang, Percy , year =. Lost in the. Transactions of the Association for Computational Linguistics , publisher =. doi:10.1162/tacl_a_00638 , urldate =

doi:10.1162/tacl_a_00638
[16]

Long-context

Li, Tianle and Zhang, Ge and Do, Quy Duc and Yue, Xiang and Chen, Wenhu , month = jun, year =. Long-context. doi:10.48550/arXiv.2404.02060 , urldate =

Pith/arXiv arXiv doi:10.48550/arxiv.2404.02060
[17]

Least-to-

Zhou, Denny and Schärli, Nathanael and Hou, Le and Wei, Jason and Scales, Nathan and Wang, Xuezhi and Schuurmans, Dale and Cui, Claire and Bousquet, Olivier and Le, Quoc and Chi, Ed , month = apr, year =. Least-to-. doi:10.48550/arXiv.2205.10625 , urldate =

Pith/arXiv arXiv doi:10.48550/arxiv.2205.10625
[18]

Advances in Neural Information Processing Systems , author =

Large. Advances in Neural Information Processing Systems , author =. 2022 , pages =

2022
[19]

Li, Yifei and Lin, Zeqi and Zhang, Shizhuo and Fu, Qiang and Chen, Bei and Lou, Jian-Guang and Chen, Weizhu , editor =. Making. Proceedings of the 61st. 2023 , pages =. doi:10.18653/v1/2023.acl-long.291 , urldate =

doi:10.18653/v1/2023.acl-long.291 2023
[20]

and Hashimoto, Tatsunori and Vinyals, Oriol and Liang, Percy and Dean, Jeff and Fedus, William , month = oct, year =

Wei, Jason and Tay, Yi and Bommasani, Rishi and Raffel, Colin and Zoph, Barret and Borgeaud, Sebastian and Yogatama, Dani and Bosma, Maarten and Zhou, Denny and Metzler, Donald and Chi, Ed H. and Hashimoto, Tatsunori and Vinyals, Oriol and Liang, Percy and Dean, Jeff and Fedus, William , month = oct, year =. Emergent. doi:10.48550/arXiv.2206.07682 , urldate =

Pith/arXiv arXiv doi:10.48550/arxiv.2206.07682
[21]

Advances in Neural Information Processing Systems , author =

Are. Advances in Neural Information Processing Systems , author =. 2023 , pages =

2023
[22]

Lu, Sheng and Bigoulaeva, Irina and Sachdeva, Rachneet and Tayyar Madabushi, Harish and Gurevych, Iryna , editor =. Are. Proceedings of the 62nd. 2024 , pages =. doi:10.18653/v1/2024.acl-long.279 , urldate =

doi:10.18653/v1/2024.acl-long.279 2024
[23]

Emergent

Berti, Leonardo and Giorgi, Flavio and Kasneci, Gjergji , month = mar, year =. Emergent. doi:10.48550/arXiv.2503.05788 , urldate =

Pith/arXiv arXiv doi:10.48550/arxiv.2503.05788
[24]

and Lu, Hongjing , month = sep, year =

Webb, Taylor and Holyoak, Keith J. and Lu, Hongjing , month = sep, year =. Emergent analogical reasoning in large language models , volume =. Nature Human Behaviour , publisher =. doi:10.1038/s41562-023-01659-w , language =

doi:10.1038/s41562-023-01659-w
[25]

arXiv preprint arXiv:2501.12948 , year=

Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning , author=. arXiv preprint arXiv:2501.12948 , year=

Pith/arXiv arXiv
[26]

arXiv preprint arXiv:2507.06261 , year=

Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities , author=. arXiv preprint arXiv:2507.06261 , year=

Pith/arXiv arXiv
[27]

Advances in Neural Information Processing Systems , author =

Language. Advances in Neural Information Processing Systems , author =. 2023 , pages =

2023
[28]

Errica, Federico and Sanvito, Davide and Siracusano, Giuseppe and Bifulco, Roberto , editor =. What. Proceedings of the 2025. 2025 , pages =. doi:10.18653/v1/2025.naacl-long.73 , urldate =

doi:10.18653/v1/2025.naacl-long.73 2025
[29]

Zhou, Han and Wan, Xingchen and Proleev, Lev and Mincu, Diana and Chen, Jilin and Heller, Katherine and Roy, Subhrajit , month = dec, year =. Batch. doi:10.48550/arXiv.2309.17249 , urldate =

Pith/arXiv arXiv doi:10.48550/arxiv.2309.17249
[30]

Claude 4.5 Sonnet , year =
[31]

Advances in Neural Information Processing Systems , author =

Response. Advances in Neural Information Processing Systems , author =. 2023 , pages =

2023
[32]

Su, Jinyan and Healey, Jennifer and Nakov, Preslav and Cardie, Claire , month = apr, year =. Between. doi:10.48550/arXiv.2505.00127 , abstract =

Pith/arXiv arXiv doi:10.48550/arxiv.2505.00127
[33]

Yang, Wenkai and Ma, Shuming and Lin, Yankai and Wei, Furu , month = oct, year =. Towards. doi:10.48550/arXiv.2502.18080 , urldate =

doi:10.48550/arxiv.2502.18080
[34]

Rebuilding Devin for Claude Sonnet 4.5: Lessons and Challenges , date =

Marcu, Theodor and. Rebuilding Devin for Claude Sonnet 4.5: Lessons and Challenges , date =
[35]

OpenAI and Agarwal, Sandhini and Ahmad, Lama and Ai, Jason and Altman, Sam and Applebaum, Andy and Arbus, Edwin and Arora, Rahul K. and Bai, Yu and Baker, Bowen and Bao, Haiming and Barak, Boaz and Bennett, Ally and Bertao, Tyler and Brett, Nivedita and Brevdo, Eugene and Brockman, Greg and Bubeck, Sebastien and Chang, Che and Chen, Kai and Chen, Mark and...

Pith/arXiv arXiv doi:10.48550/arxiv.2508.10925

[1] [1]

arXiv preprint arXiv:2506.06941 , year=

The illusion of thinking: Understanding the strengths and limitations of reasoning models via the lens of problem complexity , author=. arXiv preprint arXiv:2506.06941 , year=

Pith/arXiv arXiv

[2] [2]

International Conference on Innovative Techniques and Applications of Artificial Intelligence , pages=

Rethinking the illusion of thinking , author=. International Conference on Innovative Techniques and Applications of Artificial Intelligence , pages=. 2025 , organization=

2025

[3] [3]

Wang, Boshi and Min, Sewon and Deng, Xiang and Shen, Jiaming and Wu, You and Zettlemoyer, Luke and Sun, Huan , editor =. Towards. Proceedings of the 61st. 2023 , pages =. doi:10.18653/v1/2023.acl-long.153 , urldate =

doi:10.18653/v1/2023.acl-long.153 2023

[4] [4]

Advances in Neural Information Processing Systems , author =

Chain-of-. Advances in Neural Information Processing Systems , author =. 2022 , pages =

2022

[5] [5]

Complexity-

Fu, Yao and Peng, Hao and Sabharwal, Ashish and Clark, Peter and Khot, Tushar , month = jan, year =. Complexity-. doi:10.48550/arXiv.2210.00720 , urldate =

Pith/arXiv arXiv doi:10.48550/arxiv.2210.00720

[6] [6]

Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: long papers) , author =

Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions , url =. Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: long papers) , author =. 2023 , pages =

2023

[7] [7]

Ho, Namgyu and Schmid, Laura and Yun, Se-Young , editor =. Large. Proceedings of the 61st. 2023 , pages =. doi:10.18653/v1/2023.acl-long.830 , urldate =

doi:10.18653/v1/2023.acl-long.830 2023

[8] [8]

Wang, Peiyi and Li, Lei and Chen, Liang and Song, Feifan and Lin, Binghuai and Cao, Yunbo and Liu, Tianyu and Sui, Zhifang , month = sep, year =. Making. doi:10.48550/arXiv.2309.02144 , urldate =

Pith/arXiv arXiv doi:10.48550/arxiv.2309.02144

[9] [9]

Wang, Xuezhi and Wei, Jason and Schuurmans, Dale and Le, Quoc and Chi, Ed and Narang, Sharan and Chowdhery, Aakanksha and Zhou, Denny , month = mar, year =. Self-. doi:10.48550/arXiv.2203.11171 , urldate =

Pith/arXiv arXiv doi:10.48550/arxiv.2203.11171

[10] [10]

Levy, Mosh and Jacoby, Alon and Goldberg, Yoav , month = jul, year =. Same. doi:10.48550/arXiv.2402.14848 , urldate =

Pith/arXiv arXiv doi:10.48550/arxiv.2402.14848

[11] [11]

Sui, Yang and Chuang, Yu-Neng and Wang, Guanchu and Zhang, Jiamu and Zhang, Tianyi and Yuan, Jiayi and Liu, Hongyi and Wen, Andrew and Zhong, Shaochen and Zou, Na and Chen, Hanjie and Hu, Xia , month = aug, year =. Stop. doi:10.48550/arXiv.2503.16419 , urldate =

Pith/arXiv arXiv doi:10.48550/arxiv.2503.16419

[12] [12]

Language

Kadavath, Saurav and Conerly, Tom and Askell, Amanda and Henighan, Tom and Drain, Dawn and Perez, Ethan and Schiefer, Nicholas and Hatfield-Dodds, Zac and DasSarma, Nova and Tran-Johnson, Eli and Johnston, Scott and El-Showk, Sheer and Jones, Andy and Elhage, Nelson and Hume, Tristan and Chen, Anna and Bai, Yuntao and Bowman, Sam and Fort, Stanislav and G...

Pith/arXiv arXiv doi:10.48550/arxiv.2207.05221

[13] [13]

Yin, Zhangyue and Sun, Qiushi and Guo, Qipeng and Wu, Jiawen and Qiu, Xipeng and Huang, Xuanjing , month = may, year =. Do. doi:10.48550/arXiv.2305.18153 , urldate =

Pith/arXiv arXiv doi:10.48550/arxiv.2305.18153

[14] [14]

Advances in Neural Information Processing Systems , author =

Large. Advances in Neural Information Processing Systems , author =. 2024 , pages =. doi:10.52202/079017-2729 , language =

doi:10.52202/079017-2729 2024

[15] [15]

and Lin, Kevin and Hewitt, John and Paranjape, Ashwin and Bevilacqua, Michele and Petroni, Fabio and Liang, Percy , year =

Liu, Nelson F. and Lin, Kevin and Hewitt, John and Paranjape, Ashwin and Bevilacqua, Michele and Petroni, Fabio and Liang, Percy , year =. Lost in the. Transactions of the Association for Computational Linguistics , publisher =. doi:10.1162/tacl_a_00638 , urldate =

doi:10.1162/tacl_a_00638

[16] [16]

Long-context

Li, Tianle and Zhang, Ge and Do, Quy Duc and Yue, Xiang and Chen, Wenhu , month = jun, year =. Long-context. doi:10.48550/arXiv.2404.02060 , urldate =

Pith/arXiv arXiv doi:10.48550/arxiv.2404.02060

[17] [17]

Least-to-

Zhou, Denny and Schärli, Nathanael and Hou, Le and Wei, Jason and Scales, Nathan and Wang, Xuezhi and Schuurmans, Dale and Cui, Claire and Bousquet, Olivier and Le, Quoc and Chi, Ed , month = apr, year =. Least-to-. doi:10.48550/arXiv.2205.10625 , urldate =

Pith/arXiv arXiv doi:10.48550/arxiv.2205.10625

[18] [18]

Advances in Neural Information Processing Systems , author =

Large. Advances in Neural Information Processing Systems , author =. 2022 , pages =

2022

[19] [19]

Li, Yifei and Lin, Zeqi and Zhang, Shizhuo and Fu, Qiang and Chen, Bei and Lou, Jian-Guang and Chen, Weizhu , editor =. Making. Proceedings of the 61st. 2023 , pages =. doi:10.18653/v1/2023.acl-long.291 , urldate =

doi:10.18653/v1/2023.acl-long.291 2023

[20] [20]

and Hashimoto, Tatsunori and Vinyals, Oriol and Liang, Percy and Dean, Jeff and Fedus, William , month = oct, year =

Wei, Jason and Tay, Yi and Bommasani, Rishi and Raffel, Colin and Zoph, Barret and Borgeaud, Sebastian and Yogatama, Dani and Bosma, Maarten and Zhou, Denny and Metzler, Donald and Chi, Ed H. and Hashimoto, Tatsunori and Vinyals, Oriol and Liang, Percy and Dean, Jeff and Fedus, William , month = oct, year =. Emergent. doi:10.48550/arXiv.2206.07682 , urldate =

Pith/arXiv arXiv doi:10.48550/arxiv.2206.07682

[21] [21]

Advances in Neural Information Processing Systems , author =

Are. Advances in Neural Information Processing Systems , author =. 2023 , pages =

2023

[22] [22]

Lu, Sheng and Bigoulaeva, Irina and Sachdeva, Rachneet and Tayyar Madabushi, Harish and Gurevych, Iryna , editor =. Are. Proceedings of the 62nd. 2024 , pages =. doi:10.18653/v1/2024.acl-long.279 , urldate =

doi:10.18653/v1/2024.acl-long.279 2024

[23] [23]

Emergent

Berti, Leonardo and Giorgi, Flavio and Kasneci, Gjergji , month = mar, year =. Emergent. doi:10.48550/arXiv.2503.05788 , urldate =

Pith/arXiv arXiv doi:10.48550/arxiv.2503.05788

[24] [24]

and Lu, Hongjing , month = sep, year =

Webb, Taylor and Holyoak, Keith J. and Lu, Hongjing , month = sep, year =. Emergent analogical reasoning in large language models , volume =. Nature Human Behaviour , publisher =. doi:10.1038/s41562-023-01659-w , language =

doi:10.1038/s41562-023-01659-w

[25] [25]

arXiv preprint arXiv:2501.12948 , year=

Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning , author=. arXiv preprint arXiv:2501.12948 , year=

Pith/arXiv arXiv

[26] [26]

arXiv preprint arXiv:2507.06261 , year=

Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities , author=. arXiv preprint arXiv:2507.06261 , year=

Pith/arXiv arXiv

[27] [27]

Advances in Neural Information Processing Systems , author =

Language. Advances in Neural Information Processing Systems , author =. 2023 , pages =

2023

[28] [28]

Errica, Federico and Sanvito, Davide and Siracusano, Giuseppe and Bifulco, Roberto , editor =. What. Proceedings of the 2025. 2025 , pages =. doi:10.18653/v1/2025.naacl-long.73 , urldate =

doi:10.18653/v1/2025.naacl-long.73 2025

[29] [29]

Zhou, Han and Wan, Xingchen and Proleev, Lev and Mincu, Diana and Chen, Jilin and Heller, Katherine and Roy, Subhrajit , month = dec, year =. Batch. doi:10.48550/arXiv.2309.17249 , urldate =

Pith/arXiv arXiv doi:10.48550/arxiv.2309.17249

[30] [30]

Claude 4.5 Sonnet , year =

[31] [31]

Advances in Neural Information Processing Systems , author =

Response. Advances in Neural Information Processing Systems , author =. 2023 , pages =

2023

[32] [32]

Su, Jinyan and Healey, Jennifer and Nakov, Preslav and Cardie, Claire , month = apr, year =. Between. doi:10.48550/arXiv.2505.00127 , abstract =

Pith/arXiv arXiv doi:10.48550/arxiv.2505.00127

[33] [33]

Yang, Wenkai and Ma, Shuming and Lin, Yankai and Wei, Furu , month = oct, year =. Towards. doi:10.48550/arXiv.2502.18080 , urldate =

doi:10.48550/arxiv.2502.18080

[34] [34]

Rebuilding Devin for Claude Sonnet 4.5: Lessons and Challenges , date =

Marcu, Theodor and. Rebuilding Devin for Claude Sonnet 4.5: Lessons and Challenges , date =

[35] [35]

OpenAI and Agarwal, Sandhini and Ahmad, Lama and Ai, Jason and Altman, Sam and Applebaum, Andy and Arbus, Edwin and Arora, Rahul K. and Bai, Yu and Baker, Bowen and Bao, Haiming and Barak, Boaz and Bennett, Ally and Bertao, Tyler and Brett, Nivedita and Brevdo, Eugene and Brockman, Greg and Bubeck, Sebastien and Chang, Che and Chen, Kai and Chen, Mark and...

Pith/arXiv arXiv doi:10.48550/arxiv.2508.10925