Pith. sign in

REVIEW 3 major objections 5 minor 64 references

Organizing model reasoning into alternating scratch and conclusion blocks improves math accuracy and lets the model drop most of its working memory.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 12:07 UTC pith:7PUVEEE7

load-bearing objection Solid empirical SFT recipe with real accuracy gains and a usable pruning hook; the main soft spot is that 70B-generated outcomes are not fully isolated from pure format effects. the 3 major comments →

arxiv 2607.10386 v1 pith:7PUVEEE7 submitted 2026-07-11 cs.CL cs.AI

Structured Thoughts For Improved Reasoning And Context Pruning

Zain Sarwar , Supriyo Chakraborty , Berkcan Kapusuzoglu , Chia-Hsuan Lee , Anirban Das , Stephen Rawls , Kartik Balasubramaniam , Sambit Sahu This is my paper
classification cs.CL cs.AI
keywords structured thoughtschain-of-thoughtsupervised fine-tuningcontext pruningreasoning modelsKV-cachetry-outcome blocks
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Long chains of thought help language models solve hard math and logic problems, but they fill memory with verbose scratch work that later steps may not need. This paper shows that rewriting those traces as alternating blocks (exploratory algebra and checks) and blocks (one- or two-sentence conclusions) and fine-tuning on the reformatted data produces models that reason more accurately than the same models trained on the original free-form traces. The same format then lets the system prune every completed once its is written, keeping only the distilled conclusions in context. On two 7–8B reasoning models the structured fine-tune yields relative gains up to 8 percent; a proof-of-concept prune then reclaims roughly 85 percent of context and KV-cache memory while costing about 9 percent accuracy.

Core claim

Supervised fine-tuning on the same million-example math corpus reformatted into explicit <try>/<outcome> pairs improves average benchmark accuracy by 3.66 percent on Llama-Nemotron-8B and 8.08 percent on Qwen2.5-7B relative to ordinary SFT, and the resulting structure permits inference-time deletion of finished scratch blocks that saves about 85 percent of context length at an average 8.7 percent accuracy cost.

What carries the argument

The <try>/<outcome> alternation: each reasoning step is forced into a scratch-work block followed by a short distilled conclusion; after the conclusion is written, attention is masked (or the tokens deleted) so later steps see only the outcomes.

Load-bearing premise

The method assumes that simply splitting free-form traces at words like “Wait” or “Hmm” and asking a larger model to summarize each piece yields conclusions that later steps can rely on once the original scratch text is gone.

What would settle it

Train and evaluate the same models on an identical corpus whose step boundaries are produced by a stronger supervised segmenter or by human annotation; if the accuracy lift disappears or the prune-time drop becomes catastrophic, the claim that the structure itself (rather than the particular heuristic) is doing the work is falsified.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Structured Thoughts, reformatting free-form reasoning traces into alternating <try> (exploratory scratch work) and <outcome> (distilled conclusion) blocks. Dataset construction segments existing Llama-Nemotron Math v1.1 traces at decision cues (e.g., “Wait,” “Hmm”) and uses Llama-70B-Instruct to generate each <outcome>. Supervised fine-tuning of Llama-Nemotron-8B and Qwen2.5-7B-Instruct (s1) on the reformatted 1M-example corpus yields relative average gains of +3.66% and +8.08% respectively versus baseline SFT on the identical raw traces across seven math benchmarks (Table 1). The same structure supports pruning-aware masked SFT and inference-time deletion of completed <try> spans, producing ~85% context/memory savings at an average 8.76% accuracy cost on Llama-Nemotron-8B (Table 2). A compute-matched ablation that replaces <outcome> content with MASK tokens while preserving length recovers only part of the gain (Table 3).

Significance. If the accuracy gains are genuinely attributable to the imposed structure rather than auxiliary factors, the work supplies a lightweight, data-only recipe that improves SFT reasoning performance and simultaneously enables semantically meaningful context pruning without architectural changes to attention. The controlled same-data comparisons, two-model evaluation, compute-matched ablation, and explicit discussion of PENCIL differences are strengths; planned code release would further aid reproducibility. The pruning proof-of-concept, even with its current overhead, points toward practical memory-efficient long-horizon reasoning.

major comments (3)
  1. Section 3.2 and Table 1: the central claim that the <try>/<outcome> format itself drives the +3.66%/+8.08% gains is incompletely isolated. Structured SFT receives additional high-quality summary tokens written by Llama-70B-Instruct that the baseline never sees. The compute-matched ablation (Table 3) replaces those tokens with MASK symbols and still leaves a residual +1.69% gain, yet never tests the natural pure-format controls (identical segmentation with empty or self-generated <outcome> tags, or original tokens merely wrapped in tags). Without those, distillation from the external 70B summarizer remains a plausible confounder for the accuracy lift that later enables pruning.
  2. Section 4.5 and Table 2: pruning evaluation is reported for only one model (Llama-Nemotron-8B) and relies on the same heuristic segmentation whose mid-step cuts the authors themselves identify as unrecoverable once <try> tokens are deleted. The 8.76% average drop, while not catastrophic, is large enough that the “proof-of-concept” claim needs either multi-model confirmation or a stronger demonstration that the residual performance is still competitive with shorter unstructured baselines of comparable context length.
  3. Section 3.3–3.4: the pruning-aware mask and inference loop are implemented only in a custom HuggingFace decoder; the paper acknowledges that vLLM-style engines lack the required dynamic eviction. Because the efficiency claim (85% savings) is a co-equal contribution, the absence of any wall-clock or FLOPs comparison against an optimized baseline leaves the practicality of the method under-supported.
minor comments (5)
  1. Abstract vs. Table 2: the abstract reports an 8.67% drop while the table average is 8.76%; reconcile the figures.
  2. Figure 1 caption and Section 3.2: the segmentation heuristic is described only by example cues; a short appendix listing the full cue vocabulary and any post-processing rules would improve reproducibility.
  3. Section 4.3: learning-rate, epoch, and packing details are given, yet the precise packing strategy and whether packing interacts with the Flex-Attention mask are not stated.
  4. Related Work: PENCIL is correctly distinguished, but a brief quantitative comparison of context reduction ratios on a shared symbolic task would strengthen the contrast.
  5. Typos: “compoed” (p. 5), “structure-aware pruning” capitalization inconsistency, and the arXiv date stamp “11 Jul 2026” should be corrected.

Circularity Check

0 steps flagged

No circularity: empirical SFT/pruning results on held-out benchmarks with same-data baseline and compute-matched control; nothing reduces by construction.

full rationale

This is a standard empirical ML paper. The central claims (Table 1 relative gains of +3.66%/+8.08% from Structured SFT; Table 2 pruning trade-off of ~8.7% drop for ~85% context savings) are measured by fine-tuning on a re-formatted version of the public llama-nemotron-math-v1.1 corpus and evaluating on external held-out math benchmarks (MATH-500, AIME24, etc.). Dataset construction (Section 3.2) is a preprocessing pipeline (heuristic cue-word segmentation + external Llama-70B summarization into <outcome> tags); the resulting traces are then used for ordinary cross-entropy SFT. The compute-matched ablation (Table 3) replaces <outcome> content with MASK tokens of identical length and still reports residual gains, confirming the authors do not treat length alone as the result. There are no free parameters fitted to a subset and then re-labeled as predictions, no self-definitional equations, no uniqueness theorems imported from the authors’ own prior work, and no load-bearing self-citations that close a derivation loop. Related-work discussion of PENCIL is contrastive, not foundational. Minor design choices (tag names, cue list, summarizer prompt) are ordinary engineering decisions, not circular reductions. The skeptic concern about 70B distillation is a possible confound for causal attribution of the gains, not circularity under the stated criteria.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 1 invented entities

The central claims rest on standard transformer SFT practice plus three paper-specific design choices: cue-word segmentation, external LLM summarization into outcomes, and the try/outcome tag syntax with pruning-aware attention. No physical constants or fitted scientific free parameters appear; training hyperparameters are ordinary and not load-bearing for the scientific claim beyond enabling the runs.

free parameters (3)
  • decision-cue segmentation vocabulary
    Step boundaries are defined by scanning for phrases such as "Wait," "Hmm," "Let me try again" (Section 3.2). Choice of cues is hand-selected and directly determines try spans and therefore outcomes and pruning units.
  • outcome length / summarizer instruction
    Outcomes are generated by Llama-70B-Instruct under a fixed prompt that requests one-to-two-sentence conclusions (Appendix B). Prompt wording and model choice control what information survives pruning.
  • SFT hyperparameters (lr=1e-4, 2 epochs, max 25k tokens)
    Standard training knobs reported in Section 4.3; not theoretically derived; affect absolute scores but are shared across baseline and structured runs.
axioms (4)
  • domain assumption Standard next-token cross-entropy SFT on curated CoT traces improves multi-step math reasoning.
    Background assumption of the entire post-training literature the paper builds on (Introduction, Related Work).
  • ad hoc to paper An external instruction-tuned model can extract the main logical conclusion of a try span without introducing new facts that change the solution.
    Required for dataset construction (Section 3.2 Summarization); authors note instruction-following weaknesses of reasoning models as motivation for using Llama-70B-Instruct.
  • ad hoc to paper Once an outcome has attended to its try, future tokens need only the outcome representation (keys/values) and can safely ignore the original try tokens.
    Load-bearing for Stage 2 masking and Stage 3 pruning (Section 3.3–3.4).
  • domain assumption Causal transformer attention and RoPE positional encodings remain valid when try spans are masked or deleted without renumbering positions.
    Implementation choice stated in Section 3.4 (positions retained as gaps).
invented entities (1)
  • <try>/<outcome> alternating reasoning syntax no independent evidence
    purpose: Separate exploratory scratch work from distilled step conclusions so the model can learn where to attend and so try tokens can be pruned after the outcome is written.
    Core format introduced by the paper; not present as a standard in the cited SFT corpora. Independent evidence is only the reported benchmark deltas and pruning tradeoff, not an external physical measurement.

pith-pipeline@v1.1.0-grok45 · 17808 in / 3217 out tokens · 37729 ms · 2026-07-14T12:07:09.211929+00:00 · methodology

0 comments
read the original abstract

Large language models (LLMs) excel at generating long chains of thought, but long reasoning traces are often verbose and memory-inefficient. In this work, we introduce Structured Thoughts, a framework that organizes reasoning into alternating <try> and <outcome> blocks: <try> captures exploratory scratch work, while <outcome> contains the distilled conclusion of that step. We construct a dataset of structured thoughts by segmenting reasoning traces into <try> blocks and prompting an LLM to summarize each step into its corresponding <outcome>. Fine-tuning pretrained foundation models on this reformatted data produces models that adopt the structured reasoning style, leading to performance gains of up to 8.08\% on reasoning benchmarks compared to standard SFT. The explicit structure also enables context pruning: after each <try>/<outcome> pair, the <try> can be pruned, allowing the model to retain conclusions without keeping the full scratch work in the context. A proof-of-concept pruning implementation achieves an average of 85\% memory / context savings with an 8.67\% performance drop across mathematical tasks.

Figures

Figures reproduced from arXiv: 2607.10386 by Anirban Das, Berkcan Kapusuzoglu, Chia-Hsuan Lee, Kartik Balasubramaniam, Sambit Sahu, Stephen Rawls, Supriyo Chakraborty, Zain Sarwar.

Figure 1
Figure 1. Figure 1: Raw → Structured traces. We convert free-form solutions into block-structured traces by (i) detecting decision cues (e.g., “Wait” “Hmm”) to segment the answer into step-sized spans, (ii) wrapping each span as a <try> (scratch work), and (iii) prompting a larger instruction-tuned model to summarize that span into a concise <outcome> (the step’s takeaway). The underlying problem and final answer are unchange… view at source ↗
Figure 2
Figure 2. Figure 2: Pruning procedure. During generation the model alternates between <try> (blue) and <outcome> (green) blocks. Once an <outcome> is generated, its preceding <try> block is removed from the context. This way, only the summarized outcomes are carried forward, while earlier scratch work is discarded step by step. format now provides a method for exploring the efficiency side of reasoning. With outcomes serv￾ing… view at source ↗
Figure 3
Figure 3. Figure 3: Pruning-aware train￾ing mask. The plot illustrates how attention is restricted during train￾ing. Tokens within a <try> block (blue) are only available until their corresponding <outcome> block (green) is produced. Once an <outcome> is generated, all subsequent queries are prevented (crossed out region) from attending back to the scratch tokens in the associated <try>. Future tokens can still access <outcom… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

64 extracted references · 42 linked inside Pith

  1. [1]

    Advances in neural information processing systems , volume=

    Chain-of-thought prompting elicits reasoning in large language models , author=. Advances in neural information processing systems , volume=

  2. [2]

    Advances in neural information processing systems , volume=

    Large language models are zero-shot reasoners , author=. Advances in neural information processing systems , volume=

  3. [3]

    arXiv preprint arXiv:2303.08774 , year=

    Gpt-4 technical report , author=. arXiv preprint arXiv:2303.08774 , year=

    Pith/arXiv arXiv

  4. [4]

    arXiv preprint arXiv:2505.00949 , year=

    Llama-nemotron: Efficient reasoning models , author=. arXiv preprint arXiv:2505.00949 , year=

    arXiv

  5. [5]

    arXiv preprint arXiv:2203.11171 , year=

    Self-consistency improves chain of thought reasoning in language models , author=. arXiv preprint arXiv:2203.11171 , year=

    Pith/arXiv arXiv

  6. [6]

    Advances in neural information processing systems , volume=

    Tree of thoughts: Deliberate problem solving with large language models , author=. Advances in neural information processing systems , volume=

  7. [7]

    arXiv preprint arXiv:2004.05150 , year=

    Longformer: The long-document transformer , author=. arXiv preprint arXiv:2004.05150 , year=

    Pith/arXiv arXiv 2004

  8. [8]

    Advances in neural information processing systems , volume=

    Big bird: Transformers for longer sequences , author=. Advances in neural information processing systems , volume=

  9. [9]

    arXiv preprint arXiv:2001.04451 , year=

    Reformer: The efficient transformer , author=. arXiv preprint arXiv:2001.04451 , year=

    Pith/arXiv arXiv 2001

  10. [10]

    arXiv preprint arXiv:2006.04768 , year=

    Linformer: Self-attention with linear complexity , author=. arXiv preprint arXiv:2006.04768 , year=

    Pith/arXiv arXiv 2006

  11. [11]

    arXiv preprint arXiv:2009.14794 , year=

    Rethinking attention with performers , author=. arXiv preprint arXiv:2009.14794 , year=

    Pith/arXiv arXiv 2009

  12. [12]

    arXiv preprint arXiv:1901.02860 , year=

    Transformer-xl: Attentive language models beyond a fixed-length context , author=. arXiv preprint arXiv:1901.02860 , year=

    Pith/arXiv arXiv 1901

  13. [13]

    arXiv preprint arXiv:1911.05507 , year=

    Compressive transformers for long-range sequence modelling , author=. arXiv preprint arXiv:1911.05507 , year=

    Pith/arXiv arXiv 1911

  14. [14]

    arXiv preprint arXiv:2309.17453 , year=

    Efficient streaming language models with attention sinks , author=. arXiv preprint arXiv:2309.17453 , year=

    Pith/arXiv arXiv

  15. [15]

    Advances in Neural Information Processing Systems , volume=

    Dynamic context pruning for efficient and interpretable autoregressive transformers , author=. Advances in Neural Information Processing Systems , volume=

  16. [16]

    arXiv preprint arXiv:2405.10637 , year=

    Layer-condensed kv cache for efficient inference of large language models , author=. arXiv preprint arXiv:2405.10637 , year=

    Pith/arXiv arXiv

  17. [17]

    arXiv preprint arXiv:2407.14057 , year=

    Lazyllm: Dynamic token pruning for efficient long context llm inference , author=. arXiv preprint arXiv:2407.14057 , year=

    Pith/arXiv arXiv

  18. [18]

    arXiv preprint arXiv:2503.14337 , year=

    Pencil: Long thoughts with short memory , author=. arXiv preprint arXiv:2503.14337 , year=

    Pith/arXiv arXiv

  19. [19]

    arXiv preprint arXiv:2504.09858 , year=

    Reasoning models can be effective without thinking , author=. arXiv preprint arXiv:2504.09858 , year=

    Pith/arXiv arXiv

  20. [20]

    arXiv preprint arXiv:2506.02536 , year=

    Answer Convergence as a Signal for Early Stopping in Reasoning , author=. arXiv preprint arXiv:2506.02536 , year=

    arXiv

  21. [21]

    arXiv preprint arXiv:2504.15895 , year=

    Dynamic Early Exit in Reasoning Models , author=. arXiv preprint arXiv:2504.15895 , year=

    arXiv

  22. [22]

    arXiv preprint arXiv:2401.10480 , year=

    Escape sky-high cost: Early-stopping self-consistency for multi-step reasoning , author=. arXiv preprint arXiv:2401.10480 , year=

    Pith/arXiv arXiv

  23. [23]

    Well, Keep Thinking

    " Well, Keep Thinking": Enhancing LLM Reasoning with Adaptive Injection Decoding , author=. arXiv preprint arXiv:2503.10167 , year=

    Pith/arXiv arXiv

  24. [24]

    arXiv preprint arXiv:2505.11896 , year=

    AdaCoT: Pareto-Optimal Adaptive Chain-of-Thought Triggering via Reinforcement Learning , author=. arXiv preprint arXiv:2505.11896 , year=

    Pith/arXiv arXiv

  25. [25]

    arXiv preprint arXiv:2308.16137 , year=

    Lm-infinite: Zero-shot extreme length generalization for large language models , author=. arXiv preprint arXiv:2308.16137 , year=

    Pith/arXiv arXiv

  26. [26]

    arXiv preprint arXiv:2408.03675 , year=

    Nacl: A general and effective kv cache eviction framework for llms at inference time , author=. arXiv preprint arXiv:2408.03675 , year=

    Pith/arXiv arXiv

  27. [27]

    5-coder technical report , author=

    Qwen2. 5-coder technical report , author=. arXiv preprint arXiv:2409.12186 , year=

    Pith/arXiv arXiv

  28. [28]

    arXiv preprint arXiv:2307.08691 , year=

    Flashattention-2: Faster attention with better parallelism and work partitioning , author=. arXiv preprint arXiv:2307.08691 , year=

    Pith/arXiv arXiv

  29. [29]

    Learning to reason with llms | openai ,year=

  30. [30]

    arXiv preprint arXiv:2507.06261 , year=

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities , author=. arXiv preprint arXiv:2507.06261 , year=

    Pith/arXiv arXiv

  31. [31]

    arXiv preprint arXiv:2501.12948 , year=

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning , author=. arXiv preprint arXiv:2501.12948 , year=

    Pith/arXiv arXiv

  32. [32]

    arXiv preprint arXiv:1707.06347 , year=

    Proximal policy optimization algorithms , author=. arXiv preprint arXiv:1707.06347 , year=

    Pith/arXiv arXiv

  33. [33]

    arXiv preprint arXiv:2402.03300 , year=

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models , author=. arXiv preprint arXiv:2402.03300 , year=

    Pith/arXiv arXiv

  34. [34]

    Introducing openai O3 and O4-mini | openai , author=

  35. [35]

    Show your work: Scratchpads for intermediate computation with language models , author=

  36. [36]

    arXiv preprint arXiv:2205.10625 , year=

    Least-to-most prompting enables complex reasoning in large language models , author=. arXiv preprint arXiv:2205.10625 , year=

    Pith/arXiv arXiv

  37. [37]

    Advances in Neural Information Processing Systems , volume=

    Self-refine: Iterative refinement with self-feedback , author=. Advances in Neural Information Processing Systems , volume=

  38. [38]

    International Conference on Learning Representations (ICLR) , year=

    React: Synergizing reasoning and acting in language models , author=. International Conference on Learning Representations (ICLR) , year=

  39. [39]

    arXiv preprint arXiv:2211.12588 , year=

    Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks , author=. arXiv preprint arXiv:2211.12588 , year=

    Pith/arXiv arXiv

  40. [40]

    Understanding r1-zero-like training: A critical perspective, 2025 , author=

    2025

  41. [41]

    The Twelfth International Conference on Learning Representations , year=

    Let's verify step by step , author=. The Twelfth International Conference on Learning Representations , year=

  42. [42]

    arXiv preprint arXiv:2211.14275 , year=

    Solving math word problems with process-and outcome-based feedback , author=. arXiv preprint arXiv:2211.14275 , year=

    Pith/arXiv arXiv

  43. [43]

    arXiv preprint arXiv:2503.07572 , year=

    Optimizing test-time compute via meta reinforcement fine-tuning , author=. arXiv preprint arXiv:2503.07572 , year=

    Pith/arXiv arXiv

  44. [44]

    URL https://arxiv

    Scaling llm test-time compute optimally can be more effective than scaling model parameters, 2024 , author=. URL https://arxiv. org/abs/2408.03314 , volume=

    Pith/arXiv arXiv 2024

  45. [45]

    arXiv preprint arXiv:2501.19393 , year=

    s1: Simple test-time scaling , author=. arXiv preprint arXiv:2501.19393 , year=

    Pith/arXiv arXiv

  46. [46]

    arXiv preprint arXiv:2412.05496 , year=

    Flex attention: A programming model for generating optimized attention kernels , author=. arXiv preprint arXiv:2412.05496 , year=

    Pith/arXiv arXiv

  47. [47]

    Proceedings of the Twentieth European Conference on Computer Systems , pages=

    Hybridflow: A flexible and efficient rlhf framework , author=. Proceedings of the Twentieth European Conference on Computer Systems , pages=

  48. [48]

    arXiv preprint arXiv:2403.13372 , year=

    Llamafactory: Unified efficient fine-tuning of 100+ language models , author=. arXiv preprint arXiv:2403.13372 , year=

    Pith/arXiv arXiv

  49. [49]

    Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles , year=

    Efficient Memory Management for Large Language Model Serving with PagedAttention , author=. Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles , year=

  50. [50]

    arXiv preprint arXiv:1910.03771 , year=

    Huggingface's transformers: State-of-the-art natural language processing , author=. arXiv preprint arXiv:1910.03771 , year=

    Pith/arXiv arXiv 1910

  51. [51]

    2025 , eprint=

    Qwen2.5 Technical Report , author=. 2025 , eprint=

    2025

  52. [52]

    arXiv preprint arXiv:2308.14508 , year=

    Longbench: A bilingual, multitask benchmark for long context understanding , author=. arXiv preprint arXiv:2308.14508 , year=

    Pith/arXiv arXiv

  53. [53]

    arXiv preprint arXiv:2307.03172 , year=

    Lost in the middle: How language models use long contexts , author=. arXiv preprint arXiv:2307.03172 , year=

    Pith/arXiv arXiv

  54. [54]

    arXiv preprint arXiv:2502.07266 , year=

    When more is less: Understanding chain-of-thought length in llms , author=. arXiv preprint arXiv:2502.07266 , year=

    Pith/arXiv arXiv

  55. [55]

    arXiv preprint arXiv:2103.03874 , year=

    Measuring mathematical problem solving with the math dataset , author=. arXiv preprint arXiv:2103.03874 , year=

    Pith/arXiv arXiv

  56. [56]

    American Invitational Mathematics Examination – Mathematical Association of America , url=

  57. [57]

    MAA Invitational Competitions – Mathematical Association of America , url=

  58. [58]

    Advances in neural information processing systems , volume=

    Solving quantitative reasoning problems with language models , author=. Advances in neural information processing systems , volume=

  59. [59]

    arXiv preprint arXiv:2305.12524 , year=

    Theoremqa: A theorem-driven question answering dataset , author=. arXiv preprint arXiv:2305.12524 , year=

    Pith/arXiv arXiv

  60. [60]

    arXiv preprint arXiv:2402.14008 , year=

    Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems , author=. arXiv preprint arXiv:2402.14008 , year=

    Pith/arXiv arXiv

  61. [61]

    arXiv preprint arXiv:2110.14168 , year=

    Training verifiers to solve math word problems , author=. arXiv preprint arXiv:2110.14168 , year=

    Pith/arXiv arXiv

  62. [62]

    arXiv preprint arXiv:2505.11423 , year=

    When thinking fails: The pitfalls of reasoning for instruction-following in llms , author=. arXiv preprint arXiv:2505.11423 , year=

    Pith/arXiv arXiv

  63. [63]

    arXiv preprint arXiv:2505.14810 , year=

    Scaling reasoning, losing control: Evaluating instruction following in large reasoning models , author=. arXiv preprint arXiv:2505.14810 , year=

    Pith/arXiv arXiv

  64. [64]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Two-level transformer and auxiliary coherence modeling for improved text segmentation , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=