Pith. sign in

REVIEW 4 major objections 4 minor 35 references

A three-model loop reuses prior reasoning traces as 'stickers' to beat brute-force sampling.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Sticker-TTS uses three collaborating models to distill and reuse compact summaries (stickers) of past reasoning attempts, improving math benchmark accuracy at a claimed equivalent inference cost.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Sticker-TTS has a genuinely new mechanism, but the efficiency claims rest on an unmeasured cost assumption and should be read with caution. the 4 major comments →

arxiv 2509.05007 v2 pith:SWHPZOW6 submitted 2025-09-05 cs.AI cs.CL

Sticker-TTS: Learn to Utilize Historical Experience with a Sticker-driven Test-Time Scaling Framework

Jie Chen , Jinhao Jiang , Yingqian Min , Zican Dong , Shijie Wang , Wayne Xin Zhao , Ji-Rong Wen This is my paper
classification cs.AI cs.CL
keywords test-time scalingreasoning modelsself-consistencyhistorical experience reuseknowledge distillationself-improvementmathematical reasoningmajority voting
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Sticker-TTS argues that the main inefficiency of test-time scaling is redundancy: independent sampling and simple reuse of full reasoning traces waste compute because each new attempt starts from scratch or drowns in long context. The paper proposes replacing those strategies with a collaboration of three models—an extractor, a modifier, and a utilizer—that distill the essential conditions of a previous attempt into a compact 'sticker', correct it, and feed it into the next attempt. The claim is that N such iterations cost about as much as 2N ordinary solutions yet outperform self-consistency and reinforcement-learned baselines on AIME-24, AIME-25, and OlymMATH. If true, test-time compute can be spent revisiting past mistakes rather than repeating fresh guesses.

Core claim

The paper's central claim is that historical experience, compressed into structured 'stickers', is a more efficient test-time compute currency than raw sampling. In their framework, the Sticker Extractor turns the previous reasoning trace into a list of key conditions and the question; the Sticker Modifier checks and corrects that sticker; and the Sticker Utilizer generates a new trace using the corrected sticker plus the previous answer. After N iterations, answers from all traces are majority-voted. The paper reports that this sticker-guided loop beats self-consistency by 12.42% relative on AIME-25 with a 7B model, beats Skywork-OR1 by 9.79% relative on OlymMATH, and scales further with N—

What carries the argument

The central object is the 'sticker': a short, structured abstract of a reasoning trace listing the atomic key conditions and the question, produced by the Sticker Extractor and revised by the Sticker Modifier. The recursive loop—extract, modify, utilize—does the work of accumulating experience: each iteration injects the corrected sticker and previous answer into the generation prompt, so the next trace starts from the distilled essence of the previous one instead of from scratch or from a verbatim long chain. The two-stage training (distillation from DeepSeek-R1 and o3-mini, then self-improvement bootstrapping with dual filtering) is what makes the three components behave as reliable specia

Load-bearing premise

The paper's headline efficiency result rests on the claim that N Sticker-TTS iterations cost about the same as generating 2N long-CoT solutions; no token-level measurement is provided for the sticker extractor and modifier overhead.

What would settle it

Measure the actual total tokens consumed by Sticker-TTS for N=10 iterations on a benchmark and compare against 2N self-consistency samples (i.e., 20 long-CoT solutions). If the sticker extractor, modifier, and utilizer prompts push total tokens above that 2N budget, the reported Cons@N comparisons stop being fair; a token-budget-matched evaluation would then show whether the accuracy gain survives.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • With N=10 iterations (claimed cost comparable to Cons@20), the 7B Stage-2 model reaches 83.33 on AIME-24 Cons@20, beating every listed 7B baseline's Cons@64 except Skywork-OR1's tie; on AIME-25 it matches the best 7B Cons@20 at 63.33.
  • The 32B Stage-2 model reaches 93.33 on AIME-24 Cons@20, above all baselines' Cons@64 values except AM-Thinking-v1's 90.00 tie; on AIME-25 it ties AM-Thinking-v1 at 80.00.
  • Increasing N from 2 to 16 on OlymMATH raises Cons@N from 56.67 to 83.33, showing the method scales with iteration count.
  • Removing the Extractor, the Modifier, or both hurts AIME Cons@20, so both modules contribute to the gain.
  • The 32B Sticker Utilizer alone matches Light-R1's Cons@20 while having lower Pass@1, suggesting the trained generator is strong but unstable per-sample.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the cost equivalence holds, the same sticker abstraction could be applied to other multi-step generative tasks—code repair, planning, or tool-use—where a previous failed attempt contains reusable 'conditions' that should survive into the next attempt.
  • The dual-filter ratio (1:2 error-to-correct vs correct-to-correct) is a curation choice the paper does not vary; it plausibly trades diversity against difficulty, and its optimal value may differ on domains where initial success rates are lower or higher.
  • Because the final answer comes from majority vote over only N traces, the method's edge could shrink on problems with two equally attractive wrong answers; a weighted or confidence-based aggregation would be a natural test.
  • A cheap external validation would be to fix the trained Extractor/Modifier and swap in different base Utilizer models; if gains scale with base-model weakness, sticker guidance is acting as error correction rather than as a generic prompt improvement.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes Sticker-TTS, a test-time scaling framework that coordinates three LLM components—a Sticker Extractor, a Sticker Modifier, and a Sticker Utilizer—to iteratively distill compact 'stickers' from previous reasoning traces, correct them, and generate new traces. The final answer is obtained by majority voting over N iterations. Training uses a two-stage recipe: knowledge distillation from teacher models (DeepSeek-R1, o3-mini) followed by self-improvement bootstrapping with dual filtering. Experiments on AIME-24, AIME-25, and OlymMATH-EN-EASY with 7B and 32B models claim consistent improvements over self-consistency and RL-trained baselines under comparable inference budgets.

Significance. The conceptual contribution is interesting: representing historical experience as compact, correctable stickers is a plausible middle ground between feeding full traces (which is verbose) and feeding only answers (which lacks diagnostic information). The paper provides code/data and a concrete two-stage training recipe. If the efficiency claims hold, Sticker-TTS would offer a useful alternative to redundant sampling and prior iterative-refinement methods. However, the headline efficiency result rests on an unmeasured cost-equivalence assumption, and the benchmark sizes (30 or 100 problems) are too small to support the strong 'consistently surpasses' wording without statistical support. The central idea is worth pursuing, but the current evidence is not yet convincing at the claimed level of rigor.

major comments (4)
  1. [§3.2, Table 1] The claim that 'the total reasoning cost for N iterations is comparable to generating 2N long-CoT solutions' is load-bearing for the efficiency comparison, but it is neither derived from Algorithm 1 nor measured. Algorithm 1 calls U once for the initial response (line 2) and N times in the loop (line 12), so U alone produces N+1 long-CoT traces, not 2N. Moreover, the text calls the Sticker Modifier one of the 'main long-CoT components,' yet the Appendix B prompt asks M only for a refined abstract. Without token-level counts for E, M, and U, and for inputs/outputs, the Table 1 comparisons of Ours (N=10) against baselines at Cons@20/Cons@64 are not established. Since the abstract's concrete gains (12.42%, 9.79%, 18.75%) are all efficiency claims, this missing measurement is the single point on which the central argument depends.
  2. [Abstract / Table 1] The abstract states that Sticker-TTS 'consistently surpasses strong baselines,' but Table 1 contains multiple ties or losses at the aligned columns. For example, 7B AIME-24 Cons@20: Ours 83.33 ties Skywork-OR1; 32B AIME-25 Cons@20: Ours 80.00 is below AM-Thinking 83.33; 32B OlymMATH Pass@1: Ours 78.00 is below Skywork-OR1's 85.77. The evidence supports 'better or comparable in several settings,' not 'consistently surpasses.' Please revise the abstract and Section 3.2's 'Superior Performance' bullet to match the table, or provide stronger evidence.
  3. [§3.1, Table 1] The evaluation sets are very small: AIME-24 and AIME-25 each contain 30 problems, and OlymMATH-EN-EASY contains 100. Many reported differences are one or two problems (e.g., 80.00 vs 83.33 on a 30-problem set). No variance, confidence intervals, significance tests, or multiple seeds are reported. Given the paper's central claim of consistent improvement over strong baselines, the lack of any statistical reliability analysis weakens the conclusions. I ask that the authors report bootstrap confidence intervals or a paired test (e.g., McNemar) for the headline comparisons.
  4. [§3.3, Table 3] The text states that deeper iterations yield gains 'with the same reasoning costs,' but the cost alignment among early exit, parallel sampling, and the full N=10 loop is not specified or measured. Early exit uses fewer iterations (hence less compute), while parallel sampling with P chains each running Q iterations has a different invocation pattern (each chain has an initial U call plus Q iterations plus E/M overhead) than a single chain of P*Q iterations. Please provide an explicit cost model or measure wall-clock/token usage for each strategy before claiming equal-cost comparison.
minor comments (4)
  1. [Algorithm 1] Line 13 appears to contain a bug: `AnswerList.append(A(0)))` should presumably be `AnswerList.append(A^(k))`, since the loop is intended to append the current iteration's answer.
  2. [§3.2] The phrase 'Sticker Modifier and Optimizer are the main long-CoT components' uses 'Optimizer' inconsistently; the third component is the Sticker Utilizer. Also, the claim that stickers have 'minimal overhead' is asserted without evidence.
  3. [Table 1] The 'Ours' results are placed under the Cons@20 and Cons@64 columns even though the method uses N=10 iterations (with a claimed cost comparable to Cons@20). The column header should indicate the actual N and the cost-equivalent baseline N, e.g., 'Cons@10 (cost ≈ Cons@20)'.
  4. [Throughout] Typos and minor formatting issues: 'compute bugets' in the Introduction; 'utill' near the stopping criterion; 'utlizing' in Section 2.3; missing spaces in Table 1's body (e.g., '83.3352.50').

Circularity Check

0 steps flagged

No significant circularity: the framework is evaluated on held-out benchmarks and its components are not defined in terms of the target results.

full rationale

Sticker-TTS is not circular in the sense of deriving its conclusions from its own assumptions by construction. The method is evaluated on held-out AIME 2024, AIME 2025, and OlymMATH-EN-EASY, while training data comes from OpenThoughts and self-generated trajectories with correctness filtering—not from the test benchmarks. The three components (Extractor, Modifier, Utilizer) have distinct functional roles, and the final answer is aggregated by majority vote over iteratively generated traces; this is a concrete mechanism rather than a renaming of the evaluation metric. The only assumption that could threaten the headline efficiency claims is the cost-equivalence statement in Section 3.2 ('the total reasoning cost for N iterations is comparable to generating 2N long-CoT solutions'), but this is an unmeasured empirical approximation, not a definition or equation that makes the comparison true by construction. If the token-level cost were higher, the fairness of the budget comparison would be weakened, but that would be a correctness/validity issue, not circularity. Self-citations (e.g., the OlymMATH benchmark from the same research group) are not load-bearing; the main comparisons also rely on external AIME benchmarks and independent baselines. The Limitations section explicitly acknowledges potential error propagation and the need for further failure analysis, which further supports that the authors do not claim a logically forced result. Overall, the derivation chain is self-contained against external benchmarks, and no step reduces to its own inputs.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 1 invented entities

The central claim rests on several hand-picked thresholds and unverified assumptions about cost accounting, teacher faithfulness, and the effectiveness of filtering. None of these are fitted to the test benchmarks, but the cost-equivalence assumption and the filtering ratio materially affect the reported efficiency and performance. The invented sticker entity is central to the method but lacks independent validation beyond this paper.

free parameters (5)
  • Difficulty thresholds for data selection = [0.2, 0.5] (stage 1), [0, 0.4] (stage 2)
    Chosen to select moderate-difficulty problems from OpenThoughts; affects training data composition and final performance, no sensitivity analysis given.
  • Transition ratio (error-to-correct vs correct-to-correct) = 1:2
    Set to match a claimed natural distribution; no empirical justification is provided.
  • Correction significance window = two preceding iterations must be wrong
    Hand-picked requirement for error-to-correct cases to ensure non-trivial corrections; affects the difficulty of the curated subset.
  • Number of iterations N = 10
    Main results use N=10; Table 2 shows scaling with N but no optimality criterion or cost tradeoff analysis.
  • Sampling hyperparameters (temperature, top-p) = temperature=0.6, top-p=0.95
    Standard settings, but still choices that affect both training data generation and evaluation.
axioms (5)
  • domain assumption The inference cost of N Sticker-TTS iterations is comparable to generating 2N long-CoT solutions because stickers have minimal overhead.
    Stated in Section 3.2; no token-level measurements provided; this assumption drives the fair-comparison claim and is not verified.
  • domain assumption DeepSeek-R1 reasoning traces and o3-mini sticker extractions are faithful and high-quality.
    Assumed in Section 2.3; no faithfulness evaluation or comparison of teacher outputs is reported.
  • domain assumption Majority voting over iteratively refined answers improves final accuracy.
    Standard heuristic, but the iterative dependency between votes may violate the independence assumptions that make majority voting robust in self-consistency.
  • domain assumption The dual filtering criteria (solution validity and correction significance) produce high-quality training data that improves the framework.
    The 1:2 ratio and the two-prior-failures rule are asserted without empirical derivation or ablation of the thresholds.
  • domain assumption The base models (DeepSeek-R1-Distill and Qwen2.5) can perform the extract, modify, and utilize roles after SFT.
    The entire framework depends on the models' ability to follow the sticker prompts effectively and to benefit from iterative refinement; this is not guaranteed a priori.
invented entities (1)
  • Sticker no independent evidence
    purpose: A compact structured summary of key conditions, the question, and strategic cues extracted from a reasoning trace, used to guide subsequent reasoning iterations.
    The sticker is a new intermediate representation introduced in this paper. Its utility is demonstrated only through the paper's experiments; there are no external falsifiable predictions or downstream applications independent of the proposed framework.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Sticker-TTS: Learn to Utilize Historical Experience with a Sticker-driven Test-Time Scaling Framework." pith.science (2026). https://pith.science/paper/SWHPZOW6

@misc{pith2026250905007,
  author       = {Pith},
  title        = {Pith review of: Sticker-TTS: Learn to Utilize Historical Experience with a Sticker-driven Test-Time Scaling Framework},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SWHPZOW6}},
  note         = {Machine review of arXiv:2509.05007}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Large reasoning models (LRMs) have exhibited strong performance on complex reasoning tasks, with further gains achievable through increased computational budgets at inference. However, current test-time scaling methods predominantly rely on redundant sampling, ignoring the historical experience utilization, thereby limiting computational efficiency. To overcome this limitation, we propose Sticker-TTS, a novel test-time scaling framework that coordinates three collaborative LRMs to iteratively explore and refine solutions guided by historical attempts. At the core of our framework are distilled key conditions-termed stickers-which drive the extraction, refinement, and reuse of critical information across multiple rounds of reasoning. To further enhance the efficiency and performance of our framework, we introduce a two-stage optimization strategy that combines imitation learning with self-improvement, enabling progressive refinement. Extensive evaluations on three challenging mathematical reasoning benchmarks, including AIME-24, AIME-25, and OlymMATH, demonstrate that Sticker-TTS consistently surpasses strong baselines, including self-consistency and advanced reinforcement learning approaches, under comparable inference budgets. These results highlight the effectiveness of sticker-guided historical experience utilization. Our code and data are available at https://github.com/RUCAIBox/Sticker-TTS.

Figures

Figures reproduced from arXiv: 2509.05007 by Jie Chen, Jinhao Jiang, Ji-Rong Wen, Shijie Wang, Wayne Xin Zhao, Yingqian Min, Zican Dong.

Figure 1
Figure 1. Figure 1: The overall framework of our proposed Sticker-TTS. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

35 extracted references · 11 canonical work pages · 1 internal anchor

  1. [1]

    Forrest Bao, Miaoran Li, Rogger Luo, and Ofer Mendelevitch. 2024. https://doi.org/10.57967/hf/3240 HHEM-2.1-Open

    doi:10.57967/hf/3240 2024

  2. [2]

    Guoxin Chen, Minpeng Liao, Chengxi Li, and Kai Fan. 2024. Alphamath almost zero: process supervision without process. CoRR, abs/2405.03553

    Pith/arXiv arXiv 2024

  3. [3]

    Tianqi Chen, Bing Xu, Chiyuan Zhang, and Carlos Guestrin. 2016. Training deep nets with sublinear memory cost. CoRR, abs/1604.06174

    Pith/arXiv arXiv 2016

  4. [4]

    Zhipeng Chen, Yingqian Min, Beichen Zhang, Jie Chen, Jinhao Jiang, Daixuan Cheng, Wayne Xin Zhao, Zheng Liu, Xu Miao, Yang Lu, et al. 2025. An empirical study on eliciting and improving r1-like reasoning models. arXiv preprint arXiv:2503.04548

    Pith/arXiv arXiv 2025

  5. [5]

    Fu, Stefano Ermon, Atri Rudra, and Christopher R \' e

    Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher R \' e . 2022. Flashattention: Fast and memory-efficient exact attention with io-awareness. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022

    work page 2022

  6. [6]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    Pith/arXiv arXiv 2025

  7. [7]

    Jujie He, Jiacai Liu, Chris Yuhao Liu, Rui Yan, Chaojie Wang, Peng Cheng, Xiaoyu Zhang, Fuxiang Zhang, Jiacheng Xu, Wei Shen, Siyuan Li, Liang Zeng, Tianwen Wei, Cheng Cheng, Bo An, Yang Liu, and Yahui Zhou. 2025. Skywork open reasoner series. Notion Blog

    work page 2025

  8. [8]

    Yunjie Ji, Xiaoyu Tian, Sitong Zhao, Haotian Wang, Shuaiting Chen, Yiping Peng, Han Zhao, and Xiangang Li. 2025. Am-thinking-v1: Advancing the frontier of reasoning at 32b scale. arXiv preprint arXiv:2505.08311

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  9. [9]

    Jinhao Jiang, Zhipeng Chen, Yingqian Min, Jie Chen, Xiaoxue Cheng, Jiapeng Wang, Yiru Tang, Haoxiang Sun, Jia Deng, Wayne Xin Zhao, Zheng Liu, Dong Yan, Jian Xie, Zhongyuan Wang, and Ji-Rong Wen. 2024. https://arxiv.org/abs/2411.11694 Enhancing llm reasoning with reward-guided tree search . Preprint, arXiv:2411.11694

    Pith/arXiv arXiv 2024

  10. [10]

    Jikun Kang, Xin Zhe Li, Xi Chen, Amirreza Kazemi, and Boxing Chen. 2024. Mindstar: Enhancing math reasoning in pre-trained llms at inference time. CoRR, abs/2405.16265

    Pith/arXiv arXiv 2024

  11. [11]

    Jiaqi Li, Mengmeng Wang, Zilong Zheng, and Muhan Zhang. 2023. Loogle: Can long-context language models understand long contexts? arXiv preprint arXiv:2311.04939

    Pith/arXiv arXiv 2023

  12. [12]

    Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 . OpenReview.net

    work page 2019

  13. [13]

    Tongxu Luo, Wenyu Du, Jiaxi Bi, Stephen Chung, Zhengyang Tang, Hao Yang, Min Zhang, and Benyou Wang. 2025. Learning from peers in reasoning models. arXiv preprint arXiv:2505.07787

    Pith/arXiv arXiv 2025

  14. [14]

    MAA . 2024. American Invitational Mathematics Examination - AIME 2024

    work page 2024

  15. [15]

    MAA . 2025. American Invitational Mathematics Examination - AIME 2025

    work page 2025

  16. [16]

    OpenAI . 2024 a . https://openai.com/index/learning-to-reason-with-llms/ Learning to reason with llms . Accessed: 2025-05-03

    work page 2024

  17. [17]

    OpenAI . 2024 b . https://cdn.openai.com/o1-system-card-20241205.pdf Openai o1 system card . Accessed: 2025-05-03

    work page 2024

  18. [18]

    OpenAI . 2025. https://openai.com/index/openai-o3-mini/ Openai o3 mini . Accessed: 2025-05-03

    work page 2025

  19. [19]

    Haoxiang Sun, Yingqian Min, Zhipeng Chen, Wayne Xin Zhao, Zheng Liu, Zhongyuan Wang, Lei Fang, and Ji - Rong Wen. 2025. Challenging the boundaries of reasoning: An olympiad-level math benchmark for large language models. CoRR, abs/2503.21380

    Pith/arXiv arXiv 2025

  20. [20]

    Amir Taubenfeld, Tom Sheffer, Eran Ofek, Amir Feder, Ariel Goldstein, Zorik Gekhman, and Gal Yona. 2025. Confidence improves self-consistency in llms. arXiv preprint arXiv:2502.06233

    arXiv 2025

  21. [21]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. 2025. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599

    Pith/arXiv arXiv 2025

  22. [22]

    Open Thoughts Team. 2025. Open Thoughts

    work page 2025

  23. [23]

    Fengwei Teng, Zhaoyang Yu, Quan Shi, Jiayi Zhang, Chenglin Wu, and Yuyu Luo. 2025. https://doi.org/10.48550/ARXIV.2502.12018 Atom of thoughts for markov LLM test-time scaling . CoRR, abs/2502.12018

    work page doi:10.48550/arxiv.2502.12018 2025

  24. [24]

    Xiaoyu Tian, Sitong Zhao, Haotian Wang, Shuaiting Chen, Yunjie Ji, Yiping Peng, Han Zhao, and Xiangang Li. 2025. Think twice: Enhancing llm reasoning by scaling multi-round test-time thinking. arXiv preprint arXiv:2503.19855

    Pith/arXiv arXiv 2025

  25. [25]

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171

    Pith/arXiv arXiv 2022

  26. [26]

    Liang Wen, Yunke Cai, Fenrui Xiao, Xin He, Qi An, Zhenyu Duan, Yimin Du, Junchen Liu, Lifu Tang, Xiaowei Lv, Haosheng Zou, Yongchao Deng, Shousheng Jia, and Xiangzheng Zhang. 2025 a . https://doi.org/10.48550/ARXIV.2503.10460 Light-r1: Curriculum sft, DPO and RL for long COT from scratch and beyond . CoRR, abs/2503.10460

    Pith/arXiv arXiv doi:10.48550/arxiv.2503.10460 2025

  27. [27]

    Liang Wen, Yunke Cai, Fenrui Xiao, Xin He, Qi An, Zhenyu Duan, Yimin Du, Junchen Liu, Lifu Tang, Xiaowei Lv, et al. 2025 b . Light-r1: Curriculum sft, dpo and rl for long cot from scratch and beyond. arXiv preprint arXiv:2503.10460

    Pith/arXiv arXiv 2025

  28. [28]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R \'e mi Louf, Morgan Funtowicz, et al. 2019. Huggingface's transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771

    Pith/arXiv arXiv 2019

  29. [29]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388

    Pith/arXiv arXiv 2025

  30. [30]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tingyu X...

    Pith/arXiv arXiv 2024

  31. [31]

    Yufeng Yuan, Yu Yue, Ruofei Zhu, Tiantian Fan, and Lin Yan. 2025. What's behind ppo's collapse in long-cot? value optimization holds the secret. arXiv preprint arXiv:2503.01491

    Pith/arXiv arXiv 2025

  32. [32]

    Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. 2025. Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild. arXiv preprint arXiv:2503.18892

    Pith/arXiv arXiv 2025

  33. [33]

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian - Yun Nie, and Ji - Rong Wen. 2023. A survey of large language models. CoRR, abs/2303.18223

    Pith/arXiv arXiv 2023

  34. [34]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  35. [35]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.