{"id":"764d7914-6797-4e09-ae41-291cd6e9cdad","arxiv_id":"2505.17746","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Fast Quiet-STaR uses a curriculum to compress Quiet-STaR's token-level thoughts and an RL stage to remove them entirely, improving accuracy on four benchmarks at the same or lower inference cost.","lead":"This paper trains language models to reason with fewer or no explicit \"thought\" tokens, cutting inference time while keeping accuracy. It shows a curriculum that gradually compresses thoughts, then distills the result into standard next-token prediction models, improving accuracy without extra latency.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline NTP gains may be due to standard continued pretraining on OpenWebMath rather than the proposed curriculum/RL mechanism; the paper lacks a plain NTP control trained for the same 0.5M tokens.","rationale":"The reader's weakest assumption focused on the step-matched Quiet-STaR comparison: Fast Quiet-STaR 12-4 and 8-4 receive more total training steps than the Quiet-STaR baselines, so the observed gains at matched thought-token counts could partly reflect additional compute. That is a valid concern, but the paper's Section 4.3.1 reverse-curriculum ablation (Rev Quiet-STaR) and Table 3's no-curriculum NTP comparison provide some evidence that the easy-to-hard order specifically matters, which partially mitigates the step-matching issue. The more serious gap is at the level of the paper's headline claim: Fast Quiet-STaR NTP is compared only to a pre-trained model with no continued pretraining, so the 9% and 5.7% average accuracy improvements cannot be attributed to the curriculum or the RL distillation without a plain NTP control trained on the same 0.5M tokens. This is not a trivial omission, because continued pretraining on a domain-adjacent corpus like OpenWebMath is known to improve downstream math and commonsense performance, and the reported total training budget is small (0.5M tokens), so the control is cheap to run. The paper's own Section 4.3.2 claims to rule out 'greater data exposure,' but Figure 5 only contains Quiet-STaR-style models, not an NTP baseline, so the claim is not directly supported. The number of RL steps for the NTP stage is not specified, which makes the compute match for the proposed control approximate; nevertheless, a 0.5M-token standard NTP baseline is the decisive experiment. I therefore retain the reader's CONDITIONAL verdict: the method is plausible and the paper contains useful ablations, but the central NTP improvement needs this missing control before the claim is secure.","tokens_in":14065,"tokens_out":10648,"duration_ms":84571,"concrete_test":"Train a control model for each base model (Mistral-7B and Qwen2.5-7B): initialize from the same pre-trained checkpoint and perform standard causal next-token prediction on OpenWebMath for exactly the same total number of tokens (0.5M) and the same batch/sequence settings as the full Fast Quiet-STaR pipeline, with no thought tokens, no curriculum, and no RL. Use a comparable learning rate (e.g., 1e-6 for Mistral) and greedy decoding. Evaluate on PIQA, SIQA, CommonsenseQA, and GSM8K with the same ACC formula and report the average. If this NTP continued-pretraining control achieves average accuracy within roughly 1 point of Fast Quiet-STaR NTP, the claimed benefit of the proposed mechanism is not established; if it is more than 5 points lower, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central quantitative claim in the abstract and Section 4.2 is that Fast Quiet-STaR NTP improves average accuracy by 9% on Mistral 7B and 5.7% on Qwen2.5 7B over the pre-trained model at identical inference latency. However, Table 1 compares Fast Quiet-STaR NTP against the original pre-trained checkpoint, which has received zero additional training on OpenWebMath, while the full Fast Quiet-STaR pipeline consumes 0.5M tokens of continued pretraining (Section 4.3.4) plus an unspecified number of RL fine-tuning steps. A standard NTP model given the same 0.5M tokens of OpenWebMath could plausibly improve on these benchmarks through domain adaptation alone, and the paper includes no such control. Section 4.3.2's data-efficiency analysis (Figure 5) only tracks Quiet-STaR-style models, not a plain NTP baseline, so it does not rule out this confound. Table 3's Quiet-STaR NTP comparison is an alternative RL-based method, not a standard continued-pretraining control. Because the headline contribution is 'thinking without thought tokens,' the absence of a no-thought, same-compute NTP baseline is the most load-bearing gap: if a plain NTP model matches the reported gains, the proposed curriculum and RL-to-NTP mechanism are not responsible for the improvement.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Fast Quiet-STaR, a curriculum-learning variant of Quiet-STaR that progressively reduces the number of thought tokens (16-8 → 12-4 → 8-4) during continued pretraining, and then applies a REINFORCE-style objective (Eq. 4) to fine-tune the final 8-4 checkpoint into a standard next-token-prediction (NTP) model that does not generate explicit thoughts at inference. The authors report experiments on Mistral 7B and Qwen2.5 7B over PIQA, SIQA, CommonsenseQA, and GSM8K, claiming that Fast Quiet-STaR outperforms Quiet-STaR at equal thought-token counts and that Fast Quiet-STaR NTP improves average accuracy by 9% (Mistral) and 5.7% (Qwen2.5) over the original pre-trained models at identical inference latency.","tokens_in":14372,"tokens_out":7112,"duration_ms":60091,"significance":"The practical goal of removing thought-token overhead while retaining reasoning gains is timely and, if the empirical claims hold, would make Quiet-STaR-style training substantially more deployable. The paper also reports a useful latency analysis (Section 4.3.4) showing that the added training cost can be amortized over very few inference runs. However, the current evidence does not isolate the proposed mechanism: the Fast-vs-Quiet comparisons are not compute-matched, the headline NTP gains have no plain continued-pretraining control, and the evaluation metric is not fully specified. The qualitative thought-token examples (Figures 6-7) are suggestive but not quantified.","major_comments":[{"comment":"The central Fast Quiet-STaR vs. Quiet-STaR comparison is confounded by training steps. Implementation details state that Quiet-STaR is trained for 100 steps while Fast Quiet-STaR receives 100 steps on the 16-8 stage plus 50 steps for each subsequent stage (150 steps for 12-4 and 200 for 8-4). No step-matched Quiet-STaR baseline (e.g., Quiet-STaR 8-4 trained for 200 steps) is reported, so the improved average accuracy in Table 1 could reflect additional optimization on OpenWebMath rather than the proposed curriculum. A matched-compute baseline is needed to support the claim in Section 4.2 that Fast Quiet-STaR 'consistently outperforms Quiet-STaR' under equal thought-token counts.","section":"4.1, 4.2, Table 1"},{"comment":"The headline result in the abstract and Table 1 compares Fast Quiet-STaR NTP against the original pre-trained checkpoint, which has received zero OpenWebMath tokens, while the Fast Quiet-STaR pipeline consumes 0.5M tokens of continued pretraining (Section 4.3.4) plus RL fine-tuning. The paper does not include a plain NTP model trained on the same 0.5M OpenWebMath tokens (or the same number of gradient steps with a standard LM loss). Such a control is necessary to rule out that the reported 9%/5.7% average gains are due to domain-adaptive continued pretraining alone. The data-efficiency analysis in Figure 5 tracks only Quiet-STaR-style models and therefore does not address this confound.","section":"4.2, Table 1, 4.3.4"},{"comment":"The evaluation metric is not well defined. The accuracy formula in Section 4.1 is syntactically garbled: as typeset it reads as a ratio of a product of per-token answer likelihoods to a product over sums of candidate-answer likelihoods, which is not a standard accuracy measure and is not a faithful reproduction of the Quiet-STaR protocol (Zelikman et al., 2024) without additional definitions. All quantitative claims, including the abstract's 9% and 5.7%, rest on this metric. The authors should provide the exact computation, define the candidate set S_ans for each dataset (especially GSM8K, where no candidate set is given), and state how final answers are extracted and matched.","section":"4.1, Accuracy formula"},{"comment":"The conclusion that Fast Quiet-STaR NTP 'performs on par with Quiet-STaR' is not supported on Qwen2.5, where Quiet-STaR 16-8 achieves an average accuracy of 57.5 vs. 54.4 for Fast Quiet-STaR NTP; on Mistral the NTP model (41.0) is actually above Quiet-STaR 16-8 (39.2), but both comparisons are within a few points and no error bars, seeds, or significance tests are reported. Differences of about 2-4 points in Table 1 (e.g., Fast Quiet-STaR 12-4 vs. Quiet-STaR 12-4 on Qwen) may be within run-to-run variability. The paper should report multiple seeds or otherwise quantify variance before drawing these comparative conclusions.","section":"4.2, Table 1"}],"minor_comments":[{"comment":"Section 4.1 says Quiet-STaR is trained for 100 steps, while Section 4.3.2 says Quiet-STaR trains for 150 steps; these numbers should be reconciled.","section":"4.1 vs 4.3.2"},{"comment":"The accuracy formula uses 'Ql' and 'Q' where product symbols are intended; please typeset with \\prod and define all variables explicitly.","section":"4.1"},{"comment":"The header 'A VG ACC' appears to be a typo for 'AVG ACC'.","section":"Table 2"},{"comment":"The legend label 'Thought T okens Number' has a stray space, and the x-axis labels are not clearly tied to the curve types.","section":"Figure 1"},{"comment":"The section contains an unresolved cross-reference 'see Table (?)' that should be replaced with the actual table number.","section":"4.3.4"},{"comment":"The reference entries for 'OpenAI' and 'Qwen' begin with an empty author field ('OpenAI, :' and 'Qwen, :'), which should be cleaned up.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper proposes a potentially useful idea, but the missing controls and the uninterpretable metric make the central claims difficult to verify. In particular, the absence of a plain NTP continued-pretraining control is a critical gap for a paper whose title promises 'thinking without thought tokens.' I suggest the authors be asked to add a step-matched Quiet-STaR baseline and a same-token-budget NTP baseline, and to rewrite the metric section, before the paper can be considered. The scope fit is acceptable for a venue interested in efficient reasoning, but the empirical support needs substantial strengthening."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere is my read on Fast Quiet-STaR. The core idea is sensible: train a Quiet-STaR model with long thought traces, shrink the thought budget through an easy-to-hard curriculum, and then distill the reasoning into a plain NTP model with a REINFORCE-style loss. The reverse-curriculum ablation (Figure 3) is a real contribution—it shows that ordering matters and that the easy-to-hard schedule is doing work. The training cost claim (0.5M tokens, 54 minutes on 8 H800s) is also refreshingly concrete. If the results hold up, this is a useful efficiency patch for a method that is otherwise too slow to deploy.\n\nThe problem is that the headline numbers are not yet supported. First, the Fast versus Quiet-STaR comparison is step-confounded: Quiet-STaR gets 100 training steps, while Fast 12-4 gets 100 on 16-8 plus 50 on 12-4, and Fast 8-4 gets another 50. A step-matched Quiet-STaR baseline would settle whether the gain comes from the curriculum or just from more gradient steps. The data-efficiency figure in Section 4.3.2 only compares training curves, not a matched-budget evaluation.\n\nSecond, and more load-bearing, the Fast Quiet-STaR NTP numbers are reported against the original pretrained checkpoint, which has seen zero OpenWebMath tokens. The full NTP pipeline consumes 0.5M tokens of continued pretraining plus RL fine-tuning. A plain NTP model given the same 0.5M tokens of OpenWebMath could plausibly capture a large chunk of the reported 9% / 5.7% gains just through domain adaptation. The paper does not include that control, and Figure 5 cannot rule it out. Without it, the claim that 'thinking without thought tokens' is doing the work is unproven. That is the gap I would want closed first.\n\nTwo smaller issues: the ACC formula in Section 4.1 is garbled as printed, and the GSM8K setup (direct answer? prompt format? sampling?) is underspecified. No seeds or error bars either, but given the tuned nature of these pipelines, even a couple of seeds would help.\n\nOverall, the machinery is plausible and the curriculum ordering result is interesting. But the paper is overclaiming relative to the evidence. It deserves a serious referee, not a desk reject—the missing baselines are expensive but entirely feasible. I'd send it to review with a strong request for the step-matched Quiet-STaR and the plain-NTP same-data controls.","headline":"Plausible efficiency idea for Quiet-STaR, but the headline NTP gains are missing the step-matched and plain-continued-pretraining controls needed to prove them.","tokens_in":14891,"tokens_out":3365,"would_cite":false,"duration_ms":32188,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fast Quiet-STaR claims that a curriculum shrinking Quiet-STaR thought traces from 16 to 8 tokens, then distilling the model to plain next-token prediction with reinforcement learning, preserves the reasoning gains while eliminating…","keywords":["quiet reasoning","token-level thought traces","curriculum learning","next token prediction","reinforcement learning","inference efficiency","large language models"],"falsifier":"Train Quiet-STaR 8-4 for 200 steps with the same data, learning rate, and batching as Fast Quiet-STaR 8-4, and compare the four-benchmark average accuracy; if the step-matched baseline matches or exceeds Fast Quiet-STaR 8-4, the curriculum ordering is not what carries the gain.","tokens_in":13870,"feed_emoji":"🧠","tokens_out":12908,"duration_ms":93082,"temperature":0.7,"pith_summary":"Fast Quiet-STaR tries to show that token-level \"thinking\" does not have to be paid for at every inference step. Quiet-STaR improves reasoning by generating a short thought trace before each predicted token, but this makes time-to-first-token more than ten times slower than ordinary next-token prediction. The paper's proposal is a curriculum: train on long thought traces first, then progressively fewer thought tokens, and finally use reinforcement learning to distill the model into a plain next-token predictor. The result, Fast Quiet-STaR NTP, keeps or improves the accuracy gains while running at the same latency as the original pre-trained model, with average gains of 9.0% on Mistral 7B and 5.7% on Qwen2.5 7B.","feed_headline":"Drop thought tokens, keep reasoning: 9-point gain, zero latency cost","feed_subtitle":"A curriculum that shrinks a model's inner monologue to nothing keeps the accuracy boost of token-level thinking.","key_machinery":"The central machinery is a curriculum over Quiet-STaR's \"n-m\" configuration, where n is the number of thought tokens and m the number of ahead tokens generated per position. The model first learns under the easy 16-8 setting (verbose thoughts), then is re-trained under 12-4 and 8-4, each stage initialized from the previous checkpoint, so the harder compressed-thought settings are reached gradually. A final REINFORCE stage converts the 8-4 checkpoint into an NTP model by rewarding it for matching the 8-4 model's negative log-likelihood after thinking; this removes explicit thought-token generation while retaining the predictive signal the thoughts carried.","core_discovery":"The central claim is that a model trained in the Quiet-STaR token-level thinking paradigm can learn to reason in progressively shorter thought traces, and finally in no explicit trace at all, without losing the reasoning benefit. Starting from a 16-thought-token, 8-ahead-token Quiet-STaR configuration, Fast Quiet-STaR trains through 12-4 and 8-4 stages, then uses REINFORCE to make a standard next-token-prediction head match the 8-4 model's post-thought prediction quality. On PIQA, SIQA, CommonsenseQA, and GSM8K with Mistral 7B and Qwen2.5 7B, the resulting Fast Quiet-STaR NTP improves average accuracy by 9.0% and 5.7% over the pre-trained baselines while keeping the same time-to-first-token and generation latency as ordinary next-token prediction.","pith_inferences":["Editorial inference: because the Quiet-STaR baselines are trained for 100 steps while each Fast Quiet-STaR stage adds 50 more, the reported gains at matched thought-token counts mix the curriculum effect with extra training compute; a step-matched Quiet-STaR baseline would isolate the curriculum's contribution.","Editorial inference: the same easy-to-hard thought-compression recipe could be tested on other verbose-reasoning methods, such as chain-of-thought fine-tuning or latent-space reasoning, to see whether 'compress the reasoning, then drop it' is a general route to fast reasoning.","Editorial inference: the thought-token visualizations suggest Fast Quiet-STaR's compressed thoughts are more goal-directed than Quiet-STaR's, which would predict that the curriculum teaches a reusable abstraction skill that transfers to reasoning datasets beyond the four tested.","Editorial inference: the RL-to-NTP stage treats post-thought log-likelihood as a soft training target, so one testable extension is to regularize the NTP model against the full token distribution of the thinking model rather than only the next-token loss, and to ablate the reward baseline."],"forward_implications":["With the same number of thought tokens and therefore the same inference time, Fast Quiet-STaR 12-4 and 8-4 outperform Quiet-STaR at those settings on both base models and all four benchmarks.","Fast Quiet-STaR 8-4 on Mistral 7B beats Quiet-STaR 16-8 by 1.8% average accuracy while cutting time-to-first-token to about 41.3% of the 16-8 value.","Fast Quiet-STaR NTP keeps ordinary NTP latency and still improves average accuracy by 9.0% on Mistral 7B and 5.7% on Qwen2.5 7B; on GSM8K, it also lifts chain-of-thought majority-vote accuracy from 43.3% to 52.4% at maj@6.","The curriculum ordering is necessary: a reversed curriculum (8-4 to 12-4 to 16-8) yields no improvement, and jumping straight from Quiet-STaR 16-8 to NTP loses 5.2% average accuracy against the full curriculum.","The training cost is small (0.5M tokens, about 54 minutes on eight H800 GPUs), and the paper estimates the time saved over 67 end-to-end inference runs of Quiet-STaR 16-8 roughly offsets the entire training cost."],"supporting_citations":[{"why":"Defines Quiet-STaR's Think-Talk-Learn procedure and supplies the baseline and evaluation setup that Fast Quiet-STaR extends.","marker":"(Zelikman et al., 2024)"},{"why":"Supplies the OpenWebMath corpus used for all post-training and baseline runs.","marker":"(Paster et al., 2023)"},{"why":"Provides the REINFORCE algorithm used for thought optimization and for the RL fine-tuning that converts Fast Quiet-STaR to NTP.","marker":"(Phan et al., 2023)"},{"why":"Gives the prior evidence that language models can skip reasoning steps, motivating the compression of thought traces.","marker":"(Liu et al., 2024)"},{"why":"Grounds the curriculum-learning principle that easy-to-hard training improves learning, which the multi-stage thought-token schedule instantiates.","marker":"(Xu et al., 2020)"},{"why":"Defines the Mistral 7B base model used for the main curriculum and NTP results.","marker":"(Jiang et al., 2023)"},{"why":"Defines the Qwen2.5 7B base model that provides the second confirmation of the gains.","marker":"(Qwen et al., 2025)"}],"fun_headline_variants":["Think faster, not deeper: 9% accuracy, zero latency","Reasoning without thinking: 9-point boost, no slowdown","No thought tokens, all reasoning: 9% boost, same speed","Fast Quiet-STaR: think less, score 9% more, no delay","Silent reasoning: 9% accuracy gain, no latency hit"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that Fast Quiet-STaR's accuracy gains over Quiet-STaR come from the easy-to-hard curriculum ordering rather than from the extra training steps, because the Quiet-STaR baselines are trained for 100 steps while the Fast Quiet-STaR 12-4 and 8-4 checkpoints receive additional stages of training.","fun_headline_variants_meta":{"raw":{"variants":["Think faster, not deeper: 9% accuracy, zero latency","Reasoning without thinking: 9-point boost, no slowdown","No thought tokens, all reasoning: 9% boost, same speed","Fast Quiet-STaR: think less, score 9% more, no delay","Silent reasoning: 9% accuracy gain, no latency hit"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001378,"raw_usage":{"total_tokens":5614,"prompt_tokens":1010,"completion_tokens":4604,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":626,"completion_tokens_details":{"reasoning_tokens":4509}},"tokens_in":626,"tokens_out":4604,"duration_ms":26381,"temperature":1.0,"reasoning_tokens":4509,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:40:48.940790+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train Quiet-STaR 8-4 for 200 steps with the same data, learning rate, and batching as Fast Quiet-STaR 8-4, and compare the four-benchmark average accuracy; if the step-matched baseline matches or exceeds Fast Quiet-STaR 8-4, the curriculum ordering is not what carries the gain.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the REINFORCE algorithm used for thought optimization and for the RL fine-tuning that converts Fast Quiet-STaR to NTP."}],"review_version":1}