{"id":"0de583e2-758b-4d8a-96f5-e017535098e2","arxiv_id":"2502.01612","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Iterative self-training on a model's own correct outputs, with simple length and voting filters, lets transformers generalize to far longer arithmetic and path-finding problems than they saw in training.","lead":"This paper shows that a standard transformer can learn to solve much longer and harder versions of tasks like addition and maze solving by repeatedly training on its own correct answers, starting from easy examples. The authors report jumps from 16-digit to 100-digit reverse addition and from 5 by 5 to 9 by 9 multiplication without changing the model architecture or positional encodings.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Transcendence is load-bearing but unmeasured: the paper never reports the correctness of the self-generated labels in the reverse-addition loop behind the 100-digit headline, leaving the mechanism empirically underdetermined.","rationale":"The reader identified the transcendence property as the weakest assumption, and I agree. The paper's central result is that self-improvement enables extreme length generalization; this claim can only hold if, at every round, the model's outputs on the next difficulty level are sufficiently accurate to serve as training labels. The paper does not provide this measurement for its flagship reverse-addition experiment, even though its own analysis (Figures 5 and 22, Section 8) shows that label quality is the decisive factor between sustained improvement and error avalanche. The remaining concerns raised by the reader, such as the test-set leakage in the accelerated schedule (Section 7.2), are explicitly acknowledged and affect a secondary claim about exponential speedups, not the core phenomenon. Similarly, the lack of error bars is a reproducibility issue that would be addressed by the same multi-seed test. Therefore, the single most load-bearing concern is the unverified label-accuracy trajectory in the main self-improvement loop, and the recommended test would either confirm the transcendence assumption or force a re-interpretation of the mechanism. The verdict remains CONDITIONAL because the paper is otherwise coherent and the proposed test can settle the ambiguity.","tokens_in":66699,"tokens_out":6382,"duration_ms":60641,"concrete_test":"Re-run the reverse addition experiment (Section 4.1) with at least 3 seeds. At each self-improvement round r, compute the exact-match accuracy of the 50,000 generated labels for length 16+r against an oracle (e.g., a verified big-integer addition routine), and also record the model's accuracy on that length before fine-tuning. Report the label-accuracy-versus-round plot and the transcendence accuracy. If label accuracy stays above 95% for all rounds, the transcendence assumption is directly confirmed. If it falls below 80% before round 20 while final test accuracy still reaches 100 digits, then the success mechanism is not captured by the paper's narrative and the conclusion needs revision.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that a standard transformer can bootstrap from 10-digit to 100-digit addition without labeled data for longer lengths depends entirely on the 'transcendence' property: at each of roughly 84 self-improvement rounds, the model must generate enough correct labels at difficulty 16+r for training to improve rather than degrade performance. The paper never directly measures this label accuracy for the flagship reverse-addition experiment (Section 4.1) or for copy/reverse (Section 4.2). Instead, it shows final test accuracy (Figure 3) and an indirect analysis (Figure 5) indicating that low self-generated data accuracy hurts the next round, while Section 8 demonstrates that structured label noise can trigger an error avalanche. Without the label-accuracy trajectory, we cannot rule out that the result is a single-seed fluke or that it only holds in a narrow regime where transcendence happens to be strong. The paper's own limitations (Section 9) admit that transcendence does not hold for raw multiplication, so the property is task-dependent and must be verified, not assumed, for the headline result.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an iterative self-improvement procedure for transformer models: train on an initial labeled dataset of low difficulty, then at each round generate predictions on slightly harder instances, optionally filter those predictions with unsupervised criteria (relative length filtering and/or majority voting), and fine-tune on the combined data. The authors evaluate this recipe on reverse addition, copy/reverse, forward addition, chain-of-thought multiplication, and maze solving, reporting strong out-of-distribution extrapolation (e.g., from 10-digit to 100-digit reverse addition, from length-10 to length-120 strings, and from 5-by-5 to 10-by-10 multiplication with filtering). They also analyze an \"error avalanche\" failure mode, show that structured label noise is more harmful than uniform noise, and demonstrate that pretrained Llama models accelerate self-improvement. The framing positions the results as evidence that standard transformers, without new positional encodings or architectural changes, can achieve extreme easy-to-hard and length generalization through self-training with simple filters.","tokens_in":66795,"tokens_out":8187,"duration_ms":77330,"significance":"If the reported results hold, the paper would make a strong empirical contribution: it suggests that iterative self-training with weak, task-agnostic filtering can bootstrap a vanilla decoder-only transformer far beyond its training difficulty, complementing prior work that relies on architectural or positional-encoding modifications. Strengths include the breadth of tasks, the controlled difficulty schedules, the explicit analysis of self-generated label quality in Figure 5, the synthetic noise experiments in Section 8, and the ablation of majority-voting variants in Figure 31. The paper is generally well written and the experimental setup is described in enough detail to be reproducible. However, the significance of the headline claims is currently limited by (i) the test-set-leaked accelerated schedule used to claim exponential improvement, (ii) the absence of variance estimates for the flagship length-generalization curves, and (iii) the lack of a per-round measurement of self-generated label accuracy for the 100-digit reverse-addition trajectory that supports the central transcendence mechanism.","major_comments":[{"comment":"The claim of \"exponential improvement\" is not supported as stated because the accelerated schedule directly uses test-set accuracy to choose which difficulty levels to sample. The paper acknowledges this: \"the sampling schedule in this proof-of-concept experiment leverages information about test set accuracy to determine the extra OOD lengths to sample.\" Since the training data at each round is selected from lengths that achieve above 99% test accuracy, the reported extrapolation is partly a product of oracle knowledge of the test distribution, not of the self-improvement procedure alone. Moreover, \"exponential\" is not established: Figure 18 shows a steeply rising but concave curve with no exponential fit or functional form. Please remove or substantially qualify the exponential claim, and provide a leak-free version that selects difficulties using only a held-out validation set or a model-based proxy.","section":"§7.2, Fig. 18"},{"comment":"The headline results—reverse addition to 100 digits, copy/reverse to length 120, forward addition to length 75, and the accelerated schedules—are reported without error bars or multiple seeds. The paper's own Figure 26 shows that 11-digit OOD accuracy varies dramatically across seeds (e.g., 0.617 for seed 44 vs. 0.998 for seed 41), so the claim of \"perfect generalization\" in Figure 3 is not yet convincing. Please report mean and standard deviation across at least 5 independent seeds for the central length-generalization curves, and state explicitly how many runs underlie each plotted line.","section":"Figs. 3, 4, 10, 18"},{"comment":"The success of the reverse-addition self-improvement loop depends on the transcendence property: at each of roughly 84 rounds, the model must generate enough correct labels at difficulty 16+r to avoid an error avalanche. Figure 5 provides a scatter of data-accuracy versus next-round accuracy for some rounds, but the paper never reports the per-round label accuracy trajectory for the specific experiment behind Figure 3. Without this trajectory, one cannot verify that transcendence held throughout the loop, assess the margin of safety near each step, or rule out that the result hinges on a few early high-accuracy rounds. Please include a plot of self-generated label accuracy (and, where filtering is used, filtered-data accuracy) versus round for the reverse-addition and copy/reverse experiments.","section":"§4.1, §5.1, Fig. 5"},{"comment":"The filtering thresholds are chosen ad hoc and no sensitivity analysis is provided. Forward addition uses a length threshold of 2, multiplication uses 10 tokens, majority voting uses 4-out-of-5 consensus for multiplication and 3-out-of-3 for mazes. The paper presents these filters as \"general-purpose\" and central to success, but it does not show how performance varies with the threshold τ or with the consensus level. Without such an analysis, the positive results for forward addition, multiplication, and mazes could be knife-edge phenomena tied to particular hand-selected constants. Please add ablations over τ and over the consensus threshold for at least one task in each category.","section":"§5.2, §6.1, §6.2"}],"minor_comments":[{"comment":"The abstract states the method works \"without any changes to the positional embeddings,\" but Section 3 explicitly removes positional embeddings and uses NoPE. This is itself a deliberate design choice (indeed, NoPE is a known technique for length generalization). Please rephrase the abstract to acknowledge that NoPE is used, or clarify that \"no additional\" positional encoding beyond the absence of one is meant.","section":"Abstract, §3"},{"comment":"The fine-tuning set is written as \"D1 ∪ ⋯ ∪ Dr\" without D0, unlike the reverse-addition description. If the original labeled data is intentionally omitted, say so; if this is a typo, correct it.","section":"§4.2"},{"comment":"The symbol τ is used both for the length-filter threshold and for the majority-vote consensus threshold. Please use distinct symbols (e.g., τ_len and τ_maj) to avoid ambiguity.","section":"§5.2, Fig. 8"},{"comment":"The label \"drop-preturb\" in Figure 24 should be \"drop-perturb.\"","section":"§8.2"},{"comment":"The statement that \"the amount of extra OOD generalization increases roughly linearly\" with round is not quantified; Figure 17 shows different slopes for copy and reverse addition. A linear fit or a statement of the observed range would be more precise.","section":"§7.2"},{"comment":"For the pretrained Llama experiments, the paper says \"keep the embedding component unchanged\" after replacing the tokenizer with a character tokenizer, which is internally inconsistent because a different vocabulary size requires a different embedding. Please clarify how the embedding layer is handled.","section":"§7.3, Appendix C.3"}],"recommendation":"major_revision","confidential_remarks":"This paper fits the scope of the journal and the core phenomenon—self-improvement enabling large length extrapolation on several algorithmic tasks—is potentially publishable. The main reasons for major revision are the test-leaked accelerated schedule that undermines the exponential claim, the lack of error bars on the flagship results, and the missing per-round label-accuracy trajectory for the headline reverse-addition experiment. The threshold sensitivity analysis is also needed to substantiate the claimed generality. I recommend that the editor treat the abstract's 'exponential' and 'no changes to positional embeddings' wording as items that must be corrected during revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: this paper demonstrates something real, but the flagship claim is less clean than the abstract implies. A standard NoPE transformer, bootstrapping from 10-digit to 100-digit reverse addition and up to 9-by-9 multiplication via iterative self-training with simple unsupervised filters, is new and, as far as I can tell, holds up. The breadth across reverse/forward addition, copy/reverse, multiplication, and maze solving makes the point that this is not a one-off phenomenon.\n\nWhat the paper does well: it applies the STaR/ReST recipe without inventing a new framework, and it carefully documents where vanilla self-improvement collapses (multiplication, maze without filtering). The error avalanche analysis in Section 8 is the most valuable part: characterizing the mistake structure—digit drops near the end, off-by-one errors—and showing that injected structured noise causes a crash while uniform noise is tolerated, is a genuinely useful contribution. The authors are also honest, explicitly admitting in Section 7.2 that the accelerated schedule uses test-set accuracy and listing real limitations in Section 9, including the failure of raw multiplication.\n\nThe main soft spot, and the stress-test note gets this right: transcendence is the engine, but the paper never reports the actual accuracy of self-generated labels in the reverse-addition loop that produces the 100-digit headline. Figure 5 is indirect, and Figure 3 shows only final accuracy. The mechanism remains underdetermined without the label-quality trajectory. The authors almost certainly have the data, so this is fixable, but it is a real gap. Two smaller issues: the \"exponential improvement\" claim is partly circular because the accelerated schedule picks difficulties using test information, and the main figures lack error bars (multiplication averages over 5 seeds but shows no variance). The \"state-of-the-art\" claim would also benefit from a quantitative comparison to Abacus, FIRE, or other positional-encoding methods.\n\nWho this is for: researchers working on length generalization, weak-to-strong learning, or self-training. It deserves a serious referee. I would send it out, asking for the label-accuracy trajectories, multiple seeds with variance reported on the headline experiments, and a tempered reading of the exponential claim.","headline":"Real results with a missing measurement: the 100-digit addition headline works, but the paper never shows the label-quality trajectory that the self-improvement loop depends on.","tokens_in":67439,"tokens_out":2400,"would_cite":true,"duration_ms":23837,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By repeatedly training on its own filtered solutions to slightly harder problems, a standard transformer can learn to add 100-digit numbers and solve mazes far beyond its training distribution.","keywords":["self-improvement","length generalization","easy-to-hard generalization","transformer","arithmetic reasoning","chain-of-thought","majority voting","error avalanche"],"falsifier":"Measure the base model's exact-match accuracy on the first out-of-distribution difficulty before any self-improvement round, for example 17-digit reverse addition after training on 1-to-16-digit examples. If this accuracy is near zero, the first self-improvement round injects mostly wrong labels and the loop cannot bootstrap, whereas the paper's account predicts it is well above zero.","tokens_in":66398,"feed_emoji":"🧮","tokens_out":8076,"duration_ms":69009,"temperature":0.7,"pith_summary":"This paper argues that transformers' failure to generalize beyond their training length or difficulty does not require architectural fixes: an iterative loop in which the model labels slightly harder problems, filters those labels, and fine-tunes on them can push performance far outside the original training distribution. Starting with supervised labels for roughly 1-to-16-digit reverse addition, the authors report near-perfect accuracy on additions beyond 100 digits, and similar one-step-at-a-time gains for string copying and reversing, forward addition, chain-of-thought multiplication, and maze shortest-path finding. The method relies on the observed \"transcendence\" property: a model trained on easy instances can produce correct answers for marginally harder instances. Simple unsupervised filters, such as dropping outputs that are too short relative to the batch and keeping only majority-voted answers across several seeds, keep the self-generated data clean enough to avoid an error avalanche. The paper also reports that pretrained models improve faster and that, under filtered self-training, out-of-distribution performance can grow exponentially across rounds.","feed_headline":"Self-training takes transformers from 16-digit to 100-digit addition","feed_subtitle":"By labeling and filtering its own harder problems, a stock transformer solves tasks far beyond its training set.","key_machinery":"The load-bearing mechanism is the iterative self-improvement loop: train on initial supervised data $D_0$ of difficulty at most $d_0$; at round $r$, sample inputs of difficulty $d_r=d_0+r$ (or a finer schedule), let the current model generate labels, filter them, and fine-tune on $D_0\\cup D_1\\cup\\cdots\\cup D_r$. The two unsupervised filters are relative length filtering, which removes outputs shorter than the batch maximum minus a threshold, and majority voting, which keeps outputs agreed upon by at least a threshold fraction of several seed-trained models. This loop leverages the transcendence property: a model trained on easy problems can answer problems one difficulty step harder, giving the next round a stream of mostly correct self-generated labels.","core_discovery":"The central discovery is that the \"transcendence\" phenomenon, where a model trained on easy instances can solve slightly harder ones, can be chained into extreme length and difficulty generalization. By repeatedly collecting the model's predictions on the next difficulty level, filtering them, and continuing training on the combined old and new data, a standard transformer progresses one step at a time from its initial distribution to lengths and difficulties far beyond it. The authors demonstrate this on reverse addition, string copy and reverse, forward addition, chain-of-thought multiplication, and maze solving, with no changes to positional embeddings or the transformer architecture. They further show that unsupervised filtering based on relative output length and majority voting is sufficient to prevent label-noise avalanches, and that the safe range of difficulty the model can be asked to self-label widens with each round.","pith_inferences":["The paper leaves implicit that the recipe only needs a difficulty ladder whose rungs are one step apart; any problem family with a monotone, quantifiable notion of hardness, such as proof length, code complexity, or planning horizon, is a candidate for this kind of bootstrapping.","The error-avalanche simulations suggest that filters targeting structurally short or incomplete outputs may be more valuable than generic noise filters; outside arithmetic, truncation and dropped reasoning steps are common failure patterns, so a domain-agnostic \"completeness\" filter may substitute for exact verification.","Because the accelerated schedule in Section 7.2 used test-set accuracy to choose next-round difficulties, a practical deployment would need an unsupervised confidence proxy, such as the agreement rate across majority-voted models or the length-filter survival rate, to decide how far to push each round.","The majority-voting ablation shows that independent training trajectories matter more than simply training more data, which hints that diversity in the models being voted over is the active ingredient rather than ensemble size alone."],"forward_implications":["On reverse addition, copying, and reversing, the loop reaches near-perfect accuracy far outside the training range, such as over 100-digit addition and over 120-character strings, even without explicit filtering.","For forward addition, multiplication, and maze solving, unsupervised length filtering and majority voting are what sustain the loop; without them, accuracy collapses after a few rounds.","The safe difficulty range widens with each round, so an accelerated schedule that samples several difficulties per round reaches the same generalization in fewer rounds, e.g., 10-by-10 multiplication in 19 rounds instead of 41.","Pretrained 1B and 3B models bootstrap the loop faster than a from-scratch 14M model, so the recipe composes with model scale.","Filtered self-improvement can yield exponential growth in out-of-distribution accuracy across rounds, while the error-avalanche analysis identifies structured short-answer noise, rather than pure random label noise, as the main failure mode."],"supporting_citations":[{"why":"Supplies the transcendence property the loop relies on: models trained on easy instances can generalize slightly beyond the difficulty of their training data.","marker":"Zhang et al., 2024"},{"why":"Provides the STaR template of generating candidate labels, filtering them, and fine-tuning on the filtered set, which the self-improvement loop builds on.","marker":"Zelikman et al., 2022"},{"why":"Provides the ReST iterative self-training framework that motivates alternating between prediction collection and continued training.","marker":"Gulcehre et al., 2023"},{"why":"Establishes the No Positional Encoding (NoPE) setting that the experiments deliberately keep unmodified to show the method does not rely on positional-encoding changes.","marker":"Kazemnejad et al., 2024"},{"why":"Introduces the reverse addition task used as a primary length-generalization benchmark and motivates its difficulty for standard transformers.","marker":"Lee et al., 2023"},{"why":"Motivates why forward addition and copying are hard, providing the RASP-L framing against which the self-improvement results are compared.","marker":"Zhou et al., 2023"},{"why":"Supplies the chain-of-thought data format for multiplication that makes slight out-of-distribution generalization possible in the first place.","marker":"Deng et al., 2024"},{"why":"Shows that dataset priming with a few long examples can enable length generalization, which the paper interprets the filtering-based self-improvement as automating.","marker":"Jelassi et al., 2023"},{"why":"Supports the interpretation that deep models can tolerate large amounts of label noise, which the paper uses to explain why memorizing past mistakes does not always cause collapse.","marker":"Rolnick, 2017"}],"fun_headline_variants":["Transformers teach themselves 100-digit addition","From 10 to 100 digits: self-taught transformers","Chained self-training pushes transformers to 100-digit sums","Self-improving loop lets transformers add 100 digits","Stock transformer masters 100-digit addition via self-curriculum"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method depends on the empirical regularity that a model trained on easy problems can solve problems one step harder; if that one-step \"transcendence\" fails at any round, the self-generated labels become garbage and the loop breaks.","fun_headline_variants_meta":{"raw":{"variants":["Transformers teach themselves 100-digit addition","From 10 to 100 digits: self-taught transformers","Chained self-training pushes transformers to 100-digit sums","Self-improving loop lets transformers add 100 digits","Stock transformer masters 100-digit addition via self-curriculum"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001292,"raw_usage":{"total_tokens":5227,"prompt_tokens":849,"completion_tokens":4378,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":465,"completion_tokens_details":{"reasoning_tokens":4300}},"tokens_in":465,"tokens_out":4378,"duration_ms":28940,"temperature":1.0,"reasoning_tokens":4300,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T14:49:01.698479+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the base model's exact-match accuracy on the first out-of-distribution difficulty before any self-improvement round, for example 17-digit reverse addition after training on 1-to-16-digit examples. If this accuracy is near zero, the first self-improvement round injects mostly wrong labels and the loop cannot bootstrap, whereas the paper's account predicts it is well above zero.","supporting_citations":[{"cited_title":"Transcendence: Generative Models Can Outperform The Experts That Train Them","cited_arxiv_id":"2406.11741","evidence_quote":"Supplies the transcendence property the loop relies on: models trained on easy instances can generalize slightly beyond the difficulty of their training data."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the STaR template of generating candidate labels, filtering them, and fine-tuning on the filtered set, which the self-improvement loop builds on."}],"review_version":1}