{"id":"1f84ecde-4071-49a2-bfab-3e0fb33e9081","arxiv_id":"2504.15266","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"On four minimal graph-construction tasks, multi-token training (teacherless or diffusion) produces more diverse and original outputs than next-token training, and random seed prefixes can replace temperature as a diversity knob.","lead":"The paper builds small algorithmic puzzles that force a model to plan an entire answer before writing it, then shows that next-token-trained language models memorize or collapse while multi-token and diffusion-trained models generate more varied, original answers. It also finds that feeding a random prefix, called seed-conditioning, produces diverse outputs even with greedy decoding, matching or beating temperature sampling.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The transfer guarantee in Section 6.1 is load-bearing: the paper asserts that failure on the minimal tasks guarantees failure on more complex tasks, but provides no evidence for that monotonicity, and its own OOD limitation cuts against it.","rationale":"The reader's CONDITIONAL verdict is fair. I looked for internal flaws: metric conflations, capacity-control confounds, and top-K dependence. The paper addresses the capacity-control confound for teacherless training via seed-memorization (Fig 8); the top-K caveat is disclosed in Section 6.1(6a); and the diffusion-vs-NTP size mismatch is mitigated by the medium-size replication (Fig 29). The remaining soft spot is the inference from four minimal tasks to next-token prediction as a paradigm. The paper explicitly relies on Limitation 1's 'failure here guarantees failure in more complex tasks,' which is a universal monotonicity claim that no experiment in the paper tests. It is also in tension with Section 6.2's admission that real creative tasks are out-of-distribution. Thus the headline claim overreaches if the monotonicity fails. I recommend keeping the verdict CONDITIONAL, with the condition being a monotonicity or transfer check.","tokens_in":43683,"tokens_out":8089,"duration_ms":81044,"concrete_test":"Using the released code, run the same NTP-vs-teacherless comparison on strictly harder members of the same task family, e.g., Circle and Line Construction with N=15, M=25, and Triangle Discovery with deg=10, tri=10, matching the default training budgets and early-stopping on Eq. 1. The Section 6.1 transfer guarantee predicts that the NTP-vs-MTP gap should persist or widen as complexity grows. If the gap shrinks, vanishes, or reverses on any of these larger instances, the monotonicity claim is falsified and the paper's central conclusion should be narrowed to the tested regime. If the gap persists and widens, the transfer concern is substantially weakened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The experimental core is internally consistent: NTP is worse than MTP on the four tasks under the Eq. 1 metric, and the seed-memorization check (Fig 8) addresses the capacity-control confound for teacherless training. The load-bearing step is the leap from these tasks to a general claim about 'the creative limits of next-token prediction.' Section 6.1, Limitation 1, states: 'failure here guarantees failure in more complex tasks.' This is asserted, not demonstrated, and it is not a logical consequence of the experiments. A more complex task can engage the same nominal skill while offering different surface statistics, additional context, or alternative strategies (e.g., in-context scratchpads, chain-of-thought, or richer data) that allow an NTP model to sidestep the specific myopia diagnosed in the minimal setting. The paper's own Section 6.2.2 concedes that many real-world creative tasks are 'out-of-distribution' in nature, which the minimal tasks deliberately do not capture; if so, an in-distribution minimal-task failure cannot by itself guarantee failure on an OOD complex task. The central claim therefore holds as a statement about these four benchmarks, but not as an established general limit of next-token learning. This is a boundary-of-claim concern, not a defect in the measurements.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces four minimal algorithmic tasks (Sibling Discovery, Triangle Discovery, Circle Construction, Line Construction) that are intended as controlled abstractions of open-ended creative tasks, and a quantitative metric called algorithmic creativity (Eq. 1) that rewards coherent, original, and unique generations. Across these tasks, the authors compare standard next-token prediction (NTP) with multi-token objectives: teacherless training on a Gemma 2B model and discrete diffusion (SEDD) on smaller models. They report that multi-token training improves algorithmic creativity on the Gemma 2B model across all four tasks and reduces memorization, while diffusion improves creativity in three of four small-model tasks. They also introduce seed-conditioning, where an arbitrary random prefix is used at training and inference, and report that it induces diversity even under greedy decoding, performing comparably to or better than temperature sampling in several settings. The paper argues that next-token learning is myopic in these tasks and that no token ordering is friendly to NTP in three of the four tasks, extending prior work on the path-star problem.","tokens_in":43952,"tokens_out":4547,"duration_ms":44049,"significance":"If the results hold, the paper provides a clean, controllable test-bed for studying creative generation and supplies new empirical evidence that multi-token objectives can improve originality and diversity relative to NTP, going beyond the correctness-focused path-star result of Bachmann and Nagarajan. The strengths of the paper include the four well-specified tasks with objective coherence conditions, the seed-memorization control in Fig. 8 that addresses a capacity-control confound, extensive sensitivity analyses across hyperparameters and dataset sizes, and a partly released codebase. The main significance, however, depends on a transfer claim from these minimal tasks to general creative limits of next-token prediction, and that claim is not established by the experiments as presented.","major_comments":[{"comment":"The paper's central conclusion about 'the creative limits of next-token prediction' rests on the statement in Section 6.1, Limitation 1, that 'failure here guarantees failure in more complex tasks.' This monotonicity claim is load-bearing but is asserted rather than demonstrated, and the paper's own Section 6.2, Limitation 2, concedes that many real-world creative tasks are 'out-of-distribution' in nature, which the minimal tasks deliberately do not capture. An in-distribution failure on a minimal task does not logically imply failure on an out-of-distribution complex task, because a more complex task can offer additional context, in-context reasoning, or alternative strategies that let an NTP model bypass the specific myopia diagnosed in these four benchmarks. The authors should either provide evidence for the transfer claim, for example by studying a family of tasks of increasing complexity and showing that the NTP-vs-MTP gap persists, or explicitly reframe the contribution as a finding about these minimal algorithmic tasks.","section":"Section 6.1, Limitation 1; Section 6.2, Limitation 2; Abstract"},{"comment":"The main quantitative claims, such as the 'nearly a 5x factor' improvement for the discovery datasets and the statement that seed-conditioning 'results in algorithmic creativity comparable to temperature sampling,' are presented without error bars or per-run variability, despite the text stating that results are averaged over 4 runs. This makes it difficult to assess whether the observed differences are stable across seeds or training runs, particularly in the small-model comparisons where teacherless training shows no gain (Fig. 4) and where the seed-conditioning advantage is contingent on top-K sampling in Sibling Discovery and Triangle Discovery (Section 6.1, Limitation 6a). The authors should add error bars, confidence intervals, or per-run values to the main figures, or report statistical tests for the key comparisons.","section":"Section 4, Figs. 3-6"},{"comment":"The abstract claims that 'multi-token approaches, namely teacherless training and diffusion models, comparatively excel in producing diverse and original output' and that seed-conditioning 'works surprisingly as well as temperature sampling.' These statements are broader than the evidence reported in the paper: teacherless training shows no gains on the smaller GPT-2 model, diffusion is mildly worse than NTP on Sibling Discovery in that setting, and the seed-conditioning advantage does not appear for Sibling Discovery on GPT-2 and is contingent on top-K sampling for Sibling Discovery and Triangle Discovery (Section 6.1, Limitation 6a). The claims should be qualified to the specific model sizes and tasks in which the effect is observed, or the paper should provide a meta-analysis of when the advantage appears and when it does not.","section":"Abstract; Section 4.1; Section 6.1, Limitation 6a"}],"minor_comments":[{"comment":"The subscript N in the metric \\hat{cr}_N(T) is never defined in the main text; please define N or remove the subscript.","section":"Eq. (1)"},{"comment":"There is a typo in 'parametersGPT-2 (86M)' which should read 'parameters GPT-2 (86M)'.","section":"Section 4.1"},{"comment":"The sentence 'Likewise models like GPT4 and Claude currently underperform...' and the nearby discussion of Walsh et al. lack a full citation marker for the Walsh et al. work; please add the citation.","section":"Appendix J"},{"comment":"The x-axis labels such as 'seed10 greedy' and 'null temp5.0' are long and overlapping; consider rotating the labels, abbreviating the conditions, or using a grouped bar layout to improve readability.","section":"Figs. 18-20"}],"recommendation":"major_revision","confidential_remarks":"The experimental core is solid and well-controlled, and the seed-memorization check in Fig. 8 is a thoughtful control. The main obstacle is that the title and abstract promise a general claim about the creative limits of next-token prediction, while the evidence only establishes results on four minimal tasks; this is likely fixable by narrowing the claims or adding evidence for the transfer step."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read it. The empirical core is solid and the seed-conditioning result is genuinely worth knowing. On four synthetic tasks that require global coherence (sibling/triangle discovery, circle/line construction), next-token-trained Gemma 2B is clearly worse at generating coherent, original, diverse strings than teacherless multi-token training, and a small diffusion model beats a comparable GPT-2 on three of the four. The tasks are well-designed, the Eq. 1 metric is transparent, and the paper does the right ablations—LRs picked to favor NTP, a seed-memorization check (Fig 8), a top-K confound for GPT-2, and a decomposition of creativity into diversity vs. memorization.\n\nThe surprise is seed-conditioning: a random prefix, paired arbitrarily with each training example, makes a greedy-decoding Transformer produce diverse outputs that match or exceed temperature sampling. That is a real empirical phenomenon, even though it is not explained and does not transfer to diffusion.\n\nThe soft spot is not the measurements, it is the scope of the claim. Section 6.1 says failure on minimal tasks 'guarantees failure in more complex tasks,' and the title/abstract push 'the creative limits of next-token prediction' as a general result. That monotonicity is asserted, not shown, and Section 6.2.2—where they concede real creative tasks are out-of-distribution—undercuts it. A complex task can offer different surface statistics, scratchpads, or richer data that sidestep the specific myopia. So the honest reading is: NTP is measurably myopic on these four controlled benchmarks, and that is a useful benchmark result, not a proven general limit.\n\nMinor caveats, all disclosed: teacherless training shows no gain over NTP on the small models, and the seed-conditioning edge over temperature is partly contingent on top-K in some GPT-2 settings.\n\nWho it's for: people working on multi-token objectives, diversity in generation, or best-of-N exploration. I'd bring it to a reading group; I'd cite it. It deserves a serious referee; send it out, but ask the authors to reframe the conclusion as a benchmark and an open phenomenon rather than a limit theorem.","headline":"Well-run empirical study: multi-token training beats NTP on four synthetic creativity tasks and seed-conditioning is a real surprise, but the paper overreaches by asserting a transfer guarantee its own limitations undercut.","tokens_in":44473,"tokens_out":2207,"would_cite":true,"duration_ms":19314,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Multi-token training beats next-token prediction on four minimal creativity tasks.","keywords":["next-token prediction","teacherless training","discrete diffusion","algorithmic creativity","seed-conditioning","open-ended generation","creative leap-of-thought","memorization"],"falsifier":"Train a next-token Transformer on the same task families with a token order that reveals the hidden plan early (parent-first for sibling discovery, or an explicit permutation token for circle and line construction) under matched data, compute, and decoding; if algorithmic creativity reaches or exceeds the multi-token models, the gap is an artifact of token ordering rather than next-token learning itself. Alternatively, run the same comparison on real knowledge graphs with semantic nodes: if the multi-token advantage disappears, the transfer premise fails.","tokens_in":43483,"feed_emoji":"🎲","tokens_out":7876,"duration_ms":67453,"temperature":0.7,"pith_summary":"Next-token prediction is the standard training objective for language models, but this paper argues it is myopic for open-ended creative generation. On a new suite of four minimal algorithmic tasks, multi-token training objectives consistently beat next-token training at producing outputs that are coherent, unique, and absent from the training set. Two of the tasks, sibling discovery and triangle discovery, require recalling and combining facts stored in a graph within the model's weights; the other two, circle and line construction, require inventing adjacency lists that secretly resolve to a circle or a line. The paper also finds that for Transformers, conditioning on a random meaningless prefix (seed-conditioning) with greedy decoding matches or beats temperature sampling. If these results hold, they give concrete evidence that both the next-token loss and output-level temperature are replaceable design choices for open-ended generation.","feed_headline":"Multi-token training beats next-token prediction on four creativity tasks.","feed_subtitle":"Teacherless and diffusion training beat next-token models at diverse, original output; seeds can replace temperature.","key_machinery":"The machinery is a suite of minimal generative tasks in which coherence is a global property of the whole string, paired with a metric called algorithmic creativity that counts the fraction of generated strings that are coherent, unique, and not in the training set. Two combinational tasks store a knowledge graph in the model's weights: sibling discovery asks for a parent node whose two revealed siblings both belong to it, and triangle discovery asks for three nodes whose pairwise edges all exist. Two exploratory tasks ask for an edge list that can be rearranged, under some permutation, into a circle or a line graph. The argument rides on a mechanism the authors call the Clever Hans cheat: a next-token model can predict late tokens from earlier ones without learning the latent plan, and the simplicity bias of gradient training makes that shortcut irresistible, after which gradient starvation removes any push to learn the plan. Teacherless training and diffusion are the two multi-token alternatives used to break that shortcut.","core_discovery":"The paper's central claim is that creative leap-of-thought tasks are misaligned with next-token learning. In each task, the most natural generative rule is to settle a random latent plan first — the hidden parent node, the edges of a triangle, or the resolving permutation of an adjacency list — while the surface token order either reveals that plan last or not at all. A next-token learner can shortcut the plan by predicting late tokens from early ones, a \"Clever Hans cheat\" that starves the gradient signal for the latent structure and pushes the model toward memorizing its training set. Multi-token objectives, by contrast, force the model to commit to all tokens at once: teacherless training predicts every response token from the prompt alone, and discrete diffusion refines a fully corrupted sequence. The result is the paper's headline finding: in all four tasks, multi-token training attains higher algorithmic creativity than next-token training, and for Transformers, seed-conditioning with greedy decoding is as creative as temperature sampling.","pith_inferences":["The authors' own decomposition suggests seed-conditioning buys diversity while multi-token training buys both diversity and reduced memorization; an obvious extension is to combine them and measure whether the gains add.","If the Clever Hans mechanism is the true cause, then a testable prediction follows: revealing the plan token early, or hiding the sibling evidence, should shrink the next-token versus multi-token gap.","The paper asserts that failure on these minimal tasks guarantees failure on more complex ones, but that transfer claim is not demonstrated; scaling the same tasks to semantically meaningful knowledge graphs would test it directly.","Seed-conditioning helps Transformers but not the diffusion model, so the noise-injection benefit is tied to autoregressive decoding; a diffusion analog would be conditioning on random initial latent states rather than a token prefix."],"forward_implications":["Finetuning a transformer with a multi-token component should raise the diversity and originality of open-ended generations at fixed model size and compute, not just correctness.","Seed-conditioning gives practitioners a cheap diversity knob: train with arbitrary random prefixes, then decode greedily without tuning temperature.","Proposals that try to fix next-token prediction by reordering tokens or predicting infills will not help on the permutation-invariant tasks, since no token order makes the hidden plan local.","Verbatim memorization in open-ended generation is partly a symptom of fitting late tokens via local shortcuts, so multi-token objectives are a direct way to curb memorization.","The positive, if small, diversity gain at fixed ROUGE on XSUM summarization suggests the effect extends beyond toy tasks to realistic generation."],"supporting_citations":[{"why":"Supplies the path-star failure and the Clever Hans cheat mechanism that this paper extends from reasoning correctness to creative diversity.","marker":"Bachmann & Nagarajan (2024)"},{"why":"Supplies the teacherless multi-token training objective used as the main multi-token Transformer alternative.","marker":"Monea et al. (2023)"},{"why":"Another source of the teacherless objective that trains all response tokens from the prompt alone.","marker":"Tschannen et al. (2023)"},{"why":"Supplies the SEDD discrete diffusion model used as the second multi-token approach.","marker":"Lou et al. (2023)"},{"why":"Provides the taxonomy of combinational and exploratory creativity that the four task families are designed to isolate.","marker":"Boden (2003)"},{"why":"Provides the simplicity-bias evidence underlying the claim that next-token learners pick up the Clever Hans cheat.","marker":"Shah et al. (2020)"},{"why":"Supplies gradient starvation as the mechanism by which supervision for the latent plan is lost after the shortcut is learned.","marker":"Pezeshki et al. (2021)"},{"why":"Establishes the diversity-accuracy tradeoff in next-token graph navigation that the paper aims to improve through alternative training.","marker":"Khona et al. (2024)"}],"fun_headline_variants":["Multi-token learning boosts algorithmic creativity over next-token","Next-token predicts, multi-token creates: new test shows","Teacherless and diffusion beat next-token on creativity tasks","Seed-conditioning matches temperature for creative sampling","Multi-token goals unlock originality that next-token misses"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that these four minimal tasks faithfully capture the computational core of real creative leaps, so that next-token failure here predicts failure in real open-ended tasks; the paper asserts the one-way transfer without demonstrating it.","fun_headline_variants_meta":{"raw":{"variants":["Multi-token learning boosts algorithmic creativity over next-token","Next-token predicts, multi-token creates: new test shows","Teacherless and diffusion beat next-token on creativity tasks","Seed-conditioning matches temperature for creative sampling","Multi-token goals unlock originality that next-token misses"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000389,"raw_usage":{"total_tokens":2067,"prompt_tokens":979,"completion_tokens":1088,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":595,"completion_tokens_details":{"reasoning_tokens":1008}},"tokens_in":595,"tokens_out":1088,"duration_ms":7981,"temperature":1.0,"reasoning_tokens":1008,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:28:35.560106+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a next-token Transformer on the same task families with a token order that reveals the hidden plan early (parent-first for sibling discovery, or an explicit permutation token for circle and line construction) under matched data, compute, and decoding; if algorithmic creativity reaches or exceeds the multi-token models, the gap is an artifact of token ordering rather than next-token learning itself. Alternatively, run the same comparison on real knowledge graphs with semantic nodes: if the multi-token advantage disappears, the transfer premise fails.","supporting_citations":[],"review_version":1}