Pith. sign in

REVIEW 4 major objections 5 minor 35 references

Hierarchical Latent Prediction for Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Adding a coarse, k-step-ahead latent prediction to language model pretraining improves coding and multi-step reasoning, and speeds speculative decoding with no inference overhead.

desk verdict A plausible incremental extension of NextLat whose central representation claim is under-supported by the reported evidence; worth refereeing if experimental rigor is enforced. read the letter →

arxiv 2608.05806 v1 pith:VHLS6M2T submitted 2026-08-06 cs.CL cs.AI

Chang Shi , Tim Pearce , Manan Tomar , Siddhartha Sen , John Langford This is my paper · ORCID
classification cs.CLcs.AI
keywords hierarchicallatentpredictionlanguagemodelpretrainingnext-tokenspeculativedecodingspacedynamicsmulti-scalerepresentationlearninglong-horizonreasoning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that language models can be pre-trained more effectively by adding a hierarchical latent prediction objective on top of the standard next-token loss. Instead of only predicting the next token or the next latent one step ahead, HiLP trains a sliding-window module that summarizes the last $k$ hidden states into an 'abstract' latent, and a predictor that forecasts this abstract latent $k$ steps into the future. The authors report that this coarse-scale target reduces the error accumulation that limits single-step latent rollouts, and that models trained this way score higher on HumanEval and on symbolic multi-step reasoning benchmarks, while also accepting more tokens per step in speculative decoding. All the auxiliary machinery is discarded at inference time, so the pre-trained model runs with no added latency.

What carries the argument

The load-bearing mechanism is the higher-level 'abstract latent' $u_t = \mathrm{SWA}(h_{t-k+1:t})$, a deterministic sliding-window attention summary of the last $k$ backbone hidden states, paired with the $k$-step-ahead predictor $p_\phi(u_{t+k}|u_t)$. The $k$-step lookahead in a coarser space is what distinguishes HiLP from single-step NextLat: the higher-level pathway supervises long-horizon structure directly instead of unrolling one step at a time and compounding error. The stop-gradient operators are equally load-bearing: targets such as $\mathrm{sg}[u_{t+k}]$ are detached so that the higher-level loss shapes the SWA module and $p_\phi$ without sending gradient pressure back into the backbone latents, leaving the token-level supervision to govern $h_t$.

What would settle it

Train the identical HiLP configuration but freeze the SWA module to a fixed random projection of the last $k$ hidden states (e.g., a random linear map that is not updated), keeping $p_\phi$, the combined head, and all loss weights unchanged. If the HumanEval and speculative-decoding gains over NextLat persist, the learned hierarchy is not the cause of the reported improvement.

Watch

Extended reading notes

Core claim

HiLP's central discovery is that a second, coarser temporal scale of self-prediction can be added to latent-space pretraining without changing the inference-time model. Concretely, the backbone produces lower-level latents $h_t = G_\theta(X_{1:t})$; a sliding-window attention module produces $u_t = \mathrm{SWA}(h_{t-k+1:t})$; and a higher-level dynamics model $p_\phi$ is trained to predict $\hat{u}_{t+k} = p_\phi(u_t)$, with the target $u_{t+k}$ stop-gradiented. The total training loss combines the standard NTP loss, the NextLat-style lower-level rollout losses and KL term, the higher-level consistency loss $L_u = \mathbb{E}_t[\mathrm{SmoothL1}(\mathrm{sg}[u_{t+k}], \hat{u}_{t+k})]$, and a combined-NTP loss that conditions next-token prediction on both latents. At optimality, the losses enforce the consistency conditions $p_\phi(u_{t+k}|u_t) = P(u_{t+k}|X_{1:t})$ alongside the usual next-token and lower-transition consistencies. The paper reports HumanEval pass@1 of 11.33 (versus 10.58 for NextLat and 8.77 for NTP) and an average accepted tokens of 3.41 versus 3.26 on code data, and presents latent cross-entropy curves showing lower long-horizon rollout error.

Load-bearing premise

The load-bearing premise is that the stop-gradiented target $\mathrm{sg}[u_{t+k}]$ is a stable and useful teaching signal even though $u_{t+k}$ is produced by the same backbone and SWA modules being trained; if this moving target collapses or drifts trivially, the claimed longer-horizon representation benefit fails.

Editorial extensions

If this is right

  • If HiLP is right, latent-space auxiliary objectives for language pretraining should be designed at multiple temporal scales rather than as single-step transitions.
  • The reported speculative-decoding gains (3.41 vs 3.26 average accepted tokens on code data, with no inference overhead) imply that self-predictive pretraining can make autoregressive decoding cheaper without enlarging the deployed model.
  • The HumanEval and DCLM benchmark improvements support the view that pure teacher-forced next-token prediction under-trains long-horizon structure, and that explicit coarse-scale lookahead during training helps.
  • Because the hierarchical apparatus is dropped at inference, HiLP is a pure pretraining intervention with zero added decoding latency, so it can be layered onto existing pretraining runs as a drop-in objective.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not study whether the stop-gradiented moving target $u_{t+k}$ stays stable, so the natural next test is to measure the cosine drift or collapse of SWA latents over training; if the abstract latent becomes trivial, the hierarchy is not doing the claimed work.
  • If the hierarchy is the cause of the gains, a similar multi-scale latent prediction should transfer to other modalities, e.g., image or audio pretraining, where long-horizon coherence matters.
  • The fixed lookahead $k$ is a hyperparameter; a plausible extension is to make $k$ position-dependent, set by the confidence of $p_\phi$, which the paper lists as future work but does not test.
  • The reported gains might partly come from adding a second auxiliary loss rather than from the hierarchical structure; a control with an equal-parameter single-scale auxiliary objective would separate these explanations.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Hierarchical Latent Prediction (HiLP), an auxiliary training objective for causal language models that adds a higher-level abstract latent u_t = SWA(h_{t-k+1:t}) and a k-step-ahead predictor p_phi, along with a combined NTP head, on top of the standard NTP loss and a lower-level NextLat objective. At inference, only the trunk and the standard NTP head are retained, yielding zero additional latency. The authors pretrain 1B-parameter models on 100B tokens and report benchmark results on HumanEval and DCLM, plus speculative-decoding efficiency, claiming consistent gains over NTP, MTP, and NextLat baselines.

Significance. If the hierarchical mechanism performs as claimed, HiLP is a simple training-time intervention that could improve long-horizon latent prediction and speculative decoding without adding inference cost, and the paper includes a complete algorithmic description, hyperparameter table, and efficiency measurements. However, the central claim that HiLP 'leads to longer-horizon coherent belief state representation' is not yet established: the higher-level objective admits a trivial constant solution, the empirical comparisons lack statistical grounding, and the representation-quality evidence is partly circular. The paper's value depends on resolving these points.

major comments (4)
  1. [Section 2, Eq. (D), Algorithm 1 lines 9-13] The higher-level objective L_u = E SmoothL1(sg[u_{t+k}], p_phi(u_t)) admits a trivial constant solution: a constant u_t and a constant output of p_phi drive the loss to zero, and because the target is stop-gradiented there is no pressure to preserve information in u_t. The detached path in Eq. (E) (\tilde u_t = SWA(sg[h_{t-k+1:t}])) allows the combined head p_rho to learn to ignore \tilde u_t entirely, in which case the SWA module receives no gradient from L_cntp and condition (III) holds vacuously. The paper reports no diagnostic for u_t (e.g., effective rank, gradient norm, or probe accuracy), and Fig. 3 evaluates only lower-level latent rollout. To support the central claim of a longer-horizon coherent belief state, the authors should either add a collapse-prevention term or provide evidence that u_t is non-degenerate under the trained model.
  2. [Section 3, Tables 1-3 and Fig. 2] All reported benchmark numbers are single-run values without error bars, confidence intervals, or multiple seeds. The differences that support the main empirical claim are small (HumanEval pass@1 11.33 for HiLP vs 10.58 for NextLat; average accepted tokens 3.41 vs 3.26 on code data, and 2.59 vs 2.57 on Nemotron-ClimbMix), so without variance information the reader cannot determine whether the improvements are statistically meaningful. The DCLM symbolic/multi-step results appear only in a figure (Fig. 2) with no numerical values, preventing quantitative comparison and reproduction. Please provide seed-level numbers, confidence intervals, and tabulated DCLM values.
  3. [Section 3, Table 4 and Appendix D] The paper provides no ablations for the method's key hyperparameters: latent window size and lookahead k, rollout depth d, or loss weights (lambda_h, lambda_KL, lambda_u, lambda_cntp). More importantly, there is no ablation that removes the higher-level channel while keeping the lower-level NextLat objective, so the reported gains cannot be attributed specifically to the hierarchical component rather than to the added SWA module and combined head. The only ablation (Appendix D) concerns the input-combination implementation and does not address the hierarchical mechanism. I request an ablation of the hierarchy (e.g., setting lambda_u=0 and lambda_cntp=0, or replacing u_t with a fixed projection) to demonstrate that the higher-level latent is the source of the improvement.
  4. [Section 4, Fig. 3] The evidence for the 'longer-horizon coherent belief state' relies on latent prediction cross-entropy curves (Fig. 3), which are closely related to the training objective (Eq. D) itself. Reporting lower loss on essentially the same objective that is being optimized is partly circular and does not establish that the learned representation is more coherent. The authors should evaluate representation quality with an independent probe or a held-out task, or at minimum clarify the relation between the plotted curves and the training losses.
minor comments (5)
  1. [Section 3.1] The phrase 'show that HiLP is improve performance' should read 'shows that HiLP improves performance'.
  2. [Section 5.2 and References] The citations for CALM and Coconut both appear as 'Shao et al., 2025', but the reference list contains only one Shao et al. entry (Continuous autoregressive language models); the CALM paper is missing from the references.
  3. [Section 2, Eq. (D) and Table 4] The symbol k is used for both the SWA window size and the lookahead offset; the paper should state explicitly whether these are coupled or independently tunable.
  4. [Table 2] The per-K columns for MTP end at K=4 without a footnote; a note explaining MTP's shorter evaluated horizon would help the reader interpret the comparison.
  5. [Algorithm 1] The KL term uses a frozen copy p_sg_theta; please clarify whether this is implemented via stop-gradient on parameters or via a periodic parameter copy.

Circularity Check

1 steps flagged · score 4.0 of 10

Fig. 3’s “longer-horizon belief state” evidence replays the optimized latent losses; the benchmark results remain independent.

  1. fitted input called prediction [Section 4 (Discussion) and Fig. 3 caption]
    "The latent prediction cross-entropy losses across rollout horizons up to 8 steps ahead are shown in 3. As the curves show, HiLP preserves the near-term cross-entropy of NextLat while producing lower future prediction error at longer horizons, suggesting that the hierarchical latent mitigates multi-step error accumulation. [Fig. 3 caption:] HiLP has comparable near-term latent prediction loss and lower long-horizon future and excess cross-entropy, indicating more stable multi-step latent rollout."

    The “latent prediction cross-entropy losses” plotted in Fig. 3 are the same losses HiLP minimizes during training: Eq. (B) L_h = E_t[1/d Σ SL1(sg[h_{t+i}], ħ_{t+i})] and Eq. (D) L_u = E_t[SmoothL1(sg[u_{t+k}], û_{t+k})]. Reporting that HiLP has “lower future prediction error at longer horizons” is therefore reporting that the fitted objective is smaller on the web/code splits, not an independent measurement of whether the abstract latent u_t encodes longer-horizon structure. No external probe of u_t (e.g., effective rank, downstream probe, or controlled intervention) is provided. The HumanEval, DataComp, and speculative-decoding results are external benchmarks, so the circularity is localized to the “longer-horizon coherent belief state” claim.

full rationale

The paper’s headline empirical results are evaluated with external benchmarks: HumanEval pass@1 (11.33 vs 10.58 for NextLat), DCLM symbolic/multi-step reasoning, and speculative decoding accepted tokens (3.41 vs 3.26 on code). These comparisons are independent of the training objective and give the core practical claim genuine external grounding. The method itself is not derived from a self-citation chain; NextLat is used as an experimental baseline, and the overlapping-author citations are background rather than load-bearing. The one identifiable circular element is the evidence offered for “longer-horizon coherent belief state representation.” Fig. 3 plots latent prediction cross-entropy at rollout horizons up to 8, which is precisely the family of losses L_h (Eq. B) and L_u (Eq. D) that HiLP optimizes during pretraining. Showing that HiLP has lower future prediction error on those same curves is, to a first approximation, showing that the optimized objective is smaller; it does not by itself certify that the higher-level latent captures non-trivial multi-scale structure. That conclusion would require an independent diagnostic of the latent space. Because the benchmark tables provide independent support for the practical utility claims, the circularity is partial and localized, hence a score of 4 rather than 6 or higher.

Assumptions & free parameters 4 free parameters · 5 assumptions · 1 invented entities

The central claim rests on the chosen latent window size, lookahead, rollout depth, and loss weights, plus a set of unverified assumptions about the usefulness of a moving stop-gradiented latent target and the stability of the stop-gradient separation. No external anchor exists for the abstract latent space, and the main mechanism evidence is the training objective itself. These are not fatal, but they mean the contribution is a plausible heuristic rather than a demonstrated mechanism.

free parameters (4)
  • Latent window size and lookahead offset k = 4
    Controls temporal abstraction scale and the higher-level prediction horizon; chosen without ablation and central to the claimed benefit.
  • Lower-level rollout depth d = not stated in Table 4
    Algorithm 1 uses a rollout depth d for the lower-level losses; the value is not reported, and it directly determines the multi-step error-accumulation behavior the paper claims to improve.
  • Auxiliary loss weights (lambda_h, lambda_KL, lambda_u, lambda_cntp) = 10.0, 0.1, 1.0, 0.5
    Hand-chosen, no sensitivity analysis; these weights set the balance between token-level and latent objectives and could drive the observed effects.
  • Latent learning rate = 1e-3
    Separate learning rate for latent modules; no schedule or sensitivity reported.
assumptions (5)
  • domain assumption Language contains inherent hierarchical structure at multiple scales (characters, phrases, sentences, paragraphs) that can be exploited by hierarchical latent prediction.
    Motivates the method in Section 1 (third paragraph) and is not independently measured; a false premise would weaken the motivation, though the method could still work.
  • ad hoc to paper Latent-space self-prediction with a moving, stop-gradiented target provides a useful and stable training signal.
    Section 2, Eq. (D) and Algorithm 1 rely on u_{t+k} generated by the same network with stop-gradient; no collapse or stability analysis is provided.
  • ad hoc to paper Stop-gradient separation in the combined NTP head prevents competing optimization signals and lets the SWA module learn useful abstractions.
    Stated as design rationale after Eq. (E) in Section 2, but not verified experimentally; if false, the combined head could distort the lower latents.
  • domain assumption At optimality, the five loss terms can simultaneously satisfy the consistency conditions (I)-(IV).
    The consistency conditions are presented without proof of joint satisfiability under the chosen parameterization; this is a standard assumption for multi-task objectives, but it is not established.
  • domain assumption The hidden-state latent space is sufficiently continuous and metric that SmoothL1 distance between predicted and actual latents is a meaningful training signal.
    Inherited from NextLat and used in Eq. (B) and (D); no analysis of curvature or collapse is provided.
invented entities (1)
  • Higher-level abstract latent u_t
    purpose: A sliding-window summary of the last k lower latents, used as the state for a k-step-ahead predictor and as conditioning for the combined NTP head.
    Introduced as a new internal representation. It is removed at inference and has no external falsifiable handle; its value is only indirectly evidenced by downstream benchmark changes and by training-loss curves.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hierarchical Latent Prediction for Language Models." pith.science (2026). https://pith.science/paper/VHLS6M2T

@misc{pith2026260805806,
  author       = {Pith},
  title        = {Pith review of: Hierarchical Latent Prediction for Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VHLS6M2T}},
  note         = {Machine review of arXiv:2608.05806}
}
read the original abstract

While standard Next-Token Prediction (NTP) lays the foundation of language model pre- training, its teacher-forced training paradigm may not be optimal for long-horizon reasoning and planning. Recent works such as Multi-Token Prediction (MTP) and Next-Latent prediction (NextLat) try to mitigate the problem through predicting multiple future tokens and self-supervised prediction in the latent space. However, those auxiliary objectives either have a limited horizon or suffer from compounding error from multi-step rollout. We introduce Hierarchical Latent Prediction (HiLP), which introduces an auxiliary higher-level abstract latent to help reduce the error accumulation effect in latent-space rollouts. Experiments show that HiLP can lead to longer-horizon coherent belief state representation and demonstrate the effectiveness of our method across coding and multi-step reasoning benchmarks, and offers more speculative decoding efficiency.

Figures

Figures reproduced from arXiv: 2608.05806 by the authors.

Figure 1
Figure 1. Overview of HiLP with sliding-window size [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. DCLM symbolic/multi-step eval results [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Latent cross-entropy curves for NextLat and HiLP on web and code splits. HiLP has comparable near-term [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: glu_cross input-combination mode, which gates a projected lower latent with the conditioning latent while preserving hidden width D [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 13 canonical work pages

  1. [1]

    arXiv preprint arXiv:2412.08821 , year=

    Large concept models: Language modeling in a sentence representation space , author=. arXiv preprint arXiv:2412.08821 , year=

    arXiv

  2. [2]

    Advances in Neural Information Processing Systems , volume=

    Nemotron-climb: Clustering-based iterative data mixture bootstrapping for language model pre-training , author=. Advances in Neural Information Processing Systems , volume=

    work page

  3. [3]

    arXiv preprint arXiv:2604.03208 , year=

    Hierarchical planning with latent world models , author=. arXiv preprint arXiv:2604.03208 , year=

    arXiv

  4. [4]

    ACM Transactions on Intelligent Systems and Technology , year=

    Hierarchical multimodal llms with semantic space alignment for enhanced time series classification , author=. ACM Transactions on Intelligent Systems and Technology , year=

    work page

  5. [5]

    arXiv preprint arXiv:2510.27688 , year=

    Continuous Autoregressive Language Models , author=. arXiv preprint arXiv:2510.27688 , year=

    arXiv

  6. [6]

    Advances in Neural Information Processing Systems , volume=

    Learning discrete concepts in latent hierarchical models , author=. Advances in Neural Information Processing Systems , volume=

    work page

  7. [7]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Byte latent transformer: Patches scale better than tokens , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    work page

  8. [8]

    arXiv preprint arXiv:2507.07955 , year=

    Dynamic chunking for end-to-end hierarchical sequence modeling , author=. arXiv preprint arXiv:2507.07955 , year=

    arXiv

Show all 35 references
  1. [9]

    Advances in Neural Information Processing Systems , volume=

    Megabyte: Predicting million-byte sequences with multiscale transformers , author=. Advances in Neural Information Processing Systems , volume=

  2. [10]

    arXiv preprint arXiv:2503.21801 , year=

    Efficient joint prediction of multiple future tokens , author=. arXiv preprint arXiv:2503.21801 , year=

    arXiv

  3. [11]

    arXiv preprint arXiv:2403.06963 , year=

    The pitfalls of next-token prediction , author=. arXiv preprint arXiv:2403.06963 , year=

    arXiv

  4. [12]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    The option-critic architecture , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  5. [13]

    arXiv preprint arXiv:2511.08544 , year=

    Lejepa: Provable and scalable self-supervised learning without the heuristics , author=. arXiv preprint arXiv:2511.08544 , year=

    arXiv

  6. [14]

    arXiv preprint arXiv:2302.01318 , year=

    Accelerating large language model decoding with speculative sampling , author=. arXiv preprint arXiv:2302.01318 , year=

    arXiv

  7. [15]

    Advances in neural information processing systems , volume=

    The evolution of statistical induction heads: In-context learning markov chains , author=. Advances in neural information processing systems , volume=

  8. [16]

    Better & faster large language models via multi-token prediction, 2024 , author=

    2024

  9. [17]

    International Conference on Learning Representations , volume=

    Learning hierarchical world models with adaptive temporal abstractions from discrete latent dynamics , author=. International Conference on Learning Representations , volume=

  10. [18]

    Advances in Neural Information Processing Systems , volume=

    Deep hierarchical planning from pixels , author=. Advances in Neural Information Processing Systems , volume=

  11. [19]

    International Conference on Learning Representations , volume=

    Hierarchical world models as visual whole-body humanoid controllers , author=. International Conference on Learning Representations , volume=

  12. [20]

    International Conference on Learning Representations , volume=

    The belief state transformer , author=. International Conference on Learning Representations , volume=

  13. [21]

    2, 2022-06-27 , author=

    A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27 , author=. Open Review , volume=

    2022

  14. [22]

    International Conference on Machine Learning , pages=

    Fast inference from transformers via speculative decoding , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  15. [23]

    arXiv preprint arXiv:2603.19312 , year=

    Leworldmodel: Stable end-to-end joint-embedding predictive architecture from pixels , author=. arXiv preprint arXiv:2603.19312 , year=

    arXiv

  16. [24]

    arXiv preprint arXiv:2401.15077 , year=

    Eagle: Speculative sampling requires rethinking feature uncertainty , author=. arXiv preprint arXiv:2401.15077 , year=

    arXiv

  17. [25]

    Proceedings of the 2024 conference on empirical methods in natural language processing , pages=

    Eagle-2: Faster inference of language models with dynamic draft trees , author=. Proceedings of the 2024 conference on empirical methods in natural language processing , pages=

    2024

  18. [26]

    Advances in Neural Information Processing Systems , volume=

    Eagle-3: Scaling up inference acceleration of large language models via training-time test , author=. Advances in Neural Information Processing Systems , volume=

  19. [27]

    arXiv preprint arXiv:2510.14751 , year=

    Beyond multi-token prediction: Pretraining llms with future summaries , author=. arXiv preprint arXiv:2510.14751 , year=

  20. [28]

    arXiv preprint arXiv:2401.08898 , year=

    Bridging state and history representations: Understanding self-predictive rl , author=. arXiv preprint arXiv:2401.08898 , year=

    arXiv

  21. [29]

    Artificial intelligence , volume=

    Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning , author=. Artificial intelligence , volume=. 1999 , publisher=

    1999

  22. [30]

    International Conference on Machine Learning , pages=

    Understanding self-predictive learning for reinforcement learning , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  23. [31]

    arXiv preprint arXiv:2511.05963 , year=

    Next-latent prediction transformers learn compact world models , author=. arXiv preprint arXiv:2511.05963 , year=

    arXiv

  24. [32]

    doi:10.5281/zenodo.12608602 , url =

    Gao, Leo and Tow, Jonathan and Abbasi, Baber and Biderman, Stella and Black, Sid and DiPofi, Anthony and Foster, Charles and Golding, Laurence and Hsu, Jeffrey and Le Noac'h, Alain and Li, Haonan and McDonell, Kyle and Muennighoff, Niklas and Ociepa, Chris and Phang, Jason and...

    doi

  25. [33]

    2025 , eprint=

    DataComp-LM: In search of the next generation of training sets for language models , author=. 2025 , eprint=

    2025

  26. [34]

    2021 , eprint=

    Evaluating Large Language Models Trained on Code , author=. 2021 , eprint=

    2021

  27. [35]

    2025 , eprint=

    Roll the dice & look before you leap: Going beyond the creative limits of next-token prediction , author=. 2025 , eprint=

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.