Pith. sign in

REVIEW 5 major objections 6 minor 1 cited by

Deciphering Trajectory-Aided LLM Reasoning: An Optimization Perspective

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Reasoning chains act like pseudo-gradient descent on LLM parameters, recasting question-solving as meta-learning.

desk verdict Useful meta-learning metaphor for CoT with substantial experiments, but the pseudo-gradient claim is an analogy, not a proven mechanism. read the letter →

arxiv 2505.19815 v1 pith:CX6HJW2K submitted 2025-05-26 cs.CL cs.AI

Junnan Liu , Hongwei Liu , Linchen Xiao , Shudong Liu , Taolin Zhang , Zihan Ma , Songyang Zhang , Kai Chen This is my paper
classification cs.CLcs.AI
keywords reasoningtrajectoriesmeta-learningpseudo-gradientdescentchain-of-thoughtLLMMAMLreinforcementlearningsupervisedfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the chain of intermediate tokens an LLM generates while solving a problem is not just scaffolding but a sequence of pseudo-gradient updates to the model's own parameters: each token nudges an implicit parameter state toward the answer. On that basis it recasts the training of reasoning models as a meta-learning problem in which every question is a task, the reasoning trajectory is the inner-loop adaptation, and the final answer is the query set. If this identification holds, then established meta-learning results about support-set size, inner-loop steps, and adaptive task sampling become directly applicable to training and improving LLMs that reason.

What carries the argument

The load-bearing object is the pseudo-gradient identity (Equation 4): the reasoning trajectory is interpreted as an update to the model parameters, with each token $t_{\le i}$ contributing a step $-\eta \nabla_{\theta'_{i-1}}\mathcal{L}_q$. The paper establishes a one-step version (Proposition 2.1) showing that appending a token to the context can be mimicked by a new parameter set $\theta'_t$, using a linear re-parametrization that requires the new token embedding to lie (approximately) in the span of the existing context embeddings. This identity turns the whole training pipeline into the meta-learning objective (Equation 8), which is what lets the authors port MAML and L2O intuitions—support-set size, inner-loop length, oracle optimizers—onto LLM reasoning.

What would settle it

Compute, for a real LLM and a set of questions, the direction of the parameter change that would reproduce the activation shift caused by appending each trajectory token (by solving the Proposition 2.1 re-parametrization), and compare its cosine similarity with the true gradient of the answer loss $\nabla_\theta \mathcal{L}_q$; if the average similarity is not consistently positive, the pseudo-gradient identity fails. A simpler proxy: measure whether the negative log-probability of the correct answer decreases at every token for correct trajectories—the paper itself shows false trajectories do not exhibit a downward trend, so the claim is testable on random perturbations of trajectories.

Watch

Extended reading notes

Core claim

The paper's central proposal is RaML, a view in which the decoding-time trajectory $t$ of tokens is written as a multi-step parameter update $\theta'_i \leftarrow \theta'_{i-1} + \Delta\mathcal{M}_{\theta'_{i-1}}(I,q,t_{\le i})$ with $\Delta = -\eta \nabla_{\theta'_{i-1}}\mathcal{L}_q(\theta'_{i-1})$ (Equation 4). Training is then formalized as a meta-objective $\min_\theta \sum_{q_i\in\mathcal{Q}}\sum_{t\in\mathcal{T}_i}\mathcal{L}_{q_i}(\mathcal{M}_{\theta+\Delta\mathcal{M}_\theta(I,q,t)})$ (Equation 8), matching the inner-loop/outer-loop structure of model-agnostic meta-learning with trajectories in place of support-set gradients. The paper proves (Proposition 2.1, Appendix B) that for a simplified one-layer transformer, attending to a trajectory token can be exactly reproduced by a re-parameterized model, giving a constructive existence argument for the pseudo-gradient step. Empirically, it shows that the negative log-probability of the correct answer decreases along generated trajectories; that longer trajectories, more trajectories per question, and SFT cold-starts followed by RL improve performance; and that summarizing long trajectories preserves most of the performance at a fraction of the tokens.

Load-bearing premise

The load-bearing premise is that the effect of reading each new reasoning token is equivalent to taking a small step in the direction that would reduce the loss on the final answer; the paper's proof of this equivalence requires the new token's embedding to be (approximately) a linear combination of the previous context tokens' embeddings, and it is verified only for a single transformer layer.

Editorial extensions

If this is right

  • Supervised fine-tuning and reinforcement learning can be understood as different inner-loop optimizers: SFT learns from oracle trajectories (stable), RL explores its own trajectories (higher ceiling).
  • The number of reasoning trajectories sampled per question plays the role of support-set size: increasing it improves performance and stability, as the paper shows for both SFT and GRPO.
  • Longer trajectories are equivalent to more inner-loop optimization steps, explaining why long-CoT models perform better; reflection tokens act as gradient spikes that help escape saddle points.
  • Because each token has a different contribution to the update, it should be possible to compress trajectories by removing low-impact tokens; the paper's summarization experiments support this.
  • Meta-learning tools such as adaptive task sampling or support-set weighting can be applied directly to LLM reasoning training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the pseudo-gradient identification is exact, then token-level attribution could be measured directly: the contribution of a token is its implied $\Delta$ magnitude, which would let practitioners drop low-impact tokens during decoding without retraining.
  • The meta-learning view implies that the base model is a meta-initialization; one could then treat continual learning or domain adaptation as meta-learning across question distributions, e.g., balancing math and code data as task ratios.
  • A testable extension: interventions that insert 'reflect' tokens mid-trajectory should produce predictable jumps in answer confidence (larger local decreases in negative log-probability), which could be verified with the paper's own probe.
  • If reasoning is inner-loop adaptation, then the same mechanism should underpin in-context learning; the paper's trajectory view could be unified with the meta-optimizer account of ICL, though the paper does not make that connection.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes RaML, a framework that interprets LLM reasoning trajectories as pseudo-gradient descent updates to the model's parameters (Eq. 4), and formalizes reasoning training as a meta-learning objective (Eq. 8) in which each question is a task, the trajectory is the inner loop, and the answer is the query set. The authors provide a theoretical statement (Proposition 2.1, proved in Appendix B) intended to support the one-step pseudo-gradient update, and they complement the framework with extensive experiments: comparisons of SFT versus RL, the effect of the number of trajectories per question, token-level analysis of reflection tokens, cross-domain generalization, and a trajectory summarization study for efficiency. The paper concludes that reasoning trajectories can be usefully understood and improved through meta-learning concepts.

Significance. If the mechanistic claim of Eq. (4) were established, the paper would provide a substantive connection between LLM reasoning and meta-learning, enabling principled use of MAML/L2O techniques for model improvement. The empirical studies, particularly the scaling of trajectory count (Sec. 4.1) and the comparison of SFT and RL cold starts (Sec. 3.2), are valuable and well executed; the release of code and the use of public benchmarks are strengths. However, the central theoretical bridge from trajectory conditioning to parameter-space gradient descent is not established, and the empirical probe measures answer confidence rather than parameter updates. The paper currently overclaims a mechanism where it provides an analogy; its value as a descriptive framework is real, but it needs substantial reframing and additional evidence before the mechanistic conclusions can be accepted.

major comments (5)
  1. [§2.2, Eq. (4)] The central identification ΔM = −η∇L_q is asserted but not derived. Proposition 2.1, proved in Appendix B, shows only that for a single attention+FFN block the activation produced after appending one token can be reproduced on the original input by modified weights. The construction in Eqs. (13)–(19) applies to any appended token E_{t,0} and imposes no relation between θ′ − θ and −η∇L_q(θ). Since Eq. (4) and the subsequent meta-learning objective Eq. (8) depend entirely on this identification, the theoretical core of the paper is currently unsupported.
  2. [Appendix B, Eq. (16)] The proof uses the approximation E_{t,0} ≈ C E_{l,:}. In the case where E_{t,0} is not in the row span of E_{l,:}, the least-squares solution makes Eqs. (17)–(18) approximate, yet Proposition 2.1 is stated as an exact equality in Eq. (9). The paper neither quantifies the approximation error nor states conditions under which the equality holds exactly, so the existence theorem is not established in the form claimed.
  3. [§2.2, Empirical Examples (Figs. 2–3)] The probe measures the negative log-probability of the final answer when the answer string is appended after each prefix. This is a sequence-level confidence measure, not a measurement of the parameter-space update θ′ − θ. A decreasing curve is expected for any trajectory that gradually converges to a correct answer, and it does not distinguish gradient descent from other mechanisms such as gradual accumulation of contextual evidence. Thus the empirical evidence does not substantiate the parameter-space gradient claim in Eq. (4).
  4. [§2.3, Eq. (8) and Algorithm 2] With ΔM defined as trajectory conditioning, the objective min_θ Σ L_q(M_{θ+ΔM}(I,q,t)) is exactly the standard supervised objective over (q,t,a) triples; no inner-loop parameter update is performed and no second-order gradient is computed. Describing this as a 'pseudo second order gradient' and 'inner loop optimization' is a relabeling rather than a derivation of a MAML-style bi-level objective. The paper should either explicitly state that this is an analogy or provide a formal sense in which Eq. (8) differs from ordinary SFT.
  5. [§2.2, Proposition 2.1 setup] The reduction to a single self-attention layer and two-layer FFN while 'disregarding normalization layers and other components' is not a valid 'without loss of generality' step for modern LLMs with residual connections, layer norm, and hundreds of layers. Furthermore, the proof gives only a one-step construction and provides no induction over the |t| steps in Eq. (4). As stated, the result cannot support the multi-step pseudo-gradient claim for full models.
minor comments (6)
  1. [§2.2] The text refers to 'Theorem 2.1' when the statement is labeled Proposition 2.1; the numbering/labeling should be made consistent throughout the paper.
  2. [§3.2, Fig. 4 caption] The sentence 'We generate64 for each question' is missing a noun; it should read 'We generate 64 trajectories for each question'.
  3. [§4.2, text after Fig. 11] The phrase 'even that tThe latter has more tokens' contains a typo and should be corrected.
  4. [§3.3, Takeaway] 'reasning performance' is a typo and should be 'reasoning performance'.
  5. [Appendix C.1, Algorithm 3] The step size s is listed as an input but never used; either remove it or clarify its role.
  6. [Fig. 16 caption] The caption reads 'Qwen3' while the main text refers to Qwen3-32B; please make the model name consistent.

Circularity Check

3 steps flagged · score 7.0 of 10

Core pseudo-gradient claim is definitional: Eq. (4) names trajectory effects as loss gradients, Prop. 2.1 proves only a permissive reparametrization, and the empirical probe measures conditional answer likelihood.

  1. self definitional [§2.2, Eq. (4) and Proposition 2.1]
    "we conceptualize the reasoning trajectories of an LLM ℳ as a multi-step gradient descent process of the model's parameters 𝜃, which could be formally represented by: 𝜃′_𝑖 ← 𝜃′_{𝑖−1} + ∆ℳ_{𝜃′_{𝑖−1}}(𝐼, 𝑞, 𝑡≤𝑖), 𝜃′_0 = 𝜃, 1 ≤ 𝑖 ≤ |𝑡|, where ... ∆ℳ_{𝜃′_{𝑖−1}}(𝐼, 𝑞, 𝑡≤𝑖) = −𝜂∇_{𝜃′_{𝑖−1}} ℒ_𝑞(𝜃′_{𝑖−1}) represents the pseudo gradient update."

    Equation (4) defines the trajectory-induced change as a loss gradient, but Proposition 2.1 only shows that, for a single layer and modulo the always-approximately-valid assumption E_{t,0} ≈ C E_{l,:}, there exists some parameter set θ′ reproducing the appended-token activation. The proof constructs W′_q = P W_q and W′_k, W′_v via linear maps and a pseudoinverse that depend only on embeddings, never on ℒ_q, so the same construction works for any appended token. It therefore imposes no constraint that θ′ − θ equals −η∇_θ ℒ_q(θ).

  2. renaming known result [§2.2, 'Empirical Examples', Figs. 2–3]
    "Specifically, we calculate the negative log-probability of the answer at each position (denoted as ℒ̂) within the generated trajectories by appending Final Answer∖n∖boxed{..answer..} at each position. ... As shown in Figure 3, the negative log-probability progressively decreases along the reasoning trajectories which aligns with our definition."

    The empirical 'verification' of the pseudo-gradient update is the decrease of ℒ̂ = −log p(answer | prefix). For any trajectory that ends in the correct boxed answer, conditioning on a longer prefix containing 'Final Answer' and the answer makes the answer tokens more probable by construction, regardless of whether any parameter-space update −η∇ℒ_q is being followed. The paper even says the curve 'aligns with our definition,' confirming that the observation is the definition restated rather than a test of Eq. (4). The probe measures sequence-level confidence, not the claimed parameter change ∆M_θ, so the monotone curve is a relabeling of conditional likelihood as gradient descent.

1 more flagged steps
  1. self definitional [§2.3, Eq. (8)]
    "min_𝜃 ∑_{𝑞𝑖∈𝒬} ∑_{𝑡∈𝒯𝑖} ℒ_{𝑞𝑖}(ℳ_{𝜃+Δℳ_𝜃(𝐼,𝑞,𝑡)}) ... where 𝒯𝑖 denotes the set of reasoning trajectories corresponding to the question 𝑞𝑖 and 𝜃+∆ℳ_𝜃(𝐼, 𝑞, 𝑡) indicates the multi-step update of 𝜃 as detailed in Equation (4)."

    Because Eq. (4) already defines the trajectory's effect as a pseudo-gradient parameter update, Eq. (8) is simply that definition inserted into a MAML-style outer loss; no independent argument shows that conditioning on t produces an actual parameter change θ → θ+ΔM. The paper concedes 'the model weights have not been directly updated,' so ℳ_{θ+ΔM_θ(I,q,t)} is trajectory-conditioned decoding ℳ_θ(I,q,t) under a new name. The support-set/query-set assignment ('the gradient update associated with the latent support set is represented by the reasoning trajectories, whereas the answer denotes the query set') is terminology, not a demonstrated equivalence. The meta-learning formalization therefore inherits the definitional circularity of Eq. (4).

full rationale

The paper contains substantial standalone empirical work—training Qwen2.5-7B-Base from scratch with SFT and GRPO, cold-start gains, trajectory-count scaling, and reasoning-efficiency experiments—that would retain its value independent of the meta-learning framing, and its use of the authors' own earlier metric [60] is not load-bearing for the central claim. However, the central interpretive bridge is circular in three connected places. Equation (4) defines the trajectory-induced change as −η∇ℒ_q, so the conclusion that trajectories are pseudo-gradient descent is written into the definition. Proposition 2.1 does not supply the missing link: its construction only reparametrizes a one-layer activation for an arbitrary appended token under an approximate span assumption, and the paper then uses the word 'update' to convert this existence statement into a gradient claim. The empirical probe in Figs. 2–3 measures the negative log-probability of the final answer after each prefix, a quantity that must decrease for any correct-answer trajectory, so it is a restatement of the definition rather than evidence about parameter-space gradients. Equation (8) then renames trajectory-conditioned SFT/RL as MAML/L2O by identifying trajectories with the inner loop and answers with the query set. These are not isolated word choices: the paper's title, theoretical analysis, and 'strong connection' claim all rest on this definitional bridge. Score is 7 rather than higher because the training experiments themselves are self-contained and could support practical conclusions independently of the isomorphism.

Assumptions & free parameters 0 free parameters · 7 assumptions · 2 invented entities

No numeric parameters are fitted to data; the analysis relies on hyperparameters from prior pipelines. The load-bearing content is definitional: trajectories are declared to be pseudo-gradients (Eq. 4), the existence proof uses an approximate span assumption, and two conceptual entities (pseudo-gradient and optimal path) are introduced without direct falsifiable evidence.

assumptions (7)
  • domain assumption A single-layer transformer without normalization layers captures the essential behavior of trajectory-based reasoning (Section 2.2, Eq. 5-6).
    Proposition 2.1 is proved for one self-attention layer plus a two-layer FFN with no normalization, then generalized to full LLMs by assumption.
  • domain assumption The trajectory token embedding is approximately in the row span of the context embeddings, E_t0 approximately C E_l,: (Appendix B, Eq. 16).
    The modified key and value matrices W'_k and W'_v are constructed from this least-squares approximation; without it the one-step equality is inexact.
  • ad hoc to paper The constructed parameter change equals -eta grad theta L_q(theta) (Section 2.2, Eq. 4).
    This is the core identification of trajectories with gradient descent; no derivation connects the existence construction in Proposition 2.1 to a loss gradient.
  • domain assumption Decreasing answer negative log-probability along a trajectory is evidence for parameter-space optimization (Section 2.2, Figures 2-3).
    The plotted quantity is the loss of the final answer conditioned on partial trajectories; it is an indirect proxy, not a measurement of parameter movement.
  • domain assumption MAML and L2O theory transfer to LLM reasoning through the proposed isomorphism (Section 2.3, Section 4).
    The paper assumes the analogy between support set and trajectory count, inner-loop steps and tokens, and outer-loop training and SFT/GRPO is strong enough to import meta-learning conclusions.
  • standard math Moore-Penrose pseudoinverse and least-squares solutions exist for E_l,: (Appendix B).
    Used to define W'_k and W'_v in Eq. 19; standard linear algebra.
  • standard math Linear transformation extension theorem B.1 and universal approximation theorem hold.
    Used to construct P and to argue that alternative parameter sets exist.
invented entities (2)
  • Pseudo-gradient update Delta M_theta(I,q,t)
    purpose: Represents each reasoning trajectory token as a parameter update in Eq. (4), enabling the meta-learning analogy.
    No direct measurement of actual parameter changes during decoding is provided; only the answer-log-probability proxy is plotted.
  • Optimal inner-loop optimization path
    purpose: Explains why shorter summarized trajectories can match long reasoning in Section 4.2 and Figure 10.
    Existence is hypothesized and approximated by LLM summarization; the paper does not provide an oracle or constructive guarantee.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Deciphering Trajectory-Aided LLM Reasoning: An Optimization Perspective." pith.science (2026). https://pith.science/paper/CX6HJW2K

@misc{pith2026250519815,
  author       = {Pith},
  title        = {Pith review of: Deciphering Trajectory-Aided LLM Reasoning: An Optimization Perspective},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CX6HJW2K}},
  note         = {Machine review of arXiv:2505.19815}
}
read the original abstract

We propose a novel framework for comprehending the reasoning capabilities of large language models (LLMs) through the perspective of meta-learning. By conceptualizing reasoning trajectories as pseudo-gradient descent updates to the LLM's parameters, we identify parallels between LLM reasoning and various meta-learning paradigms. We formalize the training process for reasoning tasks as a meta-learning setup, with each question treated as an individual task, and reasoning trajectories serving as the inner loop optimization for adapting model parameters. Once trained on a diverse set of questions, the LLM develops fundamental reasoning capabilities that can generalize to previously unseen questions. Extensive empirical evaluations substantiate the strong connection between LLM reasoning and meta-learning, exploring several issues of significant interest from a meta-learning standpoint. Our work not only enhances the understanding of LLM reasoning but also provides practical insights for improving these models through established meta-learning techniques.

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Learning without training: The implicit dynamics of in-context learning

    cs.CL 2025-07 conditional novelty 6.0 of 10

    A forward pass with in-context examples is exactly equivalent to a context-free forward pass with a minimal rank-1 update to the MLP weights, with an explicit formula for that update.

Reference graph

Works this paper leans on

133 extracted references · 35 canonical work pages · cited by 1 Pith paper

  1. [1]

    On sensitivity of meta-learning to support data

    Mayank Agarwal, Mikhail Yurochkin, and Yuekai Sun. On sensitivity of meta-learning to support data. InNeurIPS, pages 20447–20460, 2021. 1, 4.1

    2021

  2. [2]

    Back to basics: Revisiting reinforce-style optimization for learning from human feedback in llms

    Arash Ahmadian, Chris Cremer, Matthias Gallé, Marzieh Fadaee, Julia Kreutzer, Olivier Pietquin, Ahmet Üstün, and Sara Hooker. Back to basics: Revisiting reinforce-style optimization for learning from human feedback in llms. InACL, pages 12248–12267. Association for Computational Linguistics, 2024. 2.4

    2024

  3. [3]

    What learning algorithm is in-context learning? investigations with linear models

    Ekin Akyürek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, and Denny Zhou. What learning algorithm is in-context learning? investigations with linear models. InICLR. OpenReview.net, 2023. 5

    2023

  4. [4]

    Hoffman, David Pfau, Tom Schaul, and Nando de Freitas

    Marcin Andrychowicz, Misha Denil, Sergio Gomez Colmenarejo, Matthew W. Hoffman, David Pfau, Tom Schaul, and Nando de Freitas. Learning to learn by gradient descent by gradient descent. InNIPS, pages 3981–3989, 2016. 1, 2, 6

    2016

  5. [5]

    Transformers as statisticians: Provable in-context learning with in-context algorithm selection

    Yu Bai, Fan Chen, Huan Wang, Caiming Xiong, and Song Mei. Transformers as statisticians: Provable in-context learning with in-context algorithm selection. InNeurIPS, 2023. 2.2, 5

    2023

  6. [6]

    Numinamath 72b cot.https://huggingface.co/ AI-MO/NuminaMath-72B-CoT, 2024

    Edward Beeching, Shengyi Costa Huang, Albert Jiang, Jia Li, Benjamin Lipkin, Zihan Qina, Kashif Rasul, Ziju Shen, Roman Soletskyi, and Lewis Tunstall. Numinamath 72b cot.https://huggingface.co/ AI-MO/NuminaMath-72B-CoT, 2024. 5

    2024

  7. [7]

    On the optimization of a synaptic learning rule

    Samy Bengio, Yoshua Bengio, Jocelyn Cloutier, and Jan Gecsei. On the optimization of a synaptic learning rule. InOptimality in Artificial and Biological Neural Networks, pages 6–8, 1992. 5

    1992

  8. [8]

    Llama-Nemotron: Efficient reasoning models.CoRR, abs/2505.00949, 2025

    Akhiad Bercovich, Itay Levy, Izik Golan, Mohammad Dabbah, Ran El-Yaniv, Omri Puny, Ido Galil, Zach Moshe, Tomer Ronen, Najeeb Nabwani, Ido Shahaf, Oren Tropp, Ehud Karpas, Ran Zilberstein, Jiaqi Zeng, Soumye Singhal, Alexander Bukharin, Yian Zhang, Tugrul Konuk, Gerald Shen, Ameya Sunil Mahabaleshwarkar, Bilal Kartal, Yoshi Suhara, Olivier Delalleau, Ziji...

    arXiv 2025

Show all 133 references
  1. [9]

    Gormley, and Graham Neubig

    Amanda Bertsch, Maor Ivgi, Uri Alon, Jonathan Berant, Matthew R. Gormley, and Graham Neubig. In-context learning with long-context models: An in-depth exploration.CoRR, abs/2405.00200, 2024. 5

    2024 arXiv

  2. [10]

    Graph of thoughts: Solving elaborate problems with large language models

    Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, and Torsten Hoefler. Graph of thoughts: Solving elaborate problems with large language models. InAAAI, pages 1768...

    2024

  3. [11]

    On the ability and limitations of transformers to recognize formal languages

    Satwik Bhattamishra, Kabir Ahuja, and Navin Goyal. On the ability and limitations of transformers to recognize formal languages. InEMNLP, pages 7096–7116. Association for Computational Linguistics,

  4. [12]

    Application of calculus of matrices to method of least squares: with special reference to geodetic calculations.Trans

    Arne Bjerhammar. Application of calculus of matrices to method of least squares: with special reference to geodetic calculations.Trans. Roy. Inst. Tech. Stockholm., 1951. B

    1951

  5. [13]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffr...

    2020

  6. [14]

    Sft or rl? an early investigation into training r1-like reasoning large vision-language models.CoRR, abs/2504.11468, 2025

    Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. Sft or rl? an early investigation into training r1-like reasoning large vision-language models.CoRR, abs/2504.11468, 2025. 3.2

    2025 arXiv

  7. [15]

    A closer look at the training strategy for modern meta-learning

    Jiaxin Chen, Xiao-Ming Wu, Yanke Li, Qimai Li, Li-Ming Zhan, and Fu-Lai Chung. A closer look at the training strategy for modern meta-learning. InNeurIPS, 2020. 4.1

    2020

  8. [16]

    Variational metric scaling for metric-based meta-learning

    Jiaxin Chen, Li-Ming Zhan, Xiao-Ming Wu, and Fu-Lai Chung. Variational metric scaling for metric-based meta-learning. InAAAI, pages 3478–3485. AAAI Press, 2020. 5

    2020

  9. [17]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondé de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott G...

    2021 arXiv

  10. [18]

    Towards reasoning era: A survey of long chain-of-thought for reasoning large language models.CoRR, abs/2503.09567, 2025

    Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. Towards reasoning era: A survey of long chain-of-thought for reasoning large language models.CoRR, abs/2503.09567, 2025. 1

    2025 arXiv

  11. [19]

    Tighter bounds on the expressivity of transformer encoders

    David Chiang, Peter Cholak, and Anand Pillay. Tighter bounds on the expressivity of transformer encoders. InICML, volume 202 ofProceedings of Machine Learning Research, pages 5544–5562. PMLR,

  12. [20]

    Le, Sergey Levine, and Yi Ma

    Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V. Le, Sergey Levine, and Yi Ma. SFT memorizes, RL generalizes: A comparative study of foundation model post-training.CoRR, abs/2501.17161, 2025. 3.2

    2025 arXiv

  13. [21]

    Gpg: A simple and strong reinforcement learning baseline for model reasoning.CoRR, abs/2504.02546, 2025

    Xiangxiang Chu, Hailang Huang, Xiao Zhang, Fei Wei, and Yong Wang. Gpg: A simple and strong reinforcement learning baseline for model reasoning.CoRR, abs/2504.02546, 2025. D

    2025

  14. [22]

    Task-robust model-agnostic meta-learning

    Liam Collins, Aryan Mokhtari, and Sanjay Shakkottai. Task-robust model-agnostic meta-learning. In NeurIPS, 2020. 4.3

    2020

  15. [23]

    How does the task landscape affect MAML performance? InCoLLAs, volume 199 ofProceedings of Machine Learning Research, pages 23–59

    Liam Collins, Aryan Mokhtari, and Sanjay Shakkottai. How does the task landscape affect MAML performance? InCoLLAs, volume 199 ofProceedings of Machine Learning Research, pages 23–59. PMLR,

  16. [24]

    Approximations by superpositions of a sigmoidal function.MCSS, 2:183–192, 1989

    George Cybenko. Approximations by superpositions of a sigmoidal function.MCSS, 2:183–192, 1989. B

    1989

  17. [25]

    Why can gpt learn in-context? language models secretly perform gradient descent as meta-optimizers

    Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Shuming Ma, Zhifang Sui, and Furu Wei. Why can gpt learn in-context? language models secretly perform gradient descent as meta-optimizers. InACL (Findings), pages 4005–4019. Association for Computational Linguistics, 2023. 2.2, 5

    2023

  18. [26]

    Gemini 2.5: Our most intelligent ai model

    Google Deepmind. Gemini 2.5: Our most intelligent ai model. https://blog.google/ technology/google-deepmind/gemini-model-thinking-updates-march-2025/ #gemini-2-5-thinking/, 2025. Accessed: 2025-03-26. 1, 5 16 Deciphering Trajectory-Aided LLM Reasoning: An Optimization Perspective

    2025

  19. [27]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei F...

    2025 arXiv

  20. [28]

    Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J

    DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei L...

    2024 arXiv

  21. [29]

    Universal transformers

    Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Lukasz Kaiser. Universal transformers. InICLR. OpenReview.net, 2019. 5

    2019

  22. [30]

    BERT: pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. InNAACL-HLT, pages 4171–4186. Association for Computational Linguistics, 2019. 2.3, 2.4

    2019

  23. [31]

    A survey on in-context learning

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, Xu Sun, Lei Li, and Zhifang Sui. A survey on in-context learning. InEMNLP, pages 1107–1128. Association for Computational Linguistics, 2024. 5

    2024

  24. [32]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zha...

    2024 arXiv

  25. [33]

    Towards revealing the mystery behind chain of thought: A theoretical perspective

    Guhao Feng, Bohang Zhang, Yuntian Gu, Haotian Ye, Di He, and Liwei Wang. Towards revealing the mystery behind chain of thought: A theoretical perspective. InNeurIPS, 2023. 1, 5

    2023

  26. [34]

    Model-agnostic meta-learning for fast adaptation of deep networks

    Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. InICML, volume 70 ofProceedings of Machine Learning Research, pages 1126–1135. PMLR, 2017. 1, 2, 6, 5

    2017

  27. [35]

    Transformers learn to achieve second-order convergence rates for in-context linear regression

    Deqing Fu, Tian-Qi Chen, Robin Jia, and Vatsal Sharan. Transformers learn to achieve second-order convergence rates for in-context linear regression. InNeurIPS, 2024. 2.2

    2024

  28. [36]

    Reddi, Stefanie Jegelka, and Sanjiv Kumar

    Khashayar Gatmiry, Nikunj Saunshi, Sashank J. Reddi, Stefanie Jegelka, and Sanjiv Kumar. Can looped transformers learn to implement multi-step gradient descent for in-context learning? InICML. OpenReview.net, 2024. 1, 5

    2024

  29. [37]

    Lee, and Dimitris Papail- iopoulos

    Angeliki Giannou, Shashank Rajput, Jy-yong Sohn, Kangwook Lee, Jason D. Lee, and Dimitris Papail- iopoulos. Looped transformers as programmable computers. InICML, volume 202 ofProceedings of Machine Learning Research, pages 11398–11442. PMLR, 2023. 1, 2.2, 5

    2023

  30. [38]

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y. Wu, Y. K. Li, Fuli Luo, Yingfei Xiong, and Wenfeng Liang. Deepseek-coder: When the large language model meets programming - the rise of code intelligence.CoRR, abs/2401.14196, 2024. 3.4

    2024 arXiv

  31. [39]

    Measuring mathematical problem solving with the MATH dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. InNeurIPS Datasets and Benchmarks, 2021. C.3

    2021

  32. [40]

    JohnHewitt, Michael Hahn, Surya Ganguli, PercyLiang, andChristopher D. Manning. Rnns cangenerate bounded hierarchical languages with optimal memory. InEMNLP, pages 1978–2010. Association for Computational Linguistics, 2020. 5

    1978

  33. [41]

    Approximation capabilities of multilayer feedforward networks.Neural Networks, 4(2):251– 257, 1991

    Kurt Hornik. Approximation capabilities of multilayer feedforward networks.Neural Networks, 4(2):251– 257, 1991. B

    1991

  34. [42]

    Hospedales, Antreas Antoniou, Paul Micaelli, and Amos J

    Timothy M. Hospedales, Antreas Antoniou, Paul Micaelli, and Amos J. Storkey. Meta-learning in neural networks: A survey.IEEE Trans. Pattern Anal. Mach. Intell., 44(9):5149–5169, 2022. 1, 2

    2022

  35. [43]

    Universal language model fine-tuning for text classification

    Jeremy Howard and Sebastian Ruder. Universal language model fine-tuning for text classification. In ACL, pages 328–339. Association for Computational Linguistics, 2018. 2.3, 2.4

    2018

  36. [44]

    Open- reasoner-zero: An open source approach to scaling up reinforcement learning on the base model.CoRR, abs/2503.24290, 2025

    Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung-Yeung Shum. Open- reasoner-zero: An open source approach to scaling up reinforcement learning on the base model.CoRR, abs/2503.24290, 2025. 3.1, 4.1, D

    2025 arXiv

  37. [45]

    Jianhao Huang, Zixuan Wang, and Jason D. Lee. Transformers learn to implement multi-step gradient descent with chain of thought.CoRR, abs/2502.21212, 2025. 1, 2.2, 5

    2025 arXiv

  38. [46]

    Qwen2.5-coder technical report.CoRR, abs/2409.12186, 2024

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Kai Dang, An Yang, Rui Men, Fei Huang, Xingzhang Ren, Xuancheng Ren, Jingren Zhou, and Junyang Lin. Qwen2.5-coder technical report.CoRR, abs/2409.12186, 2024. 3.4

    2024 arXiv

  39. [47]

    Livecodebench: Holistic and contamination free evaluation of large language models for code.CoRR, abs/2403.07974, 2024

    Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar- Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code.CoRR, abs/2403.07974, 2024. 3.1, C.3

    2024 arXiv

  40. [48]

    Xu, Jun Araki, and Graham Neubig

    Zhengbao Jiang, Frank F. Xu, Jun Araki, and Graham Neubig. How can we know what language models know.Trans. Assoc. Comput. Linguistics, 8:423–438, 2020. 1

    2020

  41. [49]

    Efficient memory management for large language model serving with pagedattention

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. InSOSP, pages 611–626. ACM, 2023. C.2 18 Deciphering Trajectory-Aided L...

    2023

  42. [50]

    Bespoke-stratos: The unreasonable effectiveness of reasoning distillation, 2025

    Bespoke Labs. Bespoke-stratos: The unreasonable effectiveness of reasoning distillation, 2025. Accessed: 2025-01-22. 4

    2025

  43. [51]

    Rupam Mahmood, Shuicheng Yan, and Zhongwen Xu

    Qingfeng Lan, A. Rupam Mahmood, Shuicheng Yan, and Zhongwen Xu. Learning to optimize for reinforcement learning.RLJ, 2:481–497, 2024. 3.2

    2024

  44. [52]

    Meta-learning with differentiable convex optimization

    Kwonjoon Lee, Subhransu Maji, Avinash Ravichandran, and Stefano Soatto. Meta-learning with differentiable convex optimization. InCVPR, pages 10657–10665. Computer Vision Foundation / IEEE,

  45. [53]

    Visualizing the loss landscape of neural nets

    Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets. InNeurIPS, pages 6391–6401, 2018. 2, 4

    2018

  46. [54]

    Learning to optimize

    Ke Li and Jitendra Malik. Learning to optimize. InICLR (Poster). OpenReview.net, 2017. 6

    2017

  47. [55]

    Learning to optimize neural nets.CoRR, abs/1703.00441, 2017

    Ke Li and Jitendra Malik. Learning to optimize neural nets.CoRR, abs/1703.00441, 2017. 6

    2017 arXiv

  48. [56]

    A survey on LLM test-time compute via search: Tasks, LLM profiling, search algorithms, and relevant frameworks.CoRR, abs/2501.10069, 2025

    Xinzhe Li. A survey on LLM test-time compute via search: Tasks, LLM profiling, search algorithms, and relevant frameworks.CoRR, abs/2501.10069, 2025. 1

    2025 arXiv

  49. [57]

    Let’s verify step by step

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. InICLR. OpenReview.net,

  50. [58]

    Ash, Surbhi Goel, Akshay Krishnamurthy, and Cyril Zhang

    Bingbin Liu, Jordan T. Ash, Surbhi Goel, Akshay Krishnamurthy, and Cyril Zhang. Transformers learn shortcuts to automata. InICLR. OpenReview.net, 2023. 1, 5

    2023

  51. [59]

    Chenghao Liu, Zhihao Wang, Doyen Sahoo, Yuan Fang, Kun Zhang, and Steven C. H. Hoi. Adaptive task sampling for meta-learning. InECCV, volume 12363 ofLecture Notes in Computer Science, pages 752–769. Springer, 2020. 1, 4.3

    2020

  52. [60]

    Are your llms capable of stable reasoning?CoRR, abs/2412.13147, 2024

    JunnanLiu, HongweiLiu, LinchenXiao, ZiyiWang, KuikunLiu, SongyangGao, WenweiZhang, Songyang Zhang, and Kai Chen. Are your llms capable of stable reasoning?CoRR, abs/2412.13147, 2024. 3.1, 4.1, C.3

    2024 arXiv

  53. [61]

    Reasoning models can be effective without thinking.CoRR, abs/2504.09858, 2025

    Wenjie Ma, Jingxuan He, Charlie Snell, Tyler Griggs, Sewon Min, and Matei Zaharia. Reasoning models can be effective without thinking.CoRR, abs/2504.09858, 2025. 3.3, 4.2

    2025 arXiv

  54. [62]

    William Merrill, Ashish Sabharwal, and Noah A. Smith. Saturated transformers are constant-depth threshold circuits.Trans. Assoc. Comput. Linguistics, 10:843–856, 2022. 1, 5

    2022

  55. [63]

    Academy, 1877

    Mansfield Merriman.A List of Writings Relating to the Method of Least Squares: With Historical and Critical Notes, volume 4. Academy, 1877. B

  56. [64]

    Metaicl: Learning to learn in context

    Sewon Min, Mike Lewis, Luke Zettlemoyer, and Hannaneh Hajishirzi. Metaicl: Learning to learn in context. InNAACL-HLT, pages 2791–2809. Association for Computational Linguistics, 2022. 5

    2022

  57. [65]

    Rethinking the role of demonstrations: What makes in-context learning work? InEMNLP, pages 11048–11064

    Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. Rethinking the role of demonstrations: What makes in-context learning work? InEMNLP, pages 11048–11064. Association for Computational Linguistics, 2022. 5

    2022

  58. [66]

    Imitate, explore, and self-improve: A reproduction report on slow-thinking reasoning systems.CoRR, abs/2412.09413, 2024

    Yingqian Min, Zhipeng Chen, Jinhao Jiang, Jie Chen, Jia Deng, Yiwen Hu, Yiru Tang, Jiapeng Wang, Xiaoxue Cheng, Huatong Song, Wayne Xin Zhao, Zheng Liu, Zhongyuan Wang, and Ji-Rong Wen. Imitate, explore, and self-improve: A reproduction report on slow-thinking reasoning system...

    2024 arXiv

  59. [67]

    Riedmiller

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin A. Riedmiller. Playing atari with deep reinforcement learning.CoRR, abs/1312.5602, 2013. 2.3

    2013 arXiv

  60. [68]

    On the reciprocal of the general algebraic matrix.Bull

    Eliakim H Moore. On the reciprocal of the general algebraic matrix.Bull. Am. Math. Soc., 26:294–295,

  61. [69]

    Candès, and Tatsunori Hashimoto

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel J. Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling. CoRR, abs/2501.19393, 2025. 4, 3.3

    2025 arXiv

  62. [70]

    In-context learning and induction heads.CoRR, abs/2209.11895, 2022

    Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac Hatfield- Dodds, Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Ka...

    2022 arXiv

  63. [71]

    GPT-4 technical report.CoRR, abs/2303.08774, 2023

    OpenAI. GPT-4 technical report.CoRR, abs/2303.08774, 2023. 1

    2023 arXiv

  64. [72]

    Learning to reason with llms

    OpenAI. Learning to reason with llms. https://openai.com/index/ learning-to-reason-with-llms/, 2024. Accessed: 2024-09-12. 1, 2.2, 5

    2024

  65. [73]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leik...

    2022

  66. [74]

    Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, ...

    2019

  67. [75]

    A generalized inverse for matrices

    Roger Penrose. A generalized inverse for matrices. InMath. Proc. Cambridge Philos. Soc., volume 51, pages 406–413. Cambridge University Press, 1955. B

    1955

  68. [76]

    A simple guard for learned optimizers

    Isabeau Prémont-Schwarz, Jaroslav Vitku, and Jan Feyereisl. A simple guard for learned optimizers. In ICML, volume 162 ofProceedings of Machine Learning Research, pages 17910–17925. PMLR, 2022. 3.2

    2022

  69. [77]

    O1 replication journey: A strategic progress report - part 1.CoRR, abs/2410.18982, 2024

    Yiwei Qin, Xuefeng Li, Haoyang Zou, Yixiu Liu, Shijie Xia, Zhen Huang, Yixin Ye, Weizhe Yuan, Hector Liu, Yuanzhi Li, and Pengfei Liu. O1 replication journey: A strategic progress report - part 1.CoRR, abs/2410.18982, 2024. 1

    2024 arXiv

  70. [78]

    A survey of efficient reasoning for large reasoning models: Language, multimodality, and beyond.CoRR, abs/2503.21614, 2025

    Xiaoye Qu, Yafu Li, Zhaochen Su, Weigao Sun, Jianhao Yan, Dongrui Liu, Ganqu Cui, Daizong Liu, Shuxian Liang, Junxian He, Peng Li, Wei Wei, Jing Shao, Chaochao Lu, Yue Zhang, Xian-Sheng Hua, Bowen Zhou, and Yu Cheng. A survey of efficient reasoning for large reasoning models: ...

    2025

  71. [79]

    Improving language understand- ing by generative pre-training.OpenAI, 2018

    Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language understand- ing by generative pre-training.OpenAI, 2018. 2.1, 2.3, 1

    2018

  72. [80]

    Manning, Stefano Ermon, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. InNeurIPS,

  73. [81]

    Kakade, and Sergey Levine

    Aravind Rajeswaran, Chelsea Finn, Sham M. Kakade, and Sergey Levine. Meta-learning with implicit gradients. InNeurIPS, pages 113–124, 2019. 5

    2019

  74. [82]

    Optimization as a model for few-shot learning

    Sachin Ravi and Hugo Larochelle. Optimization as a model for few-shot learning. InICLR. OpenRe- view.net, 2017. 1, 2

    2017

  75. [83]

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level google-proof q&a benchmark.CoRR, abs/2311.12022, 2023. 3.1, C.3

    2023 arXiv

  76. [84]

    Learning to retrieve prompts for in-context learning

    Ohad Rubin, Jonathan Herzig, and Jonathan Berant. Learning to retrieve prompts for in-context learning. InNAACL-HLT, pages 2655–2671. Association for Computational Linguistics, 2022. 5 20 Deciphering Trajectory-Aided LLM Reasoning: An Optimization Perspective

    2022

  77. [85]

    Botvinick, Daan Wierstra, and Timothy P

    Adam Santoro, Sergey Bartunov, Matthew M. Botvinick, Daan Wierstra, and Timothy P. Lillicrap. One-shot learning with memory-augmented neural networks.CoRR, abs/1605.06065, 2016. 5

    2016 arXiv

  78. [86]

    Evolutionary principles in self-referential learning, or on learning how to learn: The meta-meta-

    Jürgen Schmidhuber. Evolutionary principles in self-referential learning, or on learning how to learn: The meta-meta-. hook. Master’s thesis, Technical University of Munich, Germany, 1987. 1, 2

    1987

  79. [87]

    Proximal policy optimization algorithms.CoRR, abs/1707.06347, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.CoRR, abs/1707.06347, 2017. 2.3, 1, 2.4, 4.1

    2017 arXiv

  80. [88]

    Rethinking reflection in pre-training.CoRR, abs/2504.04022, 2025

    Darsh J Shah, Peter Rushton, Somanshu Singla, Mohit Parmar, Kurt Smith, Yash Vanjani, Ashish Vaswani, Adarsh Chaluvaraju, Andrew Hojel, Andrew Ma, et al. Rethinking reflection in pre-training.CoRR, abs/2504.04022, 2025. 1

    2025 arXiv

  81. [89]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. CoRR, abs/2402.03300, 2024. 2.4, 3.1

    2024 arXiv

  82. [90]

    Hybridflow: A flexible and efficient RLHF framework

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient RLHF framework. InEuroSys, pages 1279–1297. ACM, 2025. C.2

    2025

  83. [91]

    Route sparse autoencoder to interpret large language models.CoRR, abs/2503.08200, 2025

    Wei Shi, Sihang Li, Tao Liang, Mingyang Wan, Gojun Ma, Xiang Wang, and Xiangnan He. Route sparse autoencoder to interpret large language models.CoRR, abs/2503.08200, 2025. 1

    2025 arXiv

  84. [92]

    Co-Reyes, Rishabh Agarwal, Ankesh Anand, Piyush Patil, Xavier Garcia, Peter J

    Avi Singh, John D. Co-Reyes, Rishabh Agarwal, Ankesh Anand, Piyush Patil, Xavier Garcia, Peter J. Liu, James Harrison, Jaehoon Lee, Kelvin Xu, Aaron T. Parisi, Abhishek Kumar, Alexander A. Alemi, Alex Rizkowsky, Azade Nova, Ben Adlam, Bernd Bohnet, Gamaleldin Fathy Elsayed, Ha...

    2024

  85. [93]

    Jake Snell, Kevin Swersky, and Richard S. Zemel. Prototypical networks for few-shot learning. InNIPS, pages 4077–4087, 2017. 5

    2017

  86. [94]

    End-to-end memory networks

    Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, and Rob Fergus. End-to-end memory networks. In NIPS, pages 2440–2448, 2015. 5

    2015

  87. [95]

    Andrew Bagnell

    Gokul Swamy, Sanjiban Choudhury, Wen Sun, Zhiwei Steven Wu, and J. Andrew Bagnell. All roads lead to likelihood: The value of reinforcement learning in fine-tuning.CoRR, abs/2503.01067, 2025. 2.4

    2025

  88. [96]

    Blockmix: Meta regularization and self-calibrated inference for metric-based meta-learning

    Hao Tang, Zechao Li, Zhimao Peng, and Jinhui Tang. Blockmix: Meta regularization and self-calibrated inference for metric-based meta-learning. InACM Multimedia, pages 610–618. ACM, 2020. 5

    2020

  89. [97]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, Chuning Tang, Congcong Wang, Dehao Zhang, Enming Yuan, Enzhe Lu, Fengxiang Tang, Flood Sung, Guangda Wei, Guokun Lai, Haiqing Guo, Han Zhu, Hao Ding, ...

    2025 arXiv

  90. [98]

    Sky-t1: Train your own o1 preview model within $450.https://novasky-ai

    NovaSky Team. Sky-t1: Train your own o1 preview model within $450.https://novasky-ai. github.io/posts/sky-t1, 2025. Accessed: 2025-01-09. 4, 5 21 Deciphering Trajectory-Aided LLM Reasoning: An Optimization Perspective

    2025

  91. [99]

    Open Thoughts.https://open-thoughts.ai, 2025

    OpenThoughts Team. Open Thoughts.https://open-thoughts.ai, 2025. 4, 5, D

    2025

  92. [100]

    Qwen3: Think deeper, act faster.https://qwenlm.github.io/blog/qwen3/, April

    Qwen Team. Qwen3: Think deeper, act faster.https://qwenlm.github.io/blog/qwen3/, April

  93. [101]

    Qwq-32b: Embracing the power of reinforcement learning.https://qwenlm.github

    Qwen Team. Qwq-32b: Embracing the power of reinforcement learning.https://qwenlm.github. io/blog/qwq-32b/, March 2025. 1, 2.2, 5, C.1

    2025

  94. [102]

    Atom of thoughts for markov LLM test-time scaling.CoRR, abs/2502.12018, 2025

    Fengwei Teng, Zhaoyang Yu, Quan Shi, Jiayi Zhang, Chenglin Wu, and Yuyu Luo. Atom of thoughts for markov LLM test-time scaling.CoRR, abs/2502.12018, 2025. 1

    2025

  95. [103]

    𝜇lo: Compute-efficient meta-generalization of learned optimizers.CoRR, abs/2406.00153,

    Benjamin Thérien, Charles-Étienne Joseph, Boris Knyazev, Edouard Oyallon, Irina Rish, and Eugene Belilovsky. 𝜇lo: Compute-efficient meta-generalization of learned optimizers.CoRR, abs/2406.00153,

  96. [104]

    Sebastian Thrun and Lorien Y. Pratt. Learning to learn: Introduction and overview. InLearning to Learn, pages 3–17. Springer, 1998. 5

    1998

  97. [105]

    Zemel, and Vincent Dumoulin

    Eleni Triantafillou, Hugo Larochelle, Richard S. Zemel, and Vincent Dumoulin. Learning a universal template for few-shot dataset generalization. InICML, volume 139 ofProceedings of Machine Learning Research, pages 10424–10433. PMLR, 2021. 1

    2021

  98. [106]

    Meta-dataset: A dataset of datasets for learning to learn from few examples

    Eleni Triantafillou, Tyler Zhu, Vincent Dumoulin, Pascal Lamblin, Utku Evci, Kelvin Xu, Ross Goroshin, Carles Gelada, Kevin Swersky, Pierre-Antoine Manzagol, and Hugo Larochelle. Meta-dataset: A dataset of datasets for learning to learn from few examples. InICLR. OpenReview.ne...

    2020

  99. [107]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. InNIPS, pages 5998–6008, 2017. 2.1, 2.2, 5

    2017

  100. [108]

    Mathcoder: Seamless code integration in llms for enhanced mathe- matical reasoning

    Ke Wang, Houxing Ren, Aojun Zhou, Zimu Lu, Sichun Luo, Weikang Shi, Renrui Zhang, Linqi Song, Mingjie Zhan, and Hongsheng Li. Mathcoder: Seamless code integration in llms for enhanced mathe- matical reasoning. InICLR. OpenReview.net, 2024. 3.4

    2024

  101. [109]

    ST-MAML : A stochastic-task based method for task-heterogeneous meta-learning

    Zhe Wang, Jake Grigsby, Arshdeep Sekhon, and Yanjun Qi. ST-MAML : A stochastic-task based method for task-heterogeneous meta-learning. InUAI, volume 180 ofProceedings of Machine Learning Research, pages 2066–2074. PMLR, 2022. 4.3

    2022

  102. [110]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. InNeurIPS,

  103. [111]

    Thinking like transformers

    Gail Weiss, Yoav Goldberg, and Eran Yahav. Thinking like transformers. InICML, volume 139 of Proceedings of Machine Learning Research, pages 11080–11090. PMLR, 2021. 5

    2021

  104. [112]

    Light-r1: Curriculum sft, DPO and RL for long COT from scratch and beyond.CoRR, abs/2503.10460, 2025

    Liang Wen, Yunke Cai, Fenrui Xiao, Xin He, Qi An, Zhenyu Duan, Yimin Du, Junchen Liu, Lifu Tang, Xiaowei Lv, Haosheng Zou, Yongchao Deng, Shousheng Jia, and Xiangzheng Zhang. Light-r1: Curriculum sft, DPO and RL for long COT from scratch and beyond.CoRR, abs/2503.10460, 2025. ...

    2025 arXiv

  105. [113]

    Memory networks

    Jason Weston, Sumit Chopra, and Antoine Bordes. Memory networks. InICLR, 2015. 5

    2015

  106. [114]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020

  107. [115]

    An explanation of in-context learning as implicit bayesian inference

    Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. An explanation of in-context learning as implicit bayesian inference. InICLR. OpenReview.net, 2022. 5

    2022

  108. [116]

    Iterative preference learning from human feedback: Bridging theory and practice for RLHF under kl-constraint

    Wei Xiong, Hanze Dong, Chenlu Ye, Ziqi Wang, Han Zhong, Heng Ji, Nan Jiang, and Tong Zhang. Iterative preference learning from human feedback: Bridging theory and practice for RLHF under kl-constraint. InICML. OpenReview.net, 2024. 3.2 22 Deciphering Trajectory-Aided LLM Reaso...

    2024

  109. [117]

    Qwen2.5 technical report.CoRR, abs/2412.15115, 2024

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2024 arXiv

  110. [118]

    Qwen2.5-math technical report: Toward mathematical expert model via self-improvement

    An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, and Zhenru Zhang. Qwen2.5-math technical report: Toward mathematical expert model via se...

    2024 arXiv

  111. [119]

    Towards thinking-optimal scaling of test-time compute for LLM reasoning.CoRR, abs/2502.18080, 2025

    Wenkai Yang, Shuming Ma, Yankai Lin, and Furu Wei. Towards thinking-optimal scaling of test-time compute for LLM reasoning.CoRR, abs/2502.18080, 2025. 3.3

    2025

  112. [120]

    Meta- learning with an adaptive task scheduler

    Huaxiu Yao, Yu Wang, Ying Wei, Peilin Zhao, Mehrdad Mahdavi, Defu Lian, and Chelsea Finn. Meta- learning with an adaptive task scheduler. InNeurIPS, pages 7497–7509, 2021. 4.3

    2021

  113. [121]

    Papadimitriou, and Karthik Narasimhan

    Shunyu Yao, Binghui Peng, Christos H. Papadimitriou, and Karthik Narasimhan. Self-attention net- works can process bounded hierarchical languages. InACL/IJCNLP, pages 3770–3785. Association for Computational Linguistics, 2021. 5

    2021

  114. [122]

    Tree of thoughts: Deliberate problem solving with large language models

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. InNeurIPS, 2023. 5

    2023

  115. [123]

    Multi-objective meta learning

    Feiyang Ye, Baijiong Lin, Zhixiong Yue, Pengxin Guo, Qiao Xiao, and Yu Zhang. Multi-objective meta learning. InNeurIPS, pages 21338–21351, 2021. 5

    2021

  116. [124]

    LIMO: less is more for reasoning.CoRR, abs/2502.03387, 2025

    Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. LIMO: less is more for reasoning.CoRR, abs/2502.03387, 2025. 4, 5

    2025 arXiv

  117. [125]

    DAPO: an open-source LLM reinforcement learning system at scale.CoRR, abs/2503.14476, 2025

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wa...

    2025 arXiv

  118. [126]

    Scaling relationship on learning mathematical reasoning with large language models.CoRR, abs/2308.01825,

    Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Chuanqi Tan, and Chang Zhou. Scaling relationship on learning mathematical reasoning with large language models.CoRR, abs/2308.01825,

    arXiv

  119. [127]

    VAPO: efficient and reliable reinforcement learning for advanced reasoning tasks.CoRR, abs/2504.05118, 2025

    Yu Yue, Yufeng Yuan, Qiying Yu, Xiaochen Zuo, Ruofei Zhu, Wenyuan Xu, Jiaze Chen, Chengyi Wang, TianTian Fan, Zhengyin Du, Xiangpeng Wei, Xiangyu Yu, Gaohong Liu, Juncai Liu, Lingjun Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Ru Zhang, X...

    2025 arXiv

  120. [128]

    Reddi, and Sanjiv Kumar

    Chulhee Yun, Srinadh Bhojanapalli, Ankit Singh Rawat, Sashank J. Reddi, and Sanjiv Kumar. Are transformers universal approximators of sequence-to-sequence functions? InICLR. OpenReview.net,

  121. [129]

    Reddi, and Sanjiv Kumar

    Chulhee Yun, Yin-Wen Chang, Srinadh Bhojanapalli, Ankit Singh Rawat, Sashank J. Reddi, and Sanjiv Kumar. O(n) connections are expressive enough: Universal approximability of sparse transformers. In NeurIPS, 2020. 5

    2020

  122. [130]

    Reasoning models know when they’re right: Probing hidden states for self-verification.CoRR, abs/2504.05419, 2025

    Anqi Zhang, Yulin Chen, Jane Pan, Chen Zhao, Aurojit Panda, Jinyang Li, and He He. Reasoning models know when they’re right: Probing hidden states for self-verification.CoRR, abs/2504.05419, 2025. 3.3

    2025 arXiv

  123. [131]

    Adversarial gradient-based meta learning with metric-based test.Knowl

    Yangguang Zhang, Can Wang, Qihao Shi, Yan Feng, and Chun Chen. Adversarial gradient-based meta learning with metric-based test.Knowl. Based Syst., 263:110312, 2023. 5 23 Deciphering Trajectory-Aided LLM Reasoning: An Optimization Perspective A. Notations Table4: Illustration o...

    2023

  124. [1920]

    B 19 Deciphering Trajectory-Aided LLM Reasoning: An Optimization Perspective

  125. [2020]

    5 15 Deciphering Trajectory-Aided LLM Reasoning: An Optimization Perspective

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.