Pith. sign in

REVIEW 2 major objections 4 minor 37 references

StreamBP: Memory-Efficient Exact Backpropagation for Long Sequence Training of LLMs

T0 review · 2 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read StreamBP splits the chain rule along the sequence so that activation and logits memory drop to about one chunk's worth, while gradients stay exact.

desk verdict Good idea, honest experiments, but the central memory claim omits H_in and needs a clear fix before the 1/D scaling and 2.8–5.5x numbers can be taken at face value. read the letter →

arxiv 2506.03077 v1 pith:EOR3QYZX submitted 2025-06-03 cs.LG cs.AI

Qijun Luo , Mengqi Li , Lei Zhao , Xiao Li This is my paper
classification cs.LGcs.AI
keywords streambackpropagationexactgradientsactivationmemoryreductionlong-sequencetrainingcausalattentiongradientcheckpointingGRPODPO
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the memory bottleneck of training causal language models on very long sequences can be removed without sacrificing exact gradients. Its proposal, StreamBP, rewrites the chain rule as a sum over sequence chunks, so only one chunk's logits and layer activations are live in memory at a time, cutting activation and logits storage to roughly one D-th when D chunks are used. The same decomposition is shown to work for the three objectives used to train reasoning models—supervised fine-tuning, GRPO, and DPO—and the causal mask cuts attention-score FLOPs about in half. If the central claim is right, a single GPU can train on sequences 2.8-5.5x longer than gradient checkpointing, with comparable or faster backward passes, and the memory savings transfer directly to larger batch sizes.

What carries the argument

The load-bearing object is the linear decomposition of the chain rule, Equation (1) in the paper: $\partial L/\partial \mathrm{vec}(W) = \sum_{i=1}^D (\partial \mathrm{vec}(Z_{out}^{(i)})/\partial W)^\top \partial L/\partial \mathrm{vec}(Z_{out}^{(i)})$. This identity lets the backward pass process the output in $D$ sequential chunks, keeping memory proportional to one chunk at a time. Its companion for causal transformers is Property 3.1, which limits each chunk's needed activations to $O^{(i)}, Q^{(i)}, K^{(:i)}, V^{(:i)}$; compared with full checkpointed reforwarding, the per-chunk working set is much smaller, and the attention-score computation uses the prefix rather than the full key and value sequences.

What would settle it

Fix the sequence length, increase the number of chunks $D$, and compare peak memory and gradients of StreamBP against a full-precision standard backward pass. If the memory attributed to layer activations does not fall roughly as $1/D$, or if the gradient error exceeds the error from reordering the same floating-point additions, the claimed memory scaling or exactness fails.

Watch

Extended reading notes

Core claim

The central claim is that exact gradients can be obtained by linearly decomposing the chain rule along the sequence dimension, processing one partition at a time, and accumulating the partial contributions. Concretely, the gradient of a weight $W$ is written as a sum over chunks $i$ of $(\partial \mathrm{vec}(Z_{out}^{(i)})/\partial W)^\top \partial L/\partial \mathrm{vec}(Z_{out}^{(i)})$, so only one chunk's activations are alive at once. For transformer layers, the paper isolates Property 3.1: the $i$-th output chunk's weight gradient needs only $O^{(i)}$, $Q^{(i)}$, $K^{(:i)}$, and $V^{(:i)}$; because attention is causal, keys and values are computed once per layer and cached. The language-modeling head is streamed by partitioning logits along the sequence, with separate accumulation rules for SFT, GRPO, and DPO, where DPO requires a final scalar correction after accumulation. The paper verifies numerical equivalence to standard backpropagation at FP32 precision and reports sequence-length scaling of 2.8-5.5 times relative to gradient checkpointing on models from 4B to 32B parameters.

Load-bearing premise

The load-bearing premise is that the gradient for each sequence chunk can be formed from that chunk's query and output plus all keys and values up to that point, without also keeping the chunk's input hidden states in memory; if those inputs are required for the query, key, and value weight gradients and are not cached, the actual memory or time cost differs from the reported 1/D scaling.

Editorial extensions

If this is right

  • A fixed GPU can train on sequences 2.8-5.5 times longer than with gradient checkpointing, and 23-36 times longer than without checkpointing, across model sizes from 4B to 32B parameters.
  • The memory saving is linear in sequence length, so at a fixed sequence length the same saving becomes a batch-size multiplier that accelerates training rather than only extending length.
  • SFT, GRPO, and DPO all receive exact gradients, so StreamBP can replace the backward pass in reasoning-model training without changing the optimization target.
  • Causal attention reduces attention-score FLOPs by about half relative to standard checkpointed backward passes, and the backward-time advantage grows as the sequence length grows.
  • A distributed variant under sharded optimizer states keeps standard gradient-communication patterns and reaches 5-5.6 times longer sequences than gradient checkpointing on 8 GPUs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One design question the paper leaves implicit is how the query, key, and value projection gradients accumulate: Property 3.1 lists $O^{(i)}, Q^{(i)}, K^{(:i)}, V^{(:i)}$ as sufficient, but $dW_q, dW_k, dW_v$ formally also multiply the chunk input $H_{in}^{(i)}$, so whether the implementation caches or recomputes that input changes the precise memory-time trade-off.
  • An editor's extension: the same decomposition should transfer to any sequence layer whose per-position output Jacobian is triangular in the partition variable, such as recurrent or state-space layers, not only causal attention.
  • An editor's extension: the paper's ablation shows that very small partitions add time overhead from repeated weight loads and kernel launches, so the practical recipe is to choose the largest partition that fits in memory and fuse the per-chunk backward kernels.
  • An editor's extension: because StreamBP's memory scales linearly with sequence length, it composes naturally with sequence-parallel and activation-offloading schemes, potentially pushing single-GPU sequence lengths beyond the demonstrated 200k range.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes StreamBP, an exact backpropagation algorithm for causal transformer LLMs that partitions the sequence dimension into chunks and accumulates weight gradients chunk-by-chunk in a running sum. The method is developed for the language-modeling head under SFT, GRPO, and DPO objectives, and for transformer attention/MLP layers under a partitioned, prefix-only computation. The authors report that StreamBP reduces activation and logit memory to roughly 1/D of the standard cost, scales maximum sequence length by 2.8–5.5x compared with gradient checkpointing, and achieves comparable or faster backward time while computing mathematically exact gradients. They also propose a communication-efficient distributed version for ZeRO-style training and provide an implementation link.

Significance. If the memory and timing claims were correct, StreamBP would be a practically valuable systems contribution: it gives exact gradients (empirically verified against FP32 baselines in §B.1), supports SFT, GRPO, and DPO, ships open-source code, and improves over the logits-only Mini-Sequence Transformer baseline by also reducing transformer-layer activation memory. The exact-gradient decomposition in Eq. (1) and the head-level derivations in §3.2.1 are sound in exact arithmetic. However, the central memory-efficiency claim is currently overstated because the transformer-layer dependency analysis omits the input activations H_in required to form the attention weight gradients, and it also undercounts the cost of caching full K and V. These omissions affect the headline sequence-length scaling and speedup numbers, so the central contribution needs substantial revision before the claims can be accepted at face value.

major comments (2)
  1. [§3.2.2, Property 3.1 and “Memory efficiency of StreamBP”] Property 3.1 states that ∂H_out^(i)/∂W depends only on O^(i), Q^(i), K^(:i), and V^(:i), and the subsequent stored-activation list excludes H_in. This is not correct for the attention weight gradients: dW_q^(i) = (H_in^(i))^T dQ^(i), dW_k^(i) = (H_in^(:i))^T dK^(:i), and dW_v^(i) = (H_in^(:i))^T dV^(:i), so every partition requires input activations for the full prefix up to position iT/D. If H_in is cached per layer, the activation memory is O(Td) per layer rather than roughly 1/D of standard activation memory, and the 2.8–5.5x sequence-length gains in Figure 3 and Table 3 are measured with an unstated memory term. If H_in is instead recomputed, the additional forward FLOPs and wall-clock time are absent from the FLOPs analysis in §3.2.2 and from Tables 1, 2, and 5. Moreover, K and V are cached in full even in the stated list, costing O(Td) (or O(Td/G) with GQA) and not scaling with D. The paper must state which cost it pays for H_in and must revise the memory, scaling, and timing claims accordingly.
  2. [§3.2.2, “Computational efficiency of StreamBP”] The claimed FLOPs saving compares “the standard implementation S=QK^⊤” at 2T^2 d^2 FLOPs with StreamBP’s prefix-only computation at (1+D)T^2 d^2/D FLOPs, but a causal language model does not compute the upper-triangular entries of S in a standard causal implementation with flash attention or masked attention; such implementations already compute only about half the scores. The reported FLOPs advantage over the baseline is therefore an artifact of comparing against a full-mask dense attention baseline rather than a causal-attention baseline. The wall-clock speedups in Table 1 may still arise from reduced activation recomputation or memory pressure, but they do not support the stated “leveraging the causal structure” FLOPs argument. Please rebenchmark against a causal flash-attention checkpointing baseline or qualify the FLOPs claim.
minor comments (4)
  1. [§1 and throughout] There are several typographical errors, including “finetuing” in the introduction, “distribuetd” in contribution (C.2), and “containg” in §3.3; a careful proofread is needed.
  2. [§3.2.2, Eq. (13)] The mask partition M^(i) used in softmax(S^(i), M^(i)) is not explicitly defined; please specify its dimensions and how it is derived from the causal mask.
  3. [§3.2.2, Property 3.1] Property 3.1 is stated as an observation without proof; because it is the load-bearing dependency claim for the algorithm, a short derivation or a formal statement with a proof sketch would make the paper more self-contained.
  4. [§C.1] The text says “we draw a subfigure to interpret during the 2nd backward process,” but the referenced subfigure is not clearly located or labeled in Figure 1; please add the subfigure or remove the reference.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: StreamBP derives exact gradients by partitioning the chain rule and evaluates against external baselines; the noted Property 3.1 issue is a correctness concern, not a circular reduction.

full rationale

StreamBP is not circular. The core identity (Eq. 1) is a direct application of the chain rule: the gradient of the loss with respect to a weight is written as a sum over sequence partitions, which holds for any partition of Z_out by linearity of the transpose-Jacobian product. The language-model-head sections derive SFT, GRPO, and DPO gradients by differentiating the stated losses and accumulating per-chunk contributions; DPO's non-separable log-sigmoid term is handled by the exact gradient identity in Eq. (8)-(10), not by fitting a parameter to a target. The transformer-layer development in Eqs. (12)-(14) computes partitioned attention and MLP gradients from Q, K, V, and O and accumulates them as a running sum, which is the ordinary backward pass restricted to sequence chunks; caching K and V is an explicit implementation choice, not a circular redefinition. The reported 2.8-5.5x memory and time advantages are empirical measurements against gradient checkpointing, MsT, and standard BP, not quantities fitted from those baselines, and no load-bearing assumption is justified solely by a self-citation. The possible omission of H_in from the dependency list in Property 3.1 would be a correctness or memory-accounting flaw, but it does not make any predicted quantity equal to its own input by construction; the derivation chain from the chain rule to the gradient accumulation is mathematically exposed and externally verifiable.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

No novel physical or mathematical entity is introduced. The load-bearing inputs are the partition sizes and the causal-structure assumption; all other ingredients are standard transformer components and standard gradients.

free parameters (2)
  • transformer layer partition size ps = 500 for maximum sequence length; T/3 for time measurements
    The reported memory savings and maximum sequence length ratios scale roughly as 1/D = ps/T; the headline 2.8-5.5x numbers are tied to these hand-chosen partition sizes.
  • lm_head partition size ps_head = 100
    Logits memory is reduced by 1/D_head with D_head = T/100; the choice affects maximum sequence length and time trade-offs.
assumptions (5)
  • standard math Chain rule and linearity of Jacobians allow the gradient to be decomposed into a sum over chunks of the output (Eq. 1).
    Central identity in Section 3.1; true in exact arithmetic for any differentiable f_W.
  • domain assumption In a causal transformer, the i-th output chunk depends only on Q^(i), K^(:i), V^(:i); no future tokens contribute (Property 3.1).
    Necessary for chunk-wise attention backward; would fail for bidirectional attention unless the partition structure is changed.
  • domain assumption LayerNorm, RMSNorm, residual connections, and MLP nonlinearities act per token or are compatible with sequence partitioning.
    The paper states these are disregarded 'without loss of generality' in Section 3.2.2; a cross-token nonlinearity would break the partition.
  • domain assumption SFT, GRPO, and DPO losses or gradients are separable over sequence positions as written in Eqs. (3), (6), (8)-(10).
    The DPO case is not loss-separable but its gradient factorizes using the scalar correction in Eq. (10).
  • domain assumption FlashAttention's tiling yields operations compatible with the partitioned computations.
    The paper claims compatibility in Section 3.2.2 without citing or proving the exact numerical equivalence in the chunked setting.

how reviews work

0 comments
Cite this review

Pith. "Pith review of StreamBP: Memory-Efficient Exact Backpropagation for Long Sequence Training of LLMs." pith.science (2026). https://pith.science/paper/EOR3QYZX

@misc{pith2026250603077,
  author       = {Pith},
  title        = {Pith review of: StreamBP: Memory-Efficient Exact Backpropagation for Long Sequence Training of LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EOR3QYZX}},
  note         = {Machine review of arXiv:2506.03077}
}
read the original abstract

Training language models on long sequence data is a demanding requirement for enhancing the model's capability on complex tasks, e.g., long-chain reasoning. However, as the sequence length scales up, the memory cost for storing activation values becomes huge during the Backpropagation (BP) process, even with the application of gradient checkpointing technique. To tackle this challenge, we propose a memory-efficient and exact BP method called StreamBP, which performs a linear decomposition of the chain rule along the sequence dimension in a layer-wise manner, significantly reducing the memory cost of activation values and logits. The proposed method is applicable to common objectives such as SFT, GRPO, and DPO. From an implementation perspective, StreamBP achieves less computational FLOPs and faster BP speed by leveraging the causal structure of the language model. Compared to gradient checkpointing, StreamBP scales up the maximum sequence length of BP by 2.8-5.5 times larger, while using comparable or even less BP time. Note that StreamBP's sequence length scaling ability can be directly transferred to batch size scaling for accelerating training. We further develop a communication-efficient distributed StreamBP to effectively support multi-GPU training and broaden its applicability. Our code can be easily integrated into the training pipeline of any transformer models and is available at https://github.com/Ledzy/StreamBP.

Figures

Figures reproduced from arXiv: 2506.03077 by the authors.

Figure 1
Figure 1. Memory profile during backpropagation of Qwen 3-4B with gradient checkpointing, [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. StreamBP for transformer layer (best view in color). The stored activations of StreamBP is [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Peak BP memory cost measurement of Qwen 3-8B, 14B, and 32B models under different [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Memory cost comparison with MsT. 4.2 Training Cost Measure Sequence length scaling. We present the maximum sequence length of StreamBP and baseline methods given a single A800-80GB GPU in [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Time and memory costs of StreamBP on Qwen 3-8B with varying partition sizes (ps). [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Design of distributed StreamBP. When backward through a transformer layer, its parameter [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Peak BP memory cost measurement of Qwen 3-8B with LoRA (rank=32) on a single [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Memory profile with StreamBP under the same settings as [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

37 extracted references · 6 canonical work pages

  1. [1]

    GPT-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. GPT-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    Gqa: Training generalized multi-query transformer models from multi-head checkpoints

    Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebron, and Sumit Sanghai. Gqa: Training generalized multi-query transformer models from multi-head checkpoints. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 4895–4901, 2023

    2023

  3. [3]

    Sparks of artificial general intelligence: Early experiments with GPT-4.arXiv preprint arXiv:2303.12712, 2023

    Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. Sparks of artificial general intelligence: Early experiments with GPT-4.arXiv preprint arXiv:2303.12712, 2023

    arXiv 2023

  4. [4]

    Training deep nets with sublinear memory cost.arXiv preprint arXiv:1604.06174, 2016

    Tianqi Chen, Bing Xu, Chiyuan Zhang, and Carlos Guestrin. Training deep nets with sublinear memory cost.arXiv preprint arXiv:1604.06174, 2016

    arXiv 2016

  5. [5]

    QLoRA: Efficient finetuning of quantized LLMs.Advances in Neural Information Processing Systems, 36, 2023

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. QLoRA: Efficient finetuning of quantized LLMs.Advances in Neural Information Processing Systems, 36, 2023

    work page 2023

  6. [6]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    arXiv 2024

  7. [7]

    Open R1: A fully open reproduction of deepseek-r1, January 2025

    Hugging Face. Open R1: A fully open reproduction of deepseek-r1, January 2025

    work page 2025

  8. [8]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    arXiv 2025

Show all 37 references
  1. [9]

    LoRA: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. LoRA: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022

    2022

  2. [10]

    Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model.arXiv preprint arXiv:2503.24290, 2025

    Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung-Yeung Shum. Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model.arXiv preprint arXiv:2503.24290, 2025

    2025 arXiv

  3. [11]

    Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models.arXiv preprint arXiv:2309.14509, 2023

    Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Minjia Zhang, Shuaiwen Leon Song, Samyam Rajbhandari, and Yuxiong He. Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models.arXiv preprint arXiv:2309.14509, 2023

    2023 arXiv

  4. [12]

    Openai o1 system card.arXiv preprint arXiv:2412.16720, 2024

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card.arXiv preprint arXiv:2412.16720, 2024

    2024 arXiv

  5. [13]

    Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014. 10

    2014 arXiv

  6. [14]

    Reducing activation recomputation in large transformer models.Proceedings of Machine Learning and Systems, 5:341–353, 2023

    Vijay Anand Korthikanti, Jared Casper, Sangkug Lym, Lawrence McAfee, Michael Ander- sch, Mohammad Shoeybi, and Bryan Catanzaro. Reducing activation recomputation in large transformer models.Proceedings of Machine Learning and Systems, 5:341–353, 2023

    2023

  7. [15]

    Llms can easily learn to reason from demonstrations structure, not content, is what matters!arXiv preprint arXiv:2502.07374, 2025

    Dacheng Li, Shiyi Cao, Tyler Griggs, Shu Liu, Xiangxi Mo, Eric Tang, Sumanth Hegde, Kourosh Hakhamaneshi, Shishir G Patil, Matei Zaharia, et al. Llms can easily learn to reason from demonstrations structure, not content, is what matters!arXiv preprint arXiv:2502.07374, 2025

    2025 arXiv

  8. [16]

    Sequence paral- lelism: Long sequence training from system perspective.arXiv preprint arXiv:2105.13120, 2021

    Shenggui Li, Fuzhao Xue, Chaitanya Baranwal, Yongbin Li, and Yang You. Sequence paral- lelism: Long sequence training from system perspective.arXiv preprint arXiv:2105.13120, 2021

    2021 arXiv

  9. [17]

    Dora: Weight-decomposed low-rank adaptation

    Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen. Dora: Weight-decomposed low-rank adaptation. In Forty-first International Conference on Machine Learning, 2024

    2024

  10. [18]

    Understanding r1-zero-like training: A critical perspective.arXiv preprint arXiv:2503.20783, 2025

    Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding r1-zero-like training: A critical perspective.arXiv preprint arXiv:2503.20783, 2025

    2025 arXiv

  11. [19]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. InInternational Conference on Learning Representations

  12. [20]

    Mini-sequence transformer: Optimizing intermediate memory for long sequences training.arXiv preprint arXiv:2407.15892, 2024

    Cheng Luo, Jiawei Zhao, Zhuoming Chen, Beidi Chen, and Anima Anandkumar. Mini-sequence transformer: Optimizing intermediate memory for long sequences training.arXiv preprint arXiv:2407.15892, 2024

    2024 arXiv

  13. [21]

    Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica

    Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y . Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica. DeepScaleR: Surpassing o1-preview with a 1.5b model by scaling rl, 2025. Notion Blog

    2025

  14. [22]

    BAdam: A memory efficient full parameter optimization method for large language models.Advances in Neural Information Processing Systems, 37:24926–24958, 2024

    Qijun Luo, Hengxu Yu, and Xiao Li. BAdam: A memory efficient full parameter optimization method for large language models.Advances in Neural Information Processing Systems, 37:24926–24958, 2024

    2024

  15. [23]

    s1: Simple test-time scaling.arXiv preprint arXiv:2501.19393, 2025

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling.arXiv preprint arXiv:2501.19393, 2025

    2025 arXiv

  16. [24]

    Tinyzero

    Jiayi Pan, Junjie Zhang, Xingyao Wang, Lifan Yuan, Hao Peng, and Alane Suhr. Tinyzero. https://github.com/Jiayi-Pan/TinyZero, 2025. Accessed: 2025-01-24

    2025

  17. [25]

    Zero-offload: Democratizing billion-scale model training.arXiv preprint arXiv:2101.06840, 2021

    Jie Ren, Samyam Rajbhandari, Reza Yazdani Aminabadi, Olatunji Ruwase, Shuangyan Yang, Minjia Zhang, Dong Li, and Yuxiong He. Zero-offload: Democratizing billion-scale model training.arXiv preprint arXiv:2101.06840, 2021

    2021 arXiv

  18. [26]

    Trl: Transformer reinforce- ment learning.https://github.com/huggingface/trl, 2020

    Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, Shengyi Huang, Kashif Rasul, and Quentin Gallouédec. Trl: Transformer reinforce- ment learning.https://github.com/huggingface/trl, 2020

    2020

  19. [27]

    Reinforcement learning for reasoning in large language models with one training example.arXiv preprint arXiv:2504.20571, 2025

    Yiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren, Lucas Liu, Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, Jianfeng Gao, et al. Reinforcement learning for reasoning in large language models with one training example.arXiv preprint arXiv:2504.20571, 2025

    2025 arXiv

  20. [28]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  21. [29]

    Light-R1: Curriculum sft, dpo and rl for long cot from scratch and beyond.arXiv preprint arXiv:2503.10460, 2025

    Liang Wen, Yunke Cai, Fenrui Xiao, Xin He, Qi An, Zhenyu Duan, Yimin Du, Junchen Liu, Lifu Tang, Xiaowei Lv, et al. Light-R1: Curriculum sft, dpo and rl for long cot from scratch and beyond.arXiv preprint arXiv:2503.10460, 2025. 11

    2025 arXiv

  22. [30]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020

  23. [31]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report.arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  24. [32]

    Limo: Less is more for reasoning.arXiv preprint arXiv:2502.03387, 2025

    Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. Limo: Less is more for reasoning.arXiv preprint arXiv:2502.03387, 2025

    2025 arXiv

  25. [33]

    Demystifying long chain-of-thought reasoning in llms.arXiv preprint arXiv:2502.03373, 2025

    Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. Demystifying long chain-of-thought reasoning in llms.arXiv preprint arXiv:2502.03373, 2025

    2025 arXiv

  26. [34]

    Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476, 2025

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476, 2025

    2025 arXiv

  27. [35]

    Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? arXiv preprint arXiv:2504.13837, 2025

    Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, and Gao Huang. Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? arXiv preprint arXiv:2504.13837, 2025

    2025 arXiv

  28. [36]

    Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild.arXiv preprint arXiv:2503.18892, 2025

    Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild.arXiv preprint arXiv:2503.18892, 2025

    2025 arXiv

  29. [37]

    Model parameters

    Jiawei Zhao, Zhenyu Zhang, Beidi Chen, Zhangyang Wang, Anima Anandkumar, and Yuandong Tian. Galore: Memory-efficient llm training by gradient low-rank projection.arXiv preprint arXiv:2403.03507, 2024. 12 Contents 1 Introduction 1 2 Preliminary: Peak Memory Cost during Backprop...

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.