Pith. sign in

REVIEW 5 major objections 5 minor 42 references

Infinite Sampling: Efficient and Stable Grouped RL Training for Large Language Models

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that GRPO training can scale to much larger sample groups on a fixed GPU by decomposing each group into memory-bounded micro groups, interleaving their decoding, and scheduling them by predicted length.

desk verdict Real memory savings for large-group GRPO decoding, but the throughput and stability claims are resting on a metric that doesn't survive scrutiny. read the letter →

arxiv 2506.22950 v1 pith:RBXXRHLI submitted 2025-06-28 cs.LG

Liangyu Wang , Huanyi Xie , Xinhai Wang , Tianjin Huang , Mengdi Li , Di Wang This is my paper
classification cs.LG
keywords GRPOgroupreinforcementlearningKVcachemicrosamplinggroupscontinuouslength-awareschedulingshortest-job-firstmemory-efficientLLMtraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Infinite Sampling argues that large-group GRPO training does not have to pay memory proportional to the group size. The paper proposes splitting each prompt's group of completions into memory-sized micro groups that share a single prompt KV cache, then interleaving token generation across those micro groups so decoding slots stay busy, and finally scheduling completions by predicted length to cut idle waiting. The reported result is that peak decoding memory drops by over 50% (21.55 GB to 10.64 GB on Qwen3-1.7B with group size 32), and decoding rounds drop by up to 45% at the same average sequence length. A sympathetic reader should care because this is a direct path to training with much larger GRPO groups on a fixed GPU budget, if the step-reduction metric survives a wall-clock check.

What carries the argument

The load-bearing mechanism is a shared, bounded KV-cache pool: the prompt's prefill cache is computed once and reused by every completion, and a pool sized for g active sequences is recycled across micro groups. On top of that, continuous sampling interleaves token-level decoding across micro groups, and a length-aware scheduler first packs estimated-length samples into balanced groups with an FPTAS bin-packing approximation, then refills freed slots with the shortest remaining predicted job. A BERT-based predictor, conditioned on the first k decoded tokens concatenated to the prompt, supplies the length estimates that make the packing possible.

What would settle it

Run the full pipeline on an A100 with Qwen3-8B, GSM8K prompts, group size 32 and micro group size 4, measuring wall-clock time end to end including the BERT length predictor and scheduling bookkeeping; if the wall-clock ratio of Infinite Sampling to naive micro groups is not close to 1769/3250, or if measured peak memory during the entire pipeline (not just decoding) exceeds the reported bound, the central claims are refuted. Separately, train GRPO with both methods and compare reward curves or held-out accuracy to test the stability claim.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that the KV cache, not the optimizer state, is the bottleneck that limits group size in GRPO, and that this bottleneck can be decoupled from the group size. The paper shows that decoding G completions in a fixed-size KV pool shared across sequential micro groups makes peak memory depend only on the micro group size g, and that a token-conditioned length predictor feeding a two-stage scheduler (FPTAS global grouping plus shortest-job-first slot refill) recovers most of the throughput lost to serialization. The central measurement is 'running steps,' the number of token-generation rounds across g active slots, which the paper reports as a proxy for decoding throughput. In the main experiments the full framework matches an oracle lower bound within about 1% while preserving the average sequence length of the naive micro-group baseline.

Load-bearing premise

The load-bearing premise is that a 'running step' is a constant-cost unit of work and that equal average sequence length means equal GRPO sample quality, so the reported step and length numbers alone carry the efficiency and stability conclusions.

Editorial extensions

If this is right

  • Group size becomes a memory-free hyperparameter: a user can set G large while peak decoding memory stays pinned to the micro group size.
  • On a fixed GPU, the framework supports GRPO runs that would otherwise be infeasible, at the cost of partial serialization.
  • Decoding rounds are reduced by up to 45% (3250 to 1770 on GSM8K) while average sequence length is unchanged, so fewer rounds do not come from truncated outputs.
  • The dynamic-slot variant is fastest but biases toward short sequences, so it is unsuitable for reward-based training despite its low step count.
  • The practical scheduler lands within about 1% of the oracle that knows true lengths before decoding.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same memory-bounded, length-aware scheduling could be dropped into other multi-sample decoding workloads, such as best-of-n sampling or self-consistency evaluation, where every completion shares a prompt.
  • The paper's stability claim is not yet evidenced: a direct GRPO training run comparing reward curves, KL, and downstream accuracy against full-group decoding would settle whether preserving average length preserves training quality.
  • Because the step metric ignores scheduler and predictor cost, the real speedup depends on these overheads staying under a few percent; a wall-clock measurement is the decisive next experiment.
  • An obvious extension is to layer KV-cache compression on top of the pool to shrink the micro group further, which would compound the memory savings rather than trade against them.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes Infinite Sampling, a decoding-side framework for GRPO training that decouples the sample group size G from GPU memory by (1) decomposing large groups into micro sampling groups that reuse a shared prompt KV cache, (2) interleaving token generation across micro groups via continuous sampling, and (3) using a length-aware scheduler that combines a BERT-based prefix-conditioned length predictor with a two-stage FPTAS/SJF scheduling scheme. The experiments on Qwen3-1.7B/8B across GSM8K, MATH, and KK report peak-memory reductions of over 50% versus full-group decoding and reductions in 'Running Steps' of up to 45% relative to naive micro group decoding, while preserving average sequence length. The paper also claims that this yields efficient and stable GRPO training under tight memory budgets.

Significance. If the memory and throughput claims hold, this is a practically valuable contribution: large-group GRPO is indeed memory-bound, and a decoding scheme that makes the memory footprint independent of group size would help practitioners train with larger groups on limited hardware. The paper credits itself with a concrete system design, clear ablations, and an oracle lower bound, which is useful framing. However, the current evidence is not sufficient to support the central claims. The memory measurements are the strongest part of the paper. The throughput claim is based exclusively on a step-count metric that is not a constant-cost proxy for wall-clock time, and the stability half of the central claim is never evaluated: there is no actual GRPO training run, no reward or loss curve, and no downstream accuracy result. The predictor evaluation in Figure 5 is isolated from the scheduling pipeline, and Algorithm 2's FPTAS claim is not accompanied by a proof or even a correct algorithmic description. The paper is well written and addresses an important problem, but the load-bearing empirical claims need substantial additional validation.

major comments (5)
  1. [Section 5.1, Table 1] The central throughput claim rests on 'Running Steps' as a proxy for wall-clock time, but this metric is not a constant-cost unit. A decoding step with a long active sequence and a long KV cache can cost substantially more than a step with several short sequences, and the proposed SJF scheduler deliberately changes which sequences are batched together at each step. The paper reports no wall-clock time, no measured cost of BERT inference, prefix sampling, or SJF bookkeeping, and no end-to-end generation time. Therefore the abstract's claims of 'over 25%' and 'up to 45%' throughput improvement are not established by the data in Table 1.
  2. [Table 1, Section 5.1] Table 1 appears internally inconsistent under the paper's own definition of Running Steps. Section 5.1 defines one running step as one token-generation round across all g active decoding slots, i.e., g tokens per step. For GSM8K with Qwen3-8B and the default g=4, Dynamic-slot reports 360 steps, which would correspond to 1,440 token-slots; but 32 completions with average length 21 yield only 672 generated tokens. If slots are not always full, then the metric is not the one defined in Section 5.1, and the comparisons in Tables 1 and 2 are based on an inconsistent unit. The paper should either reconcile the definition, state the actual average number of active slots, or replace this metric with wall-clock time.
  3. [Section 5.2, Section 5.3, Section 3.4] The claim that Infinite Sampling enables 'stable GRPO training' is not tested. Section 5 reports only decoding-side ablations: memory, running steps, and average sequence length. There is no actual GRPO training run, no reward or loss curve, no evaluation of final task accuracy, and no comparison of training stability against a baseline. The statement in Table 1's 'Stability vs. Efficiency' bullet is therefore an interpretation, not a result. The micro-batched reward and backpropagation design in Section 3.4 is described but never validated, so the stability half of the central claim is unsupported by the experiments.
  4. [Section 4.2, Figure 5] The length predictor is central to the scheduler, but its evaluation is incomplete. The predictor is fine-tuned on LMSYS-Chat and then applied to GSM8K, MATH, and KK, yet Figure 5 reports only prediction error as a function of prefix length k, with no baseline (e.g., a prompt-only predictor or no predictor), no error bars, no stated error metric, and no measurement of the overhead of BERT inference and prefix sampling during training. The paper's assertion that the predictor runs with 'negligible runtime overhead' is not measured. Without such data, it is unclear how much of the step reduction in Tables 1 and 2 transfers to real throughput, and whether the predictor transfers across the distribution shift from chat data to math reasoning tasks.
  5. [Algorithm 2, Section 3.3] Algorithm 2 is described as an FPTAS for micro-group assignment, but the algorithm is a greedy list-scheduling heuristic with scaled lengths, not an FPTAS. No approximation guarantee is derived, and the loop 'for each sample ... for n=1 to N ... break' can terminate without assigning all samples if no group has enough remaining capacity, with no fallback or feasibility check. The paper's statement that FPTAS 'ensures formal guarantees on group balance' is therefore not supported by the presented algorithm. This is not merely a terminology issue: the theoretical contribution claimed in Section 3.3 and the feasibility of the decoding plan depend on this algorithm. The authors should either provide a correct FPTAS with a proof, or relabel the method as a heuristic and verify its feasibility guarantees empirically.
minor comments (5)
  1. [Section 5.1] The paper states that all metrics are averaged over all prompts, but Table 1 does not report standard deviations or the number of prompts considered; for a systems paper, this makes it hard to distinguish signal from noise.
  2. [Figure 5] Figure 5 does not specify the error metric plotted on the vertical axis, and no error bars or repeated-run variation are shown; please add the definition and variability information.
  3. [References] References [15] and [16] are the same InstructGPT paper listed twice with slightly different formatting; one duplicate should be removed.
  4. [Algorithm 3] In Algorithm 3, the condition C = {i | i not in F and i not in mask} is confusing because, after Stage 1, every sample has a mask entry; please clarify whether 'mask' means 'already scheduled' or 'already completed', and define the refill candidate set accordingly.
  5. [Section 3.3] The expansion 'FPTAS' is written as 'fixed-point approximation scheme' in the introduction, but the standard expansion is 'fully polynomial-time approximation scheme'; please correct this in the text.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the scheduler's length estimates are externally trained, the oracle lower bound is post hoc and clearly labeled, and no fitted value is renamed as a prediction.

full rationale

The derivation chain is self-contained with respect to circularity. The length-aware scheduler's estimates come from a BERT regressor fine-tuned on the external LMSYS-Chat dataset, and these estimates are used only as scheduling signals; the reported running-step counts are measured from actual decoding, not reconstructed from the fitted predictor. The 'Infinite-sampling *' oracle uses post hoc true lengths as a lower-bound reference and is explicitly labeled as a theoretical rather than practical strategy, so its proximity to the real scheduler is an empirical sanity check rather than a construction. The memory-reduction claim is a direct measurement of peak GPU usage under different active-sequence counts; reducing the number of concurrently decoded sequences necessarily reduces KV-cache memory, but this is an experimental comparison, not a prediction derived from itself. The 'Running Steps' metric is defined independently in Section 5.1; although its internal consistency and cost-equivalence can be questioned (Table 1's Dynamic-slot row appears inconsistent, and step count is not wall-clock time), those are correctness and validity concerns, not circularity. No load-bearing self-citation, imported uniqueness theorem, or ansatz-smuggling citation appears; all cited external works are standard references. The unsupported 'stable GRPO training' claim stems from missing reward/loss experiments, not from a circular reduction.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

No new physical or mathematical entities are postulated. 'Infinite Sampling', 'micro sampling groups', and 'continuous sampling' are names for system components, not invented entities in the ledger sense. The main fitted knob reported is the prefix length k; the BERT length predictor is a trained model fit on an external dataset, and its weights are not reported.

free parameters (2)
  • Prefix length k for length prediction = 16
    Chosen from Figure 5 as the knee of the prediction-error curve; the token-conditioned length estimator and all scheduling results depend on this choice.
  • FPTAS tolerance epsilon = not reported
    Algorithm 2 uses an additive tolerance epsilon that is never given a value; the near-optimality claim for micro-group assignment depends on it.
assumptions (5)
  • domain assumption Running Steps is a faithful proxy for wall-clock decoding time; each step has equal cost.
    Section 5.1 defines Running Steps; all throughput claims use it as the metric, but no wall-clock measurements are reported.
  • domain assumption Average sequence length preservation implies GRPO sample quality and training stability are preserved.
    Section 5.3 argues Infinite-sampling preserves average length matching Naive Micro Group, and uses this to claim training stability; no reward, loss, or downstream accuracy is measured.
  • ad hoc to paper Algorithm 2 implements a valid FPTAS for multiprocessor scheduling and always produces a feasible partition.
    Section 3.3, Algorithm 2: the greedy loop assigns a sample only if the bin capacity is not exceeded, and unassigned samples are not handled; no feasibility proof or reference is given.
  • domain assumption The BERT length predictor trained on LMSYS-Chat transfers to GSM8K, MATH, and KK distributions for both Qwen3 sizes.
    Section 4.2 fine-tunes on LMSYS-Chat and applies to target datasets; no evidence of distribution match is provided beyond Figure 5's error curves.
  • standard math Standard GRPO and transformer decoding formulations are correct and applicable.
    Section 2 restates GRPO (Shao et al.) and KV-cache decoding; treated as background.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Infinite Sampling: Efficient and Stable Grouped RL Training for Large Language Models." pith.science (2026). https://pith.science/paper/RBXXRHLI

@misc{pith2026250622950,
  author       = {Pith},
  title        = {Pith review of: Infinite Sampling: Efficient and Stable Grouped RL Training for Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RBXXRHLI}},
  note         = {Machine review of arXiv:2506.22950}
}
read the original abstract

Group-based reinforcement learning algorithms such as Group Reward Policy Optimization (GRPO) have proven effective for fine-tuning large language models (LLMs) with human feedback. However, generating and storing multiple responses per prompt incurs substantial memory overhead, especially as the sample group size increases, limiting scalability under constrained hardware. We propose Infinite Sampling, a framework that enables efficient and stable GRPO training by decoupling group size from GPU memory usage. It consists of: (1) micro sampling groups that decompose large groups into memory-feasible rounds; (2) continuous sampling that interleaves generation across groups to improve utilization; and (3) a length-aware scheduler combining token-conditioned sequence length prediction with a two-stage plan: global grouping via FPTAS and runtime refill via SJF. Experiments show that our Micro Sampling Groups reduce peak memory usage by over 50% compared to full-group decoding (e.g., from 21.55 GB to 10.64 GB on Qwen3-1.7B). Building on this, Infinite Sampling improves throughput by over 25% compared to the naive micro sampling group method, reducing decoding steps while maintaining full-length completions and memory usage. Our hybrid scheduling ensures efficient and stable GRPO training with larger groups under realistic GPU memory constraints.

Figures

Figures reproduced from arXiv: 2506.22950 by the authors.

Figure 1
Figure 1. Illustration of the three key components in Infinite Sampling. (a) Naive Micro Sampling Group: Samples are divided [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of Infinite Sampling. Given a query, all completions share a prefilled KV cache. (1) Micro Sampling Groups [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Two Modes of Continuous Sampling. (a) In Fixed [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Peak memory usage comparison under different group sizes and micro group configurations for Infinite Sampling. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Length prediction error vs. prefix length [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Illustration of the three key components in Infinite [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Continuous Batching in inference merges unrelated [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

42 extracted references · 5 canonical work pages

  1. [1]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin...

    arXiv 2020

  2. [2]

    Seungbeom Choi, Jeonghoe Goo, Eunjoo Jeon, Mingyu Yang, and Minsung Jang

    work page

  3. [3]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training Verifiers to Solve Math Word Problems. arXiv:2110.14168 [cs.LG] https://arxiv.org/abs/2110.14168

    arXiv 2021

  4. [4]

    Tri Dao. 2023. FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning. arXiv:2307.08691 [cs.LG] https://arxiv.org/abs/2307.08691

    arXiv 2023

  5. [5]

    Fu, Stefano Ermon, Atri Rudra, and Christopher Ré

    Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. 2022. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness. arXiv:2205.14135 [cs.LG] https://arxiv.org/abs/2205.14135

    arXiv 2022

  6. [6]

    Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J

    DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Haowei Zhang, Honghui Ding, Huaj...

    arXiv 2025

  7. [7]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv:1810.04805 [cs.CL] https://arxiv.org/abs/1810.04805

    arXiv 2019

  8. [8]

    Wei Fu, Jiaxuan Gao, Xujie Shen, Chen Zhu, Zhiyu Mei, Chuyi He, Shusheng Xu, Guo Wei, Jun Mei, Jiashu Wang, Tongkai Yang, Binhang Yuan, and Yi Wu

    work page

Show all 42 references
  1. [9]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Ab- hishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, A...

    2024 arXiv

  2. [10]

    arXiv:2505.24298 [cs.LG] https://arxiv.org/abs/2505.24298

    AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning. arXiv:2505.24298 [cs.LG] https://arxiv.org/abs/2505.24298

    arXiv

  3. [11]

    W Keith Hastings. 1970. Monte Carlo sampling methods using Markov chains and their applications. (1970)

    1970

  4. [12]

    Ozgur Guldogan, Jackson Kunde, Kangwook Lee, and Ramtin Pedarsani

  5. [13]

    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let’s Verify Step by Step. arXiv:2305.20050 [cs.LG] https://arxiv.org/abs/2305.20050

    2023 arXiv

  6. [14]

    Nicholas Metropolis and Stanislaw Ulam. 1949. The monte carlo method.Journal of the American statistical association44, 247 (1949), 335–341

    1949

  7. [15]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. arXiv:2309.06180 [cs.LG] https://arxiv.org/abs/2309.06180

    2023 arXiv

  8. [16]

    Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schul- man, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Pe- ter Welinder, Paul Christiano, Jan Leike,...

    2022 arXiv

  9. [17]

    Reiner Pope, Sholto Douglas, Aakanksha Chowdhery, Jacob Devlin, James Brad- bury, Anselm Levskaya, Jonathan Heek, Kefan Xiao, Shivani Agrawal, and Jeff Dean. 2022. Efficiently Scaling Transformer Inference. arXiv:2211.05102 [cs.LG] https://arxiv.org/abs/2211.05102

    2022 arXiv

  10. [18]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback.Advances in neural information processing systems35 (...

    2022

  11. [19]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog1, 8 (2019), 9

    2019

  12. [20]

    Manning, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. 2024. Direct Preference Optimization: Your Language Model is Secretly a Reward Model. arXiv:2305.18290 [cs.LG] https://arxiv.org/ abs/2305.18290

    2024 arXiv

  13. [21]

    Kalbarczyk, Tamer Başar, and Ravishankar K

    Haoran Qiu, Weichao Mao, Archit Patke, Shengkun Cui, Saurabh Jha, Chen Wang, Hubertus Franke, Zbigniew T. Kalbarczyk, Tamer Başar, and Ravishankar K. Iyer

  14. [22]

    arXiv:2404.08509 [cs.DC] https://arxiv.org/abs/2404.08509

    Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Prediction. arXiv:2404.08509 [cs.DC] https://arxiv.org/abs/2404.08509

    arXiv

  15. [23]

    John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel

  16. [24]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov

  17. [25]

    Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 2020. ZeRO: Memory Optimizations Toward Training Trillion Parameter Models. arXiv:1910.02054 [cs.LG] https://arxiv.org/abs/1910.02054

    2020 arXiv

  18. [26]

    Samyam Rajbhandari, Olatunji Ruwase, Jeff Rasley, Shaden Smith, and Yuxiong He. 2021. ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning. arXiv:2104.07857 [cs.DC] https://arxiv.org/abs/2104.07857

    2021 arXiv

  19. [27]

    Luohe Shi, Hongyi Zhang, Yao Yao, Zuchao Li, and Hai Zhao. 2024. Keep the Cost Down: A Review on Methods to Optimize LLM’ s KV-Cache Consumption. arXiv:2407.18003 [cs.CL] https://arxiv.org/abs/2407.18003

    2024 arXiv

  20. [28]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Liangyu Wang, Huanyi Xie, Xinhai Wang, Tianjin Huang, Mengdi Li, and Di Wang Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edoua...

    2023 arXiv

  21. [29]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yas- mine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhos- ale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy F...

    2023 arXiv

  22. [30]

    Chulin Xie, Yangsibo Huang, Chiyuan Zhang, Da Yu, Xinyun Chen, Bill Yuchen Lin, Bo Li, Badih Ghazi, and Ravi Kumar. 2025. On Memorization of Large Language Models in Logical Reasoning. arXiv:2410.23123 [cs.CL] https://arxiv. org/abs/2410.23123

    2025 arXiv

  23. [31]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. DeepSeek- Math: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv:2402.03300 [cs.CL] https://arxiv.org/abs/2402.03300

    2024 arXiv

  24. [32]

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2025. Hybridflow: A flexible and efficient rlhf framework. InProceedings of the Twentieth European Conference on Computer Systems. 1279–1297

    2025

  25. [33]

    Gonzalez, Clark Barrett, and Ying Sheng

    Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, and Ying Sheng. 2024. SGLang: Efficient Execution of Structured Language Model Programs. arXiv:2312.07104 [cs.AI] htt...

    2024 arXiv

  26. [34]

    Zhen Zheng, Xin Ji, Taosong Fang, Fanghao Zhou, Chuanjie Liu, and Gang Peng

  27. [35]

    Yinmin Zhong, Zili Zhang, Xiaoniu Song, Hanpeng Hu, Chao Jin, Bingyang Wu, Nuo Chen, Yukun Chen, Yu Zhou, Changyi Wan, et al. 2025. StreamRL: Scalable, Heterogeneous, and Elastic RL for LLMs with Disaggregated Stream Generation. arXiv preprint arXiv:2504.15930(2025). Appendix ...

    2025 arXiv

  28. [37]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jia...

    2025 arXiv

  29. [38]

    P Xing, Joseph E

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Tianle Li, Siyuan Zhuang, Zhang- hao Wu, Yonghao Zhuang, Zhuohan Li, Zi Lin, Eric. P Xing, Joseph E. Gonzalez, Ion Stoica, and Hao Zhang. 2023. LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset. arXiv:2309.11998 [cs.CL]

    2023 arXiv

  30. [41]

    arXiv:2412.03594 [cs.CL] https://arxiv.org/abs/2412.03594

    BatchLLM: Optimizing Large Batched LLM Inference with Global Prefix Sharing and Throughput-oriented Token Batching. arXiv:2412.03594 [cs.CL] https://arxiv.org/abs/2412.03594

    arXiv

  31. [2017]

    Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347 (2017)

    2017 arXiv

  32. [2018]

    arXiv:1506.02438 [cs.LG] https://arxiv.org/abs/1506.02438

    High-Dimensional Continuous Control Using Generalized Advantage Estimation. arXiv:1506.02438 [cs.LG] https://arxiv.org/abs/1506.02438

    arXiv

  33. [2024]

    arXiv:2412.04504 [cs.CL] https://arxiv.org/abs/2412.04504

    Multi-Bin Batching for Increasing LLM Inference Throughput. arXiv:2412.04504 [cs.CL] https://arxiv.org/abs/2412.04504

    arXiv

  34. [2025]

    arXiv:2505.09142 [cs.DC] https://arxiv.org/abs/2505.09142

    ELIS: Efficient LLM Iterative Scheduling System with Response Length Predictor. arXiv:2505.09142 [cs.DC] https://arxiv.org/abs/2505.09142

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.