Distributed, Parallel, and Cluster Computing · arXiv:2609.34645 v1 · submitted 28 Sep 2026 · 60 references
Nereus: Adaptive Parallelism for LLM Post-Training
The paper says
“Reinforcement learning (RL) post-training for large language models (LLMs) coordinates multiple models across generation, inference, and training on GPU clusters. Several factors may change during a run, including resource availability, sequence length, memory pressure, and stage bottlenecks.”
more of the abstract
As a consequence, an execution plan that was initially suitable can then become slow or even infeasible over time. However, adapting a job whose models share GPUs entails significant challenges: deciding whether a new plan is worth the transition cost, reusing the job's distributed state, and coordinating GPU transfers across models and stages. Nereus targets these challenges as a cost-aware runtime that adapts RL post-training jobs into efficient execution plans. Its low-overhead controller selects a memory-feasible global plan and admits the transition using a cost model calibrated against the running job. To estimate and execute a transition, Nereus represents the distributed state of each replica of a model-stage (one model in one stage) as an Elastic Model Unit. It then employs a global transition graph to order the transformations and GPU transfers of these units. In a trace built from real data, online TP/PP adaptation reduces average step latency by 27.7% relative to the initial fixed TP/PP layout with DP scaling. In a 1,000-step run reaching 1,024 GPUs, six transitions consume 0.079% of total run time. Nereus improves end-to-end 8B PPO throughput by 2.14–7.27× over OpenRLHF and by 1.10–1.47× over Verl across diverse clusters.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 29 Sep 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. A genuine systems contribution with thorough hardware evaluation; the weak link is the admission-timing premise, which the authors partially acknowledge but never stress-test under volatile drift. This line is wrong
2 major objections · 3 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 60 references say.
The paper says, at 4.3, Eq. (2); 7.3.3
The review did not quote this passage.
Pith says: Objection 1 of 2 · major
The 'when to adapt' decision rests entirely on the predicted payback ratio L_tran/Delta-L and on the EMA horizon H_x, but the evaluation validates this link only under benign, monotone drift: all three held-out traces combine gradual sequence-length growth with moderate GPU-count variation, and the only probe of admission misbehavior is the gamma=1 sweep, where over-admission to 14 transitions raises latency to 939.5 s (worse than the 861.9 s default).
more of objection 1
Because the cost model's 4.96% MAPE (Section 7.3.1) can be as large as the true latency gap between near-optimal plans (the paper's own p95 selection gap is 4.1%, Section 7.3.2), the sign of predicted Delta-L can be wrong near the optimum, and the paper reports no per-transition realized-payback audit that would establish whether each admitted transition actually repaid L_tran within H_x steps. Since contribution (1) is specifically the payback-based admission decision, I recommend adding (i) a per-decision audit recording predicted and realized savings and transition cost for every admission in the traces, and (ii) stress runs under synthetic drift with abrupt sequence-length spikes or rapid GPU revocation, where an EMA horizon is known to lag. These experiments, not further calibration fit, would settle whether the timing link is well calibrated.
The paper says, at 7.3.3
The review did not quote this passage.
Pith says: Objection 2 of 2 · major
The marginal value of the payback admission rule specifically, as opposed to replanning at all, is thin in the current evidence. Across the three held-out traces Nereus averages 858.7 s/step versus 873.8 s for 3-step adapt, a 1.7% margin that is comparable to the reported cross-trace standard deviations (5.2 s for Nereus, 12.6 s for 3-step adapt), and the larger …
more of objection 2
… headline numbers (27.7% over static, 8.3% over DynaRL-style) do not isolate the timing decision from the benefit of adapting at all. To support the claim that payback-based admission matters, the paper should report the realized-payback outcome distribution and compare against a small set of replan periods rather than a single fixed 3-step period; if the margin over periodic replanning is genuinely small in these traces, that is an important scoping statement for contribution (1).
3 minor objections
The paper says, at 7 (general)
Pith says: minor objection 1 of 3
The paper describes a 43k-line implementation but gives no artifact or code-availability statement; providing the code, or a clear statement of availability, would materially aid reproducibility of the 480-plan and three-trace experiments.
The paper says, at 7.3.3, Fig. 11
Pith says: minor objection 2 of 3
The three subplots mix the latency curve, the gap curve, the confidence-margin sweep, and a secondary GPU-count axis; a small table listing each admitted transition with its predicted Delta-L, predicted L_tran, H_x, and realized payback would make the admission behavior directly checkable and would directly address the timing concern above.
show 1 more
The paper says, at 4.3
Pith says: minor objection 3 of 3
The H_x initial value is one step until the first crossing interval is observed, and reference values reset after every replan even on rejection; a sentence on how quickly H_x stabilizes (e.g., first-interval length in the traces) would clarify early-run admission behavior.
What would settle it
Instrument the three held-out traces (Section 7.3.3) to record, for every admitted transition: the predicted Delta-L and L_tran from the cost model, the realized Delta-L from measured step latencies before and after the transition, and the realized window until the next drift crossing. Then (1) count the fraction of admitted transitions whose realized payback exceeds the realized window, i.e., transitions that did not repay before drift moved again; (2) recompute average s/step with an oracle admission rule that uses realized instead of predicted Delta-L. If a substantial fraction (above roughly 20%) of admitted transitions fail to repay in the realized window, or if oracle-based admission changes the 858.7 s/step result by more than about 5%, the payback-timing premise is confirmed as load-bearing.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 29 Sep 2026
- Reader holds up, with conditions
- Skeptic partly agrees
- Editor wrote the verdict above
- Referee major revision: 2 major objections, 3 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
Bottom line: this is a real contribution, not a repackaging. The new content is the integration for the coupled multi-model, multi-stage RL post-training case — the EMU dependency-aligned state boundary and the global transition DAG over four primitives. The pieces are individually published (Alpa/NanoFlow cost models, Pollux/Sia payback, Tenplex/Oobleck elastic state), but nobody has assembled them for RL post-training, and the model-stage replica as the state unit is the right granularity for that workload.
What earns credit: the evaluation is thorough by field standards. 480 plans executed on hardware against the empirical optimum, within 5% in all 18 settings; 72 out-of-sample calibration plans at 4.96% MAPE; held-out real-data traces; three clusters, four model sizes, three RL algorithms, plus async execution. They also verify adaptation preserves training behavior — reward and KL curves track Verl over 50 steps. That evidence is the kind a systems paper should have. Planning is cheap: 338 ms at 1,024 GPUs versus 511 s for SCIP, within 5–7% of SCIP's schedule quality. Transition overhead at scale is small: 0.079% of run time for six transitions.
The honest soft spot is the "when to adapt" decision, and the authors do not hide it. Admission rests on Eq. (2): predicted savings must repay predicted transition cost within a fraction of Hx, an EMA of past drift-crossing intervals. Near the optimum, the cost model's 4.96% MAPE can sign Delta-L wrong. Their own gamma sweep shows the risk — at gamma=1, over-admission yields 14 transitions and 939.5 s latency, worse than the 861.9 s default. That is fine for the default margin, but it reveals the timing premise is only as good as the horizon estimate, and the evaluation never stresses it: all three held-out traces have benign, monotone drift. No accelerating sequence-length growth, no abrupt GPU revocation spikes. That gap keeps the verdict conditional.
Two smaller points. No code or data shipped, so the hardware numbers can't be independently checked — relevant for a 43k-line system. And the OpenRLHF speedup (up to 7.27x) partly reflects baseline weakness; the Verl comparison (1.10–1.47x) is the cleaner evidence.
Who it's for: anyone building RL post-training infrastructure or elastic training runtimes. It deserves a serious referee. My asks on revision: a per-transition realized-payback audit, and an evaluation that varies drift dynamics — accelerating sequence growth and abrupt GPU revocation. Send it out.
The strongest objection
The central claim decomposes into two links: (a) plan selection quality and (b) transition timing under drift. Link (a) is well supported by direct hardware evidence: 480 held-out plans executed with the selected plan within 5% of the empirical optimum in all 18 settings, and 72 out-of-sample calibration plans at R^2=0.9993 / 4.96% MAPE. Link (b) rests entirely on Eq. (2)'s admission rule, which compares the predicted payback L_tran/Delta-L against gamma*H_x, where Delta-L and L_tran come from the same online-calibrated cost model and H_x is an EMA of past drift-crossing intervals. The load-bearing weakness is that in the flat region near the optimum, where many candidate plans lie within the paper's own 5% closeness bound, the cost model's 4.96% MAPE on latency can be as large as the true difference between the current and target plan: a transition admitted because predicted Delta-L>0 may have a realized negative benefit. The mechanism is visible in the paper's own gamma sweep (Section 7.3.3): at gamma=1, over-admission rises to 14 transitions and latency climbs to 939.5 s, worse than the gamma=0.5 default (861.9 s), demonstrating that admission outcomes are sensitive to the payback estimate and that wasted transitions cost more than they save. The paper's headline numbers (27.7% reduction; 858.7 vs 928.3 s/step across three traces) are averaged over traces with benign, monotone drift; the evaluation never varies the drift dynamics (accelerating sequence-length growth, abrupt GPU revocation spikes) that would stress the EMA horizon H_x or the Delta-L sign. So the timing premise — not plan quality — is where the conditional verdict hinges.
The reader's weakest_assumption identifies the payback-horizon estimate H_x in Eq. (2) as the load-bearing premise, and I agree that the 'when to adapt' decision is the weakly supported link rather than plan selection, which is strongly evidenced by the executed 480-plan and 72-plan out-of-sample evaluations. I part ways slightly in emphasis: the sharper failure mode is not only a stale EMA horizon but the sign error of predicted Delta-L in the flat region near the optimum, where a 4.96% MAPE cost model cannot reliably distinguish between plans that are, by the paper's own measurement, within 5% of each other. In that regime the admission rule can systematically admit transitions with negative realized benefit, and the paper's gamma=1 sweep (14 transitions, 939.5 s versus 861.9 s at gamma=0.5) is direct evidence that the mechanism is real and that over-admission costs more than it saves. The paper's headline numbers are averaged over three traces whose drift is benign and monotone, so robustness to volatile or accelerating drift is simply unevaluated; this is a gap, not a demonstration of failure. Because the verdict is already CONDITIONAL and my concern routes through the same admission rule the reader flagged, I recommend no change. I set agreement to partial rather than agree because I foreground the Delta-L sign-error mechanism (which the reader mentions but subordinates to H_x) and because the decisive test I propose is a per-decision realized-payback audit across admitted transitions, which the paper does not report. No ad hominem, no theatrical framing: this is a concrete, testable premise about prediction- and horizon-calibration under drift, with the paper's own sensitivity data indicating the mechanism matters.
The referee's report
The problem is real and growing: RL post-training runs are long and expensive, and drift (sequence-length growth, GPU revocation, memory pressure) can invalidate a startup plan. The system-level evidence is unusually strong on the plan-selection link: 480 plans executed directly on hardware as an empirical-optimum baseline, held-out calibration plans, cross-algorithm generality (PPO, ReMax, GRPO), and success in all 100-trial coordination runs. The state-abstraction and orchestration contributions are well engineered and carefully evaluated. The weakest link is the admission-timing decision, which is validated only under benign, monotone drift; the paper's own gamma sweep exposes the mechanism's fragility when the payback estimate is off. Strengthening that link with a per-transition realized-payback audit and harder drift dynamics is, in my view, the one load-bearing revision needed.
Pith's reading of the paper (a machine's summary, not the authors')
RL post-training coordinates several models across generation, inference, and training stages on shared GPUs, and everything about the run drifts: available GPUs come and go, the actor learns to emit longer sequences, and communication slows under contention. A parallelism plan that was right at startup can become slow or memory-infeasible within a few hundred steps, yet existing systems either fix the plan or restart through coarse checkpoint-based resharding. Nereus claims that online adaptation becomes cheap and correct when the state boundary matches the job's dependencies, with one Elastic Model Unit per model-stage replica and admission gated by a payback rule that executes a transition only when predicted savings repay its cost within a predicted drift window. On a real-data trace, online TP/PP adaptation cuts average step latency by 27.7% versus a fixed layout, six transitions in a 1,000-step run cost 0.079% of run time, and end-to-end 8B PPO throughput improves 2.14–7.27$\times$ over OpenRLHF and 1.10–1.47$\times$ over Verl. If right, a post-training job can ride drift continuously instead of gambling on a startup plan, saving a meaningful fraction of very expensive cluster time.
What carries the argument. The Elastic Model Unit (EMU) is one model-stage replica, defined as $\langle M, T_{tp\times pp}, R, \boldsymbol{\theta}, \boldsymbol{\omega}\rangle$: the model-stage identity, its tensor/pipeline layout, its GPU set, and the parameter and optimizer shards. TP and PP, which require tightly synchronized collectives, are encapsulated inside the unit; DP is left outside, so scaling is only adding or removing whole units. The plan space is covered by four primitives, Split and Merge converting between tightly coupled TP/PP layout and loosely coupled replicas, Extend broadcasting a replica onto new GPUs, and Destroy releasing a redundant copy, each hiding the underlying shard movement. Around the units sits the controller: an online-calibrated cost model predicts per-step latency and memory for candidate plans, and the payback admission rule $\Delta L > 0 \wedge \hat{L}_{\mathrm{tran}}/\Delta L \leq \gamma H_x$ admits a transition only when predicted savings repay predicted cost within a fraction $\gamma$ of the expected steps until the next drift crossing, where $H_x$ is an exponential moving average of past crossing intervals. The transition engine compiles the plan difference into a global DAG ordered Split$\to$Destroy$\to$Extend$\to$Merge, inserting resource-dependency edges so release-before-acquire avoids deadlock when model-stages share a near-capacity GPU pool. This machinery turns three coupled decisions, when to adapt, what state to reuse, and how to transition, into an online loop with measured decision overhead below 338 ms even at 1,024 GPUs.
Core claim. The paper establishes that plan adaptation becomes tractable when the state boundary matches the job's dependencies. It proposes a cost-aware controller that replans on drift, meaning resource changes, memory pressure, or sequence-length growth past a threshold, and admits a transition only when the current plan is infeasible (the urgent path) or when the predicted per-step latency savings repay the predicted transition cost within a fraction of the estimated steps until the next drift crossing (Eq. 2). State is reused at the granularity of the Elastic Model Unit: one model-stage replica holding the tightly coupled TP/PP shards, with DP exposed as the replica count, so resharding and scaling reduce to four primitives (Split, Merge, Extend, Destroy). A global transition DAG orders these primitives across coupled model-stages, adding resource-dependency edges when a transient GPU overlap blocks an acquisition, and executes the transition with GPU-direct collectives at a safe RL-step boundary. The reported results are that selected plans stay within 5% of the empirical optimum in all 18 measured settings, that the real-data trace shows a 27.7% average step-latency reduction versus a fixed TP/PP layout with DP scaling, that the adaptive policy averages 858.7 s/step versus 928.3 s for DynaRL-style admission across three held-out traces, and that six transitions in a 1,000-step run to 1,024 GPUs consume 0.079% of total run time.
The premise it rests on. The entire decision of when to transition rests on the online-calibrated estimates of latency savings and transition cost together with the EMA-based drift-window estimate $H_x$ being accurate enough that the payback rule admits only transitions whose savings are actually repaid, and if drift arrives sooner than expected under abrupt GPU revocation or a sudden sequence-length spike, the system over-admits and wastes transition time.
If the paper is right (Pith's inferences, not the authors')
A post-training job no longer needs to fix its plan at startup: it can follow drift in sequence length, GPU supply, and stage bottlenecks, capturing savings that static provisioning cannot access.
Checkpoint-based resharding and restart pauses become unnecessary for parallelism changes, because the same GPU-resident state is reused through EMU primitives, making measured transitions 3.8–16.2$\times$ faster than single-model scaling baselines.
The payback admission rule imports cluster-scheduling logic into a single coupled job, giving a principled criterion for whether any reconfiguration is worth its transition cost rather than adapting on utilization alone.
The dependency-aligned boundary works across RL algorithms (ReMax, GRPO) and both synchronous and asynchronous execution, so the mechanism is not specific to PPO.
Coordinated transitions across model-stages succeed in all overlap trials, versus 34–62% for per-component migration, because the resource-dependency edges prevent the shared-GPU deadlocks that a longer timeout cannot resolve.
The EMU boundary encodes a claim that DP is the cheap, loosely coupled parallelism dimension while TP/PP are expensive and tightly coupled; the natural test is whether context parallelism and expert parallelism, which the paper lists as a future direction, are as loosely coupled as DP or behave more like TP/PP.
The measured runs make very few transitions (two per trace, six in the 1,000-step run); the regime that would really stress the claim is a workload with frequent, genuinely independent drift events where the EMA horizon and payback rule are exercised near their limits.
The paper's own confidence-margin sweep exposes the trade: at $\gamma=1$ with no timing reserve, over-admission rises to 14 transitions and latency degrades from 861.9 s to 939.5 s, suggesting the headline gain rides on the timing model being well-calibrated, not on the state abstraction alone.
If the payback logic generalizes, the same cost-aware admission could decide whether to spin up external tool services or expand generation capacity in asynchronous RL, where drift arrives less predictably than sequence-length growth.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Reinforcement learning (RL) post-training for large language models (LLMs) coordinates multiple models across generation, inference, and training on GPU clusters. Several factors may change during a run, including resource availability, sequence length, memory pressure, and stage bottlenecks. As a consequence, an execution plan that was initially suitable can then become slow or even infeasible over time. However, adapting a job whose models share GPUs entails significant challenges: deciding whether a new plan is worth the transition cost, reusing the job's distributed state, and coordinating GPU transfers across models and stages. Nereus targets these challenges as a cost-aware runtime that adapts RL post-training jobs into efficient execution plans. Its low-overhead controller selects a memory-feasible global plan and admits the transition using a cost model calibrated against the running job. To estimate and execute a transition, Nereus represents the distributed state of each replica of a model-stage (one model in one stage) as an Elastic Model Unit. It then employs a global transition graph to order the transformations and GPU transfers of these units. In a trace built from real data, online TP/PP adaptation reduces average step latency by 27.7% relative to the initial fixed TP/PP layout with DP scaling. In a 1,000-step run reaching 1,024 GPUs, six transitions consume 0.079% of total run time. Nereus improves end-to-end 8B PPO throughput by 2.14–7.27× over OpenRLHF and by 1.10–1.47× over Verl across diverse clusters.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-09-29
The authors' main claim, as the classifier picked it out 0.97
“Nereus improves end-to-end 8B PPO throughput by 2.14--7.27$\times$ over OpenRLHF and by 1.10--1.47$\times$ over Verl across diverse clusters.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.04 · states limitation 0.29 · human participants 0.02 · controlled comparison 0.93 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.98 · mentions code or data availability 0.03
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Reinforcement learning (RL) post-training for large language models (LLMs) coordinates multiple models across generation, inference, and training on GPU clusters. | background 1.00 | 0.09 | 0.03 | asserted 1.00 |
| Several factors may change during a run, including resource availability, sequence length, memory pressure, and stage bottlenecks. | background 1.00 | 0.12 | 0.02 | hedged possibility 1.00 |
| As a consequence, an execution plan that was initially suitable can then become slow or even infeasible over time. | background 0.97 | 0.10 | 0.03 | asserted 0.89 |
| However, adapting a job whose models share GPUs entails significant challenges: deciding whether a new plan is worth the transition cost, reusing the job's distributed state, and coordinating GPU transfers across models and stages. | background 0.67 | 0.08 | 0.03 | asserted 1.00 |
| Nereus targets these challenges as a cost-aware runtime that adapts RL post-training jobs into efficient execution plans. | contribution 0.99 | 0.63 | 0.18 | asserted 1.00 |
| Its low-overhead controller selects a memory-feasible global plan and admits the transition using a cost model calibrated against the running job. | method 0.94 | 0.85 | 0.12 | asserted 1.00 |
| To estimate and execute a transition, Nereus represents the distributed state of each replica of a model-stage (one model in one stage) as an Elastic Model Unit. | method 0.99 | 0.87 | 0.04 | asserted 1.00 |
| It then employs a global transition graph to order the transformations and GPU transfers of these units. | method 0.99 | 0.85 | 0.04 | asserted 1.00 |
| In a trace built from real data, online TP/PP adaptation reduces average step latency by 27.7% relative to the initial fixed TP/PP layout with DP scaling. | result 1.00 | 0.94 | 0.20 | asserted 1.00 |
| In a 1,000-step run reaching 1,024 GPUs, six transitions consume 0.079% of total run time. | result 1.00 | 0.94 | 0.06 | asserted 1.00 |
| Nereus improves end-to-end 8B PPO throughput by 2.14--7.27$\times$ over OpenRLHF and by 1.10--1.47$\times$ over Verl across diverse clusters. | result 1.00 | 0.94 | 0.51 | asserted 1.00 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.98
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-29: 1 request, 8,291 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (15)
Showing 11 of the paper's 15 figures. Figures are the authors', reproduced from the paper.
References (60)
- [1] Advanced Micro Devices, Inc. 2026. ROCm System Management Interface (ROCm SMI).https://github.com/ROCm/rocm-systems/ tree/develop/projects/rocm-smi-lib on Pith
- [2] Erfan Darzi, Aldo Pareja, Kaveh Jalilian, and Shreeanant Bharad- waj. 2025. GPU Tail Latency Diagnosis for Serverless and HPC Workloads using eBPF. InProceedings of the 11th International Work- shop on Serverless Computing(Nashville, TN, USA)(WoSC11 ’25). Association for Compu... on PitharXiv
- [3] DeepSeek-AI. 2025. DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning.Nature645, 8081 (2025), 633–638. doi:10.1038/s41586-025-09422-z on Pithdoi:10.1038/s41586-025-09422-z
- [4] Jiangfei Duan, Ziang Song, Xupeng Miao, Xiaoli Xi, Dahua Lin, Harry Xu, Minjia Zhang, and Zhihao Jia. 2024. Parcae: Proactive, Liveput- Optimized DNN Training on Preemptible Instances. In21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24). USENIX Ass... on Pith
- [5] Wei Fu, Jiaxuan Gao, Xujie Shen, Chen Zhu, Zhiyu Mei, Chuyi He, Shusheng Xu, Guo Wei, Jun Mei, Jiashu Wang, Tongkai Yang, Binhang Yuan, and Yi Wu. 2025. AReaL: A Large-Scale Asynchronous Rein- forcement Learning System for Language Reasoning. InAdvances in Neural Information P... on Pith
- [6] Wei Gao, Yuheng Zhao, Tianyuan Wu, Shaopan Xiong, Weixun Wang, Dakai An, Lunxi Cao, Dilxat Muhtar, Zichen Liu, Haizhou Zhao, Ju Huang, Siran Yang, Yongbin Li, Wenbo Su, Jiamang Wang, Lin Qu, Bo Zheng, and Wei Wang. 2026. RollArt: Disaggregated Multi-Task Agentic RL Training at... on Pith
- [7] Diandian Gu, Yihao Zhao, Yinmin Zhong, Yifan Xiong, Zhenhua Han, Peng Cheng, Fan Yang, Gang Huang, Xin Jin, and Xuanzhe Liu. 2023. ElasticFlow: An Elastic Serverless Training Platform for Distributed Deep Learning. InProceedings of the 28th ACM International Confer- ence on Ar... on PitharXiv
- [8] Zhenyu Han, Ansheng You, Haibo Wang, Kui Luo, Guang Yang, Wenqi Shi, Menglong Chen, Sicheng Zhang, Zeshun Lan, Chunshi Deng, Huazhong Ji, Wenjie Liu, Yu Huang, Yixiang Zhang, Chenyi Pan, Jing Wang, Xin Huang, Chunsheng Li, and Jianping Wu. 2025. AsyncFlow: An Asynchronous Stre... on PithPith's reviewarXiv
- [9] Willy Herroelen, Bert De Reyck, and Erik Demeulemeester. 1998. Resource-constrained project scheduling: A survey of recent devel- opments.Computers & Operations Research25, 4 (1998), 279–302. doi:10.1016/S0305-0548(97)00055-5 on Pithdoi:10.1016/s0305-0548(97)00055-5
- [10] Christopher Hojny, Mathieu Besançon, Ksenia Bestuzheva, Sander Borst, João Dionísio, Johannes Ehls, Leon Eifler, Mohammed Ghannam, Ambros Gleixner, Adrian Göß, Alexander Hoen, Jacob von Holly- Ponientzietz, Rolf van der Hulst, Dominik Kamp, Thorsten Koch, Kevin Kofler, Jurgen ... on Pithdoi:10.48550/arxiv.2511.18580
- [11] Jian Hu, Xibin Wu, Wei Shen, Jason Klein Liu, Weixun Wang, Songlin Jiang, Haoran Wang, Hao Chen, Bin Chen, Wenkai Fang, Xianyu, Yu Cao, Haotian Xu, and Yiming Liu. 2025. OpenRLHF: A Ray-based Easy-to-use, Scalable and High-performance RLHF Framework. In Proceedings of the 2025... on Pith
- [14] Saurabh Jha, Archit Patke, Jim Brandt, Ann Gentile, Benjamin Lim, Mike Showerman, Greg Bauer, Larry Kaplan, Zbigniew Kalbarczyk, William Kramer, and Ravi Iyer. 2020. Measuring Congestion in High- Performance Datacenter Interconnects. In17th USENIX Symposium on Networked System... on Pith
- [15] Devvrit Khatri, Lovish Madaan, Rishabh Tiwari, Rachit Bansal, Sai Surya Duvvuri, Manzil Zaheer, Inderjit S. Dhillon, David Brandfon- brener, and Rishabh Agarwal. 2026. The Art of Scaling Reinforcement Learning Compute for LLMs. InThe Fourteenth International Confer- ence on Le... on Pith
- [17] Kinman Lei, Yuyang Jin, Mingshu Zhai, Kezhao Huang, Haoxing Ye, and Jidong Zhai. 2024. PUZZLE: Efficiently Aligning Large Language Models through Light-Weight Context Switch. In2024 USENIX Annual Technical Conference (USENIX ATC 24). USENIX Association, Santa Clara, CA, 127–140 on Pith
- [18] Cheng Li. 2023. LLM-Analysis: Latency and Memory Analysis of Transformer Models for Training and Inference.https://github.com/ cli99/llm-analysis on Pith
- [19] Mingzhen Li, Wencong Xiao, Hailong Yang, Biao Sun, Hanyu Zhao, Shiru Ren, Zhongzhi Luan, Xianyan Jia, Yi Liu, Yong Li, Wei Lin, and Depei Qian. 2023. EasyScale: Elastic Training with Consistent Accuracy and Improved Utilization on GPUs. InProceedings of the International Confe... on PitharXiv
- [20] Ziniu Li, Tian Xu, Yushun Zhang, Zhihang Lin, Yang Yu, Ruoyu Sun, and Zhi-Quan Luo. 2024. ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Models. InProceedings of the 41st International Conference on Machine Learning (ICML’24... on Pith
- [21] Xinyu Lian, Sam Ade Jacobs, Lev Kurilenko, Masahiro Tanaka, Stas Bekman, Olatunji Ruwase, and Minjia Zhang. 2025. Universal Check- pointing: A Flexible and Efficient Distributed Checkpointing System for Large-Scale DNN Training with Reconfigurable Parallelism. In Proceedings o... on Pith
- [22] Nandor Licker, Kevin Hu, Vladimir Zaytsev, and Lequn Chen. 2026. fabric-lib: RDMA Point-to-Point Communication for LLM Systems. In Proceedings of Machine Learning and Systems, Vol. 8. MLSys, Bellevue, WA, USA, 169–185 on Pith
- [23] Llama Team, AI @ Meta. 2024. The Llama 3 Herd of Models. arXiv:2407.21783 [cs.AI] doi:10.48550/arXiv.2407.21783 on PithPith's reviewarXivdoi:10.48550/arxiv.2407.21783
show 40 more
- [24] Luo Mai, Guo Li, Marcel Wagenländer, Konstantinos Fertakis, Andrei- Octavian Brabete, and Peter Pietzuch. 2020. KungFu: Making Training in Distributed Machine Learning Adaptive. In14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20). USENIX Associatio... on Pith
- [25] Zhiyu Mei, Wei Fu, Kaiwei Li, Guangju Wang, Huanchen Zhang, and Yi Wu. 2025. ReaL: Efficient RLHF Training of Large Language Models with Parameter Reallocation. InProceedings of Machine Learning and Systems, Vol. 7. MLSys, Santa Clara, CA, USA, 20 pages on Pith
- [26] Zizhao Mo, Huanle Xu, and Chengzhong Xu. 2024. Heet: Accelerating Elastic Training in Heterogeneous Deep Learning Clusters. InProceed- ings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2(La Jolla, CA,... on PitharXiv
- [27] Philipp Moritz, Robert Nishihara, Stephanie Wang, Alexey Tumanov, Richard Liaw, Eric Liang, Melih Elibol, Zongheng Yang, William Paul, Michael I. Jordan, and Ion Stoica. 2018. Ray: A Distributed Framework for Emerging AI Applications. In13th USENIX Symposium on Operating Syste... on Pith
- [28] NVIDIA. 2025. NeMo RL: A Scalable and Efficient Post-Training Library.https://github.com/NVIDIA-NeMo/RL. GitHub repository on Pith
- [29] NVIDIA. 2026. NVIDIA Management Library (NVML).https:// developer.nvidia.com/management-library-nvml on Pith
- [30] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wain- wright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Chris- tiano, Jan ... on Pith
- [31] Aurick Qiao, Sang Keun Choe, Suhas Jayaram Subramanya, Willie Neiswanger, Qirong Ho, Hao Zhang, Gregory R. Ganger, and Eric P. Xing. 2021. Pollux: Co-adaptive Cluster Scheduling for Goodput- Optimized Deep Learning. In15th USENIX Symposium on Operating Systems Design and Imple... on Pith
- [32] Qwen Team. 2025. Qwen3 Technical Report. arXiv:2505.09388 [cs.CL] doi:10.48550/arXiv.2505.09388 on PithPith's reviewarXivdoi:10.48550/arxiv.2505.09388
- [33] Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 2020. ZeRO: memory optimizations toward training trillion parameter models. InProceedings of the International Conference for High Performance Computing, Networking, Storage and Analy- sis (SC ’20). IEEE Press, ... on PithPith's reviewarXiv
- [34] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal Policy Optimization Algorithms. arXiv:1707.06347 [cs.LG] on PithPith's reviewarXiv
- [35] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo on Pith
- [36] Guangming Sheng, Yuxuan Tong, Borui Wan, Wang Zhang, Chaobo Jia, Xibin Wu, Yuqi Wu, Xiang Li, Chi Zhang, Yanghua Peng, Haibin Lin, Xin Liu, and Chuan Wu. 2026. Laminar: A Scalable Asynchro- nous RL Post-Training Framework. InProceedings of the 21st European Conference on Compu... on PitharXiv
- [37] Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2025. Hybrid- Flow: A Flexible and Efficient RLHF Framework. InProceedings of the Twentieth European Conference on Computer Systems(Rotterdam, Netherlands)(EuroSy... on PitharXiv
- [38] Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. 2020. Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism. arXiv:1909.08053 [cs.CL] 14 Nereus: Adaptive Parallelism for LLM Post-Training Songl... on PithPith's reviewarXiv
- [39] Megatron Team. 2024. Dist Checkpointing Package.https://docs. nvidia.com/megatron-core/developer-guide/nightly/api-guide/core/ dist_checkpointing.html on Pith
- [40] PyTorch Team. 2024. Distributed Checkpointing Recipe.https://docs. pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html on Pith
- [41] John Thorpe, Pengzhan Zhao, Jonathan Eyolfson, Yifan Qiao, Zhihao Jia, Minjia Zhang, Ravi Netravali, and Guoqing Harry Xu. 2023. Bam- boo: Making Preemptible Instances Resilient for Affordable Training of Large DNNs. In20th USENIX Symposium on Networked Systems Design and Impl... on Pith
- [42] Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beech- ing, Tristan Thrush, Nathan Lambert, Shengyi Huang, Kashif Rasul, and Quentin Gallouédec. 2020. TRL: Transformer Reinforcement Learning on Pith
- [43] Marcel Wagenländer, Guo Li, Bo Zhao, Luo Mai, and Peter Pietzuch on Pith
- [44] Borui Wan, Mingji Han, Yiyao Sheng, Yanghua Peng, Haibin Lin, Mo- fan Zhang, Zhichao Lai, Menghan Yu, Junda Zhang, Zuquan Song, Xin Liu, and Chuan Wu. 2025. ByteCheckpoint: a unified checkpointing system for large foundation model development. InProceedings of the 22nd USENIX ... on Pith
- [45] Tenplex: Dynamic Parallelism for Deep Learning using Paralleliz- able Tensor Collections. InProceedings of the ACM SIGOPS 30th Sym- posium on Operating Systems Principles(Austin, TX, USA)(SOSP ’24). Association for Computing Machinery, New York, NY, USA, 195–210. doi:10.1145/3... on PitharXiv
- [46] Yuanqing Wang, Hao Lin, Junhao Hu, Chunyang Zhu, Quanlu Zhang, Zhen Guo, Yuchen Zhang, Xu Fu, Si Xu, Bo Dai, Zixiao Huang, Chao Yu, Boxun Li, Guohao Dai, Zhi Yang, and Yu Wang. 2026. DynaRL: Flex- ible and Dynamic Scheduling of Large-Scale Reinforcement Learning Training. In20... on Pith
- [47] Weixun Wang, Shaopan Xiong, Gengru Chen, Wei Gao, Sheng Guo, Yancheng He, Ju Huang, Jiaheng Liu, Zhendong Li, Xiaoyang Li, Zichen Liu, Haizhou Zhao, Dakai An, Lunxi Cao, Qiyang Cao, Wanxi Deng, Feilei Du, Yiliang Gu, Jiahe Li, Xiang Li, Mingjie Liu, Yijia Luo, Zihe Liu, Yadao ... on PithPith's reviewarXivdoi:10.48550/arxiv.2506.06122
- [48] Christin Whitton, William Jones, Craig Walker, Vanessa Job, Steven Senator, and Nathan DeBardeleben. 2025. Are We There Yet? Predict- ing the Queue Wait Times for HPC Jobs . In2025 IEEE International Con- ference on Cluster Computing (CLUSTER). IEEE Computer Society, Los Alami... on PitharXiv
- [49] Zhuang Wang, Zhen Jia, Shuai Zheng, Zhen Zhang, Xinwei Fu, T. S. Eu- gene Ng, and Yida Wang. 2023. GEMINI: Fast Failure Recovery in Distributed Training with In-Memory Checkpoints. InProceedings of the 29th Symposium on Operating Systems Principles(Koblenz, Ger- many)(SOSP ’23... on PitharXiv
- [50] Zhanghao Wu, Wei-Lin Chiang, Ziming Mao, Zongheng Yang, Eric J. Friedman, Scott Shenker, and Ion Stoica. 2024. Can’t Be Late: Optimiz- ing Spot Instance Savings under Deadlines. In21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24). USENIX Associatio... on Pith
- [51] Bo Wu, Sid Wang, Yunhao Tang, Jia Ding, Eryk Helenowski, Liang Tan, Tengyu Xu, Tushar Gowda, Zhengxing Chen, Chen Zhu, Xi- aocheng Tang, Yundi Qian, Beibei Zhu, and Rui Hou. 2025. LlamaRL: A Distributed Asynchronous Reinforcement Learning Framework for Efficient Large-scale LL... on PithPith's reviewarXivdoi:10.48550/arxiv.2505.24034
- [52] Youshao Xiao, Zhenglei Zhou, Fagui Mao, Weichang Wu, Shangchun Zhao, Lin Ju, Lei Liang, Xiaolu Zhang, and Jun Zhou. 2025. FlexRLHF: A Flexible Placement and Parallelism Framework for Efficient RLHF Training. In2025 IEEE International Parallel and Distributed Process- ing Sympo... on PitharXiv
- [53] Miguel G. Xavier, Kassiano J. Matteussi, Fabian Lorenzo, and Cesar A. F. De Rose. 2016. Understanding performance interference in multi- tenant cloud databases and web applications. In2016 IEEE International Conference on Big Data (Big Data). IEEE, Washington, DC, USA, 2847– on Pith
- [54] Shaoxun Zeng, Tingxu Ren, Jiwu Shu, and Youyou Lu. 2026. GPU Checkpoint/Restore Made Fast and Lightweight. In24th USENIX Con- ference on File and Storage Technologies (FAST’26). USENIX Association, Santa Clara, CA, 239–254 on Pith
- [55] Lianmin Zheng, Zhuohan Li, Hao Zhang, Yonghao Zhuang, Zhifeng Chen, Yanping Huang, Yida Wang, Yuanzhong Xu, Danyang Zhuo, Eric P. Xing, Joseph E. Gonzalez, and Ion Stoica. 2022. Alpa: Au- tomating Inter- and Intra-Operator Parallelism for Distributed Deep Learning. In16th USEN... on Pith
- [56] Zhewei Yao, Reza Yazdani Aminabadi, Olatunji Ruwase, Samyam Rajbhandari, Xiaoxia Wu, Ammar Ahmad Awan, Jeff Rasley, Minjia Zhang, Conglong Li, Connor Holmes, Zhongzhu Zhou, Michael Wyatt, Molly Smith, Lev Kurilenko, Heyang Qin, Masahiro Tanaka, Shuai Che, Shuaiwen Leon Song, a... on PithPith's reviewarXivdoi:10.48550/arxiv.2308.01320
- [57] Yinmin Zhong, Zili Zhang, Bingyang Wu, Shengyu Liu, Yukun Chen, Changyi Wan, Hanpeng Hu, Lei Xia, Ranchen Ming, Yibo Zhu, and Xin Jin. 2025. Optimizing RLHF Training for Large Language Models with Stage Fusion. In22nd USENIX Symposium on Networked Systems Design and Implementa... on Pith
- [58] Yuzhen Zhou, Jiajun Li, Yusheng Su, Gowtham Ramesh, Zilin Zhu, Xiang Long, Chenyang Zhao, Jin Pan, Xiaodong Yu, Ze Wang, Kan- grui Du, Jialian Wu, Ximeng Sun, Jiang Liu, Qiaolin Yu, Hao Chen, Zicheng Liu, and Emad Barsoum. 2025. APRIL: Active Partial Rollouts in Reinforcement ... on Pithdoi:10.48550/arxiv.2509.18521
- [59] Yinmin Zhong, Zili Zhang, Xiaoniu Song, Hanpeng Hu, Chao Jin, Bingyang Wu, Nuo Chen, Yukun Chen, Yu Zhou, Changyi Wan, Hongyu Zhou, Yimin Jiang, Yibo Zhu, and Daxin Jiang. 2025. StreamRL: Scalable, Heterogeneous, and Elastic RL for LLMs with Disaggregated Stream Generation. ar... on PithPith's reviewarXivdoi:10.48550/arxiv.2504.15930
- [60] Zilin Zhu, Chengxing Xie, Xin Lv, and slime Contributors. 2025. slime: An LLM post-training framework for RL Scaling.https://github.com/ THUDM/slime. GitHub repository. Corresponding author: Xin Lv. 15 on Pith
- [62] Kan Zhu, Yufei Gao, Yilong Zhao, Liangyu Zhao, Gefei Zuo, Yile Gu, Dedong Xie, Tian Tang, Qinyu Xu, Zihao Ye, Keisuke Kamahori, Chien- Yu Lin, Ziren Wang, Stephanie Wang, Arvind Krishnamurthy, and Baris Kasikci. 2025. NanoFlow: Towards Optimal Large Language Model Serving Thro... on Pith
- [2024] DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv:2402.03300 [cs.CL] on PithPith's reviewarXiv
- [2852] doi:10.1109/BigData.2016.7840933 on PitharXiv
Cited by 2 Pith papers
Reviewed papers on Pith that cite this one. The full citation record
- NeMo-DCR: Bit-Exact Delta-Compressed Refit for Scalable Agentic RL at Trillion-Parameter Scale cs.DC · Oct 2026
- A Survey on RLOps for Large Language Models cs.SE · Nov 2026
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment