Pith.

Distributed, Parallel, and Cluster Computing · arXiv:2609.34645 v1 · submitted 28 Sep 2026 · 60 references

Nereus: Adaptive Parallelism for LLM Post-Training

Songlin Jiang, Tuo Shi, Sitong Zhang, Zeke Wang, Mario Di Francesco, Bo Zhao

The paper says

“Reinforcement learning (RL) post-training for large language models (LLMs) coordinates multiple models across generation, inference, and training on GPU clusters. Several factors may change during a run, including resource availability, sequence length, memory pressure, and stage bottlenecks.”

more of the abstract

As a consequence, an execution plan that was initially suitable can then become slow or even infeasible over time. However, adapting a job whose models share GPUs entails significant challenges: deciding whether a new plan is worth the transition cost, reusing the job's distributed state, and coordinating GPU transfers across models and stages. Nereus targets these challenges as a cost-aware runtime that adapts RL post-training jobs into efficient execution plans. Its low-overhead controller selects a memory-feasible global plan and admits the transition using a cost model calibrated against the running job. To estimate and execute a transition, Nereus represents the distributed state of each replica of a model-stage (one model in one stage) as an Elastic Model Unit. It then employs a global transition graph to order the transformations and GPU transfers of these units. In a trace built from real data, online TP/PP adaptation reduces average step latency by 27.7% relative to the initial fixed TP/PP layout with DP scaling. In a 1,000-step run reaching 1,024 GPUs, six transitions consume 0.079% of total run time. Nereus improves end-to-end 8B PPO throughput by 2.14–7.27× over OpenRLHF and by 1.10–1.47× over Verl across diverse clusters.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 29 Sep 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. A genuine systems contribution with thorough hardware evaluation; the weak link is the admission-timing premise, which the authors partially acknowledge but never stress-test under volatile drift. This line is wrong

2 major objections · 3 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 60 references say.

The paper says, at 4.3, Eq. (2); 7.3.3

The review did not quote this passage.

Pith says: Objection 1 of 2 · major

The 'when to adapt' decision rests entirely on the predicted payback ratio L_tran/Delta-L and on the EMA horizon H_x, but the evaluation validates this link only under benign, monotone drift: all three held-out traces combine gradual sequence-length growth with moderate GPU-count variation, and the only probe of admission misbehavior is the gamma=1 sweep, where over-admission to 14 transitions raises latency to 939.5 s (worse than the 861.9 s default).

more of objection 1

Because the cost model's 4.96% MAPE (Section 7.3.1) can be as large as the true latency gap between near-optimal plans (the paper's own p95 selection gap is 4.1%, Section 7.3.2), the sign of predicted Delta-L can be wrong near the optimum, and the paper reports no per-transition realized-payback audit that would establish whether each admitted transition actually repaid L_tran within H_x steps. Since contribution (1) is specifically the payback-based admission decision, I recommend adding (i) a per-decision audit recording predicted and realized savings and transition cost for every admission in the traces, and (ii) stress runs under synthetic drift with abrupt sequence-length spikes or rapid GPU revocation, where an EMA horizon is known to lag. These experiments, not further calibration fit, would settle whether the timing link is well calibrated.

The paper says, at 7.3.3

The review did not quote this passage.

Pith says: Objection 2 of 2 · major

The marginal value of the payback admission rule specifically, as opposed to replanning at all, is thin in the current evidence. Across the three held-out traces Nereus averages 858.7 s/step versus 873.8 s for 3-step adapt, a 1.7% margin that is comparable to the reported cross-trace standard deviations (5.2 s for Nereus, 12.6 s for 3-step adapt), and the larger …

more of objection 2

… headline numbers (27.7% over static, 8.3% over DynaRL-style) do not isolate the timing decision from the benefit of adapting at all. To support the claim that payback-based admission matters, the paper should report the realized-payback outcome distribution and compare against a small set of replan periods rather than a single fixed 3-step period; if the margin over periodic replanning is genuinely small in these traces, that is an important scoping statement for contribution (1).

3 minor objections

The paper says, at 7 (general)

Pith says: minor objection 1 of 3

The paper describes a 43k-line implementation but gives no artifact or code-availability statement; providing the code, or a clear statement of availability, would materially aid reproducibility of the 480-plan and three-trace experiments.

The paper says, at 7.3.3, Fig. 11

Pith says: minor objection 2 of 3

The three subplots mix the latency curve, the gap curve, the confidence-margin sweep, and a secondary GPU-count axis; a small table listing each admitted transition with its predicted Delta-L, predicted L_tran, H_x, and realized payback would make the admission behavior directly checkable and would directly address the timing concern above.

show 1 more

The paper says, at 4.3

Pith says: minor objection 3 of 3

The H_x initial value is one step until the first crossing interval is observed, and reference values reset after every replan even on rejection; a sentence on how quickly H_x stabilizes (e.g., first-interval length in the traces) would clarify early-run admission behavior.

What would settle it

Instrument the three held-out traces (Section 7.3.3) to record, for every admitted transition: the predicted Delta-L and L_tran from the cost model, the realized Delta-L from measured step latencies before and after the transition, and the realized window until the next drift crossing. Then (1) count the fraction of admitted transitions whose realized payback exceeds the realized window, i.e., transitions that did not repay before drift moved again; (2) recompute average s/step with an oracle admission rule that uses realized instead of predicted Delta-L. If a substantial fraction (above roughly 20%) of admitted transitions fail to repay in the realized window, or if oracle-based admission changes the 858.7 s/step result by more than about 5%, the payback-timing premise is confirmed as load-bearing.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 29 Sep 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic partly agrees 39 s · 28,345 tokens read, 5,080 written
  3. Editor wrote the verdict above 15 s · 29,445 tokens read, 2,369 written
  4. Referee major revision: 2 major objections, 3 minor 42 s · 28,701 tokens read, 6,833 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Bottom line: this is a real contribution, not a repackaging. The new content is the integration for the coupled multi-model, multi-stage RL post-training case — the EMU dependency-aligned state boundary and the global transition DAG over four primitives. The pieces are individually published (Alpa/NanoFlow cost models, Pollux/Sia payback, Tenplex/Oobleck elastic state), but nobody has assembled them for RL post-training, and the model-stage replica as the state unit is the right granularity for that workload.

What earns credit: the evaluation is thorough by field standards. 480 plans executed on hardware against the empirical optimum, within 5% in all 18 settings; 72 out-of-sample calibration plans at 4.96% MAPE; held-out real-data traces; three clusters, four model sizes, three RL algorithms, plus async execution. They also verify adaptation preserves training behavior — reward and KL curves track Verl over 50 steps. That evidence is the kind a systems paper should have. Planning is cheap: 338 ms at 1,024 GPUs versus 511 s for SCIP, within 5–7% of SCIP's schedule quality. Transition overhead at scale is small: 0.079% of run time for six transitions.

The honest soft spot is the "when to adapt" decision, and the authors do not hide it. Admission rests on Eq. (2): predicted savings must repay predicted transition cost within a fraction of Hx, an EMA of past drift-crossing intervals. Near the optimum, the cost model's 4.96% MAPE can sign Delta-L wrong. Their own gamma sweep shows the risk — at gamma=1, over-admission yields 14 transitions and 939.5 s latency, worse than the 861.9 s default. That is fine for the default margin, but it reveals the timing premise is only as good as the horizon estimate, and the evaluation never stresses it: all three held-out traces have benign, monotone drift. No accelerating sequence-length growth, no abrupt GPU revocation spikes. That gap keeps the verdict conditional.

Two smaller points. No code or data shipped, so the hardware numbers can't be independently checked — relevant for a 43k-line system. And the OpenRLHF speedup (up to 7.27x) partly reflects baseline weakness; the Verl comparison (1.10–1.47x) is the cleaner evidence.

Who it's for: anyone building RL post-training infrastructure or elastic training runtimes. It deserves a serious referee. My asks on revision: a per-transition realized-payback audit, and an evaluation that varies drift dynamics — accelerating sequence growth and abrupt GPU revocation. Send it out.

The strongest objection

The central claim decomposes into two links: (a) plan selection quality and (b) transition timing under drift. Link (a) is well supported by direct hardware evidence: 480 held-out plans executed with the selected plan within 5% of the empirical optimum in all 18 settings, and 72 out-of-sample calibration plans at R^2=0.9993 / 4.96% MAPE. Link (b) rests entirely on Eq. (2)'s admission rule, which compares the predicted payback L_tran/Delta-L against gamma*H_x, where Delta-L and L_tran come from the same online-calibrated cost model and H_x is an EMA of past drift-crossing intervals. The load-bearing weakness is that in the flat region near the optimum, where many candidate plans lie within the paper's own 5% closeness bound, the cost model's 4.96% MAPE on latency can be as large as the true difference between the current and target plan: a transition admitted because predicted Delta-L>0 may have a realized negative benefit. The mechanism is visible in the paper's own gamma sweep (Section 7.3.3): at gamma=1, over-admission rises to 14 transitions and latency climbs to 939.5 s, worse than the gamma=0.5 default (861.9 s), demonstrating that admission outcomes are sensitive to the payback estimate and that wasted transitions cost more than they save. The paper's headline numbers (27.7% reduction; 858.7 vs 928.3 s/step across three traces) are averaged over traces with benign, monotone drift; the evaluation never varies the drift dynamics (accelerating sequence-length growth, abrupt GPU revocation spikes) that would stress the EMA horizon H_x or the Delta-L sign. So the timing premise — not plan quality — is where the conditional verdict hinges.

The reader's weakest_assumption identifies the payback-horizon estimate H_x in Eq. (2) as the load-bearing premise, and I agree that the 'when to adapt' decision is the weakly supported link rather than plan selection, which is strongly evidenced by the executed 480-plan and 72-plan out-of-sample evaluations. I part ways slightly in emphasis: the sharper failure mode is not only a stale EMA horizon but the sign error of predicted Delta-L in the flat region near the optimum, where a 4.96% MAPE cost model cannot reliably distinguish between plans that are, by the paper's own measurement, within 5% of each other. In that regime the admission rule can systematically admit transitions with negative realized benefit, and the paper's gamma=1 sweep (14 transitions, 939.5 s versus 861.9 s at gamma=0.5) is direct evidence that the mechanism is real and that over-admission costs more than it saves. The paper's headline numbers are averaged over three traces whose drift is benign and monotone, so robustness to volatile or accelerating drift is simply unevaluated; this is a gap, not a demonstration of failure. Because the verdict is already CONDITIONAL and my concern routes through the same admission rule the reader flagged, I recommend no change. I set agreement to partial rather than agree because I foreground the Delta-L sign-error mechanism (which the reader mentions but subordinates to H_x) and because the decisive test I propose is a per-decision realized-payback audit across admitted transitions, which the paper does not report. No ad hominem, no theatrical framing: this is a concrete, testable premise about prediction- and horizon-calibration under drift, with the paper's own sensitivity data indicating the mechanism matters.

The referee's report

The problem is real and growing: RL post-training runs are long and expensive, and drift (sequence-length growth, GPU revocation, memory pressure) can invalidate a startup plan. The system-level evidence is unusually strong on the plan-selection link: 480 plans executed directly on hardware as an empirical-optimum baseline, held-out calibration plans, cross-algorithm generality (PPO, ReMax, GRPO), and success in all 100-trial coordination runs. The state-abstraction and orchestration contributions are well engineered and carefully evaluated. The weakest link is the admission-timing decision, which is validated only under benign, monotone drift; the paper's own gamma sweep exposes the mechanism's fragility when the payback estimate is off. Strengthening that link with a per-transition realized-payback audit and harder drift dynamics is, in my view, the one load-bearing revision needed.

Pith's reading of the paper (a machine's summary, not the authors')

RL post-training coordinates several models across generation, inference, and training stages on shared GPUs, and everything about the run drifts: available GPUs come and go, the actor learns to emit longer sequences, and communication slows under contention. A parallelism plan that was right at startup can become slow or memory-infeasible within a few hundred steps, yet existing systems either fix the plan or restart through coarse checkpoint-based resharding. Nereus claims that online adaptation becomes cheap and correct when the state boundary matches the job's dependencies, with one Elastic Model Unit per model-stage replica and admission gated by a payback rule that executes a transition only when predicted savings repay its cost within a predicted drift window. On a real-data trace, online TP/PP adaptation cuts average step latency by 27.7% versus a fixed layout, six transitions in a 1,000-step run cost 0.079% of run time, and end-to-end 8B PPO throughput improves 2.14–7.27$\times$ over OpenRLHF and 1.10–1.47$\times$ over Verl. If right, a post-training job can ride drift continuously instead of gambling on a startup plan, saving a meaningful fraction of very expensive cluster time.

What carries the argument. The Elastic Model Unit (EMU) is one model-stage replica, defined as $\langle M, T_{tp\times pp}, R, \boldsymbol{\theta}, \boldsymbol{\omega}\rangle$: the model-stage identity, its tensor/pipeline layout, its GPU set, and the parameter and optimizer shards. TP and PP, which require tightly synchronized collectives, are encapsulated inside the unit; DP is left outside, so scaling is only adding or removing whole units. The plan space is covered by four primitives, Split and Merge converting between tightly coupled TP/PP layout and loosely coupled replicas, Extend broadcasting a replica onto new GPUs, and Destroy releasing a redundant copy, each hiding the underlying shard movement. Around the units sits the controller: an online-calibrated cost model predicts per-step latency and memory for candidate plans, and the payback admission rule $\Delta L > 0 \wedge \hat{L}_{\mathrm{tran}}/\Delta L \leq \gamma H_x$ admits a transition only when predicted savings repay predicted cost within a fraction $\gamma$ of the expected steps until the next drift crossing, where $H_x$ is an exponential moving average of past crossing intervals. The transition engine compiles the plan difference into a global DAG ordered Split$\to$Destroy$\to$Extend$\to$Merge, inserting resource-dependency edges so release-before-acquire avoids deadlock when model-stages share a near-capacity GPU pool. This machinery turns three coupled decisions, when to adapt, what state to reuse, and how to transition, into an online loop with measured decision overhead below 338 ms even at 1,024 GPUs.

Core claim. The paper establishes that plan adaptation becomes tractable when the state boundary matches the job's dependencies. It proposes a cost-aware controller that replans on drift, meaning resource changes, memory pressure, or sequence-length growth past a threshold, and admits a transition only when the current plan is infeasible (the urgent path) or when the predicted per-step latency savings repay the predicted transition cost within a fraction of the estimated steps until the next drift crossing (Eq. 2). State is reused at the granularity of the Elastic Model Unit: one model-stage replica holding the tightly coupled TP/PP shards, with DP exposed as the replica count, so resharding and scaling reduce to four primitives (Split, Merge, Extend, Destroy). A global transition DAG orders these primitives across coupled model-stages, adding resource-dependency edges when a transient GPU overlap blocks an acquisition, and executes the transition with GPU-direct collectives at a safe RL-step boundary. The reported results are that selected plans stay within 5% of the empirical optimum in all 18 measured settings, that the real-data trace shows a 27.7% average step-latency reduction versus a fixed TP/PP layout with DP scaling, that the adaptive policy averages 858.7 s/step versus 928.3 s for DynaRL-style admission across three held-out traces, and that six transitions in a 1,000-step run to 1,024 GPUs consume 0.079% of total run time.

The premise it rests on. The entire decision of when to transition rests on the online-calibrated estimates of latency savings and transition cost together with the EMA-based drift-window estimate $H_x$ being accurate enough that the payback rule admits only transitions whose savings are actually repaid, and if drift arrives sooner than expected under abrupt GPU revocation or a sudden sequence-length spike, the system over-admits and wastes transition time.

If the paper is right (Pith's inferences, not the authors')

A post-training job no longer needs to fix its plan at startup: it can follow drift in sequence length, GPU supply, and stage bottlenecks, capturing savings that static provisioning cannot access.

Checkpoint-based resharding and restart pauses become unnecessary for parallelism changes, because the same GPU-resident state is reused through EMU primitives, making measured transitions 3.8–16.2$\times$ faster than single-model scaling baselines.

The payback admission rule imports cluster-scheduling logic into a single coupled job, giving a principled criterion for whether any reconfiguration is worth its transition cost rather than adapting on utilization alone.

The dependency-aligned boundary works across RL algorithms (ReMax, GRPO) and both synchronous and asynchronous execution, so the mechanism is not specific to PPO.

Coordinated transitions across model-stages succeed in all overlap trials, versus 34–62% for per-component migration, because the resource-dependency edges prevent the shared-GPU deadlocks that a longer timeout cannot resolve.

The EMU boundary encodes a claim that DP is the cheap, loosely coupled parallelism dimension while TP/PP are expensive and tightly coupled; the natural test is whether context parallelism and expert parallelism, which the paper lists as a future direction, are as loosely coupled as DP or behave more like TP/PP.

The measured runs make very few transitions (two per trace, six in the 1,000-step run); the regime that would really stress the claim is a workload with frequent, genuinely independent drift events where the EMA horizon and payback rule are exercised near their limits.

The paper's own confidence-margin sweep exposes the trade: at $\gamma=1$ with no timing reserve, over-admission rises to 14 transitions and latency degrades from 861.9 s to 939.5 s, suggesting the headline gain rides on the timing model being well-calibrated, not on the state abstraction alone.

If the payback logic generalizes, the same cost-aware admission could decide whether to spin up external tool services or expand generation capacity in asynchronous RL, where drift arrives less predictably than sequence-length growth.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Reinforcement learning (RL) post-training for large language models (LLMs) coordinates multiple models across generation, inference, and training on GPU clusters. Several factors may change during a run, including resource availability, sequence length, memory pressure, and stage bottlenecks. As a consequence, an execution plan that was initially suitable can then become slow or even infeasible over time. However, adapting a job whose models share GPUs entails significant challenges: deciding whether a new plan is worth the transition cost, reusing the job's distributed state, and coordinating GPU transfers across models and stages. Nereus targets these challenges as a cost-aware runtime that adapts RL post-training jobs into efficient execution plans. Its low-overhead controller selects a memory-feasible global plan and admits the transition using a cost model calibrated against the running job. To estimate and execute a transition, Nereus represents the distributed state of each replica of a model-stage (one model in one stage) as an Elastic Model Unit. It then employs a global transition graph to order the transformations and GPU transfers of these units. In a trace built from real data, online TP/PP adaptation reduces average step latency by 27.7% relative to the initial fixed TP/PP layout with DP scaling. In a 1,000-step run reaching 1,024 GPUs, six transitions consume 0.079% of total run time. Nereus improves end-to-end 8B PPO throughput by 2.14–7.27× over OpenRLHF and by 1.10–1.47× over Verl across diverse clusters.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-09-29

The authors' main claim, as the classifier picked it out 0.97

“Nereus improves end-to-end 8B PPO throughput by 2.14--7.27$\times$ over OpenRLHF and by 1.10--1.47$\times$ over Verl across diverse clusters.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.04 · states limitation 0.29 · human participants 0.02 · controlled comparison 0.93 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.98 · mentions code or data availability 0.03

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Reinforcement learning (RL) post-training for large language models (LLMs) coordinates multiple models across generation, inference, and training on GPU clusters. background 1.00 0.09 0.03 asserted 1.00
Several factors may change during a run, including resource availability, sequence length, memory pressure, and stage bottlenecks. background 1.00 0.12 0.02 hedged possibility 1.00
As a consequence, an execution plan that was initially suitable can then become slow or even infeasible over time. background 0.97 0.10 0.03 asserted 0.89
However, adapting a job whose models share GPUs entails significant challenges: deciding whether a new plan is worth the transition cost, reusing the job's distributed state, and coordinating GPU transfers across models and stages. background 0.67 0.08 0.03 asserted 1.00
Nereus targets these challenges as a cost-aware runtime that adapts RL post-training jobs into efficient execution plans. contribution 0.99 0.63 0.18 asserted 1.00
Its low-overhead controller selects a memory-feasible global plan and admits the transition using a cost model calibrated against the running job. method 0.94 0.85 0.12 asserted 1.00
To estimate and execute a transition, Nereus represents the distributed state of each replica of a model-stage (one model in one stage) as an Elastic Model Unit. method 0.99 0.87 0.04 asserted 1.00
It then employs a global transition graph to order the transformations and GPU transfers of these units. method 0.99 0.85 0.04 asserted 1.00
In a trace built from real data, online TP/PP adaptation reduces average step latency by 27.7% relative to the initial fixed TP/PP layout with DP scaling. result 1.00 0.94 0.20 asserted 1.00
In a 1,000-step run reaching 1,024 GPUs, six transitions consume 0.079% of total run time. result 1.00 0.94 0.06 asserted 1.00
Nereus improves end-to-end 8B PPO throughput by 2.14--7.27$\times$ over OpenRLHF and by 1.10--1.47$\times$ over Verl across diverse clusters. result 1.00 0.94 0.51 asserted 1.00

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.98

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-29: 1 request, 8,291 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (15)
Figure 1 of the paper
Figure 1. Nereus overview. (a) 1 The controller replans on drift and admits the transition to S ∗ only if the current plan is infeasible or the savings repay the transition cost (§4). 2 Both plans are represented as EMUs, one per model-stage replica, with TP/PP inside the unit and DP as the replica count (§5). 3 The transition engine compiles the plan difference into a global transition DAG, adding resource-dependency edges when transient GPU overlap blocks an acquisition (§6). (b) Sequence-length growth reshards the five model-stages, with Merge in generation and inference, and Split in…
Figure 3 of the paper
Figure 3. Frequent GPU availability fluctuations in shared cloud environments (p3.2xlarge nodes) over ten hours. 0 200 400 600 800 1000 Step 0 2000 4000 6000 8000 Length (tokens) (a) Sequence-length growth 500 1000 1500 Training time (s) 1 2 4 2 4 1 2 1 8 4 2 2 2 2 4 4 4 1 2 4 2 1 4 4 4 1 8 8 2 2 4 2 4 8 4 1 4 4 2 2 4 4 DP TP PP Seq. 1K 2K 4K 8K 8 GPUs 16 GPUs 32 GPUs (b) Best plan shifts with length
Figure 4 of the paper
Figure 4. Generated sequence length grows during training (a), changing which measured training plan is fastest (b). Missing data points indicate out-of-memory (OOM) errors. data, TP partitions tensors within a layer, and PP partitions layers into sequential pipeline stages [33, 38]. For example, raising TP for actor generation changes how actor weights must be resharded after training. Accelerating the actor has little effect if the critic remains the bottleneck. Such coupling exists in both synchronous and asynchronous frameworks. Synchronous systems such as Verl (HybridFlow) [37] leave…
Figure 5 of the paper
Figure 5. Core EMU primitives for online adaptation: Split (a) and Merge (b) reshard state by converting between tightly coupled TP/PP structure and loosely coupled replicas, while Extend (c) and Destroy (d) scale resources by creating or removing replicas. For the example in §5.1, Extend adds replicas for critic training, and Extend followed by Merge increases TP for actor generation without changing its DP degree. are expressed, transformed, and evaluated online, while the controller (§4) selects among these choices. Example. Consider actor generation and critic training for 8B models…
Figure 6 of the paper
Figure 6. Constructing the global transition DAG from S to S ∗ : Nereus first translates each model-stage’s change into a local DAG, then adds cross-model-stage dependency edges under transient GPU overlap. 6.1 Decomposition and DAG Construction The current and target global plans are S = {𝜋M | M ∈ M} and S ∗ = {𝜋 ∗ M | M ∈ M}. Their runtime representations E (S) and E (S∗ ) contain the corresponding EMU collections 𝐸M = Φ(𝜋M) and 𝐸 ∗ M = Φ(𝜋 ∗ M ) defined in §5. Nereus sum￾marizes the change for model-stage M as ΔM = (𝐸M→𝐸 ∗ M ), which captures TP/PP resharding, DP scaling, and GPU…
Figure 7 of the paper
Figure 7. Llama-3.1-8B PPO performance at fixed GPU bud￾gets on Cluster #2. For transition efficiency (§7.4), we compare with UCP [21], MCP [39], Gemini [47], Tenplex [43], and Oobleck [12]. These baselines move the same parameter and optimizer state, and their setup time is counted. We isolate DynaRL-style admis￾sion on held-out traces (§7.3.3) and DynaRL’s per-component migration in the coordination comparison (§7.4.4). Models and RL algorithms. Beyond Llama-3.1-8B, we eval￾uate Qwen3-14B, Qwen3-32B [32], and Llama-3.3-70B [23]. Generality experiments replace PPO with ReMax [20] or GRPO…
Figure 8 of the paper
Figure 8. Strong scalability on Cluster #1. 8 16 32 64 GPU Count 0 20,000 40,000 60,000 Throughput (token/s) NEREUS Verl 8B 14B 32B 70B
Figure 11 of the paper
Figure 11. Controller policies under held-out sequence￾length and GPU-supply drift: step latency, gap to the best plan at each step, and sensitivity to the confidence margin. All traces are built from real data (Figs. 3 and 4a). In the first trace (
Figure 13 of the paper
Figure 13. Transition planning across 32–1,024 GPUs. reaching 1,024 GPUs on Cluster #1 consume 49.5 s of 62,353 s (0.079%). The largest one, which also doubles the GPUs from 512 to 1,024, takes 31.55 s, versus 1,629 s for UCP. Split and Extend dominate it, and Merge and Destroy cost little. 7.4.2 Parallelism-Resharding Cost. We measure TP/PP resharding using Split and Merge for models from 8B to 70B parameters. We compare against MCP and Tenplex. MCP (checkpoint-based) and Tenplex (shard-level) are the clos￾est coarse- and fine-grained alternatives to EMU resharding. Gemini and Oobleck appear…
Figure 14 of the paper
Figure 14. Step latency on Cluster #3 for ReMax and GRPO (eight samples per prompt) and asynchronous RL. faster at this scale, where SCIP’s planning time exceeds Nereus’s step latency (Fig. 8a). The relative gap to SCIP in transition completion time stays within 6.9% at every scale (5.1% at 1,024 GPUs). This gap measures schedule quality within the same formulation, whereas §7.3.2 reports target￾plan quality against the empirical optimum. 7.4.4 Cross-Model-Stage Coordination. For the three transition types of Tab. 5, we compare Nereus on Cluster #3 with DynaRL’s per-component migration, which…
Figure 15 of the paper
Figure 15. Training behavior over the first 50 steps: reward against wall-clock time (left), reward and PPO KL estimate against step (middle, right). Laminar [36] (Fig. 14c). Here, step boundaries are weight syn￾chronizations between training and rollout, with adaptation subject to the safe-boundary conditions in §4.1. 7.5.2 Training Behavior Under Adaptation. Over the first 50 steps, Nereus tracks Verl’s reward and PPO KL es￾timate (the mean log-probability drop of sampled tokens from the rollout policy to the updated policy,

Showing 11 of the paper's 15 figures. Figures are the authors', reproduced from the paper.

References (60)
  1. [1] Advanced Micro Devices, Inc. 2026. ROCm System Management Interface (ROCm SMI).https://github.com/ROCm/rocm-systems/ tree/develop/projects/rocm-smi-lib on Pith
  2. [2] Erfan Darzi, Aldo Pareja, Kaveh Jalilian, and Shreeanant Bharad- waj. 2025. GPU Tail Latency Diagnosis for Serverless and HPC Workloads using eBPF. InProceedings of the 11th International Work- shop on Serverless Computing(Nashville, TN, USA)(WoSC11 ’25). Association for Compu... on PitharXiv
  3. [3] DeepSeek-AI. 2025. DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning.Nature645, 8081 (2025), 633–638. doi:10.1038/s41586-025-09422-z on Pithdoi:10.1038/s41586-025-09422-z
  4. [4] Jiangfei Duan, Ziang Song, Xupeng Miao, Xiaoli Xi, Dahua Lin, Harry Xu, Minjia Zhang, and Zhihao Jia. 2024. Parcae: Proactive, Liveput- Optimized DNN Training on Preemptible Instances. In21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24). USENIX Ass... on Pith
  5. [5] Wei Fu, Jiaxuan Gao, Xujie Shen, Chen Zhu, Zhiyu Mei, Chuyi He, Shusheng Xu, Guo Wei, Jun Mei, Jiashu Wang, Tongkai Yang, Binhang Yuan, and Yi Wu. 2025. AReaL: A Large-Scale Asynchronous Rein- forcement Learning System for Language Reasoning. InAdvances in Neural Information P... on Pith
  6. [6] Wei Gao, Yuheng Zhao, Tianyuan Wu, Shaopan Xiong, Weixun Wang, Dakai An, Lunxi Cao, Dilxat Muhtar, Zichen Liu, Haizhou Zhao, Ju Huang, Siran Yang, Yongbin Li, Wenbo Su, Jiamang Wang, Lin Qu, Bo Zheng, and Wei Wang. 2026. RollArt: Disaggregated Multi-Task Agentic RL Training at... on Pith
  7. [7] Diandian Gu, Yihao Zhao, Yinmin Zhong, Yifan Xiong, Zhenhua Han, Peng Cheng, Fan Yang, Gang Huang, Xin Jin, and Xuanzhe Liu. 2023. ElasticFlow: An Elastic Serverless Training Platform for Distributed Deep Learning. InProceedings of the 28th ACM International Confer- ence on Ar... on PitharXiv
  8. [8] Zhenyu Han, Ansheng You, Haibo Wang, Kui Luo, Guang Yang, Wenqi Shi, Menglong Chen, Sicheng Zhang, Zeshun Lan, Chunshi Deng, Huazhong Ji, Wenjie Liu, Yu Huang, Yixiang Zhang, Chenyi Pan, Jing Wang, Xin Huang, Chunsheng Li, and Jianping Wu. 2025. AsyncFlow: An Asynchronous Stre... on PithPith's reviewarXiv
  9. [9] Willy Herroelen, Bert De Reyck, and Erik Demeulemeester. 1998. Resource-constrained project scheduling: A survey of recent devel- opments.Computers & Operations Research25, 4 (1998), 279–302. doi:10.1016/S0305-0548(97)00055-5 on Pithdoi:10.1016/s0305-0548(97)00055-5
  10. [10] Christopher Hojny, Mathieu Besançon, Ksenia Bestuzheva, Sander Borst, João Dionísio, Johannes Ehls, Leon Eifler, Mohammed Ghannam, Ambros Gleixner, Adrian Göß, Alexander Hoen, Jacob von Holly- Ponientzietz, Rolf van der Hulst, Dominik Kamp, Thorsten Koch, Kevin Kofler, Jurgen ... on Pithdoi:10.48550/arxiv.2511.18580
  11. [11] Jian Hu, Xibin Wu, Wei Shen, Jason Klein Liu, Weixun Wang, Songlin Jiang, Haoran Wang, Hao Chen, Bin Chen, Wenkai Fang, Xianyu, Yu Cao, Haotian Xu, and Yiming Liu. 2025. OpenRLHF: A Ray-based Easy-to-use, Scalable and High-performance RLHF Framework. In Proceedings of the 2025... on Pith
  12. [14] Saurabh Jha, Archit Patke, Jim Brandt, Ann Gentile, Benjamin Lim, Mike Showerman, Greg Bauer, Larry Kaplan, Zbigniew Kalbarczyk, William Kramer, and Ravi Iyer. 2020. Measuring Congestion in High- Performance Datacenter Interconnects. In17th USENIX Symposium on Networked System... on Pith
  13. [15] Devvrit Khatri, Lovish Madaan, Rishabh Tiwari, Rachit Bansal, Sai Surya Duvvuri, Manzil Zaheer, Inderjit S. Dhillon, David Brandfon- brener, and Rishabh Agarwal. 2026. The Art of Scaling Reinforcement Learning Compute for LLMs. InThe Fourteenth International Confer- ence on Le... on Pith
  14. [17] Kinman Lei, Yuyang Jin, Mingshu Zhai, Kezhao Huang, Haoxing Ye, and Jidong Zhai. 2024. PUZZLE: Efficiently Aligning Large Language Models through Light-Weight Context Switch. In2024 USENIX Annual Technical Conference (USENIX ATC 24). USENIX Association, Santa Clara, CA, 127–140 on Pith
  15. [18] Cheng Li. 2023. LLM-Analysis: Latency and Memory Analysis of Transformer Models for Training and Inference.https://github.com/ cli99/llm-analysis on Pith
  16. [19] Mingzhen Li, Wencong Xiao, Hailong Yang, Biao Sun, Hanyu Zhao, Shiru Ren, Zhongzhi Luan, Xianyan Jia, Yi Liu, Yong Li, Wei Lin, and Depei Qian. 2023. EasyScale: Elastic Training with Consistent Accuracy and Improved Utilization on GPUs. InProceedings of the International Confe... on PitharXiv
  17. [20] Ziniu Li, Tian Xu, Yushun Zhang, Zhihang Lin, Yang Yu, Ruoyu Sun, and Zhi-Quan Luo. 2024. ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Models. InProceedings of the 41st International Conference on Machine Learning (ICML’24... on Pith
  18. [21] Xinyu Lian, Sam Ade Jacobs, Lev Kurilenko, Masahiro Tanaka, Stas Bekman, Olatunji Ruwase, and Minjia Zhang. 2025. Universal Check- pointing: A Flexible and Efficient Distributed Checkpointing System for Large-Scale DNN Training with Reconfigurable Parallelism. In Proceedings o... on Pith
  19. [22] Nandor Licker, Kevin Hu, Vladimir Zaytsev, and Lequn Chen. 2026. fabric-lib: RDMA Point-to-Point Communication for LLM Systems. In Proceedings of Machine Learning and Systems, Vol. 8. MLSys, Bellevue, WA, USA, 169–185 on Pith
  20. [23] Llama Team, AI @ Meta. 2024. The Llama 3 Herd of Models. arXiv:2407.21783 [cs.AI] doi:10.48550/arXiv.2407.21783 on PithPith's reviewarXivdoi:10.48550/arxiv.2407.21783
show 40 more
  1. [24] Luo Mai, Guo Li, Marcel Wagenländer, Konstantinos Fertakis, Andrei- Octavian Brabete, and Peter Pietzuch. 2020. KungFu: Making Training in Distributed Machine Learning Adaptive. In14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20). USENIX Associatio... on Pith
  2. [25] Zhiyu Mei, Wei Fu, Kaiwei Li, Guangju Wang, Huanchen Zhang, and Yi Wu. 2025. ReaL: Efficient RLHF Training of Large Language Models with Parameter Reallocation. InProceedings of Machine Learning and Systems, Vol. 7. MLSys, Santa Clara, CA, USA, 20 pages on Pith
  3. [26] Zizhao Mo, Huanle Xu, and Chengzhong Xu. 2024. Heet: Accelerating Elastic Training in Heterogeneous Deep Learning Clusters. InProceed- ings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2(La Jolla, CA,... on PitharXiv
  4. [27] Philipp Moritz, Robert Nishihara, Stephanie Wang, Alexey Tumanov, Richard Liaw, Eric Liang, Melih Elibol, Zongheng Yang, William Paul, Michael I. Jordan, and Ion Stoica. 2018. Ray: A Distributed Framework for Emerging AI Applications. In13th USENIX Symposium on Operating Syste... on Pith
  5. [28] NVIDIA. 2025. NeMo RL: A Scalable and Efficient Post-Training Library.https://github.com/NVIDIA-NeMo/RL. GitHub repository on Pith
  6. [29] NVIDIA. 2026. NVIDIA Management Library (NVML).https:// developer.nvidia.com/management-library-nvml on Pith
  7. [30] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wain- wright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Chris- tiano, Jan ... on Pith
  8. [31] Aurick Qiao, Sang Keun Choe, Suhas Jayaram Subramanya, Willie Neiswanger, Qirong Ho, Hao Zhang, Gregory R. Ganger, and Eric P. Xing. 2021. Pollux: Co-adaptive Cluster Scheduling for Goodput- Optimized Deep Learning. In15th USENIX Symposium on Operating Systems Design and Imple... on Pith
  9. [32] Qwen Team. 2025. Qwen3 Technical Report. arXiv:2505.09388 [cs.CL] doi:10.48550/arXiv.2505.09388 on PithPith's reviewarXivdoi:10.48550/arxiv.2505.09388
  10. [33] Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 2020. ZeRO: memory optimizations toward training trillion parameter models. InProceedings of the International Conference for High Performance Computing, Networking, Storage and Analy- sis (SC ’20). IEEE Press, ... on PithPith's reviewarXiv
  11. [34] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal Policy Optimization Algorithms. arXiv:1707.06347 [cs.LG] on PithPith's reviewarXiv
  12. [35] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo on Pith
  13. [36] Guangming Sheng, Yuxuan Tong, Borui Wan, Wang Zhang, Chaobo Jia, Xibin Wu, Yuqi Wu, Xiang Li, Chi Zhang, Yanghua Peng, Haibin Lin, Xin Liu, and Chuan Wu. 2026. Laminar: A Scalable Asynchro- nous RL Post-Training Framework. InProceedings of the 21st European Conference on Compu... on PitharXiv
  14. [37] Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2025. Hybrid- Flow: A Flexible and Efficient RLHF Framework. InProceedings of the Twentieth European Conference on Computer Systems(Rotterdam, Netherlands)(EuroSy... on PitharXiv
  15. [38] Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. 2020. Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism. arXiv:1909.08053 [cs.CL] 14 Nereus: Adaptive Parallelism for LLM Post-Training Songl... on PithPith's reviewarXiv
  16. [39] Megatron Team. 2024. Dist Checkpointing Package.https://docs. nvidia.com/megatron-core/developer-guide/nightly/api-guide/core/ dist_checkpointing.html on Pith
  17. [40] PyTorch Team. 2024. Distributed Checkpointing Recipe.https://docs. pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html on Pith
  18. [41] John Thorpe, Pengzhan Zhao, Jonathan Eyolfson, Yifan Qiao, Zhihao Jia, Minjia Zhang, Ravi Netravali, and Guoqing Harry Xu. 2023. Bam- boo: Making Preemptible Instances Resilient for Affordable Training of Large DNNs. In20th USENIX Symposium on Networked Systems Design and Impl... on Pith
  19. [42] Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beech- ing, Tristan Thrush, Nathan Lambert, Shengyi Huang, Kashif Rasul, and Quentin Gallouédec. 2020. TRL: Transformer Reinforcement Learning on Pith
  20. [43] Marcel Wagenländer, Guo Li, Bo Zhao, Luo Mai, and Peter Pietzuch on Pith
  21. [44] Borui Wan, Mingji Han, Yiyao Sheng, Yanghua Peng, Haibin Lin, Mo- fan Zhang, Zhichao Lai, Menghan Yu, Junda Zhang, Zuquan Song, Xin Liu, and Chuan Wu. 2025. ByteCheckpoint: a unified checkpointing system for large foundation model development. InProceedings of the 22nd USENIX ... on Pith
  22. [45] Tenplex: Dynamic Parallelism for Deep Learning using Paralleliz- able Tensor Collections. InProceedings of the ACM SIGOPS 30th Sym- posium on Operating Systems Principles(Austin, TX, USA)(SOSP ’24). Association for Computing Machinery, New York, NY, USA, 195–210. doi:10.1145/3... on PitharXiv
  23. [46] Yuanqing Wang, Hao Lin, Junhao Hu, Chunyang Zhu, Quanlu Zhang, Zhen Guo, Yuchen Zhang, Xu Fu, Si Xu, Bo Dai, Zixiao Huang, Chao Yu, Boxun Li, Guohao Dai, Zhi Yang, and Yu Wang. 2026. DynaRL: Flex- ible and Dynamic Scheduling of Large-Scale Reinforcement Learning Training. In20... on Pith
  24. [47] Weixun Wang, Shaopan Xiong, Gengru Chen, Wei Gao, Sheng Guo, Yancheng He, Ju Huang, Jiaheng Liu, Zhendong Li, Xiaoyang Li, Zichen Liu, Haizhou Zhao, Dakai An, Lunxi Cao, Qiyang Cao, Wanxi Deng, Feilei Du, Yiliang Gu, Jiahe Li, Xiang Li, Mingjie Liu, Yijia Luo, Zihe Liu, Yadao ... on PithPith's reviewarXivdoi:10.48550/arxiv.2506.06122
  25. [48] Christin Whitton, William Jones, Craig Walker, Vanessa Job, Steven Senator, and Nathan DeBardeleben. 2025. Are We There Yet? Predict- ing the Queue Wait Times for HPC Jobs . In2025 IEEE International Con- ference on Cluster Computing (CLUSTER). IEEE Computer Society, Los Alami... on PitharXiv
  26. [49] Zhuang Wang, Zhen Jia, Shuai Zheng, Zhen Zhang, Xinwei Fu, T. S. Eu- gene Ng, and Yida Wang. 2023. GEMINI: Fast Failure Recovery in Distributed Training with In-Memory Checkpoints. InProceedings of the 29th Symposium on Operating Systems Principles(Koblenz, Ger- many)(SOSP ’23... on PitharXiv
  27. [50] Zhanghao Wu, Wei-Lin Chiang, Ziming Mao, Zongheng Yang, Eric J. Friedman, Scott Shenker, and Ion Stoica. 2024. Can’t Be Late: Optimiz- ing Spot Instance Savings under Deadlines. In21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24). USENIX Associatio... on Pith
  28. [51] Bo Wu, Sid Wang, Yunhao Tang, Jia Ding, Eryk Helenowski, Liang Tan, Tengyu Xu, Tushar Gowda, Zhengxing Chen, Chen Zhu, Xi- aocheng Tang, Yundi Qian, Beibei Zhu, and Rui Hou. 2025. LlamaRL: A Distributed Asynchronous Reinforcement Learning Framework for Efficient Large-scale LL... on PithPith's reviewarXivdoi:10.48550/arxiv.2505.24034
  29. [52] Youshao Xiao, Zhenglei Zhou, Fagui Mao, Weichang Wu, Shangchun Zhao, Lin Ju, Lei Liang, Xiaolu Zhang, and Jun Zhou. 2025. FlexRLHF: A Flexible Placement and Parallelism Framework for Efficient RLHF Training. In2025 IEEE International Parallel and Distributed Process- ing Sympo... on PitharXiv
  30. [53] Miguel G. Xavier, Kassiano J. Matteussi, Fabian Lorenzo, and Cesar A. F. De Rose. 2016. Understanding performance interference in multi- tenant cloud databases and web applications. In2016 IEEE International Conference on Big Data (Big Data). IEEE, Washington, DC, USA, 2847– on Pith
  31. [54] Shaoxun Zeng, Tingxu Ren, Jiwu Shu, and Youyou Lu. 2026. GPU Checkpoint/Restore Made Fast and Lightweight. In24th USENIX Con- ference on File and Storage Technologies (FAST’26). USENIX Association, Santa Clara, CA, 239–254 on Pith
  32. [55] Lianmin Zheng, Zhuohan Li, Hao Zhang, Yonghao Zhuang, Zhifeng Chen, Yanping Huang, Yida Wang, Yuanzhong Xu, Danyang Zhuo, Eric P. Xing, Joseph E. Gonzalez, and Ion Stoica. 2022. Alpa: Au- tomating Inter- and Intra-Operator Parallelism for Distributed Deep Learning. In16th USEN... on Pith
  33. [56] Zhewei Yao, Reza Yazdani Aminabadi, Olatunji Ruwase, Samyam Rajbhandari, Xiaoxia Wu, Ammar Ahmad Awan, Jeff Rasley, Minjia Zhang, Conglong Li, Connor Holmes, Zhongzhu Zhou, Michael Wyatt, Molly Smith, Lev Kurilenko, Heyang Qin, Masahiro Tanaka, Shuai Che, Shuaiwen Leon Song, a... on PithPith's reviewarXivdoi:10.48550/arxiv.2308.01320
  34. [57] Yinmin Zhong, Zili Zhang, Bingyang Wu, Shengyu Liu, Yukun Chen, Changyi Wan, Hanpeng Hu, Lei Xia, Ranchen Ming, Yibo Zhu, and Xin Jin. 2025. Optimizing RLHF Training for Large Language Models with Stage Fusion. In22nd USENIX Symposium on Networked Systems Design and Implementa... on Pith
  35. [58] Yuzhen Zhou, Jiajun Li, Yusheng Su, Gowtham Ramesh, Zilin Zhu, Xiang Long, Chenyang Zhao, Jin Pan, Xiaodong Yu, Ze Wang, Kan- grui Du, Jialian Wu, Ximeng Sun, Jiang Liu, Qiaolin Yu, Hao Chen, Zicheng Liu, and Emad Barsoum. 2025. APRIL: Active Partial Rollouts in Reinforcement ... on Pithdoi:10.48550/arxiv.2509.18521
  36. [59] Yinmin Zhong, Zili Zhang, Xiaoniu Song, Hanpeng Hu, Chao Jin, Bingyang Wu, Nuo Chen, Yukun Chen, Yu Zhou, Changyi Wan, Hongyu Zhou, Yimin Jiang, Yibo Zhu, and Daxin Jiang. 2025. StreamRL: Scalable, Heterogeneous, and Elastic RL for LLMs with Disaggregated Stream Generation. ar... on PithPith's reviewarXivdoi:10.48550/arxiv.2504.15930
  37. [60] Zilin Zhu, Chengxing Xie, Xin Lv, and slime Contributors. 2025. slime: An LLM post-training framework for RL Scaling.https://github.com/ THUDM/slime. GitHub repository. Corresponding author: Xin Lv. 15 on Pith
  38. [62] Kan Zhu, Yufei Gao, Yilong Zhao, Liangyu Zhao, Gefei Zuo, Yile Gu, Dedong Xie, Tian Tang, Qinyu Xu, Zihao Ye, Keisuke Kamahori, Chien- Yu Lin, Ziren Wang, Stephanie Wang, Arvind Krishnamurthy, and Baris Kasikci. 2025. NanoFlow: Towards Optimal Large Language Model Serving Thro... on Pith
  39. [2024] DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv:2402.03300 [cs.CL] on PithPith's reviewarXiv
  40. [2852] doi:10.1109/BigData.2016.7840933 on PitharXiv
Cited by 2 Pith papers

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment