{"id":"537eba81-0e85-4164-a4d6-72bcf3bf8326","arxiv_id":"2505.04519","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Pangu Ultra MoE, a 718B-parameter MoE model trained on 6,000 Ascend NPUs, reaches 30.0% MFU and matches DeepSeek R1 on a selected set of math, coding, and medical benchmarks.","lead":"Huawei's Pangu team reports training Pangu Ultra MoE, a 718-billion-parameter Mixture-of-Experts model, on 6,000 Ascend NPUs at 30.0% model FLOPs utilization. They claim benchmark performance comparable to DeepSeek R1 on math, coding, and medical tasks, backed by a simulation-based architecture search and system optimizations.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 30.0% MFU headline is unauditable because the FLOPs-per-token formula is never disclosed, and a different accounting convention could shift the number by tens of percent before any comparison to GPU training runs is meaningful.","rationale":"The reader's conditional verdict is driven by the unverifiability of the MFU and benchmark claims, and my stress-test narrows the decisive unknown to the MFU accounting. The manuscript contains internal admissions that support this concern: Section 2.3 states that simulated durations are 'consistently shorter than the measured data' with 'ideally overlapped communications'; Section 5.2 marks several DeepSeek rows as the authors' own testing; and no MFU formula or raw training configuration is given in Sections 4 or 5. These are not signs of misconduct; they simply make the headline quantitative claim unauditable. I do not think the paper should be rejected on this basis, because the engineering techniques are described in substantial detail and the 4.2B pilot simulation validation provides one independent anchor, but acceptance of the 30.0% claim should be conditional on disclosure or independent reconstruction. The in-sample re-validation on Pangu Ultra MoE itself (Table 1, row 2) is a further weakness but is less load-bearing than the missing MFU formula, because even a perfect simulator would not resolve the accounting ambiguity. Benchmark comparability is secondary: a few points on one or two tables would not overturn the paper's systems contribution, whereas an undefined MFU convention undermines the headline result itself. I therefore recommend keeping the reader's CONDITIONAL verdict.","tokens_in":32606,"tokens_out":5198,"duration_ms":50711,"concrete_test":"Require the authors to publish the exact MFU formula used: the per-token FLOPs expression (which terms are counted: activated routed experts, shared expert, attention/MLA, MTP, routing and permute operations, auxiliary loss, recomputation multiplier; and whether communication, pipeline bubbles, or load-imbalance idle time are included), plus global batch size, sequence length, precision, and peak FLOPs per Ascend 910B. Then recompute MFU from the reported 1.46M TPS and also recompute under an activated-FLOPs-only convention and a full-parameter MoE convention. If the three values differ by more than about 5 MFU percentage points, the headline number must be restated with a clearly defined convention before it can be compared with GPU training runs.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline claim is '30.0% MFU when training Pangu Ultra MoE... on 6K Ascend NPUs' (Section 1). This number is meaningful only if the FLOPs-per-token expression used to compute it is fixed, and the paper never states that expression. For a 718B MoE with 256 experts, top-k=8, a shared expert, MLA, MTP, fine-grained recomputation, tensor swapping, and dynamic expert placement, plausible accounting choices diverge sharply: counting only activated routed-expert GEMMs versus counting routed-plus-shared-plus-attention-plus-MTP FLOPs, including or excluding recomputation work, and treating communication, pipeline bubbles, or load-imbalance idle cycles as overhead can change MFU by tens of percent. Section 5.1 does not give global batch size, sequence length, training duration, or peak FP16 FLOPs of Ascend 910B, so the reported TPS of 1.46M cannot be used to reconstruct MFU independently. Without the formula, '30.0%' is not comparable to MFU numbers reported for GPU-based MoE training, and Table 5's 58.7% relative improvement is a statement about an unspecified baseline. Separately, Table 6 mixes official scores with the authors' own evaluations (asterisked rows), so 'comparable to DeepSeek R1' is also not fully auditable; however, the MFU accounting is the more load-bearing of the two because it is the paper's central quantitative claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper reports the design and training of Pangu Ultra MoE, a 718B-parameter Mixture-of-Experts language model with 61 layers, hidden size 7680, 256 routed experts, top-8 routing, a shared expert, MLA, and MTP, trained on 6K Ascend NPUs. The authors propose a simulation-based architecture search over approximately 10,000 configurations and select a model with 61 layers, hidden size 7680, 256 experts, using parallelism TP=8, PP=16, VPP=2, EP=4, MBS=2. On the systems side, they describe hierarchical EP All-to-All communication, adaptive pipe overlap, fine-grained recomputation, tensor swapping, and a planner/executor for dynamic expert placement. They report 30.0% MFU and 1.46M tokens/sec on 6K Ascend NPUs, a 58.7% relative MFU improvement over their baseline, and benchmark results across C-Eval, MMLU, AIME, GPQA, LiveCodeBench, and medical datasets that are said to be comparable to DeepSeek R1. They also include analyses of auxiliary-loss granularity, drop-and-pad versus dropless training, expert specialization, router scale, and expert co-activation.","tokens_in":32801,"tokens_out":10349,"duration_ms":92475,"significance":"If the reported efficiency and benchmark numbers hold, the paper is a significant engineering contribution: it provides one of the first public accounts of training a near-trillion-parameter MoE to near-frontier quality on a non-NVIDIA accelerator stack, and the system techniques (hierarchical EP all-to-all, adaptive pipe overlap, fine-grained recomputation, tensor swapping, dynamic placement) are described in enough detail to be useful to other systems groups. The authors are also transparent about simulation-accuracy limits and mark which benchmark scores were obtained by their own testing, which is more candid than many industrial reports. However, the paper does not ship code, checkpoints, training logs, or a full data recipe, and the central quantitative claims currently rest on undisclosed accounting choices: no MFU formula, no global batch size or sequence length, no evaluation protocol for asterisked benchmark scores, and an in-sample simulation validation. The methodology is sound in outline, but the headline numbers are not yet auditable.","major_comments":[{"comment":"The paper never states the FLOPs-per-token formula used to compute MFU. For the reported 718B MoE with 256 routed experts, top-8 activation, one shared expert, MLA, MTP, fine-grained recomputation, tensor swapping, and dynamic expert placement, the distinction between counting only activated routed-expert GEMMs and counting attention/shared-expert/MTP/recomputation FLOPs can shift the MFU by tens of percent. Section 5.1 does not report the global batch size, sequence length, total training tokens, or the peak FLOPs of the Ascend 910B used in the denominator, so the TPS value of 1.46M cannot be used to reconstruct or audit the 30.0% figure. In addition, Table 5's cumulative '58.7% relative increase' is presented as a baseline of 1, while the Section 1 baseline of 18.9% MFU is stated on 4K NPUs; because the cluster size and parallelism differ, the relative improvement is not a controlled comparison. The authors should disclose the exact MFU formula and all experimental inputs, or the central claim is not comparable to MFU numbers reported for GPU baselines.","section":"Section 1 / Section 5.1 / Table 5"},{"comment":"The simulation is validated on the final model it selected. The second row of Table 1 uses 'Pangu Ultra MoE Training' itself, giving 90.1% accuracy; this is an in-sample check and does not establish that the simulator correctly ranks the ~10,000 candidate configurations or the eight high-throughput models in Figure 4. The first row (4.2B, 128 NPUs) is out-of-sample but uses GQA, 4 experts, and a 6-layer model, so it does not exercise MLA, 256-expert routing, MTP, or the 6K-NPU communication patterns of the target configuration. The architecture-selection claim needs either a held-out validation on at least one additional 718B-class configuration or an explicit analysis of how simulation error varies across the candidate set, for example by comparing predicted versus measured performance of several selected candidates.","section":"Section 2.3 / Table 1"},{"comment":"The comparison to DeepSeek R1 is not fully auditable. Several DeepSeek numbers are asterisked as obtained by the authors' own testing, but the paper gives no evaluation protocol: prompt templates, decoding settings (temperature, top-p, sampling), answer extraction, number of runs or seeds, and contamination checks are all absent. Without these details and without run-to-run variability, differences such as AIME2024 81.3 versus 79.8 and LiveCodeBench 64.3 versus 65.9 cannot be distinguished from evaluation noise. The claim that Pangu Ultra MoE is comparable to DeepSeek R1 should be treated as provisional; the authors should provide a complete evaluation protocol or rely on official reported scores.","section":"Section 5.2 / Table 6"},{"comment":"The paper claims that dynamic expert placement gives a 'relative 10% improvement in MFU' and reduces device-level load imbalance by 80%-90%, but no ablation or measurement description is provided, and Table 5's cumulative MFU improvements do not list this item. If the Table 5 baseline is the same 18.9% MFU from Section 1, the arithmetic 18.9% x 1.587 = 30.0% already accounts for the full headline gain; adding a further 10% relative improvement would exceed the reported MFU unless it is already concealed inside one of Table 5's categories or the baselines differ. The authors should reconcile these numbers and provide an explicit ablation for the dynamic placement mechanism.","section":"Section 4.4 / Table 5"}],"minor_comments":[{"comment":"The header 'Huawei Proprietary - Restricted Distribution' appears on a public arXiv submission; if the report is intended to be public, this marking should be removed or explained.","section":"Page 1"},{"comment":"There are typos: 'memomy access bandwidth' should be 'memory access bandwidth', and 'This prevents any bottleneck from happening' is ungrammatical and should be rephrased.","section":"Section 2.4"},{"comment":"The term 'Tiling = 4' in the CoC discussion is undefined; please explain what tiling denotes and how the 4x reduction in communication exposure is measured.","section":"Section 4.1"},{"comment":"'Umpermute' should be 'Unpermute' in the Figure 9 legend and in Section 4.5.","section":"Figure 9 / Section 4.5"},{"comment":"The benchmark list in the opening paragraph omits C-SimpleQA, although it appears in Table 6; add it to the list.","section":"Section 5.2"},{"comment":"The figure axis labels are rendered as '/uni...' escape sequences in the submitted text; the figure needs to be regenerated so expert IDs are legible.","section":"Figure 12"},{"comment":"References [58] and [59] duplicate the same Qwen2.5 technical report, and [51] and [52] duplicate the Megatron-LM citation; consolidate to avoid confusion.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is an industrial systems report with no code or data release, and it is marked 'Huawei Proprietary - Restricted Distribution' on its first page while being posted to arXiv; the editor may wish to verify that publication is authorized and that the proprietary marking does not impede the review process. The main risk to acceptance is not the plausibility of the engineering but the absence of a fully specified MFU accounting and evaluation protocol; these are fixable in revision. Given the significance of the 30.0% MFU claim on a non-NVIDIA stack, I would require the disclosures described in the major comments before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is the first public report I know of that shows a near-trillion-parameter MoE trained to near-frontier quality on a non-NVIDIA stack at this scale, and that alone makes it worth reading. The engineering is built from known components — hierarchical EP communication, VPP/1F1B overlap, dynamic expert placement, simulation-driven design — but the integration into a working 6K-NPU recipe is real work, and the authors describe it clearly. I was most impressed by the pilot studies on expert granularity and shared experts, and by the EP-group auxiliary loss variant, which seems like a sensible middle-ground that others might adopt. The scheduling diagrams are unusually specific, which is rare for a paper with no code release.\n\nThe soft spots are mostly about accounting. The MFU formula is never given. For an MoE with recompute, tensor swapping, dynamic routing, and MTP heads, the choice of which FLOPs to count can move the number by tens of percent. Without that formula, the 30.0% MFU is not comparable to GPU-based MFU numbers, and Table 5's 58.7% relative improvement is relative to an unnamed baseline. The stress-test note's concern lands: this is the load-bearing issue. The simulator validation in Table 1 includes a re-validation on the exact model the simulator selected, which is in-sample. The earlier 4.2B out-of-sample check partially compensates, so I'd call that a caveat, not a fatal flaw. The benchmark table mixes official scores with the authors' own runs, but they do mark the asterisks; still, the \"comparable to DeepSeek R1\" claim would be easier to trust with a fully public protocol. No code or weights, expected for a proprietary report, but it limits the paper to a recipe rather than a reproducible result.\n\nNone of this makes the central claim implausible. The paper shows clear thinking and honest caveats about its simulation, and the engineering is coherent on its own terms. I would send this to a systems venue with referees who know MoE training, and I'd specifically push the authors to disclose the FLOPs formula and to separate in-sample from out-of-sample validation. The 30% MFU should be treated as a claim to verify, not as a fact to cite.","headline":"A credible industrial recipe for training a 718B MoE on Ascend NPUs, but the headline 30% MFU is unauditable without the FLOPs formula and the simulator re-validation on the final model is in-sample.","tokens_in":699,"tokens_out":729,"would_cite":false,"duration_ms":39213,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 718-billion-parameter mixture-of-experts model can be trained on Ascend NPUs at 30.0% model FLOPs utilization, with benchmark scores comparable to DeepSeek R1.","keywords":["Mixture of Experts","Ascend NPU","model FLOPs utilization","expert parallelism","load balancing","simulation-based architecture search","memory optimization","large language model training"],"falsifier":"Take the paper's own reported numbers—718B total parameters, roughly 39B activated per token, and 1.46M tokens per second—and recompute the FLOPs used per second, then divide by the number of Ascend NPUs times their peak floating-point throughput; if the resulting 30.0% MFU only holds under a lenient FLOPs formula, the comparison to prior MFU reports fails. Independently, rerunning the evaluation suite with a public harness against officially published DeepSeek R1 scores would settle whether the benchmark parity is real.","tokens_in":32199,"feed_emoji":"🧠","tokens_out":8974,"duration_ms":81646,"temperature":0.7,"pith_summary":"The paper is a system-and-architecture recipe for training a near-trillion-parameter mixture-of-experts language model on Ascend NPUs rather than on GPU clusters. It claims that careful model-shape selection guided by a hardware-aware simulator, together with communication, memory, and load-balancing optimizations, lets a 718-billion-parameter MoE reach 30.0% model FLOPs utilization on 6K Ascend NPUs while producing benchmark scores comparable to DeepSeek R1. The central assertion is that the sparse-model efficiency promised by MoE can be materialized on a non-NVIDIA accelerator stack. If true, it means the practical ceiling on large-model training is not set by the choice of accelerator vendor, and that simulation can replace much of the expensive trial-and-error in choosing model configurations.","feed_headline":"A 718B-parameter MoE trains at 30% of peak compute on Ascend NPUs","feed_subtitle":"The model matches DeepSeek R1 on math and knowledge benchmarks, showing frontier-scale training is not GPU-only.","key_machinery":"The load-bearing machinery is a two-part design loop. First, a bottom-up simulator models operator throughput, communication, memory, and overlaps on Ascend 910B hardware, validated to 88.9% and 90.1% accuracy on two training runs, and is used to prune a search space of about 10,000 architectures down to the Pangu Ultra MoE configuration. Second, the system-side optimizations turn that configuration into a working 6K-NPU run: hierarchical EP all-to-all communication (a global inter-node AllGather followed by an intra-node All-to-All exchange) to cut inter-node traffic, an adaptive pipe overlap schedule that hides forward/backward communication behind computation, fine-grained recomputation plus tensor swapping to manage activation memory, fused operators, and a planner/executor pair that predicts expert load and physically moves experts across devices. These mechanisms jointly convert sparse-model dynamic routing from a throughput tax into a recovered efficiency gain.","core_discovery":"The paper's central discovery is a repeatable training recipe for large sparse MoE models on Ascend NPUs. It combines simulation-based architecture search that screens roughly 10,000 candidate configurations by predicted throughput and selects 61 layers, hidden size 7680, 256 routed experts plus one shared expert, MLA attention, and a specific 5D parallelism plan (TP = 8, PP = 16, VPP = 2, EP = 4), with system optimizations including hierarchical EP all-to-all communication, adaptive pipe overlap, fine-grained recomputation with tensor swapping, and dynamic device-level expert placement. On this basis the authors report training Pangu Ultra MoE, 718B total parameters with 39B activated per token, at 30.0% MFU and 1.46M tokens per second on 6K Ascend NPUs, up from an 18.9% MFU baseline on 4K NPUs, with benchmark performance comparable to or better than DeepSeek R1 on most evaluated tasks, especially medical benchmarks. The paper also reports that dropless routing outperforms drop-and-pad increasingly as models grow, and that an EP-group auxiliary loss strikes the best performance/efficiency balance among sequence-, micro-batch-, EP-group-, and DP-group-level losses.","pith_inferences":["Editorial extension: because the paper never states the FLOPs formula behind MFU, a reader who re-derives efficiency from the stated 1.46M tokens per second, 718B parameters, and the Ascend peak compute could get a different number; a standardized formula would be needed for a fair cross-vendor comparison.","Editorial extension: the hierarchical all-to-all scheme and dynamic expert placement are developed for training, but the same mechanisms should carry over to MoE inference on the same hardware, where routing is still dynamic and memory is tighter.","Editorial extension: the simulator's 85-90% accuracy suggests the architecture-search workflow itself is portable; other accelerator vendors could run the same screening on their own compute-to-memory ratios, and the prediction that larger hidden sizes fit compute-heavy chips is directly testable."],"forward_implications":["If the 30.0% MFU holds under the same accounting used for GPU reports, near-trillion-parameter MoE training is no longer tied to NVIDIA accelerators; other large vendors' stacks can host frontier-scale runs.","Simulation-based architecture search at 85-90% accuracy can substitute for much full-scale hyperparameter experimentation, so future model/hardware co-design can be done before hardware exists in quantity.","Dropless routing with device-level expert placement is preferable to token dropping at scale: with capacity factor 1.5, the 718B model dropped 8% of tokens versus 6% for the 20B model, meaning token dropping is increasingly harmful as models grow.","The EP-group auxiliary loss offers a middle ground between sequence-level and global-level balancing, yielding near-global-level loss with lower communication cost.","The reported system optimizations cumulatively raise MFU by 58.7% over baseline, implying similar headroom may exist on other under-optimized stacks for MoE training."],"supporting_citations":[{"why":"Serves as the principal quality baseline; the paper's \"comparable to DeepSeek R1\" claim is judged against this model's scores.","marker":"[10]"},{"why":"Supplies the DeepSeek-V3 architecture components (MLA attention, MTP layers, dual-pipe overlap) that the design adapts, plus the DeepSeek-V3-0324 scores used in Table 6.","marker":"[33]"},{"why":"Provides the evidence on expert granularity and the device-level balance loss that motivate the MoE block and the EP-group auxiliary loss.","marker":"[9]"},{"why":"Defines expert parallelism and shared-expert concepts and is the origin of the drop-and-pad strategy the paper rejects at scale.","marker":"[26]"},{"why":"Establishes the auxiliary-loss and expert-capacity framework against which the paper compares its load-balancing loss.","marker":"[11]"},{"why":"Supplies the dropless training baseline that supports the paper's choice of dropless routing for large MoE models.","marker":"[12]"},{"why":"Underlies the parallel-training operations and activation recomputation mechanisms that the paper reworks.","marker":"[52]"},{"why":"Provides the balance-batch-size taxonomy of load-balancing losses that the EP-group auxiliary loss is designed to improve upon.","marker":"[45]"},{"why":"Supplies the high-level co-design methodology that the paper's simulator-based architecture search is built around.","marker":"[18]"}],"fun_headline_variants":["718B MoE hits 30% MFU on Ascend NPUs","Recipe for training 718B MoE on Ascend NPUs","Ascend NPUs train 718B MoE matching DeepSeek R1","Pangu Ultra MoE: 718B sparse LLM trains at 30% MFU","Simulation-driven MoE training hits 30% MFU on 6K NPUs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The single load-bearing premise is that the efficiency metric behind the 30.0% figure (the share of peak computing power actually used) and the benchmark numbers produced by the authors' own testing are counted with the same conventions used for DeepSeek's published figures; if either convention differs, the headline comparison is not apples-to-apples.","fun_headline_variants_meta":{"raw":{"variants":["718B MoE hits 30% MFU on Ascend NPUs","Recipe for training 718B MoE on Ascend NPUs","Ascend NPUs train 718B MoE matching DeepSeek R1","Pangu Ultra MoE: 718B sparse LLM trains at 30% MFU","Simulation-driven MoE training hits 30% MFU on 6K NPUs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000303,"raw_usage":{"total_tokens":1822,"prompt_tokens":1101,"completion_tokens":721,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":717,"completion_tokens_details":{"reasoning_tokens":614}},"tokens_in":717,"tokens_out":721,"duration_ms":5887,"temperature":1.0,"reasoning_tokens":614,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:26:12.107100+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the paper's own reported numbers—718B total parameters, roughly 39B activated per token, and 1.46M tokens per second—and recompute the FLOPs used per second, then divide by the number of Ascend NPUs times their peak floating-point throughput; if the resulting 30.0% MFU only holds under a lenient FLOPs formula, the comparison to prior MFU reports fails. Independently, rerunning the evaluation suite with a public harness against officially published DeepSeek R1 scores would settle whether the benchmark parity is real.","supporting_citations":[],"review_version":1}