Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

A Mixture-of-Experts model's tolerance to compression error is layer-dependent: shallow and deep experts absorb bounded noise, while middle-layer experts are the most sensitive and drive reasoning accuracy.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

In a 26-layer MoE model, injecting Gaussian weight errors into middle-layer experts hurts math accuracy most, while deep-layer errors can sometimes improve instruction compliance.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection The paper asks the right question about where MoE compression error hurts, but its answer rests on a Gaussian noise surrogate, not on any actual compressor, so the headline claims about SZ3/CuSZp are not yet supported. the 4 major comments →

arxiv 2509.07727 v1 pith:KXGEKNEU submitted 2025-09-09 cs.LG cs.DC

MoE-Compression: How the Compression Error of Experts Affects the Inference Accuracy of MoE Model?

Songkai Ma , Zhaorui Zhang , Sheng Di , Benben Liu , Xiaodong Yu , Xiaoyi Lu , Dan Wang This is my paper
classification cs.LG cs.DC
keywords Mixture of Expertserror-bounded lossy compressionexpert offloadingerror sensitivityMoE inferenceGaussian error simulationlayer-wise robustnessGSM8K
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks which experts in a Mixture-of-Experts language model can be lossily compressed without hurting inference, and answers that sensitivity is layered: bounded error injected into shallow-layer experts barely moves accuracy, error in middle-layer experts damages reasoning, and error in deep-layer experts can slightly improve instruction-following. The authors propose using error-bounded lossy compressors such as SZ3 and CuSZp to shrink non-activated experts before offloading them from GPU memory to main memory, replacing quantization's uncontrollable error with a guaranteed bound. To measure where error matters, they simulate compressor noise as Gaussian perturbations scaled to a percentage of each expert's mean absolute weight and run controlled experiments on the 26-layer Moonlight MoE with GSM8K and Math benchmarks, separating instruction-compliance accuracy from pure reasoning accuracy. If the pattern holds, a compression-aware offloading system could assign tight error bounds only to middle-layer experts and apply larger bounds—or even deliberate noise—elsewhere, cutting PCIe transfer costs with minimal accuracy loss.

Core claim

On the paper's own terms, the central discovery is a layer-dependent error-sensitivity profile for MoE inference. In the Moonlight architecture (26 expert layers, 64 experts per layer, top-6 routing), the authors inject bounded errors into expert weights and measure both Instruction Compliance Accuracy (ICA) and Pure Inference Accuracy (PIA). Single-expert errors have little effect until weights are fully randomized; perturbing the most-activated experts mostly lowers ICA, not PIA. When all experts in a layer are perturbed, layer 13 drops to the lowest accuracy (PIA 0.65 on GSM8K), while layer 26 stays near baseline (PIA 0.90) and can exceed baseline ICA; shallow layers sit in between. The p

What carries the argument

The load-bearing object is the controlled error-injection experiment: compression error is modeled as independent Gaussian noise N(0, e_hat), with e_hat set to 10%-80% of an expert's mean absolute L1 weight, and then applied at seven scopes—single expert, most-activated expert, top-k experts, all experts in a layer, and grouped layers—across a 26-layer MoE. The analysis separates Instruction Compliance Accuracy (ICA) from Pure Inference Accuracy (PIA), and uses activation-frequency heatmaps, imbalance scores, entropy, and Gini coefficient to characterize which experts actually carry the workload. This machinery lets the paper attribute accuracy changes to particular layers and to the functio

Load-bearing premise

The whole sensitivity map is computed by adding independent random noise with a chosen spread to each expert weight; if real error-bounded compressors produce clustered or weight-dependent errors instead, the measured layer pattern may not transfer to actual compressed inference.

What would settle it

Run SZ3 or CuSZp on Moonlight's middle-layer experts at an 80% mean-absolute weight error bound and measure GSM8K ICA/PIA; if layer-13 PIA stays near baseline instead of falling to about 0.65, the Gaussian noise model is not reproducing real compression error. A cheaper check is to compute the spatial autocorrelation of SZ3/CuSZp reconstruction error maps: strong correlation or error concentrated in large-weight entries would falsify the i.i.d. noise assumption.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • An MoE offloading system that compresses non-activated experts should apply the tightest error bounds to middle-layer experts; this is where bounded error most directly lowers reasoning accuracy.
  • Shallow-layer experts can tolerate relatively large bounded error for pure reasoning, but instruction-following accuracy is more fragile there, so compression policy should track both ICA and PIA.
  • Deep-layer experts can be compressed aggressively; in some settings adding bounded error actually raised instruction-compliance accuracy relative to baseline.
  • Perturbing high-frequency experts across many layers at the highest tested bound can cause total output failure, so error bounds must be coordinated across layers, not set per expert in isolation.
  • The functional view of layers (attention, reasoning, output integration) is a better predictor of error impact than network depth alone.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A direct test, not run in the paper: apply SZ3 or CuSZp to actual Moonlight experts at the same 30%-80% mean-absolute bounds and re-measure ICA/PIA by layer; if real compressor errors are spatially correlated or concentrated in large weights, the middle-layer sensitivity pattern may shift.
  • The layer-sensitivity profile suggests a per-layer error-bound schedule for offloading: set large bounds for shallow and deep experts and near-lossless bounds for middle experts, then adjust the schedule to meet an end-to-end accuracy budget without retraining.
  • The observed deep-layer benefit raises a testable deployment-time hypothesis: adding controlled noise to deep experts acts as implicit ensemble regularization, which could be evaluated by measuring calibration or out-of-distribution robustness under deep-layer noise.
  • The paper studies one architecture and two math benchmarks; whether the same edge-sensitive profile appears in conversational or code-generation MoE models, or in models with different layer counts, is an open generalization question.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper addresses expert offloading in MoE inference and proposes using error-bounded lossy compression (SZ3, CuSZp) to reduce data-transfer overhead. Because it does not run any actual compressor, it models compression error by adding i.i.d. Gaussian noise N(0, e_hat) to expert weights, with e_hat set to 30%, 50%, or 80% of the mean absolute weight of the target expert. Using the Moonlight model on GSM8K and MATH, it measures Instruction Compliance Accuracy (ICA) and Pure Inference Accuracy (PIA) across seven perturbation scenarios: a single expert, the most frequently activated expert, top-k experts, all experts in one layer, high-frequency experts in layer groups, and a second dataset. The paper concludes that shallow-layer experts tolerate bounded errors, middle-layer experts are highly sensitive, and deep-layer experts can sometimes improve; it recommends protecting middle layers and allowing larger errors in shallow/deep layers during compression-aware offloading.

Significance. If the reported layer-sensitivity pattern were established for real error-bounded lossy compressors, it would be practically valuable for designing compression-aware MoE offloading and for choosing per-layer error bounds. The seven-scenario perturbation taxonomy and the explicit separation of instruction compliance from pure reasoning accuracy are useful structuring choices, and the question is timely. However, the central evidence is entirely synthetic: no SZ3/CuSZp execution appears anywhere in the paper, no code or reproducibility artifacts are provided, and the functional attributions to attention, reasoning, and instruction following are asserted rather than measured. The paper is best read as a plausible design hypothesis, not an established result.

major comments (4)
  1. [Section 3.1 and all experiments (Tables 3-9)] The central error model is not equivalent to error-bounded lossy compression. The text states: 'To simulate the compression errors of most current state-of-the-art compressors, such as SZ3, CuSZp, etc., we randomly generated n errors which follows the normal distributions N~(0, e_hat) and add these errors to the expert parameters.' SZ3 and CuSZp guarantee a maximum pointwise error bound; Gaussian noise with standard deviation e_hat is unbounded, so the tested perturbations are not error-bounded in the compressor sense. Moreover, SZ3/CuSZp errors are spatially structured and data-dependent, not i.i.d. Gaussian. Therefore the abstract's claim about 'bounded errors' and conclusion ❽ in Section 4 are not supported for actual compression. A revision must either run actual compressors on expert tensors and re-derive the sensitivity pattern, or provide empirical evidence that the Gaussian surro
  2. [Tables 3-9] All accuracy tables report single runs without error bars, confidence intervals, or seed information. Several conclusions rest on very small differences, e.g., Table 4 shows PIA of 0.96 vs 0.95, and Table 8 shows ICA of 0.62 vs 0.60. With typical benchmark sizes, these differences are within sampling noise. The claim that deep-layer errors 'can sometimes lead to improvements' is particularly sensitive to this issue because the observed improvements are a few points. The authors should report repeated runs or bootstrap confidence intervals, and should establish that the reported patterns are not artifacts of a particular evaluation run.
  3. [Section 3.4 / Section 4, conclusion ❽] The abstract and conclusion ❽ state that 'experts in the shallow layers... exhibit minimal degradation in inference accuracy when subjected to bounded errors.' This is contradicted by the all-experts-in-a-layer experiment: Table 6 shows that perturbing all 64 experts in layer 1 with the 80%-mean-absolute-weight error drops ICA from 0.86 to 0.33 and PIA from 0.96 to 0.71. The conclusion is only true for a single expert or a small subset of experts at moderate error bounds. The paper must qualify the claim to distinguish per-expert perturbations from large-scale perturbations; as written, the central takeaway overstates the evidence.
  4. [Sections 3.2-3.6] The generalization claims are far broader than the experimental coverage. Only one MoE architecture (Moonlight) and two mathematics datasets are used, yet the paper makes architectural claims about 'attention mechanisms,' 'core reasoning,' and 'instruction following' without probing those functions directly. The recommendation in conclusion ❼ that 'the introduction of controllable data errors during training and inference can serve as an effective optimization strategy' goes beyond the experiments, which never train with noise and never test non-math tasks. At minimum, the authors should either add at least one additional MoE model and a non-mathematical reasoning task, or explicitly reframe the conclusions as observations about Moonlight on math benchmarks.
minor comments (5)
  1. [Section 3.2.2 and Table 4] There is an inconsistency in the identity of the most frequently activated expert. The text identifies 'expert 12' in the first layer, but the formulas and Table 4 refer to 'expert-26' and 'Layer1 (Expert-26)'. Please correct the numbering so the perturbed expert is unambiguous.
  2. [Table 7] The row label 'Group3 (L17-L126)' appears to contain a typo; it should likely read 'Group3 (L17-L26)'. Also, the meaning of '/' in the table should be stated explicitly in the caption (the text explains it, but the caption alone is ambiguous).
  3. [Tables 3, 4, and 5] The baseline ICA differs across tables for the same model and dataset: Table 3 reports 0.86, Table 4 reports 0.86, but Table 5 reports 0.85. If these are separate evaluation runs, this should be stated; if not, the inconsistency should be fixed.
  4. [Notation, Section 3.1] The notation 'N~(0, e_hat)' is ambiguous: e_hat is described as an error bound, but used as a standard deviation. Please write N(0, e_hat^2) if e_hat is the standard deviation, or define the parameter clearly.
  5. [Table 1 and Figure 3] Table 1 lacks source columns for the quantization methods and some entries are difficult to interpret (e.g., '4.98%↑' and negative speedup signs). Figure 3 would benefit from a colorbar label and a caption explaining the aggregation over questions.

Circularity Check

0 steps flagged

No significant circularity: the sensitivity results are empirical measurements under an explicit synthetic error model, not derivations that reduce to their own inputs.

full rationale

The paper's central claim—that shallow-layer experts tolerate bounded errors, middle-layer experts are sensitive, and deep-layer experts can benefit—is obtained by injecting synthetic Gaussian perturbations into expert weights and measuring downstream accuracy. This is an empirical sensitivity study, not a derivation whose conclusion is equivalent to its inputs. The error magnitudes e_hat are set as fractions of each expert's mean absolute weight before inference; they are not fitted to the accuracy outcomes, and no quantity called a 'prediction' is statistically forced by a fitted parameter. The only circularity-adjacent concern is the paper's assertion that SZ3/CuSZp compression errors follow N(0, e_hat) (Section 3.1, Section 3.2.1). That is an unvalidated surrogate-model assumption: real error-bounded compressors guarantee maximum pointwise errors and produce structured, tensor-dependent reconstruction errors, not independent unbounded Gaussian noise. However, an invalid or unvalidated modeling assumption is a correctness/validity threat, not a circular reduction. The paper does not define its sensitivity conclusion in terms of that assumption, nor does it fit the assumption to the measured accuracies. The citations to SZ3, CuSZp, and prior error-bounded compression work are background support for the existence and properties of compressors; they are not load-bearing self-citations that supply the paper's central result. No equation is reused as both assumption and conclusion, and no fitted value is renamed as a prediction. Therefore, under the hard rule requiring a quoted reduction to the paper's own inputs, no circular step is exhibited.

Axiom & Free-Parameter Ledger

1 free parameters · 3 axioms · 0 invented entities

No invented entities. The paper's central claim rests on the hand-chosen error scale and on two unvalidated assumptions: Gaussian compression error and equivalence of weight noise to compression. These assumptions are not benchmarked against actual SZ3/CuSZp output.

free parameters (1)
  • Gaussian noise scale e_hat = p * mean(|theta|) = p in {0.3, 0.5, 0.8} (and 0.1 for expert-0)
    Chosen by the authors as a proxy for compression error amplitude; not derived from actual SZ3/CuSZp error behavior and not fitted to compressor output.
axioms (3)
  • domain assumption Compression error of SZ3/CuSZp can be modeled as i.i.d. Gaussian noise with standard deviation equal to the error bound e_hat.
    Stated in Section 3.1: 'To simulate the compression errors... we randomly generated n errors which follows the normal distributions N(0, e_hat)'. SZ3/CuSZp are deterministic error-bounded compressors; their error distribution is not established as Gaussian.
  • domain assumption Injecting noise into expert weights before inference is equivalent to compressing and decompressing those experts during offloading.
    The methodology replaces actual compression with weight perturbation, implicitly assuming equivalence. Compression also affects data layout, decompression latency, and possibly the router inputs; none of these are modeled.
  • domain assumption Activation frequency identifies the functional role and importance of an expert.
    Section 3.1 hypothesizes that frequently activated experts matter more; the paper uses this to select experts for perturbation, but does not validate the hypothesis independently.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of MoE-Compression: How the Compression Error of Experts Affects the Inference Accuracy of MoE Model?." pith.science (2026). https://pith.science/paper/KXGEKNEU

@misc{pith2026250907727,
  author       = {Pith},
  title        = {Pith review of: MoE-Compression: How the Compression Error of Experts Affects the Inference Accuracy of MoE Model?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KXGEKNEU}},
  note         = {Machine review of arXiv:2509.07727}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

With the widespread application of Mixture of Experts (MoE) reasoning models in the field of LLM learning, efficiently serving MoE models under limited GPU memory constraints has emerged as a significant challenge. Offloading the non-activated experts to main memory has been identified as an efficient approach to address such a problem, while it brings the challenges of transferring the expert between the GPU memory and main memory. We need to explore an efficient approach to compress the expert and analyze how the compression error affects the inference performance. To bridge this gap, we propose employing error-bounded lossy compression algorithms (such as SZ3 and CuSZp) to compress non-activated experts, thereby reducing data transfer overhead during MoE inference. We conduct extensive experiments across various benchmarks and present a comprehensive analysis of how compression-induced errors in different experts affect overall inference accuracy. The results indicate that experts in the shallow layers, which are primarily responsible for the attention mechanism and the transformation of input tokens into vector representations, exhibit minimal degradation in inference accuracy when subjected to bounded errors. In contrast, errors in the middle-layer experts, which are central to model reasoning, significantly impair inference accuracy. Interestingly, introducing bounded errors in the deep-layer experts, which are mainly responsible for instruction following and output integration, can sometimes lead to improvements in inference accuracy.

Figures

Figures reproduced from arXiv: 2509.07727 by Benben Liu, Dan Wang, Sheng Di, Songkai Ma, Xiaodong Yu, Xiaoyi Lu, Zhaorui Zhang.

Figure 1
Figure 1. Figure 1: The activation frequency for the first layer. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The activation frequency for layer-26. Activation Frequency Heatmap for the Experts. To pro￾vide a more intuitive visualization, we present in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Comparison of aggregated expert layers 3.2 Error Sensitivity Analysis for a Single Expert in Different Layers To investigate the impact of parameter errors on model inference accuracy, we conducted a series of experiments to evaluate the error sensitivity of different experts and provide a comprehensive analysis of the experimental results. 3.2.1 Error Sensitivity Analysis for the Expert in the First Layer… view at source ↗
Figure 5
Figure 5. Figure 5: Heat map of activation frequency of experts for the [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. SplitFT: An Adaptive Federated Split Learning System For LLMs Fine-Tuning

    cs.DC 2026-04 unverdicted novelty 5.0

    SplitFT adapts cut-layer selection and reduces LoRA rank per client in federated split learning to improve efficiency and performance when fine-tuning LLMs on heterogeneous devices and data.

Reference graph

Works this paper leans on

45 extracted references · 17 canonical work pages · cited by 1 Pith paper · 2 internal anchors

  1. [1]

    Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J Hewett, Mojan Javaheripi, Piero Kauff- mann, et al. 2024. Phi-4 technical report.arXiv preprint arXiv:2412.08905(2024)

    Pith/arXiv arXiv 2024

  2. [2]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report.arXiv preprint arXiv:2303.08774 (2023)

    Pith/arXiv arXiv 2023

  3. [3]

    Shiyi Cao, Shu Liu, Tyler Griggs, Peter Schafhalter, Xiaoxuan Liu, Ying Sheng, Joseph E Gonzalez, Matei Zaharia, and Ion Stoica. 2025. Moe-lightning: High- throughput moe inference on memory-constrained gpus. InProceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1. 715–730

    2025

  4. [4]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168(2021)

    Pith/arXiv arXiv 2021

  5. [5]

    Sheng Di, Jinyang Liu, Kai Zhao, Xin Liang, Robert Underwood, Zhaorui Zhang, Milan Shah, Yafan Huang, Jiajun Huang, Xiaodong Yu, et al. 2025. A survey on error-bounded lossy compression for scientific datasets.ACM computing surveys 57, 11 (2025), 1–38

    work page 2025

  6. [6]

    Haojie Duanmu, Xiuhong Li, Zhihang Yuan, Size Zheng, Jiangfei Duan, Xingcheng Zhang, and Dahua Lin. 2025. MxMoE: Mixed-precision Quanti- zation for MoE with Accuracy and Performance Co-Design.arXiv preprint arXiv:2505.05799(2025)

    Pith/arXiv arXiv 2025

  7. [7]

    Artyom Eliseev and Denis Mazur. 2023. Fast inference of mixture-of-experts language models with offloading.arXiv preprint arXiv:2312.17238(2023)

    Pith/arXiv arXiv 2023

  8. [8]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al . 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948(2025)

    Pith/arXiv arXiv 2025

  9. [9]

    Xin He, Shunkang Zhang, Yuxin Wang, Haiyan Yin, Zihao Zeng, Shaohuai Shi, Zhenheng Tang, Xiaowen Chu, Ivor Tsang, and Ong Yew Soon. 2024. Expertflow: Optimized expert activation and token allocation for efficient mixture-of-experts inference.arXiv preprint arXiv:2410.17954(2024)

    arXiv 2024

  10. [10]

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset.arXiv preprint arXiv:2103.03874(2021)

    Pith/arXiv arXiv 2021

  11. [11]

    Xing Hu, Zhixuan Chen, Dawei Yang, Zukang Xu, Chen Xu, Zhihang Yuan, Sifan Zhou, and Jiangyong Yu. 2025. MoEQuant: Enhancing Quantization for Mixture- of-Experts Large Language Models via Expert-Balanced Sampling and Affinity Guidance.arXiv preprint arXiv:2505.03804(2025)

    Pith/arXiv arXiv 2025

  12. [12]

    Beichen Huang, Yueming Yuan, Zelei Shao, and Minjia Zhang. 2025. MiLo: Efficient Quantized MoE Inference with Mixture of Low-Rank Compensators. arXiv preprint arXiv:2504.02658(2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  13. [13]

    Jiajun Huang, Sheng Di, Xiaodong Yu, Yujia Zhai, Jinyang Liu, Ken Raffenetti, Hui Zhou, Kai Zhao, Zizhong Chen, Franck Cappello, et al. 2023. C-Coll: Intro- ducing error-bounded lossy compression into MPI collectives.arXiv preprint arXiv:2304.03890(2023)

    Pith/arXiv arXiv 2023

  14. [14]

    Jiajun Huang, Sheng Di, Xiaodong Yu, Yujia Zhai, Zhaorui Zhang, Jinyang Liu, Xiaoyi Lu, Ken Raffenetti, Hui Zhou, Kai Zhao, et al. 2024. An optimized error- controlled mpi collective framework integrated with lossy compression. In2024 IEEE International Parallel and Distributed Processing Symposium (IPDPS). IEEE, 752–764

    work page 2024

  15. [15]

    Jiajun Huang, Sheng Di, Xiaodong Yu, Yujia Zhai, Zhaorui Zhang, Jinyang Liu, Xiaoyi Lu, Ken Raffenetti, Hui Zhou, Kai Zhao, et al. 2025. ZCCL: Significantly improving collective communication with error-bounded lossy compression. arXiv preprint arXiv:2502.18554(2025)

    Pith/arXiv arXiv 2025

  16. [16]

    Yafan Huang, Sheng Di, Guanpeng Li, and Franck Cappello. 2024. cuSZp2: A GPU lossy compressor with extreme throughput and optimized compression ratio. InSC24: International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, 1–18

    work page 2024

  17. [17]

    Yafan Huang, Sheng Di, Xiaodong Yu, Guanpeng Li, and Franck Cappello. 2023. cuszp: An ultra-fast gpu error-bounded lossy compression framework with opti- mized end-to-end performance. InProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. 1–13

    work page 2023

  18. [18]

    Ranggi Hwang, Jianyu Wei, Shijie Cao, Changho Hwang, Xiaohu Tang, Ting Cao, and Mao Yang. 2024. Pre-gated moe: An algorithm-system co-design for fast and scalable mixture-of-expert inference. In2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA). IEEE, 1018–1031

    2024

  19. [19]

    Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al . 2024. Mixtral of experts.arXiv preprint arXiv:2401.04088(2024)

    Pith/arXiv arXiv 2024

  20. [20]

    Rui Kong, Yuanchun Li, Qingtian Feng, Weijun Wang, Xiaozhou Ye, Ye Ouyang, Linghe Kong, and Yunxin Liu. 2023. SwapMoE: Serving off-the-shelf MoE- based large language models with tunable memory budget.arXiv preprint arXiv:2308.15030(2023)

    Pith/arXiv arXiv 2023

  21. [21]

    Xin Liang, Kai Zhao, Sheng Di, Sihuan Li, Robert Underwood, Ali M Gok, Jiannan Tian, Junjing Deng, Jon C Calhoun, Dingwen Tao, et al. 2022. Sz3: A modular framework for composing prediction-based error-bounded lossy compressors. IEEE Transactions on Big Data9, 2 (2022), 485–498

    2022

  22. [22]

    Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, et al . 2024. Deepseek- v2: A strong, economical, and efficient mixture-of-experts language model.arXiv preprint arXiv:2405.04434(2024)

    Pith/arXiv arXiv 2024

  23. [23]

    Jingyuan Liu, Jianlin Su, Xingcheng Yao, Zhejun Jiang, Guokun Lai, Yulun Du, Yidao Qin, Weixin Xu, Enzhe Lu, Junjie Yan, et al. 2025. Muon is scalable for LLM training.arXiv preprint arXiv:2502.16982(2025)

    Pith/arXiv arXiv 2025

  24. [24]

    Jiacheng Liu, Peng Tang, Wenfeng Wang, Yuhang Ren, Xiaofeng Hou, Pheng- Ann Heng, Minyi Guo, and Chao Li. 2024. A survey on inference optimization techniques for mixture of experts models.arXiv preprint arXiv:2412.14219(2024)

    Pith/arXiv arXiv 2024

  25. [25]

    Qianli Liu, Zhaorui Zhang, Xin Yao, and Benben Liu. 2025. HLoRA: Efficient federated learning system for LLM heterogeneous fine-tuning.arXiv preprint arXiv:2503.00813(2025)

    Pith/arXiv arXiv 2025

  26. [26]

    Yuanjian Liu, Sheng Di, Jiajun Huang, Zhaorui Zhang, Kyle Chard, and Ian Foster

    work page

  27. [27]

    Xudong Lu, Qi Liu, Yuhui Xu, Aojun Zhou, Siyuan Huang, Bo Zhang, Junchi Yan, and Hongsheng Li. 2024. Not all experts are equal: Efficient expert prun- ing and skipping for mixture-of-experts large language models.arXiv preprint arXiv:2402.14800(2024)

    Pith/arXiv arXiv 2024

  28. [28]

    Rishov Sarkar, Hanxue Liang, Zhiwen Fan, Zhangyang Wang, and Cong Hao

  29. [29]

    Jiho Shin, Hoeseok Yang, and Youngmin Yi. 2025. SparseInfer: Training-free Pre- diction of Activation Sparsity for Fast LLM Inference. In2025 Design, Automation & Test in Europe Conference (DATE). IEEE, 1–7

    work page 2025

  30. [30]

    Xiaoniu Song, Zihang Zhong, Rong Chen, and Haibo Chen. 2024. Promoe: Fast moe-based llm serving using proactive caching.arXiv preprint arXiv:2410.22134 (2024)

    Pith/arXiv arXiv 2024

  31. [31]

    Peng Tang, Jiacheng Liu, Xiaofeng Hou, Yifei Pu, Jing Wang, Pheng-Ann Heng, Chao Li, and Minyi Guo. 2024. Hobbit: A mixed precision expert offloading system for fast moe inference.arXiv preprint arXiv:2411.01433(2024)

    Pith/arXiv arXiv 2024

  32. [32]

    Qwen Team. 2024. Qwen2 technical report.arXiv preprint arXiv:2407.10671 (2024)

    Pith/arXiv arXiv 2024

  33. [33]

    Dan Wang, Boan Liu, Rui Lu, Zhaorui Zhang, and Shuntao Zhu. 2025. StoreLLM: Energy Efficient Large Language Model Inference with Permanently Pre-stored Attention Matrices. InProceedings of the 16th ACM International Conference on Future and Sustainable Energy Systems. 398–406

    work page 2025

  34. [34]

    Zhanhao Xie, Yuexiao Ma, Xiawu Zheng, Fei Chao, Wanchen Sui, Yong Li, Shen Li, and Rongrong Ji. 2025. Automated Fine-Grained Mixture-of-Experts Quantization. InFindings of the Association for Computational Linguistics: ACL 2025. 27024– 27037

    work page 2025

  35. [35]

    Haotian Xu, Zhaorui Zhang, Sheng Di, Benben Liu, Khalid Ayed Alharthi, and Jiannong Cao. 2024. Fedfa: a fully asynchronous training paradigm for federated learning.arXiv preprint arXiv:2404.11015(2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  36. [36]

    Tairan Xu, Leyang Xue, Zhan Lu, Adrian Jackson, and Luo Mai. 2025. MoE-Gen: High-Throughput MoE Inference on a Single GPU with Module-Based Batching. arXiv preprint arXiv:2503.09716(2025)

    Pith/arXiv arXiv 2025

  37. [37]

    Leyang Xue, Yao Fu, Zhan Lu, Luo Mai, and Mahesh Marina. 2024. Moe-infinity: Offloading-efficient moe model serving.arXiv preprint arXiv:2401.14361(2024)

    Pith/arXiv arXiv 2024

  38. [38]

    Ping Zhang, Zhaorui Zhang, Sheng Di, Yao Xin, and Benben Liu. 2025. CLLoRA: An approach to measure the effects of the context length for LLM fine-tuning. arXiv preprint arXiv:2502.18910(2025)

    Pith/arXiv arXiv 2025

  39. [39]

    Zhaorui Zhang, Sheng Di, Benben Liu, Zhuoran Ji, Guanpeng Li, Xiaoyi Lu, Amelie Chi Zhou, Khalid Ayed Alharthi, and Jiannong Cao. 2025. FedEFsz: Fair Cross-Silo Federated Learning System with Error-Bounded Lossy Compression. IEEE Transactions on Parallel and Distributed Systems(2025)

    work page 2025

  40. [40]

    Zhaorui Zhang, Sheng Di, Kai Zhao, Sian Jin, Dingwen Tao, Zhuoran Ji, Benben Liu, Khalid Ayed Alharthi, Jiannong Cao, and Franck Cappello. 2025. FedCSpc: A Cross-Silo Federated Learning System with Error-Bounded Lossy Parameter Compression.IEEE Transactions on Parallel and Distributed Systems(2025)

    work page 2025

  41. [41]

    Zhaorui Zhang, Zhuoran Ji, and Choli Wang. 2022. Momentum-driven adaptive synchronization model for distributed DNN training on HPC clusters.J. Parallel and Distrib. Comput.159 (2022), 65–84

    work page 2022

  42. [42]

    Zhaorui Zhang and Choli Wang. 2021. SaPus: Self-adaptive parameter update strategy for DNN training on Multi-GPU clusters.IEEE Transactions on Parallel and Distributed Systems33, 7 (2021), 1569–1580

    work page 2021

  43. [43]

    Zhaorui Zhang and Choli Wang. 2022. MIPD: An adaptive gradient sparsification framework for distributed DNNs training.IEEE Transactions on Parallel and Distributed Systems33, 11 (2022), 3053–3066

    work page 2022

  44. [2023]

    In2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD)

    Edge-moe: Memory-efficient multi-task vision transformer architecture with task-level sparsity via mixture-of-experts. In2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD). IEEE, 01–09

    work page

  45. [2025]

    Ocelot: An Interactive, Efficient Distributed Compression-As-a-Service Platform With Optimized Data Compression Techniques.IEEE Transactions on Parallel and Distributed Systems(2025)

    work page 2025

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.