Pith. sign in

REVIEW 3 major objections 5 minor 28 references

HD-MoE: Hybrid and Dynamic Parallelism for Mixture-of-Expert LLMs with 3D Near-Memory Processing

T0 review · 3 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that hybrid expert placement—splitting hot experts across nodes and keeping cold experts whole—plus runtime pre-broadcast of predicted overloaded experts reduces MoE inference latency on 3D near-memory processors by 1.1x–1.

desk verdict HD-MoE is a genuinely useful engineering pipeline for MoE inference on 3D NMP, but the LP's communication constraint doesn't match the validated estimator, so the placement results are on shakier ground than the 1.1-1.8x speedups suggest. read the letter →

arxiv 2509.09420 v1 pith:IDNOLVHF submitted 2025-09-11 cs.PF

Haochen Huang , Shuzhang Zhong , Zhe Zhang , Shuangchen Li , Dimin Niu , Hongzhong Zheng , Runsheng Wang , Meng Li This is my paper
classification cs.PF
keywords Mixture-of-Experts3DNear-MemoryProcessingExpertParallelismTensorHybridDynamicSchedulingLoadBalancingLLMInference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that neither Tensor Parallelism nor Expert Parallelism is the right way to run Mixture-of-Experts LLMs on 3D near-memory processors, because TP over-communicates and EP leaves compute imbalanced. It proposes HD-MoE, a two-part strategy: an offline placement search that splits high-frequency experts across nodes (TP-style) and keeps low-frequency experts whole (EP-style) while balancing per-node compute and per-link traffic, plus an online scheduler that predicts which expert will be overloaded next, pre-broadcasts it to all nodes, and routes tokens to the least-loaded node holding that expert. The authors report speedups of 1.1x–1.8x over TP, 1.1x–1.5x over EP, and 1.0x–1.4x over a compute-balanced hybrid across three MoE models and three hardware configurations.

What carries the argument

Node-Link Balance: a two-stage optimizer. Node Balance is a linear program with continuous placement variables P_ic (fraction of expert i on node c) and binary presence flags Z_ic; it minimizes t_comp + 2*gamma*t_comm_hat, where t_comm_hat is a linear node-traffic estimator and gamma is a scalar fitted by regression to the paper's own discrete-event mesh simulator. Link Balance then uses Bayesian optimization to map logical balanced clusters onto physical 2D-mesh nodes, minimizing link congestion. At runtime, a priority score prio_ic = 2 P_ic f_hat_i IS / comp selects the overload candidate, and an alpha-beta communication model yields the optimal pre-broadcast chunk size.

What would settle it

Run the paper's discrete-event simulator (or a cycle-accurate 3D NMP simulator) on the exact placements produced by the Node Balance LP and compare the simulated latency to gamma*t_comm_hat; if the ordering of placements by predicted latency does not match the ordering by simulated latency, the reported node-balance speedups are artifacts of the surrogate.

Watch

Extended reading notes

Core claim

On a 3D NMP accelerator with distributed banks, allowing the expert-to-node assignment to be continuous rather than binary—so an expert can be partly on one node and partly on another—enables a placement that balances compute load and communication volume simultaneously. The optimization runs in two stages: a linear program minimizes per-node compute plus a node-traffic communication surrogate, and then Bayesian optimization searches over physical mappings of the logical clusters to the 2D mesh to reduce link congestion. At runtime, the paper exploits temporal locality in expert activation to pre-broadcast the predicted hottest expert and dispatch tokens to underloaded nodes that already hol

Load-bearing premise

The paper assumes its linear regression from node traffic to communication latency (gamma fitted with R^2=0.96) remains accurate for the unusual, optimizer-chosen placements, even though the LP's communication constraint sums over experts differently than the formula that was validated.

Editorial extensions

If this is right

  • If HD-MoE is right, MoE inference on memory-stacked accelerators no longer forces the TP-versus-EP trade-off; the same expert can be split or replicated depending on measured activation frequency.
  • The offline mapping runs once per model and hardware configuration, so the several-hour search cost is amortized over all subsequent inference queries.
  • The dynamic pre-broadcast step makes inference latency more stable across diverse workloads (math, coding, roleplay) than a static placement.
  • The method's gains are largest when expert activation is skewed and the compute-to-bandwidth ratio is mid-range; at extreme ratios, the advantage narrows.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The LP optimizes a communication objective (a sum over experts per group) that differs from the product-based estimator validated against the simulator; the reported node-balance gains therefore depend on an untested extrapolation that a direct validation could either confirm or overturn.
  • The pre-broadcast idea could transfer to any distributed-memory system with predictable expert co-activation—for example, multi-GPU servers—where the memory overhead of full expert replication is acceptable.
  • A testable extension: replace the gamma-fitted surrogate with the actual discrete-event simulator during optimization for a small model and compare the selected placements; if they differ substantially, the offline mapping is sensitive to surrogate error.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper targets MoE LLM inference on 3D near-memory-processing (NMP) accelerators with distributed memory. It proposes HD-MoE, which combines an offline hybrid Tensor/Expert Parallelism placement with an online dynamic scheduler. The offline stage performs node-level linear programming (LP) to balance compute and communication, followed by Bayesian-optimization-based link-level physical mapping. The online stage predicts expert load, pre-broadcasts the hottest experts, and routes tokens to available copies without extra communication. Using a custom discrete-event simulator and three MoE models (Mixtral, DeepSeek-V2-Lite, Qwen2-57B), the paper reports end-to-end TBT speedups of 1.1x--1.8x over TP, 1.1x--1.5x over EP, and 1.0x--1.4x over a Compute-Balanced TP-EP hybrid, with ablations for each component.

Significance. The problem is timely and relevant: MoE inference on distributed NMP accelerators is an underexplored deployment scenario, and the proposed decomposition into node-level placement and link-level mapping is a sensible approach. If the results are robust, the work would be a useful step toward automated mapping for this emerging class of hardware. The release of code and the attempt to provide a simple analytic communication surrogate are also strengths, as are the ablation experiments and load/link utilization visualizations. However, the core communication objective is internally inconsistent (Eq. 2 vs. Eq. 8), and the surrogate is calibrated on the same simulator used for the headline results. The reported speedups should therefore be treated as provisional until the communication model is corrected and re-validated.

major comments (3)
  1. [IV-B2, Eq. (2); IV-C1, Eq. (8)] The objective minimized by the LP is not the estimator that was validated, and Eq. (2) is inconsistent with the surrounding text. Eq. (2) charges node c for group g via the product \prod_{i in g} ceil(P_ic), which is 1 only if every expert in g has a nonzero placement on c; the text says it should be 1 if any expert in g is placed on c. Eq. (8) instead charges the sum \sum_{i in g} Z_ic, i.e., once per expert. These formulations agree only for singleton groups, whereas the evaluated models have e=2, 6, and 8 experts per token. Thus the LP over-counts co-location of experts within the same routing group and does not minimize the communication cost used for calibration. Since Sec. V-C1 and Fig. 10 attribute the Node-Balance speedup to reduced communication volume, the placement results need to be regenerated with a correct per-group indicator (e.g., a binary OR) and re-validated; the curre
  2. [IV-B2, Eq. (3), Fig. 5] The scalar gamma is fitted by linear regression between the analytic estimator and the same discrete-event simulator that later produces all reported TBT numbers. Consequently, the end-to-end speedups are not an independent test of the communication model; they only show that the simulator can be approximated by a scalar on the training placements. No held-out validation is reported for the hybrid placements actually selected by the LP, which can lie far from the fitted distribution, and a single global scalar cannot capture changes in topology, batch size, and expert co-location. Please report per-scenario gamma values, confidence intervals, and validation on held-out placements; otherwise, the communication reduction attributed to Node Balance in Fig. 10 is a property of the surrogate rather than an established hardware behavior.
  3. [IV-C1, Eqs. (9)--(10)] The statement that constraint (9) prunes the search 'without sacrificing optimality' is not justified. With RCC = t_comp_TP / t_TP_comm (Eq. 10), the right-hand side of (9) bounds a node's compute load by t_comp_TP + t_TP_comm, while the objective in Eq. (12) evaluates TP as t_comp_TP + 2 t_TP_comm. A placement with compute load between t_comp_TP + t_TP_comm and t_comp_TP + 2 t_TP_comm and very low communication could still beat TP, so (9) can exclude the true optimum of the stated objective. This changes the feasible set and is not merely a presentation issue. Please provide a proof of the optimality-preservation claim, or remove the claim and report whether the LP solution changes when the constraint is relaxed.
minor comments (5)
  1. [Eqs. (14)--(18)] The chunk size is denoted c, which conflicts with the node index c used throughout the paper and in Table II. Please rename the chunk size (e.g., s) to avoid ambiguity.
  2. [Figs. 8--9] The panels are labeled 'Normalized TBT' and 'Speedup' without specifying the baseline for normalization/speedup. Please clarify in the captions whether speedup is relative to TP or to each baseline separately.
  3. [IV-B1] Typo: '3D PNM' should be '3D NMP'.
  4. [Table III] The ASTRA-sim comparison table does not list the model parameters (hidden size, batch size, mesh size) used for the ring all-reduce, nor the units of the 'Latency' column. Without these, the analytical and simulated latency values cannot be reproduced.
  5. [V-B] The text says the offline optimization 'typically takes several hours,' but no details are given about the hardware on which the solver runs, the LP solver used, or the Bayesian optimization budget. One sentence on scalability would help readers judge practical applicability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the placement optimizer is evaluated by a simulator rather than by its own fitted surrogate, and self-citations are background only.

full rationale

The central claim (HD-MoE speedups over TP/EP/hybrid baselines) is not definitionally forced by any fitted parameter or self-citation. The offline LP minimizes an analytical objective (Eqs. 5–12) that uses a communication surrogate t_comm = γ·t_hat_comm (Eq. 3), with γ fitted by linear regression to the authors' discrete-event simulator. However, the reported end-to-end TBT speedups in Sec. V-B come from that simulator itself, not from the fitted surrogate, so the final evaluation does not reduce to the fitted γ. The simulator also evaluates the baselines, making the comparison a direct simulation study rather than a re-statement of the LP objective. Self-citations (e.g., [7], [13], [22], [24]) appear as background or motivation and are not load-bearing for the derivation. The skeptical observation that the LP constraint in Eq. 8 uses a sum over experts while the validated estimator in Eq. 2 uses a product is a genuine modeling/validation mismatch, but it is a correctness and robustness concern, not a circular reduction: the optimized quantity is not the validated quantity, but the final results are not obtained by plugging the LP objective back into itself. No equation is defined in terms of the claim it is used to prove, and no fitted parameter is renamed as a prediction. Therefore the paper exhibits no significant circularity.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The central contribution, the placement and scheduling optimizer, rests on a small set of fitted or assumed quantities: gamma calibrates the communication surrogate to the simulator, alpha/beta set broadcast cost, k sets the per-layer broadcast budget, and factors 2 and 4 encode dispatch/gather and FP32 traffic. The paper also assumes layer-to-layer expert predictability, simulator fidelity for irregular all-to-all traffic, and that the Eq. 9 pruning bound never excludes the global optimum. No new physical entities are postulated; pre-broadcast is a scheduling operation on existing experts.

free parameters (6)
  • gamma (communication scaling coefficient) = fit by linear regression; numeric value not stated
    tcomm = gamma * t_comm_hat is fit between the node-traffic estimator and the paper's own discrete-event simulator (Sec. IV-B2, Eq. 3, Fig. 5), then used inside the LP objective and constraints.
  • alpha, beta (broadcast latency/bandwidth coefficients) = treated as known; not measured
    The pre-broadcast chunk-size optimum (Eq. 18) and lower bound (Eq. 17) come from the alpha-beta model; the paper does not state how alpha and beta are obtained for the 3D NMP NoC.
  • pre-broadcast budget k per layer = 2 experts (Fig. 15a), 5 experts (Fig. 15b)
    k is a runtime allowance derived from the previous layer's latency; it is a per-scenario knob that directly scales the dynamic-scheduling benefit.
  • dispatch/gather factor 2 in LP objective = 2
    Eq. 12 doubles tcomm to account for all-to-all dispatch and gather; the symmetric-cost assumption is asserted, not measured.
  • FP32 traffic factor 4 = 4 bytes per element
    Eq. 2 and Eq. 8 charge 4*f_g*B*h per group transfer under an FP32 assumption; lower-precision MoE inference would change volumes.
  • RCC pruning bound in constraint 9 = derived as BW*IS*e/(2D*comp)
    Constraint 9 caps per-node workload at (1/RCC+1)*e/D and is asserted to prune 'without sacrificing optimality'; the ratio mixes bytes and FLOPs and is a heuristic cap.
assumptions (6)
  • domain assumption The discrete-event simulator faithfully models irregular all-to-all latency on a 2D-mesh 3D NMP NoC.
    All communication numbers in the evaluation come from this simulator; it is validated against ASTRA-sim only for ring all-reduce (Table III), not for the irregular all-to-all pattern that motivates the paper.
  • domain assumption Expert activation in layer L+1 is predictable from layer L's routing (gating similarity).
    Motivation section (Opportunity 2, Fig. 3d) asserts prediction accuracy; the dynamic scheduling stage depends on it, but the accuracy values are not reported numerically.
  • domain assumption The linear surrogate tcomm = gamma * t_comm_hat (Eq. 3) extrapolates to placements the LP selects.
    gamma is fit once on simulation points (Fig. 5, R^2=0.96); the optimizer then explores a much larger placement space, including partitioned experts, without re-validation.
  • ad hoc to paper The pruning constraint (Eq. 9) never excludes the global optimum.
    Claimed in Sec. IV-C1 ('without sacrificing optimality') with no proof; the bound (1/RCC+1)*e/D is a heuristic cap on per-node load.
  • domain assumption Expert co-activation affinity (Fig. 3c) is stable enough to inform grouping and co-location.
    Used to justify the group-based formulation in the communication model; the affinity matrix is presented for Qwen2 but no stability analysis is given.
  • domain assumption High-frequency experts can be split across nodes (continuous P_ic) and stitched by all-reduce without breaking the compute-balance model.
    The LP treats P_ic as continuous proportions (Eq. 1, Eq. 6), which is tensor-parallel-style splitting; the extra all-reduce cost of split experts is folded into the surrogate rather than modeled explicitly.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HD-MoE: Hybrid and Dynamic Parallelism for Mixture-of-Expert LLMs with 3D Near-Memory Processing." pith.science (2026). https://pith.science/paper/IDNOLVHF

@misc{pith2026250909420,
  author       = {Pith},
  title        = {Pith review of: HD-MoE: Hybrid and Dynamic Parallelism for Mixture-of-Expert LLMs with 3D Near-Memory Processing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IDNOLVHF}},
  note         = {Machine review of arXiv:2509.09420}
}
read the original abstract

Large Language Models (LLMs) with Mixture-of-Expert (MoE) architectures achieve superior model performance with reduced computation costs, but at the cost of high memory capacity and bandwidth requirements. Near-Memory Processing (NMP) accelerators that stack memory directly on the compute through hybrid bonding have demonstrated high bandwidth with high energy efficiency, becoming a promising architecture for MoE models. However, as NMP accelerators comprise distributed memory and computation, how to map the MoE computation directly determines the LLM inference efficiency. Existing parallel mapping strategies, including Tensor Parallelism (TP) and Expert Parallelism (EP), suffer from either high communication costs or unbalanced computation utilization, leading to inferior efficiency. The dynamic routing mechanism of MoE LLMs further aggravates the efficiency challenges. Therefore, in this paper, we propose HD-MoE to automatically optimize the MoE parallel computation across an NMP accelerator. HD-MoE features an offline automatic hybrid parallel mapping algorithm and an online dynamic scheduling strategy to reduce the communication costs while maximizing the computation utilization. With extensive experimental results, we demonstrate that HD-MoE achieves a speedup ranging from 1.1x to 1.8x over TP, 1.1x to 1.5x over EP, and 1.0x to 1.4x over the baseline Hybrid TP-EP with Compute-Balanced parallelism strategies.

Figures

Figures reproduced from arXiv: 2509.09420 by the authors.

Figure 1
Figure 1. Expert Deployment and Computation Timeline in (a) Tensor [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. MoE structure and two parallel strategies [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. (a) Expert Activation Frequency, (b) Expert Activation Overlap [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (8 more)
Figure 5
Figure 5. Figure 5: The fitting results illustrating the relationship between [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]
Figure 6
Figure 6. Figure 6: (a) TP: balanced computation but heavy communication; (b) [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 7
Figure 7. Figure 7: (a) Static deployment, (b) Computation load detection, (c) [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: End-to-end Speedup for Different Hardware Configurations [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: End-to-end Speedup for Different Mesh Shapes [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 14
Figure 14. Figure 14: Visualization of Link-Level Resource Utilization With and [PITH_FULL_IMAGE:figures/full_fig_p008_14.png]
Figure 11
Figure 11. Figure 11: Node Balancing Speedup in Computation for DeepSeekMoE [PITH_FULL_IMAGE:figures/full_fig_p008_11.png]
Figure 13
Figure 13. Figure 13: Link Balancing Speedup for DeepSeekMoE improving both computation and communication efficiency [PITH_FULL_IMAGE:figures/full_fig_p008_13.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

28 extracted references · 14 linked inside Pith

  1. [1]

    Deepseek-v3 technical report,

    A. Liu, B. Feng, B. Xue, B. Wang, B. Wu, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruanet al., “Deepseek-v3 technical report,”arXiv preprint arXiv:2412.19437, 2024

    arXiv 2024

  2. [2]

    Mixtral of experts,

    A. Q. Jiang, A. Sablayrolles, A. Roux, A. Mensch, B. Savary, C. Bamford, D. S. Chaplot, D. d. l. Casas, E. B. Hanna, F. Bressandet al., “Mixtral of experts,”arXiv preprint arXiv:2401.04088, 2024

    arXiv 2024

  3. [3]

    1.1 computing’s energy problem (and what we can do about it),

    M. Horowitz, “1.1 computing’s energy problem (and what we can do about it),” in2014 IEEE international solid-state circuits conference digest of technical papers (ISSCC). IEEE, 2014, pp. 10–14

    2014

  4. [4]

    Near- memory computing on fpgas with 3d-stacked memories: Ap- plications, architectures, and optimizations,

    V . Iskandar, M. A. A. E. Ghany, and D. Goehringer, “Near- memory computing on fpgas with 3d-stacked memories: Ap- plications, architectures, and optimizations,”ACM Transactions on Reconfigurable Technology and Systems, vol. 16, no. 1, pp. 1–32, 2022

    2022

  5. [5]

    Megatron-lm: Training multi-billion param- eter language models using model parallelism,

    M. Shoeybi, M. Patwary, R. Puri, P. LeGresley, J. Casper, and B. Catanzaro, “Megatron-lm: Training multi-billion param- eter language models using model parallelism,”arXiv preprint arXiv:1909.08053, 2019

    arXiv 1909

  6. [6]

    Gshard: Scaling giant models with conditional computation and automatic sharding,

    D. Lepikhin, H. Lee, Y . Xu, D. Chen, O. Firat, Y . Huang, M. Krikun, N. Shazeer, and Z. Chen, “Gshard: Scaling giant models with conditional computation and automatic sharding,” arXiv preprint arXiv:2006.16668, 2020

    arXiv 2006

  7. [7]

    Adapmoe: Adaptive sensitivity-based expert gating and man- agement for efficient moe inference,

    S. Zhong, L. Liang, Y . Wang, R. Wang, R. Huang, and M. Li, “Adapmoe: Adaptive sensitivity-based expert gating and man- agement for efficient moe inference,” inProceedings of the 43rd IEEE/ACM International Conference on Computer-Aided Design, 2024, pp. 1–9

    2024

  8. [8]

    Task scheduling for efficient inference of large language models on single moderate gpu systems,

    W. Lin, X. Pan, S. Shi, X. Wang, and X. Chu, “Task scheduling for efficient inference of large language models on single moderate gpu systems,”arXiv preprint arXiv:2411.15715, 2024

    arXiv 2024

Show all 28 references
  1. [9]

    Daop: Data-aware offloading and predictive pre-calculation for efficient moe in- ference,

    Y . Zhang, S. Aggarwal, and T. Mitra, “Daop: Data-aware offloading and predictive pre-calculation for efficient moe in- ference,”arXiv preprint arXiv:2501.10375, 2024

    2024 arXiv

  2. [10]

    Hobbit: A mixed precision expert offloading system for fast moe inference,

    P. Tang, J. Liu, X. Hou, Y . Pu, J. Wang, P.-A. Heng, C. Li, and M. Guo, “Hobbit: A mixed precision expert offloading system for fast moe inference,”arXiv preprint arXiv:2411.01433, 2024

    2024 arXiv

  3. [11]

    Qwen2. 5 technical report,

    A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Weiet al., “Qwen2. 5 technical report,” arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  4. [12]

    Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model,

    A. Liu, B. Feng, B. Wang, B. Wang, B. Liu, C. Zhao, C. Dengr, C. Ruan, D. Dai, D. Guoet al., “Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model,” arXiv preprint arXiv:2405.04434, 2024

    2024 arXiv

  5. [13]

    Hybrimoe: Hybrid cpu-gpu scheduling and cache management for efficient moe inference,

    S. Zhong, Y . Sun, L. Liang, R. Wang, R. Huang, and M. Li, “Hybrimoe: Hybrid cpu-gpu scheduling and cache management for efficient moe inference,”arXiv preprint arXiv:2504.05897, 2025

    2025 arXiv

  6. [14]

    Eps-moe: Expert pipeline scheduler for cost-efficient moe inference,

    Y . Qian, F. Li, X. Ji, X. Zhao, J. Tan, K. Zhang, and X. Cai, “Eps-moe: Expert pipeline scheduler for cost-efficient moe inference,” 2025. [Online]. Available: https://arxiv.org/abs/2410.12247

    2025 arXiv

  7. [15]

    Moetuner: Optimized mixture of expert serving with balanced expert placement and token routing,

    S. Go and D. Mahajan, “Moetuner: Optimized mixture of expert serving with balanced expert placement and token routing,” arXiv preprint arXiv:2502.06643, 2025

    2025 arXiv

  8. [16]

    Shortcut-connected expert parallelism for accelerating mixture- of-experts,

    W. Cai, J. Jiang, L. Qin, J. Cui, S. Kim, and J. Huang, “Shortcut-connected expert parallelism for accelerating mixture- of-experts,” 2024. [Online]. Available: https://arxiv.org/abs/ 2404.05019

    2024 arXiv

  9. [17]

    Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity,

    W. Fedus, B. Zoph, and N. Shazeer, “Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity,”Journal of Machine Learning Research, vol. 23, no. 120, pp. 1–39, 2022

    2022

  10. [18]

    Fastermoe: modeling and optimizing training of large-scale dynamic pre-trained models,

    J. He, J. Zhai, T. Antunes, H. Wang, F. Luo, S. Shi, and Q. Li, “Fastermoe: modeling and optimizing training of large-scale dynamic pre-trained models,” inProceedings of the 27th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, 2022, pp. 120–134

    2022

  11. [19]

    Netmoe: Accelerating moe training through dynamic sample placement,

    X. Liu, Y . Wang, F. Fu, X. Miao, S. Zhu, X. Nie, and B. Cui, “Netmoe: Accelerating moe training through dynamic sample placement,” inThe Thirteenth International Conference on Learning Representations

  12. [20]

    Samsung pim/pnm for transfmer based ai: Energy efficiency on pim/pnm cluster,

    J. H. Kim, Y . Ro, J. So, S. Lee, S.-h. Kang, Y . Cho, H. Kim, B. Kim, K. Kim, S. Parket al., “Samsung pim/pnm for transfmer based ai: Energy efficiency on pim/pnm cluster,” in2023 IEEE Hot Chips 35 Symposium (HCS). IEEE Computer Society, 2023, pp. 1–31

    2023

  13. [21]

    A stacked embedded dram array for lpddr4/4x using hybrid bond- ing 3d integration with 34gb/s/1gb 0.88 pj/b logic-to-memory interface,

    B. Fujun, J. Xiping, W. Song, Y . Bing, T. Jie, Z. Fengguo, W. Chunjuan, W. Fan, L. Xiaodong, Y . Guoqinget al., “A stacked embedded dram array for lpddr4/4x using hybrid bond- ing 3d integration with 34gb/s/1gb 0.88 pj/b logic-to-memory interface,” in2020 IEEE International E...

    2020

  14. [22]

    184qps/w 64mb/mm 2 3d logic-to-dram hybrid bonding with process-near-memory engine for recommendation system,

    D. Niu, S. Li, Y . Wang, W. Han, Z. Zhang, Y . Guan, T. Guan, F. Sun, F. Xue, L. Duanet al., “184qps/w 64mb/mm 2 3d logic-to-dram hybrid bonding with process-near-memory engine for recommendation system,” in2022 IEEE International Solid- State Circuits Conference (ISSCC), vol....

    2022

  15. [23]

    Exploiting similarity opportunities of emerging vision ai models on hybrid bonding architecture,

    Z. Yue, H. Wang, J. Fang, J. Deng, G. Lu, F. Tu, R. Guo, Y . Li, Y . Qin, Y . Wanget al., “Exploiting similarity opportunities of emerging vision ai models on hybrid bonding architecture,” in2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA). IEE...

    2024

  16. [24]

    H2-llm: Hardware-dataflow co- exploration for heterogeneous hybrid-bonding-based low-batch llm inference,

    C. Li, Y . Yin, X. Wu, J. Zhu, Z. Gao, D. Niu, Q. Wu, X. Si, Y . Xie, C. Zhanget al., “H2-llm: Hardware-dataflow co- exploration for heterogeneous hybrid-bonding-based low-batch llm inference,” inProceedings of the 52nd Annual International Symposium on Computer Architecture, ...

    2025

  17. [25]

    Qwen2 technical report,

    A. Yang, B. Yang, B. Hui, B. Zheng, B. Yu, C. Zhou, C. Li, C. Li, D. Liu, F. Huanget al., “Qwen2 technical report,”arXiv preprint arXiv:2407.10671, 2024, version: 2024-07. [Online]. Available: https://arxiv.org/abs/2407.10671

    2024 arXiv

  18. [26]

    Collec- tive communication on architectures that support simultaneous communication over multiple links,

    E. Chan, R. Van De Geijn, W. Gropp, and R. Thakur, “Collec- tive communication on architectures that support simultaneous communication over multiple links,” inProceedings of the eleventh ACM SIGPLAN symposium on Principles and practice of parallel programming, 2006, pp. 2–11

    2006

  19. [27]

    Astra- sim: Enabling sw/hw co-design exploration for distributed dl training platforms,

    S. Rashidi, S. Sridharan, S. Srinivasan, and T. Krishna, “Astra- sim: Enabling sw/hw co-design exploration for distributed dl training platforms,” in2020 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS). IEEE, 2020, pp. 81–92

    2020

  20. [28]

    Judging llm-as- a-judge with mt-bench and chatbot arena,

    L. Zheng, W.-L. Chiang, Y . Sheng, S. Zhuang, Z. Wu, Y . Zhuang, Z. Lin, Z. Li, D. Li, E. Xinget al., “Judging llm-as- a-judge with mt-bench and chatbot arena,”Advances in Neural Information Processing Systems, vol. 36, pp. 46 595–46 623, 2023

    2023

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.