Pith. sign in

REVIEW 3 major objections 35 references

HCRMap: Pressure-Aware Hot-Expert Residency Mapping for 3.5D MoE Chiplet Inference

T0 review · 3 major / 0 minor · reviewed 2026-07-14 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read On 3.5D MoE chips, the real bottleneck is where hot expert weights live and which replica gets each token—not just how far tokens travel.

desk verdict Solid 3.5D MoE systems method with a real multi-tier residency + two-timescale design; headline 33–47% gains rest on author-ported baselines inside one simulator. read the letter →

arxiv 2607.11586 v1 pith:6TWXXKDD submitted 2026-07-13 cs.AI

Yongqin Zhang This is my paper
classification cs.AI
keywords Mixture-of-Experts3.5DchipletsruntimemappinghotexpertresidencyhierarchicalmemoryMoEinferencetokenroutingpressure-awarescheduling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Mixture-of-Experts models keep most weights idle and send each token to only a few experts, but a small set of hot experts gets most of the work. On 3.5D multi-chiplet packages with stacked SRAM, local HBM, and shared DRAM, that skew piles pressure on queues, banks, package links, and I/O—not just compute. The paper argues the right control problem is multi-level residency: decide which hot experts deserve scarce near-tier copies, which should sit in intermediate HBM, and which can stay far, then assign each routed token group to a live replica under current pressure. HCRMap does this with a slow, masked residency controller that promotes, retains, demotes, or evicts under capacity and migration limits, plus a fast deterministic mapper that scores queue, path, bank, and streaming cost. On a shared simulator substrate and eight MoE models, the design cuts full-model latency by roughly a third to nearly half versus three reimplemented expert-management baselines in both prefill and decode.

What carries the argument

HCRMap’s two-timescale pressure-aware mapper: a slow masked Double-DQN residency loop that edits replica sets and tier/chiplet/bank placement under hard feasibility masks, and a fast normalized pressure-proxy loop that assigns each routed token group to a legal resident replica.

What would settle it

Re-run the same eight models and routing traces on a real 3.5D multi-chiplet platform, or against original Hydra/MoEntwine/PIMoE cost models, and check whether HCRMap still cuts geometric-mean prefill and decode end-to-end latency by roughly 33–47%.

Watch

Extended reading notes

Core claim

The central claim is that pressure-aware multi-level hot-expert residency mapping—not pure communication minimization or unconditional replication—is what reduces end-to-end 3.5D MoE inference latency. By coupling bounded cross-tier promotion/demotion with live token-to-replica assignment under compute, link, bank, tier, and migration pressure, HCRMap jointly lowers the dominant queueing and memory-service segments of the critical path.

Load-bearing premise

The reported gains assume that reimplemented baselines and the simulator’s model of 3.5D links, banks, and memory tiers are faithful enough that the same latency gaps would appear against the original systems on real hardware.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 0 minor

Summary. The paper proposes HCRMap, a two-timescale pressure-aware residency mapper for MoE inference on 3.5D multi-chiplet systems. Motivated by expert hotness skew that couples compute queues, multi-tier weight streaming (SRAM/HBM/DRAM), bank service, and package-link pressure, it adds a bounded multi-level residency substrate (bank-isolated replica regions, versioned directory, chunked migration, pressure counters) and couples a masked Double-DQN slow loop (promote/retain/demote/evict under feasibility masks) with a deterministic fast token-to-replica mapper under a normalized pressure proxy. On LEGOSim with eight MoE models, under a unified 3.5D substrate, HCRMap reports geometric-mean end-to-end latency reductions of 43.6%/43.0% vs Hydra, 34.5%/33.1% vs MoEntwine, and 46.7%/46.0% vs PIMoE in prefill/decode, with stage attribution, pressure-signal ablations, and capacity/budget sensitivity supporting the mechanism.

Significance. If the reported gains hold under faithful baseline policies and a credible 3.5D cost model, the work is a meaningful systems contribution: it reframes MoE expert management from communication minimization or unconditional replication into multi-level residency control under coupled package pressure, which is a natural fit for 3.5D hierarchical memory. Strengths include a detailed hardware/software model (§4), an explicit constrained formulation (§5), concrete algorithms with legal action masks and versioned migration (§6), multi-model evaluation, stage-level attribution (Fig. 5), pressure ablations (Fig. 6), and capacity/budget sensitivity (Fig. 7). The same-window counterfactual reward for residency training is a reasonable design choice for credit assignment. The main significance risk is external validity of the large headline deltas, which rest entirely on author-reimplemented baselines inside one simulator.

major comments (3)
  1. §2.3 and §7.1: The central claim (Abstract; §7.2) is a 33–47% geometric-mean E2E latency reduction versus Hydra, MoEntwine, and PIMoE on a unified LEGOSim 3.5D substrate. Those systems were designed for different cost models (popularity/distance chiplet placement; wafer-mesh cold-link migration hiding; NPU–PIM throttle-aware offload). The manuscript states that their expert-management policies were reimplemented inside the same LEGOSim event flow, but it does not specify which native mechanisms were preserved, approximated, or disabled under the paper’s SRAM/HBM/DRAM cliff, bank service (Eqs. 9–12), and package-link overlap with dispatch/gather (Eqs. 5–8, 19). Without a porting appendix (feature checklist, intended vs realized cost model, and at least one sanity check that each port recovers the qualitative behavior claimed in the original work on a shared workload), the residual queuein
  2. §6.5–§6.7 and free parameters: Token-assignment weights (ω_q, ω_p, ω_b, ω_s), placement weights (α_p, β_p, χ_p, ψ_p, η_p), reward penalties (μ1, μ2), EWMA α, U_max, A_min, and migration safety η_u are fixed during evaluation but largely unablated. §7.3 ablates pressure *signals* and selected components (no-fast-map, no-HBM), and §7.4 sweeps copy budget and SRAM/HBM capacity, but not the scalar weights that define the fast proxy (Eq. 39) and placement cost (Eq. 36). Because the fast loop is the high-frequency decision path and the paper already shows that removing it costs ~50% latency, the claim that gains come from pressure-aware control needs either (i) a weight-sensitivity study or (ii) an explicit statement that weights were held fixed across all eight models with a single setting, plus that setting disclosed. Otherwise the multi-model consistency may partly reflect tuning freedom ra
  3. §7.1–§7.2 evaluation framing: Two different improvement regimes are reported—matched resident-copy capacity vs a strongest fixed-replica baseline (~25% latency / ~20% EDP) and the unified four-policy comparison (~33–47% vs named systems). The abstract leads with the larger named-system deltas. The paper should (a) define the matched-capacity baseline precisely (how many copies, placement rule, token assignment), (b) report both regimes with equal prominence in the abstract/conclusion, and (c) state clearly that the named-system comparison is policy comparison under a common 3.5D substrate, not a claim that original Hydra/MoEntwine/PIMoE hardware would lose by those margins. Without that framing, readers will over-read the abstract numbers as cross-system hardware results.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: claimed latency gains are event-simulator measurements, not algebraic or definitional reductions of the control objective.

full rationale

HCRMap is a systems paper whose central claims are empirical end-to-end latency and EDP deltas on LEGOSim against reimplemented baselines (Abstract; §7.2). The optimization objective (Eq. 20) minimizes routed MoE completion cost T_MoE_l(M_w), but reported results are full-window Finish_full − Start_full (Eq. 16) and stage attributions from the event graph, not closed-form evaluations of that objective. The slow controller is a masked Double DQN trained offline with a same-window counterfactual reward (Eqs. 31–33) and frozen at evaluation; that is standard simulator-in-the-loop RL, not a fit that forces the headline percentages by construction. Placement and token costs (Eqs. 36, 39) are explicit pressure proxies used only to rank legal actions; physical latency remains simulator-measured. There is no uniqueness theorem, ansatz smuggled via self-citation, or renaming of a known law as a derivation. Concerns about author-ported Hydra/MoEntwine/PIMoE fidelity are external-validity/correctness risks, not circular reductions of inputs to outputs.

Assumptions & free parameters 7 free parameters · 6 assumptions · 3 invented entities

The central latency claims depend on a stack of domain modeling choices and many hand-set control weights rather than a parameter-free derivation. The invented control substrate is the paper’s contribution; free parameters in placement, token cost, reward, and RL training are load-bearing for the reported numbers even though evaluation freezes the policy.

free parameters (7)
  • Token-assignment weights ω_q, ω_p, ω_b, ω_s
    Fixed pressure-proxy weights that rank legal replicas; they directly shape fast mapping and thus measured latency (§6.7).
  • Placement cost weights α_p, β_p, χ_p, ψ_p, η_p
    Hand-chosen coefficients in C_place for source distance, queue, bank, capacity, and diversity (§6.5).
  • Reward penalties μ1, μ2 and discount γ
    Scale migration exposure and churn penalties in the counterfactual reward that trains the slow controller (§6.4).
  • EWMA smoothing α for expert demand
    Controls how quickly hotness tracks bursts vs. persistent experts and thus migration triggers (§6.2).
  • U_max residency updates per window and A_min minimum residency age
    Hard bounds that prevent churn; set to U_max=1 in evaluation (§6.6, §7.5).
  • Near-tier SRAM budget 64 MB and 1.3× resident-copy memory budget
    Hardware/capacity knobs that define the capacity cliff the method is designed around (Intro, Table 1).
  • Migration pressure weight α_mig / β_mig and link safety threshold η_u
    Weight and throttle migration relative to foreground traffic; change when migrations are allowed (§4.4, §6.8).
assumptions (6)
  • domain assumption LEGOSim event-level model of 3.5D compute, NoP/D2D, bank service, and energy is an adequate surrogate for real package latency and EDP.
    All quantitative claims are produced only inside LEGOSim (§7.1).
  • domain assumption Expert hotness skew is persistent across serving windows and can be tracked by EWMA of routed demand.
    Motivation and slow-loop state assume stable hot sets rather than pure one-window noise (§1, §6.2).
  • domain assumption Reimplemented Hydra/MoEntwine/PIMoE policies on the shared 3.5D substrate fairly represent those systems’ expert-management intent under identical constraints.
    Comparative claims in Abstract and §7.2 rest on this equivalence (§2.3).
  • domain assumption Inference expert weights are read-only, so versioned directories without coherence are correct.
    Used to justify non-blocking chunked promotion/demotion (§4.3, §6.8).
  • ad hoc to paper Masked Double DQN with same-window counterfactual reward yields a stable online residency policy when frozen.
    Core control method; not a standard theorem, introduced as the slow loop (§6.3–6.4).
  • standard math Standard discrete-event / ILP-style resource accounting (capacity, bandwidth, critical-path max) applies to the MoE stage graph.
    Latency and capacity equations in §4–5 use conventional max/sum resource models.
invented entities (3)
  • HCRMap two-timescale pressure-aware residency mapper
    purpose: Jointly decide multi-tier expert replica edits and token-to-replica assignment under live 3.5D pressure.
    Primary proposed system; evidence is internal simulation only.
  • Bounded multi-level hot-expert residency substrate (bank-isolated regions, versioned directory, chunked transfer, pressure counters)
    purpose: Make warm experts demotable to HBM instead of far DRAM and support legal masked updates.
    Architectural substrate claimed as contribution 1; not independently measured outside LEGOSim.
  • Runtime pressure state P_w = (P_load, P_link, P_bank, P_tier, P_mig)
    purpose: Summarize compute, interconnect, bank, capacity, and migration pressure for control.
    Paper-defined state vector connecting hardware model to mapper (§4.5).

how reviews work

0 comments
Cite this review

Pith. "Pith review of HCRMap: Pressure-Aware Hot-Expert Residency Mapping for 3.5D MoE Chiplet Inference." pith.science (2026). https://pith.science/paper/6TWXXKDD

@misc{pith2026260711586,
  author       = {Pith},
  title        = {Pith review of: HCRMap: Pressure-Aware Hot-Expert Residency Mapping for 3.5D MoE Chiplet Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6TWXXKDD}},
  note         = {Machine review of arXiv:2607.11586}
}
read the original abstract

Mixture-of-Experts (MoE) large language models (LLM) activate only a small number of experts during inference, but token routing introduces persistent expert hotness skew: a small set of hot experts continuously receives most tokens, while the remaining experts are lightly loaded. On 3.5D multi-chiplet systems, this skew not only causes compute imbalance but also amplifies pressure on communication, memory bandwidth, I/O, and execution queues. Therefore, the core problem is not simply to reduce token movement, but to dynamically place and reuse hot expert replicas across different memory tiers. This paper proposes HCRMap, a hot expert residency mapping framework for pressure-aware expert replica management in 3.5D MoE inference. Based on expert hotness, weight loading cost, migration overhead, and runtime resource pressure, HCRMap dynamically determines which experts should be promoted, retained, demoted, or evicted. It then maps routed token groups to suitable resident replicas, thereby jointly mitigating communication, memory, and queue bottlenecks. Experimental results show that HCRMap reduces end-to-end latency by 43.6% and 43.0% over Hydra in the prefill and decode stages, respectively; by 34.5% and 33.1% over MoEntwine; and by 46.7% and 46.0% over PIMoE.

Figures

Figures reproduced from arXiv: 2607.11586 by the authors.

Figure 1
Figure 1. MoE FFN serving pipeline for selected experts. they can be placed at different residency levels according to runtime hotness, streaming cost, and resource pressure. Existing MoE systems optimize expert execution along separate axes: placement and routing that use popularity, affinity, or co-activation to cut dispatch latency; replica￾tion or shadow replicas that relieve hot-expert queueing; and heterogeneous executi… view at source ↗
Figure 2
Figure 2. Motivation observations from LEGOSim on the shared 3.5D substrate: (a) communication-oriented placement; (b) added resident copies; (c) reduced memory capacity [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Mechanism sketches: (a) residual hot-load con￾centration; (b) replica-enabled load spreading; (c) capacity￾constrained re-concentration. however, the same hot routing pattern simultaneously de￾termines which D2D or NoP path each token takes, which memory tier supplies the expert weights, which memory banks are accessed, and whether shared I/O remains con￾tinuously occupied. Therefore, the hot-expert problem in a 3.5… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: illustrates the 3.5D multi-chiplet substrate used by the model. The figure grounds the abstractions used below: logic tiles form the compute-chiplet set , local SRAM/HBM/DRAM regions form tiered bank sets 𝑡,𝑐 , and mesh links, switch-fabric paths, D2D links, and memo…
Figure 5
Figure 5. Figure 5: Normalized end-to-end latency across eight MoE models: (a) prefill; (b) decode. Stacked bars show stage-level attribution. Lower is better. migration-exposure awareness allows useful copies to be created at the wrong time, making migration traffic visible to foreground…
Figure 6
Figure 6. Figure 6: Pressure-signal ablation normalized to full HCRMap: (a) prefill; (b) decode. Lower is better [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: Sensitivity study: (a) resident-copy budget; (b) SRAM/HBM-capacity scale. Lower is better [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Spatial compute-pressure heatmaps: (a) Hydra; (b) MoEntwine; (c) PIMoE; (d) HCRMap. grid regions, whereas HCRMap produces a smoother and less concentrated pressure distribution. 7.5. Metadata Footprint After the performance, ablation, and sensitivity studies, we quanti…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

35 extracted references · 3 canonical work pages

  1. [1]

    Fedus, B

    W. Fedus, B. Zoph, N. Shazeer, Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity, Journal of Machine Learning Research 23 (120) (2022) 1–39

    2022

  2. [2]

    A. Q. Jiang, et al., Mixtral of experts (2024).arXiv:2401.04088

    arXiv 2024

  3. [3]

    D.Dai,etal.,DeepSeekMoE:Towardsultimateexpertspecialization in mixture-of-experts language models (2024).arXiv:2401.06066

    arXiv 2024

  4. [4]

    Yang, et al., Qwen3 technical report (2025).arXiv:2505.09388

    A. Yang, et al., Qwen3 technical report (2025).arXiv:2505.09388

    arXiv 2025

  5. [5]

    Hwang, W

    C. Hwang, W. Cui, Y. Xiong, Z. Yang, Z. Liu, H. Hu, Z. Wang, R. Salas, J. Jose, P. Ram, J. Chau, P. Cheng, F. Yang, M. Zhou, Y.Xiong,Tutel:Adaptivemixture-of-expertsatscale,in:Proceedings of Machine Learning and Systems, 2023, pp. 1–19

    2023

  6. [6]

    S. He, H. Zhu, J. Zheng, L. Wu, B. Jiao, Q. Liu, X. Zeng, C. Chen, Hydra: Harnessing expert popularity for efficient mixture-of-expert inferenceonchipletsystem,in:ProceedingsoftheDesignAutomation Conference, 2025

    2025

  7. [7]

    Q. Zhu, X. Ye, Y. Liu, H. Ouyang, C. Song, PROBE: Co-balancing computation and communication in MoE inference via real-time predictive prefetching (2026).arXiv:2602.00509

    arXiv 2026

  8. [8]

    X. Wei, C. Jin, T. Dai, Y. Zhong, S. Yu, C. Yang, B. Wu, Z. Zhang, J.Mai,Q.Zhu,Z.Li,Y.Liu,G.Luo,UltraEP:UnleashMoEtraining and inference on rack-scale nodes with near-optimal load balancing (2026).arXiv:2606.04101

    arXiv 2026

Show all 35 references
  1. [9]

    Q. Wang, Z. Zhang, S. Li, H. Zheng, Z. Liang, Y. Lin, R. Wang, R. Huang, LaMoSys3.5D: Enabling 3.5D-IC-based large language model inference serving systems via hardware/software co-design (2025).arXiv:2512.08731. Y. Zhang:Preprint submitted to ElsevierPage 14 of 15 Pressure-Aw...

    2025

  2. [10]

    S. Luo, H. Ye, P. Li, J. Qin, J. Peng, Y. Zhao, Y. Cao, T. Chen, Mozart: Modularized and efficient MoE training on 3.5D wafer-scale chiplet architectures, in: Advances in Neural Information Processing Systems, 2025

    2025

  3. [11]

    Liang, K.-T

    S.Ma,H.Li,W.Zhang,Y.Tan,P.Dong,Y.Liu,L.Liu,Y.Jiao,X.Liu, L. Liang, K.-T. Cheng, Expert streaming: Accelerating low-batch MoE inference via multi-chiplet architecture and dynamic expert trajectory scheduling (2026).arXiv:2603.27624

    2026

  4. [12]

    A.Zhao,Z.Cai,Z.Song,L.Yu,H.Fan,J.Wu,Y.Wang,N.Vijayku- mar, CRAFT: Fine-grained cost-aware expert replication for efficient mixture-of-experts serving (2026).arXiv:2603.28768

    2026 arXiv

  5. [13]

    Y. Han, L. Pan, J. Peng, Z. Tao, H. Zhu, W. Zhang, Y. Zhang, GRACE-MoE: Grouping and replication with locality-aware routing for efficient distributed MoE inference (2025).arXiv:2509.25041

    2025 arXiv

  6. [14]

    H.Yu,X.Cui,H.Zhang,H.Wang,H.Wang,Taminglatency-memory trade-offinmoe-basedllmservingviafine-grainedexpertoffloading, in: Proceedings of the 21st European Conference on Computer Sys- tems, 2026.doi:10.1145/3767295.3769319

    2026 doi

  7. [15]

    K. Li, W. Huang, Q. Wang, L. Zheng, X. Liao, H. Jin, J. Xue, Diff- moe:Efficientbatchedmoeinferencewithpriority-drivendifferential expert caching, in: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2025.doi:10.1145...

    2025 doi

  8. [16]

    T. Kim, K. Choi, Y. Cho, J. Cho, H.-J. Lee, J. Sim, Monde: Mixture of near-data experts for large-scale sparse models, in: Proceedings of the Design Automation Conference, 2024

    2024

  9. [17]

    X.Tang,J.Hou,D.Jiang,T.Wei,J.Liu,J.Deng,H.Wang,Q.Yang, H.Shang,C.Li,Y.Hu,S.Yin,MoEntwine:Unleashingthepotential of wafer-scale chips for large-scale expert parallel inference, in: Pro- ceedingsoftheIEEEInternationalSymposiumonHigh-Performance Computer Architecture, 2026

    2026

  10. [18]

    L. Wu, H. Zhu, S. He, X. Lin, X. Zeng, C. Chen, PIMoE: Towards efficient MoE transformer deployment on NPU-PIM system through throttle-awaretaskoffloading,in:ProceedingsoftheDesignAutoma- tion Conference, 2025

    2025

  11. [19]

    T. Lin, C. Qiu, X. Wang, L. Wang, Z. Zheng, Y. Jiang, A. K. Singh, J.Yin,S.Qiu,X.Li,X.Tang,J.Song,M.Zhang,K.Ren,Legosim:A unifiedparallelsimulationframeworkformulti-chipletheterogeneous integration, in: Proceedings of the 58th IEEE/ACM International SymposiumonMicroarchitectur...

    2025 doi

  12. [20]

    Accessed: 2026-07-05 (2026)

    NVIDIA,ScalingexpertparallelisminTensorRT-LLM:Performance status and optimization, TensorRT-LLM documentation. Accessed: 2026-07-05 (2026). URLhttps://nvidia.github.io/TensorRT-LLM/blogs/tech_blog/blog0 8_Scaling_Expert_Parallelism_in_TensorRT-LLM_part2.html

    2026

  13. [21]

    He, A Time Scaling Theory for Multi-Layer Electronic Systems, ChinaXiv:202605.00224 (May 2026)

    T. He, A Time Scaling Theory for Multi-Layer Electronic Systems, ChinaXiv:202605.00224 (May 2026)

    2026 arXiv

  14. [22]

    525–534.doi:10.1109/ICPADS.2015.72

    P.Petoumenos,L.Mukhanov,Z.Wang,H.Leather,D.S.Nikolopou- los, Power capping: What works, what does not, in: 2015 IEEE 21st International Conference on Parallel and Distributed Systems (ICPADS), 2015, pp. 525–534.doi:10.1109/ICPADS.2015.72

    2015 doi

  15. [23]

    Haidar, H

    A. Haidar, H. Jagode, P. Vaccaro, A. YarKhan, S. Tomov, J. Don- garra, Investigating power capping toward energy-efficient scientific applications,ConcurrencyandComputation:PracticeandExperience (2018).doi:10.1002/cpe.4485

    2018 doi

  16. [24]

    Accessed: 2026-06-30 (2023)

    UniversalChipletInterconnectExpressConsortium,UCIe:Overview and usage models, Hot Chips 2023 Tutorial, includes UCIe data- rate, per-module bandwidth, latency, and energy-efficiency targets. Accessed: 2026-06-30 (2023). URLhttps://hc2023.hotchips.org/assets/program/tutorials/u...

    2023

  17. [25]

    Accessed: 2026-06-30 (2026)

    Universal Chiplet Interconnect Express Consortium, UCIe specifica- tions,specificationportalforUCIerevisionsandsupporteddatarates. Accessed: 2026-06-30 (2026). URLhttps://www.uciexpress.org/specifications

    2026

  18. [26]

    J. Wuu, R. Agarwal, M. Ciraula, C. Dietz, B. Johnson, D. Johnson, R.Schreiber,R.Swaminathan,W.Walker,S.Naffziger,3DV-Cache: The implementation of a hybrid-bonded 64MB stacked cache for a 7nm x86-64 CPU, in: 2022 IEEE International Solid-State Circuits Conference (ISSCC), Vol. ...

    2022

  19. [27]

    Horowitz, 1.1 computing’s energy problem (and what we can do aboutit),in:2014IEEEInternationalSolid-StateCircuitsConference DigestofTechnicalPapers(ISSCC),Vol.57,IEEE,2014,pp.10–14

    M. Horowitz, 1.1 computing’s energy problem (and what we can do aboutit),in:2014IEEEInternationalSolid-StateCircuitsConference DigestofTechnicalPapers(ISSCC),Vol.57,IEEE,2014,pp.10–14. doi:10.1109/ISSCC.2014.6757323

    2014 doi

  20. [28]

    Accessed: 2026-06-30 (2023)

    Advanced Micro Devices, Inc., Alveo U280 Data Center Accelerator CardUserGuide,uG1314,Revision1.1.DocumentstheU280HBM2 subsystem with 8 GB capacity and 460 GB/s bandwidth. Accessed: 2026-06-30 (2023). URLhttps://docs.amd.com/r/en-US/ug1314-alveo-u280-reconfig-a ccel

    2026

  21. [29]

    O’Connor, N

    M. O’Connor, N. Chatterjee, D. Lee, J. Wilson, A. Agrawal, S. W. Keckler, W. J. Dally, Fine-grained DRAM: Energy-efficient DRAM for extreme bandwidth systems, in: Proceedings of the 50th An- nual IEEE/ACM International Symposium on Microarchitecture (MICRO-50),AssociationforCo...

    2017 doi

  22. [30]

    URLhttps://static6.arrow.com/aropdfconversion/fc2b144ccf06116 0504edd742d01cb58fdda91bb/ddr5_sdram_core.pdf

    Micron Technology, Inc., DDR5 SDRAM Core Product Data Sheet, usedforDDR5data-rateandbandwidthparameters.Accessed:2026- 06-30 (2024). URLhttps://static6.arrow.com/aropdfconversion/fc2b144ccf06116 0504edd742d01cb58fdda91bb/ddr5_sdram_core.pdf

    2026

  23. [31]

    URLhttps://www.databricks.com/blog/introducing-dbrx-new-state -art-open-llm

    Databricks, Introducing DBRX: A new state-of-the-art open LLM, accessed: 2026-07-05 (2024). URLhttps://www.databricks.com/blog/introducing-dbrx-new-state -art-open-llm

    2026

  24. [32]

    DeepSeek-AI, DeepSeek-V2: A strong, economical, and efficient mixture-of-experts language model (2024).arXiv:2405.04434

    2024 arXiv

  25. [33]

    DeepSeek-AI, DeepSeek-V3 technical report (2024).arXiv:2412.194 37

    2024

  26. [34]

    URLhttps://mistral.ai/news/mixtral-8x22b

    MistralAI,Cheaper,better,faster,stronger:Mixtral8x22B,accessed: 2026-07-05 (2024). URLhttps://mistral.ai/news/mixtral-8x22b

    2026

  27. [35]

    URLhttps://huggingface.co/Qwen/Qwen1.5-MoE-A2.7B Y

    Qwen Team, Qwen1.5-MoE-A2.7B, accessed: 2026-07-05 (2024). URLhttps://huggingface.co/Qwen/Qwen1.5-MoE-A2.7B Y. Zhang:Preprint submitted to ElsevierPage 15 of 15

    2026

Pith tools

Reviewed July 14, 2026 · model on record in the stance chip above.