Pith. sign in

REVIEW 4 major objections 5 minor 37 references

OrderMoE turns distributed MoE inference from exact remote expert access into similarity-aware local substitution, cutting average latency by about half with quality retention above 99.3%.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 18:49 UTC pith:6JSP7FCC

load-bearing objection Genuinely new edge-MoE idea, but same-group experts aren't guaranteed pairwise-interchangeable, which likely inflates the headline latency/communication gains; still worth refereeing. the 4 major comments →

arxiv 2607.17154 v1 pith:6JSP7FCC submitted 2026-07-19 cs.NI cs.DCcs.LG

OrderMoE: An expert similarity driven distributed edge MoE inference

Xin Yuan , Ning Li , Quan Chen , Wenchao Xu , Athanasios V. Vasilakos , Song Guo , Haijun Zhang This is my paper
classification cs.NI cs.DCcs.LG
keywords mixture-of-expertsedge inferenceexpert similarityrouter logitsdistributed deploymentsubstitute executiontoken trajectorylatency-quality tradeoff
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

OrderMoE tries to establish that the main bottleneck of serving mixture-of-experts (MoE) models across bandwidth-limited edge servers—the need to send each token to the server holding the exact expert the router picked—can be largely avoided. The paper builds on the observation that experts in the same MoE layer are functionally similar, especially in deeper layers, and that this similarity is visible in the router's logits. It groups similar experts, spreads them across servers, and at runtime lets a token use a local similar expert as a substitute when the routed expert is remote, subject to a quality budget. If the argument holds, edge deployments can cut average latency, tail latency, and cross-server traffic by large margins while keeping inference quality loss small and controllable. The paper further claims the choice between local substitution and remote exact execution should be made with a look-ahead over the token's future trajectory, not layer by layer in isolation.

Core claim

OrderMoE's core discovery is that same-layer experts in mixture-of-experts models are interchangeable enough to serve as substitutes for one another, and that this interchangeability is predictable from a cheap signal: the cosine similarity between rows of the layer's routing-logits matrix—the score table the router uses to pick experts for each token. Using calibration tokens, the system computes similarity, groups experts by a layer-dependent threshold (stricter in shallow layers, looser in deep layers), and deploys representatives of each group across heterogeneous edge servers so every server holds a local candidate for most routed experts. At runtime, a budget-aware algorithm chooses, f

What carries the argument

The load-bearing machinery is the expert similarity model plus the trajectory-aware selection rule. Similarity is defined via the row vectors of the router-logits matrix, so each expert's routing response over calibration tokens becomes a vector; cosine similarity between rows defines the feasible substitute set for each target expert. This transforms the problem from 'find the exact expert' to 'find a feasible substitute,' and the paper's deployment disperses same-group experts across servers so that substitutes are usually local. The runtime selection then weighs per-step transmission and computation delay against substitution quality loss—a monotone function of 1 minus cosine similarity,

Load-bearing premise

Everything rests on the assumption that two experts are interchangeable exactly to the degree that their router-logits rows are cosine-similar—substitution damage is treated as monotone in 1 minus cosine similarity and additive across layers and tokens, and the runtime quality budget is enforced against that proxy, not against measured task quality, so a miscalibrated proxy could silently degrade outputs while still looking budget-compliant.

What would settle it

Corrupt the similarity signal: build groups from randomly permuted logits rows while keeping all other machinery and budgets identical, and compare downstream task quality; if quality degradation stays the same, similarity is not carrying the benefit. Directly, for many expert pairs across layers, measure the true task-quality delta of substituting one for the other on a held-out set and check whether that delta is monotone in 1 minus cosine similarity and additive across layers—a single layer where a 'similar' expert causes a large drop would falsify the proxy.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Cross-server token transmission in MoE inference is not an unavoidable cost of sparsity; with similarity-aware dispersal, the majority of tokens (83 percent in the reported runs) can be executed on the server that already holds them.
  • Because exact execution remains a feasible fallback, the method can guarantee its declared per-user substitution quality budget by construction; the only question is how well that budget tracks true task quality.
  • Latency no longer grows with the number of edge servers the way exact-execution placement does; more servers add memory that extends local coverage instead of spreading target experts further away.
  • A modest look-ahead horizon (three layers in the reported configuration) captures most of the trajectory-aware benefit; larger horizons add decision overhead with little additional latency reduction.
  • The layer-depth dependence of expert similarity is exploitable: deeper layers can be grouped more aggressively, while shallow layers keep stricter similarity thresholds to protect specialized experts.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's quality guarantee is internal to its substitution-quality proxy; an independent test would calibrate the 1-minus-cosine-similarity loss function against measured downstream task degradation per substitution, and if that mapping is not monotone or not additive across layers, real quality could drift while the budget remains satisfied.
  • The same similarity signal could be used during training or fine-tuning—for example, regularizing routers to produce more clustered expert assignments—which would make substitutability a designed property rather than an observed one, potentially improving the latency-quality frontier further.
  • Because the method relies on calibration statistics (similarity, activation frequencies, expert transitions), it inherits a sensitivity to distribution shift; a testable extension would be periodic re-grouping or online recalibration as request patterns drift.
  • The framework's benefits should grow with model scale, since more experts per layer create more grouping and substitution opportunities; the paper observes this on larger models, which suggests it is particularly relevant as open MoE models continue to grow.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes OrderMoE, a distributed edge MoE inference framework that exploits expert similarity to reduce cross-server token transmission. Experts in each MoE layer are grouped by cosine similarity of router-logits row vectors; groups are dispersed across heterogeneous edge servers with importance-aware redundancy. At runtime, a quality-budgeted, look-ahead server-expert selection algorithm decides whether a token should be executed by a local substitute expert or by its remote target expert, while allowing the token to remain on the remote server for subsequent layers. The problem is formulated as a multi-objective optimization with memory, workload, and quality constraints, and is decomposed into an offline grouping/deployment algorithm and an online selection algorithm. Evaluation on an 8-server testbed with Switch-Base-8E, Qwen-MoE-A2.7B, and Mixtral-8x7B reports a 51.0% average latency reduction over EdgeShard and 40.1% over MoE², cross-server communication of 1.5 GB per thousand tokens, and quality retention above 99.3%.

Significance. The central idea is timely and potentially impactful: treating same-layer expert similarity as a deployment primitive can turn a remote exact-expert invocation into a local near-equivalent substitution, which is well matched to bandwidth-limited edge environments. The paper gives a clean problem formulation, a scalable decomposition into offline grouping/deployment and online token-trajectory selection, and a broad evaluation across multiple models, tasks, and ablations. The formal feasibility and complexity properties are useful, and the real-testbed evaluation is a strength. However, the current “quality guarantee” is a property of an uninstantiated proxy loss, and the grouping algorithm does not fully implement the substitutability predicate used in the deployment objective. These gaps directly affect the reported latency and quality claims, so the paper needs concrete fixes before the results can be accepted.

major comments (4)
  1. [III.E, Eq. (18); Sec. V.B, Eqs. (39)-(40)] The substitution quality loss Q_l(r,k) is defined as φ((1-Sim_l(r,k))/2) for k≠r, but φ is only stated to be non-decreasing; no concrete form or calibration procedure is given. Consequently Q_i^max in constraint (c.8) and the per-token budget Q̄_i in Eq. (39) have undefined units, and the online feasible set A_{i,q,l} in Eq. (40) cannot be instantiated. Property 2 guarantees only that the uninstantiated proxy loss stays under budget; this is tautological with respect to the proxy and does not by itself bound measured quality. The empirical quality results in Sec. VI.C are the only grounding for the “small and controllable quality degradation” claim. Please specify φ, define Q_i^max in measurable units, and ideally calibrate the proxy against measured downstream quality (perplexity, F1, GSM8K).
  2. [III.B, Eq. (7); IV.A, Eqs. (24)-(26); IV.C, Eq. (32)] The grouping algorithm assigns each non-dominant expert j to the most similar center c with Sim_l(j,c) ≥ θ_l. Cosine similarity is not transitive, so two non-dominant experts assigned to the same center can have pairwise similarity below θ_l. Thus the group does not implement the feasible-substitute relation of Eq. (7), and the statement in Sec. IV.A that group members “can serve as feasible substitute experts for each other” is not guaranteed by the algorithm. The coverage indicator y_{m,l,g} in Eq. (32) counts any group member as covering the group, and the deployment score (30) and redundancy benefit (33) optimize using this indicator; a server hosting a non-dominant member may not be able to serve a token routed to another member. This can overstate local substitution coverage and inflate the reported latency and communication gains. Please construct groups as cliques or connected co
  3. [IV.A, Eq. (6); VI.A] Eq. (6) gives θ_l = θ_max + (θ_min - θ_max)·(l-1)/(N-1). With the experimental values θ_max=0.5 and θ_min=0.9 in Sec. VI.A, this yields θ_1=0.5 and θ_N=0.9, i.e., shallow layers get the looser threshold and deep layers get the stricter one. This is the opposite of the design described in the text immediately after Eq. (6) and of the layer-awareness argument in Sec. III.B. The formula or the labels need to be corrected; this affects grouping and all downstream experimental results.
  4. [Abstract/Introduction; Sec. VI] The abstract and introduction claim that OrderMoE “significantly reduces … tail latency,” but no tail-latency result appears in the evaluation. Section VI.B reports average latency, throughput, and communication; Sec. VI.I reports SLA satisfaction ratio, not tail latency. Please add p95/p99 latency measurements or remove the tail-latency claim from the summary. This is a load-bearing claim in the paper's own framing.
minor comments (5)
  1. [VI.J, Fig. 12(b)] The narrative is self-contradictory: “falls from 216.1 ms at w=0 to a minimum of 190.3 ms at w=2, then rises to 188.4 ms at w=4.” Since 188.4 is smaller than 190.3, the described U-shape is unclear. Please reconcile the text and figure.
  2. [VI.C/D] The “quality retention” metric in Sec. VI.D is not defined. For WikiText-103, a perplexity increase of 0.3 (Sec. VI.C) does not obviously translate to a 99.3% retention ratio. Please state how retention is computed for each task and model.
  3. [General experimental methodology] The experimental results in Figs. 4-6 are reported without error bars, confidence intervals, or repeated runs. Given the small testbed and the precision of the headline numbers (51.0%, 40.1%, 1.5 GB), some measure of variability or at least multiple seeds/runs is needed to assess the comparisons.
  4. [Notation, Eq. (18)-(21)] The symbol Q_l is used for both the per-layer substitution loss in Eq. (18) and the total user loss in Eq. (20); the notation is confusing. Consider using a lowercase symbol for the per-layer loss.
  5. [V.C, Property 3] The claim that the look-ahead selection is “no worse than the myopic selection in expectation” assumes that calibration transition statistics match the runtime routing distribution and that Φ is an unbiased estimate of future cost. Please state these assumptions explicitly in the property, or soften the formal claim. The current proof does not account for the shrinking feasible set as the per-token quality budget is consumed.

Circularity Check

1 steps flagged

Formal 'quality guarantee' is tautological w.r.t. the proxy-loss budget; core latency/quality measurements remain independent.

specific steps
  1. self definitional [Sec. V.B, Eq. (40); Sec. V.F, Property 2]
    "the runtime feasible candidate set of token q in the l-th MoE layer is defined as: A_{i,q,l} = {(m,k) | x_{m,l,k}=1, e_{l,k} in V_{l,r_l(q)}, Q_acc + Q_l(r_l(q),k) <= Q_bar} (40) ... Property 2 (Quality Guarantee). Algorithm 2 always satisfies the per-user quality constraint (c.8), i.e., Q_i <= Q_max."

    The guaranteed inequality is exactly the membership condition imposed in (40): every selected pair already satisfies Q_acc + Q_l <= Q_bar, so the proof's conclusion Q_i <= Q_max is the same budget inequality summed over layers and tokens. The 'quality' being guaranteed is the proxy loss defined in Eq. (18), not measured downstream quality. Thus Property 2 is enforced by construction rather than an independent bound on inference quality. This does not undermine the empirical quality claim, which is separately measured on WikiText/SQuAD/GSM8K, but the formal guarantee adds no information beyond the algorithm's own constraint.

full rationale

The main derivation chain is not circular: expert similarity is computed from router-logits on calibration data (Eqs. 3-4), grouping follows from the similarity threshold (Eqs. 24-26), deployment is determined by memory/importance/coverage (Eqs. 27-34), and runtime selection is a measured cost minimization over the feasible set (Eqs. 37-43). The reported latency/communication/throughput numbers are testbed measurements, and the quality-retention figures come from external benchmarks, so the central claims have independent content. The only load-bearing step that reduces to its own input is Property 2: it proves the algorithm respects the same per-token budget that the feasible set (40) already enforces, with quality measured as the proxy loss from Eq. (18). This is a formal tautology with respect to the proxy, but it is not the basis of the empirical quality claim. Self-citations [13] and [26]-[28] are background/related-work pointers, not load-bearing arguments. The non-transitivity gap between the grouping assignment (25) and the pairwise substitutability predicate (7) is a soundness/correctness concern about whether coverage (32) overstates feasible substitution, not a circularity, so it is not counted here.

Axiom & Free-Parameter Ledger

8 free parameters · 6 axioms · 0 invented entities

The central claim rests on the empirical premise that router-logit similarity predicts functional substitutability of experts, plus a calibrated-quality proxy that is never concretely instantiated. The system depends on seven-plus hyperparameters (thresholds, weights, horizon, memory budget) that are either hand-set or left unspecified, so the contribution is an algorithmic framework whose quantitative gains are not reproducible from the paper alone.

free parameters (8)
  • similarity threshold bounds θ_min, θ_max = 0.9, 0.5
    Layer-aware threshold in Eq. (6) interpolates between 0.9 (shallow) and 0.5 (deep); values chosen by hand in Sec. VI.A and shown to shift the latency-quality tradeoff in Fig. 8.
  • look-ahead horizon H = 3 (w=2)
    Selected after observing that H=3 is the knee of the latency-vs-overhead curve in Sec. VI.E and VI.J.
  • memory budget ratio = 2.0
    Total deployed memory over single-model-replica memory; set in Sec. VI.A and evaluated in Sec. VI.G.
  • importance weights α1, α2 = unspecified
    Eq. (28) combines activation frequency and representativeness; no values given.
  • deployment score weights λ1, λ2 = unspecified
    Eq. (30) balances memory load and same-group dispersion; values not stated.
  • quality loss function φ(·) = undefined
    Eq. (18) defines the substitution quality loss through a non-decreasing φ that is never concretely specified; therefore Q_i_max and the per-token budget (39) are not operational.
  • per-user quality budget Q_i_max = unspecified
    Constraint (c.8) requires Q_i≤Q_i_max, but Q_i_max is not set in the experimental setup.
  • weight adaptation step size κ = unspecified
    Eq. (45) updates ω1, ω2 with step κ; no value given.
axioms (6)
  • domain assumption Cosine similarity between router-logits row vectors of same-layer experts captures functional substitutability.
    Introduced in Sec. III.B (Eqs. 3-4) and used to build feasible substitute sets (7-8); relies on [15],[16]; not proven for general MoE models, only empirically supported on three benchmarks.
  • domain assumption Experts in deeper MoE layers are more similar than in shallow layers.
    Used in Eq. (6) to make the grouping threshold layer-aware; cited to [16], not re-established.
  • domain assumption Routers and non-expert modules are replicated on every edge server, so routing decisions are available locally.
    Stated in Sec. III.C; if false, additional communication would be needed for routing, altering the delay model.
  • domain assumption Calibration set statistics (logits, activation frequency, expert transition probabilities) are representative of runtime token distribution.
    Similarity (4), activation frequency (5), transition probability (41) are all computed on a held-out calibration set; the quality guarantee (Property 3) assumes the empirical transition distribution matches runtime.
  • domain assumption Transmission and computation delays are known and additive; per-server load constraint (c.9) bounds queueing implicitly.
    Delay model in Eqs. (13)-(16) uses fixed bandwidth/latency and workload/capacity; no queueing model beyond the per-slot capacity bound.
  • domain assumption Substituting an expert only affects the token representation through a scalar quality loss that can be budgeted independently per token.
    The quality model (18)-(21) treats quality loss as additive over layers and tokens; in reality, representation errors can compound across layers, so additivity is a strong assumption.

pith-pipeline@v1.3.0-alltime-deepseek · 24521 in / 13789 out tokens · 117637 ms · 2026-08-01T18:49:21.824094+00:00 · methodology

0 comments
read the original abstract

Although mixture-of-experts, MoE, models have been increasingly adopted to scale large language models with moderate computation cost, it remains challenging to deploy MoE inference over resource-constrained and bandwidth-limited edge infrastructures. Existing distributed MoE serving methods mainly rely on exact expert placement, caching, replication, or communication scheduling, while overlooking the functional similarity among experts, which provides an opportunity to reduce cross-server token transmission. Therefore, this paper introduces a similarity-aware expert allocation and distributed deployment framework, dubbed OrderMoE, which aims to accelerate edge MoE inference while balancing inference latency, communication overhead, server workload, and inference quality. OrderMoE first constructs an expert similarity model based on router-induced logits representations and partitions experts in each MoE layer into multiple similarity groups. Then, it develops a similarity-aware expert grouping and deployment strategy to improve local similarity coverage across edge servers. Since reducing remote expert invocation and preserving exact inference quality are conflicting objectives, OrderMoE further designs a quality-aware and trajectory-aware runtime server-expert selection algorithm to decide whether a token should invoke its remote target expert or use a feasible local substitute expert. Experimental results on a real distributed edge testbed show that OrderMoE significantly reduces average latency, tail latency, cross-server traffic, and remote expert invocation ratio, while introducing only small and controllable inference quality degradation.

Figures

Figures reproduced from arXiv: 2607.17154 by Athanasios V. Vasilakos, Haijun Zhang, Ning Li, Quan Chen, Song Guo, Wenchao Xu, Xin Yuan.

Figure 2
Figure 2. Figure 2: Fig.2 [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Fig.3 [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 4 linked inside Pith

  1. [1]

    Mobile Edge Intelligence for Large Language Models: A Contemporary Survey,

    G. Qu, Q. Chen, W. Wei, Z. Lin, X. Chen, K. Huang, “Mobile Edge Intelligence for Large Language Models: A Contemporary Survey,” IEEE Communications Surveys & Tutorials, vol.27, no.6, 2025, pp: 3820-3860

    2025

  2. [2]

    A Review on Edge Large Language Models: Design, Execution, and Applications,

    Y. Z h e n g , Y. C h e n , B . Q i a n , X . S h i , Y. S h u , J . C h e n , “A Review on Edge Large Language Models: Design, Execution, and Applications,” ACM Computing Surveys, vol.57, no.8, 2025, pp: 1-35

    2025

  3. [3]

    Mixtral of Experts,

    A. Q. Jiang, A. Sablayrolles, A. Roux, A. Mensch, B. Savary, et al, “Mixtral of Experts,” arXiv:2401.04088,

    Pith/arXiv arXiv

  4. [4]

    SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot,

    E. Frantar, D. Alistarh, “SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot,” ICML, Hawaii, USA, 2023, pp: 10323-10337

    2023

  5. [5]

    Survey on Efficient Large Language Models: Principles, Algorithms, Applications, and Open Issues,

    J. Cheng, H. Kang, Y . Shao, N. Li, P. Chen, R. Wang, “Survey on Efficient Large Language Models: Principles, Algorithms, Applications, and Open Issues,” IEEE Transactions on Neural Networks and Learning Systems, vol.37, no. 5, 2026, pp: 2025-2045

    2026

  6. [6]

    SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression,

    X. Wang, Y . Zheng, Z. Wan, M. Zhang, “SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression,” ICLR, Singapore, 2025, pp: 1-21

    2025

  7. [7]

    FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning,

    T. Dao, “FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning,” ICLR, Vienna, Austria, 2024, pp: 1-14

    2024

  8. [8]

    Accelerating Edge Inference for Distributed MoE Models with Latency-Optimized Expert Placement,

    T. Wu, L. Wang, Z. Wen, X. Zhang, X. Chen, J. Duan, X. Zhang, J. Zuo, “Accelerating Edge Inference for Distributed MoE Models with Latency-Optimized Expert Placement,” arXiv:2508.12851, 2025, pp: 1-11

    Pith/arXiv arXiv 2025

  9. [9]

    Optimal Expert Selection for Distributed Mixture-of-Experts at the Wireless Edge,

    S. Qin, H. Wu, H. Du, K. Huang, “Optimal Expert Selection for Distributed Mixture-of-Experts at the Wireless Edge,” arXiv:2503.13421, 2025, pp: 1-13

    Pith/arXiv arXiv 2025

  10. [10]

    Scalable Training of Mixture-of-Experts Models with Megatron Core,

    Z. Yan, H. Bai, D. Liu, T. Liu, H. Liu, P. Li, et al., “Scalable Training of Mixture-of-Experts Models with Megatron Core,” arXiv:2603.07685, 2026, pp: 1-88

    arXiv 2026

  11. [11]

    An Extensible Software Transport Layer for GPU Networking,

    Y. Z h o u , Z . C h e n , M , Z . M a o , C . L a o , S . Ya n g , e t a l ., “An Extensible Software Transport Layer for GPU Networking,” arXiv:2504.17307, 2025, pp: 1-20

    Pith/arXiv arXiv 2025

  12. [13]

    The MoE-Empowered Edge LLMs Deployment: Architecture, Challenges, and Opportunities,

    N. Li, S. Guo, T. Zhang, M. Li, Z. Hong, Q. Zhou, X. Yuan, H. Zhang, “The MoE-Empowered Edge LLMs Deployment: Architecture, Challenges, and Opportunities,” IEEE Communications Magazines, vol.63, no.12, 2025, pp: 164-171

    2025

  13. [14]

    Serving Long-Context LLMs at the Mobile Edge: Test-time Reinforcement Learning based Model Caching and Inference Offloading

    M Xu, D. Niyato, C.G. Brinton, “Serving Long-Context LLMs at the Mobile Edge: Test-time Reinforcement Learning based Model Caching and Inference Offloading” IEEE Transactions on Networking, vol.34, 2026, pp: 3808-3823

    2026

  14. [15]

    Layerwise Recurrent Router for Mixture-of-Experts,

    Z. Qiu, Z. Huang, S. Cheng, Y . Zhou, Z. Wang, I. Titov, J. Fu, “Layerwise Recurrent Router for Mixture-of-Experts,” ICLR, Singapore, 2025, pp. 1-27

    2025

  15. [16]

    Geometric Regularization in MoEs: The Disconnect Between Weights and Activations in MoE Models,

    H. Kim, “Geometric Regularization in MoEs: The Disconnect Between Weights and Activations in MoE Models,” arXiv:2601.00457, 2026, pp: 1-6

    arXiv 2026

  16. [17]

    Distributed inference acceleration with adaptive DNN partitioning and offloading,

    T. Mohammed, C. Joe-Wong, R. Babbar, M. D. Francesco, “Distributed inference acceleration with adaptive DNN partitioning and offloading,” IEEE INFOCOM, Toronto, Canada, 2020, pp: 854-863

    2020

  17. [18]

    PArtNNer: Platform-Agnostic Adaptive Edge-Cloud DNN Partitioning for Minimizing End-to-End Latency,

    S. K. Ghosh, A. Raha, V . Rahunathan, A. Raghunathan, “PArtNNer: Platform-Agnostic Adaptive Edge-Cloud DNN Partitioning for Minimizing End-to-End Latency,” ACM Transactions on Embedded Computing System, vol. 23, no. 1, 2024, pp: 1-38

    2024

  18. [19]

    DNN Partitioning for Cooperative Inference in Edge Intelligence: Modeling, Solutions, Toolchains,

    Y. H a o , N . D i n g , W. X i a , H . G e , L . X u, “DNN Partitioning for Cooperative Inference in Edge Intelligence: Modeling, Solutions, Toolchains,” ACM Computing Surveys, vol.58, no.8, 2026, pp: 1-34

    2026

  19. [20]

    Edge-LLM: A Collaborative Framework for Large Language Model Serving in Edge Computing,

    F. Cai, D. Yuan, Z. Yang, L. Cui, “Edge-LLM: A Collaborative Framework for Large Language Model Serving in Edge Computing,” IEEE ICWS, Shenzhen, China, 2024, pp: 1-11. 17 IEEE TRANSACTIONS ON JOURNAL NAME, MANUSCRIPT ID

    2024

  20. [21]

    EdgeShard: Efficient LLM Inference via Collaborative Edge Computing,

    M. Zhang, X. Shen, J. Cao, Z. Cui, S. Jiang, “EdgeShard: Efficient LLM Inference via Collaborative Edge Computing,” IEEE Internet Things Journal, vol.12, no.10, 2025, pp: 13119-13131

    2025

  21. [22]

    Communication-Efficient Distributed On-Device LLM Inference Over Wireless Networks,

    K. Zhang, H. He, S. Song, J. Zhang, K.B. Lateief, “Communication-Efficient Distributed On-Device LLM Inference Over Wireless Networks,” IEEE Journal of Selected Topics in Signal Processing, vol.19, no.7, 2025, pp: 1301-1317

    2025

  22. [23]

    Jupiter: Fast and Resource-Efficient Collaborative Inference of Generative LLMs on Edge Devices,

    S. Ye, B. Ouyang, L. Zeng, T. Qian, X. Chu, J. Tang, X. Chen, “Jupiter: Fast and Resource-Efficient Collaborative Inference of Generative LLMs on Edge Devices,” IEEE INFOCOM, London, UK, 2025, pp: 1-10

    2025

  23. [24]

    Birds in Cages: Edge Inference Allocation for Distributed LLM Deployment,

    J. Zhu, L. Zhao, F. Xiao, L. Duan, “Birds in Cages: Edge Inference Allocation for Distributed LLM Deployment,” IEEE IWQoS, Gold Coast, Australia, 2025, pp: 1-6

    2025

  24. [25]

    Cloud-Edge-End Collaborative Inference in Mobile Networks: Challenges and Solutions,

    X. Zheng, W. Zhang, C. Hu, L. Zhu, C. Zhang, “Cloud-Edge-End Collaborative Inference in Mobile Networks: Challenges and Solutions,” IEEE Communications Magazine, vol.39, no.4, 2025, pp:90-96

    2025

  25. [26]

    ERA: A QoE-Aware Collaborative Inference Algorithm for NOMA-based Edge Intelligence,

    X. Yuan, N. Li, Q. Chen, W. Xu, S. Guo, “ERA: A QoE-Aware Collaborative Inference Algorithm for NOMA-based Edge Intelligence,” IEEE Transactions on Mobile Computing, vol.25, no.2, 2026, pp: 2303-2319

    2026

  26. [27]

    Mobility and Cost Aware Inference Accelerating Algorithm for Edge Intelligence,

    X. Yuan, N. Li, K. Wei, W. Xu, Q. Chen, H. Chen, S. Guo, “Mobility and Cost Aware Inference Accelerating Algorithm for Edge Intelligence,” IEEE Transactions on Mobile Computing, vol.24, no.3, 2025, pp: 1530 -

    2025

  27. [28]

    High Efficiency Inference Accelerating Algorithm for NOMA-based Edge Intelligence,

    X. Yuan, N. Li, T. Zhang, M. Li, Y. Chen, J. F. M. Ortega, S. Guo, “High Efficiency Inference Accelerating Algorithm for NOMA-based Edge Intelligence,” IEEE Transactions on Wireless Communications, vol.23, no.11, 2024, pp: 17539-17556

    2024

  28. [29]

    AdapMoE: Adaptive Sensitivity-Based Expert Gating and Management for Efficient MoE Inference,

    S. Zhong, L. Liang, Y . Wang, R. Wang, R. Huang, M. Li, “AdapMoE: Adaptive Sensitivity-Based Expert Gating and Management for Efficient MoE Inference,” ACM ICCAD, Munich, Germany, 2024, pp: 1-9

    2024

  29. [30]

    SlimCaching: Edge Caching of Mixture-of-Experts for Distributed Inference,

    Q. Chen, X. Chen, K. Huang, “SlimCaching: Edge Caching of Mixture-of-Experts for Distributed Inference,” IEEE Transactions on Mobile Computing, vol. 25, no. 3, 2026, pp: 10924-10938

    2026

  30. [31]

    Diff-MoE: Efficient Batched MoE Inference with Priority-Driven Differential Expert Caching,

    K. Li, W. Huang, Q. Wang, L. Zheng, X. Liao, H. Jin, J. Xue, “Diff-MoE: Efficient Batched MoE Inference with Priority-Driven Differential Expert Caching,” ACM SC, USA, 2025, pp: 1951-1965

    2025

  31. [32]

    PROBE: Co-Balancing Computation and Communication in MoE Inference via Real-Time Predictive Prefetching,

    Q. Zhu, X. Ye, Y . Liu, H. Ouyang, C. Song, “PROBE: Co-Balancing Computation and Communication in MoE Inference via Real-Time Predictive Prefetching,” arXiv preprint arXiv:2602.00509, 2026, pp: 1-14

    arXiv 2026

  32. [33]

    MegaScale-Infer: Efficient Mixture-of-Experts Model Serving with Disaggregated Expert Parallelism,

    R. Zhu, Z. Jiang, C. Jin, P. Wu, C.A. Stuardo, D. Wang, X. Zhang, H. Zhou, H. Wei, Y . Cheng, “MegaScale-Infer: Efficient Mixture-of-Experts Model Serving with Disaggregated Expert Parallelism,” ACM SIGCOMM, CoimBra, Portugal, 2025, pp: 592-608

    2025

  33. [34]

    N. Xue, Y . Sun, Z. Chen, M. Tao, X. Xu, L. Qian, S. Cui, P. Zhang, “WDMoE: Wireless Distributed Mixture of Experts for Large Language Models,’ IEEE GLOBECOM, Cape Town, South Africa, 2024, pp: 1-6

    2024

  34. [35]

    MoE2: Optimizing Collaborative Inference for Edge Large Language Models,

    L. Jin, Y . Zhang, Y . Li, S. Wang, H.H. Yang, J. Wu, M. Zhang, “MoE2: Optimizing Collaborative Inference for Edge Large Language Models,” IEEE Transactions on Networking, vol.34, 2026, pp: 4637-4652

    2026

  35. [36]

    Breaking the MoE LLM Trilemma: Dynamic Expert Clustering with Structured Compression,

    P. Zhu, N. Yang, B. Tian, J. Wei, W. Zhang, H. Zhang, P. Lv, “Breaking the MoE LLM Trilemma: Dynamic Expert Clustering with Structured Compression,” ICML, Seoul, South Korea, 2026, pp: 1-10

    2026

  36. [37]

    A Survey on Inference Optimization Techniques for Mixture of Experts Models,

    J. Liu, P. Tang, W. Wang, Y . Ren, X. Hou, P.A. Heng, M. Guo, C. Li, “A Survey on Inference Optimization Techniques for Mixture of Experts Models,” ACM Computing Survey, vol. 58, no. 10, 2026, pp: 1-37

    2026

  37. [38]

    Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems,

    X. Miao, G. Oliaro, Z. Zhang, X. Cheng, H. Jin, T. Chen, Z. Jia, “Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems,” ACM Computing Survey, vol. 58, no.1, 2025, pp: 1-37

    2025