Pith. sign in

REVIEW 4 major objections 5 minor 61 references

This paper claims that a Mixture-of-Experts head scattered across wireless clients can be routed by compact statistics and aggregated physically by the channel itself, making MoE fusion latency and bandwidth independent of the number of exp

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 20:34 UTC pith:R7HOY6SB

load-bearing objection Novel mapping of AirComp onto MoE aggregation, but the experiments never simulate the channel, so the headline accuracy claims are unsupported. the 4 major comments →

arxiv 2607.16562 v1 pith:R7HOY6SB submitted 2026-07-18 cs.DC

AirMoE: Statistic-Augmented Over-the-Air MoE for Collaborative Intelligence

Wei-Bin Kou , Jingreng Lei , Guangxu Zhu , Yujiu Yang This is my paper
classification cs.DC
keywords Mixture of Expertsover-the-air computingedge intelligencesemantic segmentationfeature retrieval libraryJensen-Shannon divergencewireless aggregationdistributed inference
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

AirMoE is a method for running a Mixture-of-Experts model across a cloud server and many wirelessly connected edge devices, where the cloud hosts a frozen large backbone and each device hosts a specialized expert. The paper argues that the two classic MoE operations—routing (which experts to use) and aggregation (how to fuse their outputs)—become prohibitive communication bottlenecks in this setting, and that both can be redesigned. Routing is replaced by each client uploading a tiny prototype-derived statistic instead of raw features, and aggregation is performed in one shot by having selected experts transmit simultaneously, letting the wireless channel superimpose their power-scaled outputs into the desired weighted sum. The paper further proves convergence guarantees for the channel-perturbed training and reports that AirMoE outperforms both MoE baselines and single-model competitors on semantic segmentation across four datasets. The central insight is that the analog multiple-access channel itself can compute the MoE fusion, turning a communication bottleneck into a computational resource.

Core claim

The paper's central claim is that over-the-air computing can realize the statistically reweighted MoE fusion in a single transmission. With channel-inversion power control, each activated client transmits its expert output pre-scaled by its aggregation weight divided by its channel coefficient; the receiver then applies one common scaling, and the superposition of waveforms yields exactly the intended weighted sum plus a zero-mean noise term. The paper combines this physical-layer aggregation with a digitized, statistic-based routing mechanism: each client stores a Feature Retrieval Library of prototypes, retrieves a prototype-induced statistic from a cloud-broadcast compact query, reports i

What carries the argument

The key mechanism is the over-the-air aggregation identity: by setting the transmit pre-scaling b_j = sqrt(rho) beta_j / gamma_j, the received signal, after projection and receiver scaling, equals the MoE weighted sum plus zero-mean real-equivalent noise. This turns waveform superposition into the MoE fusion operator, and makes aggregation cost invariant to the number of activated clients. The supporting machinery is the Feature Retrieval Library (FRL)—each client keeps a small set of prototype vectors updated by attention-based read-then-update—and the JS-divergence routing score that selects clients in a shared normalized distribution space, keeping routing digital and low-rate while aggre

Load-bearing premise

The entire aggregation mechanism depends on each client knowing its own uplink channel coefficient exactly, without phase errors or synchronization mismatch, so that its transmitted signal arrives aligned and the received super-position is exactly the intended weighted sum.

What would settle it

A concrete test: run AirMoE with channel estimation errors (e.g., Gaussian perturbation to gamma_j) or with imperfect timing synchronization, and measure the received sum's deviation from the target MoE weighted sum. If the deviation grows beyond the noise floor and downstream segmentation quality drops sharply, the core over-the-air aggregation claim fails under realistic channel conditions.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If AirMoE is correct, a distributed MoE head can be served over bandwidth-limited wireless links with aggregation cost that does not scale with the number of experts, enabling larger expert pools on edge devices.
  • The convergence analysis shows that channel noise inflates gradient variance linearly but imposes a squared floor on achievable stationarity; this makes the communication-computation tradeoff quantitative and suggests power control and receive-SNR floor as primary design levers.
  • The statistic-based routing removes the need to upload raw latent features for routing decisions, which could be applied to other distributed inference tasks beyond segmentation where feature transmission is the bottleneck.
  • The adversarial-condition results (e.g., gains concentrated on low-mIoU datasets) suggest that AirMoE's benefit grows with task difficulty and expert heterogeneity, pointing to its use in diverse, real-world edge deployments.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • An untested but plausible extension is to apply AirMoE's statistic-based routing and over-the-air aggregation to other tasks such as object detection or multi-label classification, where the same weighted-sum fusion structure appears.
  • A testable prediction of the paper's theory is that increasing the output dimension of experts raises the stationarity floor quadratically; this implies that reducing output dimensionality (e.g., via low-rank heads or sparsification) should be prioritized over merely reducing noise to improve over-the-air training accuracy.
  • The paper's reliance on perfect channel state information suggests that an empirical study of AirMoE under realistic channel estimation errors would likely show graceful degradation up to a point, but a hard breakdown once the alignment constraint fails; this is a direction the authors explicitly flag for future work.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes AirMoE, a cloud-edge wireless Mixture-of-Experts framework. A cloud-hosted frozen large model extracts latent features and broadcasts a compact query to clients; each client maintains a Feature Retrieval Library (FRL) of prototypes and reports a normalized prototype-induced statistic. The cloud selects the TopK clients by Jensen–Shannon divergence and computes reciprocal-distance aggregation weights. Selected clients then transmit their outputs simultaneously over a multiple-access channel with channel-inversion power control, so that the channel superposition approximates the weighted MoE fusion in one analog transmission. The authors provide convergence and ε-stationarity analysis under channel noise, with an explicit noise-induced floor, and report semantic segmentation experiments on Cityscapes, CamVid, Apolloscapes, and CARLA ADV, comparing against MoE and single-model baselines.

Significance. If the empirical claims were fully supported, the paper would make a useful contribution: it identifies a natural mapping from MoE weighted aggregation to over-the-air computation, achieves aggregation latency and bandwidth independent of the number of selected experts, and gives an explicit decomposition of channel-noise effects on optimization (variance inflation and an irreducible stationarity floor). The algebraic construction in Eqs. (22)–(23) is clean, and Lemma 1’s decomposition σ_eff² = σ² + κσ_c²/(2ρ_min) is instructive. These are genuine strengths. However, the central experimental validation is currently missing the wireless channel, and the theoretical claims contain some gaps. The significance is therefore conditional on repair of these load-bearing issues.

major comments (4)
  1. [Section VI-A3 and Table II] The experimental setup never specifies the wireless channel. There is no path-loss/fading model, no noise power σ_c², no receiver noise, no per-client power budget P0, no truncation threshold γ_th, no channel-inversion pre-scaling in the simulation, and no CSI error model. Algorithm 1 includes over-the-air steps (lines 10–12), but Table II reports no SNR, noise level, or channel realization. Since Lemma 4 and Fig. 5 state that channel noise creates an irreducible accuracy floor, the reported 44.74 mIoU on Cityscapes, and the claimed gains over SoftMoE/NonLinearMoE, cannot be attributed to the over-the-air aggregation mechanism. The experiments as presented validate only the FRL-based routing and JS weighting; the distinctive AirComp component is untested.
  2. [Section VII and Eq. (22)] The perfect-CSI and ideal-synchronization assumption is acknowledged as a limitation, but it is load-bearing for the main mechanism. Eq. (22) requires b_j = sqrt(ρ)β_j/γ_j with exact knowledge of γ_j and no phase/alignment error. If CSI is imperfect or synchronization is imperfect, the received signal is no longer the intended weighted sum, Eq. (23) fails, and the entire physical-layer aggregation claim collapses. The paper provides no sensitivity analysis with respect to CSI error or synchronization offset, even in simulation. This is not a minor caveat; it is the central mechanism of the system.
  3. [Theorem 2 and Remark 1] Theorem 2 fixes the FRL state M, whereas Algorithm 1 updates M online via Eqs. (11)–(12). Remark 1 states that for η small relative to α_t, M can be treated as quasi-static, but no two-timescale convergence theorem is proved; only boundedness of M is established in Theorem 1. As written, the theory does not cover the algorithm that is implemented. The authors should either provide a genuine two-timescale convergence analysis or explicitly restrict the theorem to the fixed-M setting and soften the claim that it guarantees convergence of AirMoE.
  4. [Lemma 4 and Eq. (37)] The claim that an ε-stationary point of the noiseless objective is reachable iff ε > ε_floor = 2B_σ² is not justified by the displayed argument. The inequality ‖∇eL‖² ≤ 2‖∇Lbar‖² + 2B_σ² is a one-sided upper bound; it implies a sufficient condition for reaching ε-stationarity of eL and the absence of a guarantee below the floor, but it does not prove that no finite T can reach a point with ‖∇eL‖² ≤ ε for ε ≤ ε_floor. The proof in Appendix G would need to establish a matching lower bound on the gradient bias to support the word "iff" and the "irreducible floor" interpretation. Otherwise the statement should be weakened to a worst-case/guarantee-based claim.
minor comments (5)
  1. [Eq. (23)] The derivation implicitly assumes the expert outputs y_j are real-valued, or that taking the real part after receiver scaling projects onto the signal space without loss. This should be stated explicitly.
  2. [Section VI-A3] The comparison protocol is reasonable for MoE baselines (same frozen ViT, same expert architecture, same training protocol), but the single-model competitors are trained from scratch while the MoE models use a pretrained frozen ViT. This makes the MoE-vs-single-model comparison uneven; the claim that MoE-based models are inherently better should be tempered.
  3. [Fig. 5 and Section VI-C] Fig. 5 is presented as the relationship between iteration complexity and channel parameters, but the parameter values (σ_c², ρ_min) are abstract. Since no simulation settings are given, the figure is purely illustrative. This is acceptable, but the caption should say so explicitly.
  4. [References] References [45] and [54] point to the same arXiv preprint (arXiv:2405.14131) and should be merged or distinguished. Several references also lack venue details.
  5. [Algorithm 1] Line 12 writes "Simultaneous transmission ⇒ r^(i)" but nowhere in the algorithm is the channel realization, noise power, or fading model instantiated. This reflects the same missing wireless simulation as in VI-A3.

Circularity Check

0 steps flagged

No significant circularity: the over-the-air aggregation identity is a deliberate transmit-scaling construction, and the convergence analysis is a standard nonconvex SGD argument under explicit assumptions.

full rationale

The paper's central physical claim, Eq. (23), is obtained by substituting the channel-inversion power-control choice Eq. (22) into the received-signal model Eq. (4). This is an algebraic identity that defines b_j so that the channel superposition equals the desired MoE weighted sum; it is an engineering construction, not a prediction fitted from data. The aggregation weights β_j are defined via the reciprocal-JS formula Eq. (21), and the claim that the channel 'computes the statistically reweighted fusion' restates this construction rather than deriving a result from fitted parameters. The convergence analysis (Lemmas 1–4, Theorems 2–3) is a standard nonconvex SGD argument on the noise-smoothed objective Lbar, with bias and variance bounds derived from stated assumptions (bounded states, channel truncation floor, smoothness); no constants are fitted to experimental data, and no convergence 'prediction' is obtained from a parameter fitted to the same experiments. The experimental comparison in Table II uses fixed public benchmarks and standard baselines, so the reported gains are not forced by construction. The manuscript contains several self-citations (e.g., [14], [38]–[41]) in related-work and AirComp contexts, but none is load-bearing for the paper's central claims; the over-the-air mechanism relies on the standard AirComp identity and the convergence proof is self-contained. The skeptical concern that Table II does not report a channel model or SNR is an experimental-validity issue, not a circularity: it means the distinctive over-the-air component may be untested, but it does not make any derivation equivalent to its own inputs by construction. The paper itself acknowledges the idealized analog front-end as a limitation, again a scope caveat rather than a circular step. Overall, no prediction reduces by definition to a fitted input, no uniqueness theorem is imported from the authors' prior work, and no ansatz is smuggled in via self-citation.

Axiom & Free-Parameter Ledger

7 free parameters · 5 axioms · 1 invented entities

The paper's central mechanism relies on several untested or under-specified design choices: the over-the-air channel parameters are not reported in experiments, the convergence proofs are for a smoothed surrogate rather than the actual TopK routing, perfect CSI is assumed, and key hyperparameters are chosen without justification and even outperform the defaults in the paper's own ablations.

free parameters (7)
  • Number of experts N = 10 in main results
    Chosen by hand; ablation Table III shows N=7 gives mIoU 45.45 vs 44.74 for N=10, so the default is not the best found.
  • TopK K = 5 in main results
    Chosen by hand; ablation Table IV shows K=3 gives mIoU 45.85 vs 44.74 for K=5.
  • FRL prototypes per expert K_j = 16 in main results
    Chosen by hand; ablation Table V shows 4 prototypes gives mIoU 45.59 vs 44.74 for 16.
  • Temperature tau and stability epsilon in routing = not specified
    Hand-chosen hyperparameters in Eqs. (15)-(16); values not reported.
  • FRL update rate eta = not specified
    Kept fixed throughout the paper, but value never reported (Section VI-A).
  • Regularization weights lambda_LB, lambda_FRL = 0.01 and 1e-4 in main results
    Ablations Table VI and VII show better values (0.04 and 4e-4) give mIoU 45.44 vs 44.74.
  • Channel noise power, power budget, truncation threshold = not reported in experiments
    The wireless channel parameters central to the over-the-air mechanism (sigma_c, P0, gamma_th) are absent from Section VI.
axioms (5)
  • domain assumption Over-the-air noise is zero-mean, symmetric, and independent with bounded covariance (Assumption 4)
    Assumed to derive the unbiased-gradient and variance-inflation lemmas; reasonable for the model but unverified in experiments.
  • domain assumption Perfect channel state information and ideal synchronization at each client (Eq. 22)
    The channel-inversion power control assumes clients know gamma_j exactly; the authors later call this an idealized front-end limitation.
  • ad hoc to paper The analysis applies to a smooth surrogate or fixed activation set, not the actual TopK routing
    Section V preamble states this; the convergence theorems therefore do not cover the algorithm as described with hard TopK and online FRL updates.
  • ad hoc to paper FRL state M is fixed during convergence analysis (Theorem 2 and Remark 1)
    The paper handwaves the online FRL updates as quasi-static; no proof is provided for the coupled system.
  • domain assumption Smoothness, Lipschitz conditions on loss and forward maps (Assumptions 1, 3, 5)
    Standard for nonconvex SGD analysis; not verified for the actual segmentation losses and networks.
invented entities (1)
  • Feature Retrieval Library (FRL) no independent evidence
    purpose: Stores prototype-induced statistics per client to enable communication-efficient routing.
    The FRL is a learned memory structure with only internal experimental validation; no external benchmark or independent test is provided.

pith-pipeline@v1.3.0-alltime-deepseek · 27713 in / 11122 out tokens · 116003 ms · 2026-08-01T20:34:21.832414+00:00 · methodology

0 comments
read the original abstract

Mixture of Experts (MoE) are increasingly deployed over wireless cloud-edge networks, as a single edge device lacks sufficient resources to host large-scale models locally. In this distributed architecture, a cloud-hosted pretrained Large Model (LM) acts as a shared backbone for latent feature extraction, while heterogeneous experts deployed across distributed, wirelessly-connected clients collaboratively form the task head. However, deploying MoE over wireless links exposes two coupled bottlenecks. On the one hand, routing which clients to activate generally overloads bandwidth-limited uplinks due to required raw feature transmission. On the other hand, aggregating the activated experts' outputs over wireless links is hindered by channel noise and poor scalability. To break these bottlenecks, we propose a statistic-augmented over-the-air MoE (AirMoE) paradigm. Specifically, on the routing side, each client queries its local Feature Retrieval Library (FRL) with a cloud-broadcast compact query, retrieves a prototype-induced statistic, and reports it digitally to the cloud, drastically reducing uplink traffic; the cloud then selects the most relevant clients by aligning these statistics with the LM-extracted features via Jensen--Shannon (JS) divergence. On the aggregating side, selected experts simultaneously transmit their outputs over the multiple-access channel, which physically computes the reweighted sum via waveform superposition, with reweighting coefficients realized through channel-aware power control. The two mechanisms are thus decoupled both algorithmically and physically. We further provide theoretical analyses on convergence and iteration complexity. Taking semantic segmentation task as an example, extensive experiments demonstrate that AirMoE outperforms MoE baselines and single-model competitors. Ablations further confirm the effectiveness of each incorporated component.

Figures

Figures reproduced from arXiv: 2607.16562 by Guangxu Zhu, Jingreng Lei, Wei-Bin Kou, Yujiu Yang.

Figure 1
Figure 1. Figure 1: Illustration of the two coupled bottlenecks in wireless MoE. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the proposed AirMoE. On the other hand, aggregating admits an elegant physical￾layer solution via Over-the-air computing (AirComp) [8]–[10], which leverages the waveform-superposition property of the shared multiple-access channel to sum signals. This property is particularly suited to the wireless MoE aggregating, since, when routed experts transmit their analog-modulated symbols concurrently,… view at source ↗
Figure 3
Figure 3. Figure 3: Detailed illustration of the proposed AirMoE. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Convergence comparison of the proposed AirMoE against all baselines on Cityscapes dataset. [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Illustration of the relationship of ϵ-stationarity complexity iteration with respect to various involved factors. anced expert utilization broadens semantic coverage so that its overall mIoU and mF1 remain superior despite the tiny pre￾cision gap. RQ3 therefore reveals that AirMoE’s mechanism improves coverage of under-represented regions rather than merely shifting a decision threshold. RQ4: Where does Ai… view at source ↗
Figure 6
Figure 6. Figure 6: Relationship visualization between ViT-extracted features and expert [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

61 extracted references · 7 linked inside Pith

  1. [1]

    Multi-objective isac for low-altitude economy based on multi-task deep reinforcement learning with mixture of experts,

    X. Ye, H. Lin, X. Song, Y . Wu, and L. Fu, “Multi-objective isac for low-altitude economy based on multi-task deep reinforcement learning with mixture of experts,”IEEE Transactions on Mobile Computing, pp. 1–17, 2026

    2026

  2. [2]

    Edge computing: Vision and challenges,

    W. Shi, J. Cao, Q. Zhang, Y . Li, and L. Xu, “Edge computing: Vision and challenges,”IEEE Internet of Things Journal, vol. 3, no. 5, pp. 637–646, 2016

    2016

  3. [3]

    A comprehensive survey of mixture-of-experts: Algorithms, theory, and applications,

    S. Mu and S. Lin, “A comprehensive survey of mixture-of-experts: Algorithms, theory, and applications,”arXiv preprint arXiv:2503.07137, 2025

    arXiv 2025

  4. [4]

    Scaling vision with sparse mixture of experts,

    C. Riquelme, J. Puigcerver, B. Mustafa, M. Neumann, R. Jenatton, A. Susano Pinto, D. Keysers, and N. Houlsby, “Scaling vision with sparse mixture of experts,”Advances in Neural Information Processing Systems (NeurIPS), vol. 34, pp. 8583–8595, 2021

    2021

  5. [5]

    Wdmoe: Wireless distributed large language models with mixture of experts,

    N. Xue, Y . Sun, Z. Chen, M. Tao, X. Xu, L. Qian, S. Cui, and P. Zhang, “Wdmoe: Wireless distributed large language models with mixture of experts,” inGLOBECOM 2024-2024 IEEE Global Communications Conference. IEEE, 2024, pp. 2707–2712

    2024

  6. [6]

    Mixture-of-experts for distributed edge computing with channel-aware gating function,

    Q. Song, S. Jing, S. Zhang, S. Zhang, and C. Huang, “Mixture-of-experts for distributed edge computing with channel-aware gating function,” in 2025 IEEE International Conference on Communications Workshops (ICC Workshops). IEEE, 2025, pp. 1353–1358

    2025

  7. [7]

    A unified distributed algorithm for hybrid near-far field activity detection in cell- free massive mimo,

    J. Lei, Y . Li, Z. Wang, Q. Lin, Y .-F. Liu, and Y .-C. Wu, “A unified distributed algorithm for hybrid near-far field activity detection in cell- free massive mimo,”IEEE Transactions on Wireless Communications, vol. 25, pp. 18 259–18 275, 2026

    2026

  8. [8]

    Broadband analog aggregation for low-latency federated edge learning,

    G. Zhu, Y . Wang, and K. Huang, “Broadband analog aggregation for low-latency federated edge learning,”IEEE Transactions on Wireless Communications, vol. 19, no. 1, pp. 491–506, 2020

    2020

  9. [9]

    Machine learning at the wireless edge: Distributed stochastic gradient descent over-the-air,

    M. M. Amiri and D. G ¨und¨uz, “Machine learning at the wireless edge: Distributed stochastic gradient descent over-the-air,”IEEE Transactions on Signal Processing, vol. 68, pp. 2155–2169, 2020

    2020

  10. [10]

    Over-the-air computing for wire- less data aggregation in massive IoT,

    G. Zhu, J. Xu, K. Huang, and S. Cui, “Over-the-air computing for wire- less data aggregation in massive IoT,”IEEE Wireless Communications, vol. 28, no. 4, pp. 57–65, 2021

    2021

  11. [11]

    Computation-efficient federated prompt-tuning with vision-language foundation model compression over resource-constrained edge networks,

    Z. Hu, J. Yan, and Y .-J. A. Zhang, “Computation-efficient federated prompt-tuning with vision-language foundation model compression over resource-constrained edge networks,”IEEE Transactions on Mobile Computing, pp. 1–17, 2026

    2026

  12. [12]

    Fast ai model partition for split learning over edge networks,

    Z. Li, W. Wu, S. Wu, and X. Shen, “Fast ai model partition for split learning over edge networks,”IEEE Transactions on Mobile Computing, pp. 1–15, 2026

    2026

  13. [13]

    Efficient layer-granularity unloading for llms in edge computing,

    Z. Li, Z. Tang, J. Guo, W. Jia, and W. Zhao, “Efficient layer-granularity unloading for llms in edge computing,”IEEE Transactions on Mobile Computing, pp. 1–13, 2026

    2026

  14. [14]

    pfedlvm: A large vision model (lvm)-driven and latent feature-based personalized federated learning framework in autonomous driving,

    W.-B. Kou, Q. Lin, M. Tang, S. Xu, R. Ye, Y . Leng, S. Wang, G. Li, Z. Chen, G. Zhu, and Y .-C. Wu, “pfedlvm: A large vision model (lvm)-driven and latent feature-based personalized federated learning framework in autonomous driving,”IEEE Transactions on Intelligent Transportation Systems, vol. 26, no. 10, pp. 15 915–15 931, 2025

    2025

  15. [15]

    BottleNet: A deep learning architecture for intelligent mobile cloud computing services,

    A. E. Eshratifar, A. Esmaili, and M. Pedram, “BottleNet: A deep learning architecture for intelligent mobile cloud computing services,” inProc. IEEE/ACM Int. Symp. Low Power Electronics and Design (ISLPED), 2019, pp. 1–6

    2019

  16. [16]

    Split computing and early exiting for deep learning applications: Survey and research challenges,

    Y . Matsubara, M. Levorato, and F. Restuccia, “Split computing and early exiting for deep learning applications: Survey and research challenges,” ACM Computing Surveys, vol. 55, no. 5, pp. 1–30, 2022

    2022

  17. [17]

    BranchyNet: Fast inference via early exiting from deep neural networks,

    S. Teerapittayanon, B. McDanel, and H. T. Kung, “BranchyNet: Fast inference via early exiting from deep neural networks,” inProc. Int. Conf. Pattern Recognition (ICPR), 2016, pp. 2464–2469

    2016

  18. [18]

    Communication-computation trade-off in resource-constrained edge inference,

    J. Shao and J. Zhang, “Communication-computation trade-off in resource-constrained edge inference,”IEEE Communications Magazine, vol. 58, no. 12, pp. 20–26, 2020

    2020

  19. [19]

    JointDNN: An efficient training and inference engine for intelligent mobile cloud com- puting services,

    A. E. Eshratifar, M. S. Abrishami, and M. Pedram, “JointDNN: An efficient training and inference engine for intelligent mobile cloud com- puting services,” inIEEE Transactions on Mobile Computing, vol. 20, no. 2, 2021, pp. 565–576

    2021

  20. [20]

    EdgeMoE: Fast on-device inference of moe-based large language models,

    R. Yi, L. Guo, S. Wei, A. Zhou, S. Wang, and M. Xu, “EdgeMoE: Fast on-device inference of moe-based large language models,”arXiv preprint arXiv:2308.14352, 2023

    Pith/arXiv arXiv 2023

  21. [21]

    LLM-QAT: Data-free quantization aware training for large language models,

    Z. Liu, B. Oguz, C. Zhao, E. Chang, P. Stock, Y . Mehdad, Y . Shi, R. Krishnamoorthi, and V . Chandra, “LLM-QAT: Data-free quantization aware training for large language models,” inFindings of the Association for Computational Linguistics (ACL), 2024, pp. 467–484

    2024

  22. [22]

    Understanding complex-valued transformer for modulation recognition,

    J. Lei, Y . Li, L.-Y . Yung, Y . Leng, Q. Lin, and Y .-C. Wu, “Understanding complex-valued transformer for modulation recognition,”IEEE Wireless Communications Letters, vol. 13, no. 12, pp. 3523–3527, 2024

    2024

  23. [23]

    LoRA: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” inProc. Int. Conf. Learning Representations (ICLR), 2022

    2022

  24. [24]

    A federated rec- ommendation system framework based on variational autoencoder with mixture of experts,

    Y . Xiao, Y . Lu, F. Tang, R. Wang, and G. Wang, “A federated rec- ommendation system framework based on variational autoencoder with mixture of experts,”IEEE Transactions on Mobile Computing, vol. 25, no. 6, pp. 8597–8610, 2026

    2026

  25. [25]

    M 3vit: Mixture-of-experts vision transformer for efficient multi- task learning with model-accelerator co-design,

    Z. Fan, R. Sarkar, Z. Jiang, T. Chen, K. Zou, Y . Cheng, C. Hao, Z. Wang et al., “M 3vit: Mixture-of-experts vision transformer for efficient multi- task learning with model-accelerator co-design,”Advances in Neural Information Processing Systems, vol. 35, pp. 28 441–28 457, 2022

    2022

  26. [26]

    Outrageously large neural networks: The sparsely-gated mixture-of-experts layer,

    N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, and J. Dean, “Outrageously large neural networks: The sparsely-gated mixture-of-experts layer,”arXiv preprint arXiv:1701.06538, 2017

    Pith/arXiv arXiv 2017

  27. [27]

    Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity,

    W. Fedus, B. Zoph, and N. Shazeer, “Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity,”Journal of Machine Learning Research (JMLR), vol. 23, no. 120, pp. 1–39, 2022

    2022

  28. [28]

    GShard: Scaling giant models with condi- tional computation and automatic sharding,

    D. Lepikhin, H. Lee, Y . Xu, D. Chen, O. Firat, Y . Huang, M. Krikun, N. Shazeer, and Z. Chen, “GShard: Scaling giant models with condi- tional computation and automatic sharding,” inProc. Int. Conf. Learning Representations (ICLR), 2021

    2021

  29. [29]

    GLaM: Efficient scaling of language models with mixture- of-experts,

    N. Du, Y . Huang, A. M. Dai, S. Tong, D. Lepikhin, Y . Xu, M. Krikun, Y . Zhou, A. W. Yu, O. Firat, B. Zoph, L. Fedus, M. Bosma, Z. Zhou, T. Wang, Y . E. Wang, K. Webster, M. Pellat, K. Robinson, K. Meier- Hellstern, T. Duke, L. Dixon, K. Zhang, Q. V . Le, Y . Wu, Z. Chen, and C. Cui, “GLaM: Efficient scaling of language models with mixture- of-experts,” ...

    2022

  30. [30]

    Mixtral of experts,

    A. Q. Jiang, A. Sablayrolles, A. Roux, A. Mensch, B. Savary, C. Bam- ford, D. S. Chaplot, D. de las Casas, E. B. Hanna, F. Bressand, G. Lengyel, G. Bour, G. Lample, L. R. Lavaud, L. Saulnier, M.-A. Lachaux, P. Stock, S. Subramanian, S. Yang, S. Antoniak, T. Le Scao, T. Gervet, T. Lavril, T. Wang, T. Lacroix, and W. El Sayed, “Mixtral of experts,”arXiv pre...

    Pith/arXiv arXiv 2024

  31. [31]

    DeepSeekMoE: Towards ultimate expert specialization in mixture-of-experts language models,

    D. Dai, C. Deng, C. Zhao, R. X. Xu, H. Gao, D. Chen, J. Li, W. Zeng, X. Yu, Y . Wu, Z. Xie, Y . K. Li, P. Huang, F. Luo, C. Ruan, Z. Sui, and W. Liang, “DeepSeekMoE: Towards ultimate expert specialization in mixture-of-experts language models,”arXiv preprint arXiv:2401.06066, 2024. JOURNAL OF LATEX CLASS FILES, VOL. 18, NO. 9, SEPTEMBER 2020 17

    Pith/arXiv arXiv 2024

  32. [32]

    BASE layers: Simplifying training of large, sparse models,

    M. Lewis, S. Bhosale, T. Dettmers, N. Goyal, and L. Zettlemoyer, “BASE layers: Simplifying training of large, sparse models,” inProc. Int. Conf. Machine Learning (ICML), 2021, pp. 6265–6274

    2021

  33. [33]

    Mixture-of-experts with expert choice routing,

    Y . Zhou, T. Lei, H. Liu, N. Du, Y . Huang, V . Y . Zhao, A. M. Dai, Z. Chen, Q. V . Le, and J. Laudon, “Mixture-of-experts with expert choice routing,” inAdvances in Neural Information Processing Systems (NeurIPS), vol. 35, 2022, pp. 7103–7114

    2022

  34. [34]

    Tutel: Adaptive mixture-of-experts at scale,

    C. Hwang, W. Cui, Y . Xiong, Z. Yang, Z. Liu, H. Hu, Z. Wang, R. Salas, J. Jose, P. Ram, J. Chau, P. Cheng, F. Yang, M. Yang, and Y . Xiong, “Tutel: Adaptive mixture-of-experts at scale,” inProc. Machine Learning and Systems (MLSys), vol. 5, 2023

    2023

  35. [35]

    Theory of mixture-of-experts for mobile edge computing,

    H. Li and L. Duan, “Theory of mixture-of-experts for mobile edge computing,” inIEEE INFOCOM 2025-IEEE Conference on Computer Communications. IEEE, 2025, pp. 1–10

    2025

  36. [36]

    Accelerating distributed MoE training and inference with Lina,

    J. Li, Y . Jiang, Y . Zhu, C. Wang, and H. Xu, “Accelerating distributed MoE training and inference with Lina,” inProc. USENIX Annual Technical Conference (ATC), 2023, pp. 945–959

    2023

  37. [37]

    Computation over multiple-access channels,

    B. Nazer and M. Gastpar, “Computation over multiple-access channels,” IEEE Transactions on Information Theory, vol. 53, no. 10, pp. 3498– 3516, 2007

    2007

  38. [38]

    Fast-convergent and communication-alleviated heterogeneous hierar- chical federated learning in autonomous driving,

    W.-B. Kou, Q. Lin, M. Tang, R. Ye, S. Wang, G. Zhu, and Y .-C. Wu, “Fast-convergent and communication-alleviated heterogeneous hierar- chical federated learning in autonomous driving,”IEEE Transactions on Intelligent Transportation Systems, vol. 26, no. 7, pp. 10 496–10 511, 2025

    2025

  39. [39]

    Communication resources constrained hierarchical federated learning for end-to-end autonomous driving,

    W.-B. Kou, S. Wang, G. Zhu, B. Luo, Y . Chen, D. W. Kwan Ng, and Y .-C. Wu, “Communication resources constrained hierarchical federated learning for end-to-end autonomous driving,” in2023 IEEE/RSJ Inter- national Conference on Intelligent Robots and Systems (IROS), 2023, pp. 9383–9390

    2023

  40. [40]

    Fedrc: A rapid-converged hierarchical federated learning framework in street scene semantic understanding,

    W.-B. Kou, Q. Lin, M. Tang, S. Wang, G. Zhu, and Y .-C. Wu, “Fedrc: A rapid-converged hierarchical federated learning framework in street scene semantic understanding,” in2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024, pp. 2578– 2585

    2024

  41. [41]

    Fedema: Federated exponential moving averaging with negative entropy regularizer in autonomous driving,

    W.-B. Kou, G. Zhu, B. Cheng, S. Wang, M. Tang, and Y .-C. Wu, “Fedema: Federated exponential moving averaging with negative entropy regularizer in autonomous driving,” in2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2025, pp. 10 190– 10 197

    2025

  42. [42]

    Machine learning at the wireless edge: Distributed stochastic gradient descent over-the-air,

    M. M. Amiri and D. G ¨und¨uz, “Machine learning at the wireless edge: Distributed stochastic gradient descent over-the-air,”IEEE Trans. Signal Process., vol. 68, pp. 2155–2169, 2020

    2020

  43. [43]

    Federated learning via over- the-air computation,

    K. Yang, T. Jiang, Y . Shi, and Z. Ding, “Federated learning via over- the-air computation,”IEEE Transactions on Wireless Communications, vol. 19, no. 3, pp. 2022–2035, 2020

    2022

  44. [44]

    Optimized power control design for over-the-air federated edge learning,

    X. Cao, G. Zhu, J. Xu, and S. Cui, “Optimized power control design for over-the-air federated edge learning,”IEEE J. Sel. Areas Commun., vol. 40, no. 1, pp. 342–358, 2022

    2022

  45. [46]

    StableMoE: Stable routing strategy for mixture of experts,

    D. Dai, L. Dong, S. Ma, B. Zheng, Z. Sui, B. Chang, and F. Wei, “StableMoE: Stable routing strategy for mixture of experts,” inProc. Annu. Meeting Assoc. Comput. Linguistics (ACL), 2022, pp. 7085–7095

    2022

  46. [47]

    Hash layers for large sparse models,

    S. Roller, S. Sukhbaatar, A. Szlam, and J. Weston, “Hash layers for large sparse models,” inAdvances in Neural Information Processing Systems (NeurIPS), vol. 34, 2021, pp. 17 555–17 566

    2021

  47. [48]

    The cityscapes dataset for semantic urban scene understanding,

    M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benen- son, U. Franke, S. Roth, and B. Schiele, “The cityscapes dataset for semantic urban scene understanding,” inProc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016

    2016

  48. [49]

    Segmentation and recognition using structure from motion point clouds,

    G. J. Brostow, J. Shotton, J. Fauqueur, and R. Cipolla, “Segmentation and recognition using structure from motion point clouds,” inCom- puter Vision–ECCV 2008: 10th European Conference on Computer Vision, Marseille, France, October 12-18, 2008, Proceedings, Part I 10. Springer, 2008, pp. 44–57

    2008

  49. [50]

    The apolloscape open dataset for autonomous driving and its application,

    P. Wang, X. Huang, X. Cheng, D. Zhou, Q. Geng, and R. Yang, “The apolloscape open dataset for autonomous driving and its application,” IEEE transactions on pattern analysis and machine intelligence, 2019

    2019

  50. [51]

    Carla: An open urban driving simulator,

    A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V . Koltun, “Carla: An open urban driving simulator,” inProceedings of The 1st Annual Conference on Robot Learning, 2017, pp. 1–16

    2017

  51. [52]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gellyet al., “An image is worth 16x16 words: Transformers for image recognition at scale,”arXiv preprint arXiv:2010.11929, 2020

    Pith/arXiv arXiv 2010

  52. [53]

    Rethinking atrous convolution for semantic image segmentation,

    L.-C. Chen, G. Papandreou, F. Schroff, and H. Adam, “Rethinking atrous convolution for semantic image segmentation,”arXiv preprint arXiv:1706.05587, 2017

    Pith/arXiv arXiv 2017

  53. [54]

    Statistical advantages of perturbing cosine router in sparse mixture of experts,

    H. Nguyen, P. Akbarian, T. Pham, T. Nguyen, S. Zhang, and N. Ho, “Statistical advantages of perturbing cosine router in sparse mixture of experts,”arXiv preprint arXiv:2405.14131, 2024

    Pith/arXiv arXiv 2024

  54. [55]

    From sparse to soft mixtures of experts,

    J. Puigcerver, C. Riquelme Ruiz, B. Mustafa, and N. Houlsby, “From sparse to soft mixtures of experts,” inInternational Conference on Learning Representations, vol. 2024, 2024, pp. 28 435–28 445

    2024

  55. [56]

    Bisenet v2: Bilateral network with guided aggregation for real-time semantic segmentation,

    C. Yu, C. Gao, J. Wang, G. Yu, C. Shen, and N. Sang, “Bisenet v2: Bilateral network with guided aggregation for real-time semantic segmentation,”International Journal of Computer Vision, vol. 129, pp. 3051–3068, 2021

    2021

  56. [57]

    Segnet: A deep con- volutional encoder-decoder architecture for image segmentation,

    V . Badrinarayanan, A. Kendall, and R. Cipolla, “Segnet: A deep con- volutional encoder-decoder architecture for image segmentation,”IEEE transactions on pattern analysis and machine intelligence, vol. 39, no. 12, pp. 2481–2495, 2017

    2017

  57. [58]

    Segformer: Simple and efficient design for semantic segmentation with transformers,

    E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo, “Segformer: Simple and efficient design for semantic segmentation with transformers,” inNeural Information Processing Systems (NeurIPS), 2021

    2021

  58. [59]

    Attanet: Attention-augmented network for fast and accurate scene parsing,

    Q. Song, K. Mei, and R. Huang, “Attanet: Attention-augmented network for fast and accurate scene parsing,” inProceedings of the AAAI conference on artificial intelligence, vol. 35, no. 3, 2021, pp. 2567– 2575

    2021

  59. [60]

    Domain adaptive and general- izable network architectures and training strategies for semantic image segmentation,

    L. Hoyer, D.-X. Dai, and L. Van Gool, “Domain adaptive and general- izable network architectures and training strategies for semantic image segmentation,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023

    2023

  60. [61]

    Topformer: Token pyramid transformer for mobile semantic segmentation,

    W. Zhang, Z. Huang, G. Luo, T. Chen, X. Wang, W. Liu, G. Yu, and C. Shen, “Topformer: Token pyramid transformer for mobile semantic segmentation,” inProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, 2022, pp. 12 083–12 093

    2022

  61. [62]

    Seaformer: Squeeze- enhanced axial transformer for mobile semantic segmentation,

    Q. Wan, Z. Huang, J. Lu, Y . Gang, and L. Zhang, “Seaformer: Squeeze- enhanced axial transformer for mobile semantic segmentation,” inThe eleventh international conference on learning representations, 2023

    2023