Pith. sign in

REVIEW 3 major objections 5 minor 31 references

QoS-Efficient Serving of Multiple Mixture-of-Expert LLMs Using Partial Runtime Reconfiguration

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single GPU can serve multiple fine-tuned mixture-of-experts LLMs at near-single-model throughput by sharing similar experts and swapping only non-expert layers per request.

desk verdict Real systems idea with a useful consolidation trick; the paper oversells TTFT and the Switch quality test is too easy, but the core is solid and it deserves peer review. read the letter →

arxiv 2505.06481 v1 pith:Y2NK3GZI submitted 2025-05-10 cs.LG cs.DC

HamidReza Imani , Jiaxin Peng , Peiman Mohseni , Abdolah Amirany , Tarek El-Ghazawi This is my paper · ORCID
classification cs.LGcs.DC
keywords mixture-of-expertsLLMinferenceexpertconsolidationmodelmergingruntimereconfigurationmulti-tenantGPUqualityofservicememory
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims it can serve several fine-tuned mixture-of-experts LLMs from a single GPU without the usual performance collapse. Its proposal is to load one consolidated set of expert parameters chosen by similarity across the models, and to re-load only the non-expert transformer layers whenever the request switches models. On an A100 with two Mixtral-8x7B variants, the approach sustains throughput comparable to serving one model, adds roughly half a second of time-to-first-token, and cuts turnaround time by 85% compared with NVIDIA MIG. Quality benchmarks stay close to the individually fine-tuned models, and experiments with four Switch Transformer variants show the quality advantage over simple weight averaging grows with the number of merged models.

What carries the argument

The system has two mechanisms: similarity-based expert consolidation and runtime partial reconfiguration. Consolidation computes pairwise L2 distances between flattened expert weights at every (layer, expert) coordinate across all model pairs, ranks coordinates by total distance, and assigns each coordinate to one model in round-robin order until the GPU expert capacity is filled, so the loaded experts are equally representative of all served models. Reconfiguration swaps only the non-expert layers (attention, normalization, embedding, head) when the requested model changes, because those parameters are always active and contribute about 14 percent of the output; expert parameters are fetched from host memory only on a miss, preventing further quality loss.

What would settle it

Run the same serving scheme on a pair of fine-tuned MoE variants whose same-position expert distances are artificially inflated (for example, by fine-tuning on disjoint domains or rotating expert vectors) while keeping benchmark performance intact, and measure whether perplexity and instruction-following scores collapse; if quality drops sharply even when the loaded expert is the closest one available, the consolidation premise fails.

Watch

Extended reading notes

Core claim

The central discovery is that fine-tuned MoE variants share enough structure at the expert level to be served from a common pool: experts at the same (layer, expert) position have L2 distances between 3.6 and 4.9, while cross-position distances range from 150 to 250, so a round-robin selection of the most similar experts yields a unified layout that represents all models. At inference time the system swaps only the non-expert parameters (about 3 GB, taking roughly one second over PCIe) when the target model changes, and otherwise routes each request's selected experts through the preloaded pool, fetching a true target-model expert only on a miss. The paper reports that this preserves output quality: perplexity on WikiText, C4, and PTB stays near the base model, MT-Bench average reaches 8.16 versus 8.13 for the instruct model, and ROUGE-1 on SAMSum degrades only from 0.49 to 0.46 when four models are merged, while the averaging baseline drops to 0.25.

Load-bearing premise

Everything rests on the assumption that experts at the same (layer, expert) position in different fine-tuned variants are interchangeable enough that substituting a foreign expert while keeping the target model's non-expert layers preserves task quality.

Editorial extensions

If this is right

  • A single 80GB GPU can serve at least two Mixtral-8x7B variants with throughput equal to serving one model at the same aggregate request rate, and with 85 percent lower average turnaround time than a two-instance NVIDIA MIG partition.
  • Switching the served model adds about 1.2 seconds of non-expert reload latency, but because this happens only when consecutive requests target different models, the average time-to-first-token increase is roughly 0.5 seconds.
  • The expert-sharing scheme keeps quality close to the dedicated models: perplexity stays within about 0.04 of the base model and MT-Bench scores within 0.03 of the instruct model in the two-variant Mixtral experiments.
  • When the number of served variants grows from two to four on Switch Transformer, ROUGE-1 on the SAMSum task falls from 0.49 to 0.46, whereas weight averaging falls from 0.42 to 0.25, showing the method's advantage widens with more models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the same-position expert similarity pattern holds across other MoE families, the same consolidated layout could serve arbitrary collections of fine-tuned variants without retraining, making it a practical alternative to per-tenant GPU partitioning.
  • The round-robin assignment of expert slots by similarity rank is a heuristic; a policy that routes each request to the most similar loaded expert, or that caches the most conflicting experts, could push quality even closer to the dedicated models.
  • Because the non-expert swap cost is fixed per model switch, batching requests by target model would amortize the reload latency and push throughput beyond the reported single-model-equivalent figure.
  • The quality measurements do not isolate which expert positions tolerate substitution; a per-position sensitivity map derived from the distance data could predict when sharing becomes unsafe without running benchmarks.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a single-GPU serving system for multiple fine-tuned Mixture-of-Expert (MoE) LLMs. To reduce memory, it consolidates experts across models by ranking (layer, expert) positions by L2 distance between the corresponding expert weights from different models, then loading experts in round-robin order up to a capacity C. To preserve output quality, it swaps only the non-expert layers at runtime when a request targets a different model, while using the preloaded shared experts (including foreign experts) whenever possible and fetching the target model's own expert on a miss. The authors evaluate QoS with Poisson arrivals on an A100 for two Mixtral-8x7B variants, reporting throughput comparable to serving a single model, an average TTFT increase from 0.89 s to 1.41 s, and an 85% average turnaround-time reduction versus NVIDIA MIG. Output quality is assessed with perplexity, MT-Bench, MMLU, HellaSwag, and TruthfulQA for Mixtral, and with SAMSum ROUGE scores for up to four Switch Transformer Base-8 variants, comparing against weight averaging.

Significance. If the claims hold, the paper addresses a real deployment problem: hosting several fine-tuned MoE LLMs on one GPU without time-slicing or MIG-style partitioning. The strengths are concrete: the system is implemented and measured, the code is provided, quality is evaluated on standard external benchmarks rather than fitted to the objective, and the comparison includes both a single-model throughput baseline and a hardware-partitioning baseline. The four-model Switch experiment is a useful stress test of scalability. The main risks are that the 'negligible' TTFT claim is contradicted by the paper's own Table 2, that the central memory-saving contribution is never quantified, and that the quality evaluation does not isolate the effect of the L2-similarity ranking. These are fixable with additional measurements and revised wording, and the core idea of runtime non-expert reconfiguration with shared experts is a reasonable and potentially useful systems contribution.

major comments (3)
  1. [Abstract; Section 5.1, Table 2] The claim that the increase in time-to-first-token is 'negligible' is not supported by the reported numbers. Table 2 shows average TTFT rising from 0.89 s for single-model serving to 1.41 s for the proposed system, an increase of 0.52 s (roughly 58% of the baseline), and Section 5.1 states that the non-expert swap itself takes approximately 1.2 s. Since TTFT is one of the two QoS metrics in the title and the abstract explicitly contrasts with 'serving a single model', this wording should be revised to a quantitative statement, e.g., the absolute increase in seconds and its relation to the MIG baseline, rather than 'negligible'.
  2. [Section 3.1, Algorithm 1; Section 5.1, Table 1] The paper's central memory-sharing contribution is not quantified. The loader takes an expert capacity C as input, but the evaluation never reports C, the resulting GPU memory footprint of the consolidated layout, or the measured expert hit/miss distribution during the QoS experiments. Table 1 gives per-layer latencies for 0, 1, and 2 expert hits, but without knowing how often each case occurs in the Mixtral and Switch workloads, the end-to-end throughput results cannot be interpreted. Please report C, the GPU memory used by the proposed layout versus single-model offloading and MIG, and measured hit rates for the evaluated workloads.
  3. [Sections 5.2 and 5.3, Tables 3-4] The evaluation does not isolate the effect of the L2-similarity ranking on output quality. The proposed map is compared only against weight averaging; without a control in which experts are assigned to positions randomly (or by a deliberately poor similarity measure), the observed quality retention cannot be attributed to similarity-based consolidation rather than to general expert redundancy. In addition, Table 3 reports point estimates only, with no variance or number of MT-Bench judging runs, and the Mixtral evidence uses only two variants. A per-layer or per-position analysis of which substitutions are tolerated would directly test the load-bearing assumption and would strengthen the scalability claims.
minor comments (5)
  1. [Table 2] The caption says the values are normalized to their maximum values, but the table reports values in seconds; the caption and the text in Section 5.1 should be made consistent.
  2. [Abstract; Section 5.1] The abstract claims an '85% average reduction in turnaround time' compared to MIG, but Table 2 implies 8.78 s versus 49.67 s, i.e., about 82.3%; please clarify how the average is computed and which arrival rates are included.
  3. [Figure 4] The throughput curves are described as averages over five runs, but no error bars or variance are shown; adding them would help assess whether the proposed system and single-model serving differ at the ridge points.
  4. [Section 4.2] There is a typo: 'we asses the performance' should be 'we assess the performance'.
  5. [Algorithm 2] The comment 'Load targetModel non-experts on GPU' should say 'non-expert layers' to avoid confusion with the expert offloading described elsewhere.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: all central quality and throughput claims are measured empirically on external benchmarks, and the L2-distance expert layout is a design heuristic rather than a fitted parameter disguised as a prediction.

full rationale

The paper's derivation chain is not circular. The expert-consolidation layout is produced by a fixed, offline heuristic: experts are ranked by pairwise L2 distance at the same (layer, expert) coordinate and assigned to models in round-robin order (Algorithm 1). This heuristic motivates the design, but the paper does not claim that the L2 distances predict output quality; instead, the resulting consolidated model is evaluated directly on external benchmarks (WikiText2, PTB, C4, MT-Bench, MMLU, HellaSwag, TruthfulQA, and SAMSum ROUGE scores). The quality results, including the proposed model's perplexity (3.85/7.33/13.90) and MT-Bench average (8.16), are empirical measurements of the actual runtime configuration, not quantities derived from the similarity scores. Likewise, the QoS claims (throughput, TTFT, turnaround time) are measured with PyTorch timings against NVIDIA MIG and single-model baselines; the observed TTFT overhead is a reported measurement, not an identity. There is no fitted parameter that is later renamed as a prediction, no 'uniqueness theorem' imported from the authors' prior work, and no load-bearing argument that reduces to a self-citation. The self-citations (El-Araby et al., 2009; Huang et al., 2010; Imani et al., 2024) appear in related-work or background contexts and do not carry the paper's central claims. The main caveat is evidential rather than circular: expert-substitution quality is demonstrated only on the tested model pairs and favorable configurations, but that is a completeness limitation, not a circular-structure defect. Therefore, the correct circularity finding is none, score 0.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claims rest on architectural homogeneity of the served models and on the empirical interchangeability of same-position experts; neither is derived from first principles, and the paper supplies only limited validation across two model families.

free parameters (1)
  • Expert capacity C = not stated
    Number of expert slots on GPU; an input to Algorithm 1, determined by available memory, never quantified in the paper, so the claimed memory savings cannot be verified.
assumptions (4)
  • domain assumption All served models share identical architecture, layer count, expert count, and tokenizer.
    Stated in Section 1 as 'fine-tuned LLMs with identical architectures'; required for the consolidated layout and expert-position mapping to make sense.
  • domain assumption Same-position experts across fine-tuned variants are functionally similar, with L2 weight distance as a valid proxy.
    Motivated by Figure 2 for Mixtral, but used as the basis for Algorithm 1 without a formal or task-level justification for generality.
  • domain assumption Non-expert layers can be swapped at runtime without affecting the validity of the shared expert computations.
    The system replaces non-expert layers per request (Section 3.2); the paper assumes attention and normalization layers from the target model compose correctly with foreign experts, validated only empirically.
  • domain assumption The inference and offloading model of Eliseev and Mazur applies, i.e., PCIe transfer of missing experts is the dominant latency cost.
    The system leverages this offloading model to justify expert miss handling and the MIG comparison; it is not re-derived here.

how reviews work

0 comments
Cite this review

Pith. "Pith review of QoS-Efficient Serving of Multiple Mixture-of-Expert LLMs Using Partial Runtime Reconfiguration." pith.science (2026). https://pith.science/paper/Y2NK3GZI

@misc{pith2026250506481,
  author       = {Pith},
  title        = {Pith review of: QoS-Efficient Serving of Multiple Mixture-of-Expert LLMs Using Partial Runtime Reconfiguration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y2NK3GZI}},
  note         = {Machine review of arXiv:2505.06481}
}
read the original abstract

The deployment of mixture-of-experts (MoE) large language models (LLMs) presents significant challenges due to their high memory demands. These challenges become even more pronounced in multi-tenant environments, where shared resources must accommodate multiple models, limiting the effectiveness of conventional virtualization techniques. This paper addresses the problem of efficiently serving multiple fine-tuned MoE-LLMs on a single-GPU. We propose a serving system that employs \textit{similarity-based expert consolidation} to reduce the overall memory footprint by sharing similar experts across models. To ensure output quality, we introduce \textit{runtime partial reconfiguration}, dynamically replacing non-expert layers when processing requests from different models. As a result, our approach achieves a competitive output quality while maintaining throughput comparable to serving a single model while incurring a negligible increase in time-to-first-token (TTFT). Experiments on a server with a single NVIDIA A100 GPU (80GB) using Mixtral-8x7B models demonstrate an 85\% average reduction in turnaround time compared to NVIDIA's multi-instance GPU (MIG). Furthermore, experiments on Google's Switch Transformer Base-8 model with up to four variants demonstrate the scalability and resilience of our approach in maintaining output quality compared to other model merging baselines, highlighting its effectiveness.

Figures

Figures reproduced from arXiv: 2505.06481 by the authors.

Figure 1
Figure 1. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Expert-to-expert distance between Mixtral base and instruct model: Despite being fine-tuned for different tasks, experts in the same positions exhibit similarity. However, when handling an MoE LLM, the model does not completely fit in the accelerator’s memory. Originally proposed in (Eliseev & Mazur, 2023), offloading techniques load the non-expert parameters of the model to GPU initially and fill the remaining spac… view at source ↗
Figure 3
Figure 3. Inference process for each layer of the consolidated model: In the case of an expert hit, the inference is performed using the already loaded expert, which may not necessarily belong to the requested model. However, in the case of a miss, the corresponding expert is loaded from the host’s memory to preserve the quality of the generated output. Its architecture comprises 32 layers, each with 8 experts, and utilizes t… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The throughput of each serving approach is measured in completed requests per minute. Since NVIDIA MIG utilizes two independent processes to handle requests for different models, the arrival rate for each instance is half of the value shown on the x-axis, and the repor…
Figure 5
Figure 5. Figure 5: ROUGE-1 scores (higher is better) on the SAMSum dataset for individual models and their combinations, comparing the proposed approach with the model-merging baseline. Model layouts are illustrated beneath each bar. In our approach, a portion of the expert parameters (s…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 4 canonical work pages

  1. [1]

    L., Almeida, D., Altenschmidt, J., Altman, S., 9 QoS-Efficient Serving of Multiple Mixture-of-Expert LLMs Using Partial Runtime Reconfiguration Anadkat, S., et al

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., 9 QoS-Efficient Serving of Multiple Mixture-of-Expert LLMs Using Partial Runtime Reconfiguration Anadkat, S., et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774,

    arXiv

  2. [10]

    URL http://dx.doi.org/ 10.18653/v1/D19-5409

    18653/v1/d19-5409. URL http://dx.doi.org/ 10.18653/v1/D19-5409. He, J., Vero, M., Krasnopolska, G., and Vechev, M. Instruc- tion tuning for secure code generation. arXiv preprint arXiv:2402.09497,

    arXiv

  3. [11]

    Mixture of experts with mixture of precisions for tuning quality of service

    Imani, H., Amirany, A., and El-Ghazawi, T. Mixture of experts with mixture of precisions for tuning quality of service. arXiv preprint arXiv:2407.14417,

    arXiv

  4. [12]

    Izmailov, P., Podoprikhin, D., Garipov, T., Vetrov, D., and Wilson, A. G. Averaging weights leads to wider optima and better generalization. arXiv preprint arXiv:1803.05407,

    arXiv

  5. [13]

    Data- less knowledge fusion by merging weights of language models

    Jin, X., Ren, X., Preotiuc-Pietro, D., and Cheng, P. Data- less knowledge fusion by merging weights of language models. arXiv preprint arXiv:2212.09849,

    arXiv

  6. [15]

    J., Awan, A

    10 QoS-Efficient Serving of Multiple Mixture-of-Expert LLMs Using Partial Runtime Reconfiguration Kim, Y . J., Awan, A. A., Muzio, A., Salinas, A. F. C., Lu, L., Hendy, A., Rajbhandari, S., He, Y ., and Awadalla, H. H. Scalable and efficient moe training for multitask multilingual models. arXiv preprint arXiv:2109.10465,

    arXiv

  7. [16]

    K., El-Araby, E., and El-Ghazawi, T

    Li, T., Narayana, V . K., El-Araby, E., and El-Ghazawi, T. Gpu resource sharing and virtualization on high perfor- mance computing systems. In 2011 International Confer- ence on Parallel Processing, pp. 733–742. IEEE,

    work page 2011

  8. [18]

    Truthfulqa: Measuring how models mimic human falsehoods

    Lin, S., Hilton, J., and Evans, O. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958, 2021b. Liu, J., Tang, P., Wang, W., Ren, Y ., Hou, X., Heng, P.-A., Guo, M., and Li, C. A survey on inference optimization techniques for mixture of experts models. arXiv preprint arXiv:2412.14219,

    arXiv

Show all 31 references
  1. [19]

    A., MacIntyre, R., Bies, A., Ferguson, M., Katz, K., and Schasberger, B

    Marcus, M., Kim, G., Marcinkiewicz, M. A., MacIntyre, R., Bies, A., Ferguson, M., Katz, K., and Schasberger, B. The penn treebank: Annotating predicate argument structure. In Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994,

    1994

  2. [20]

    Pointer sentinel mixture models

    Merity, S., Xiong, C., Bradbury, J., and Socher, R. Pointer sentinel mixture models. arXiv preprint arXiv:1609.07843,

    arXiv

  3. [22]

    Learning more generalized experts by merg- ing experts in mixture-of-experts

    Park, S. Learning more generalized experts by merg- ing experts in mixture-of-experts. arXiv preprint arXiv:2405.11530,

    arXiv

  4. [23]

    Outrageously large neural networks: The sparsely-gated mixture-of-experts layer

    Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q., Hinton, G., and Dean, J. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. arXiv preprint arXiv:1701.06538,

    arXiv

  5. [26]

    Wortsman, M., Ilharco, G., Gadre, S

    doi: 10.1109/Cloud-Summit61220.2024.00007. Wortsman, M., Ilharco, G., Gadre, S. Y ., Roelofs, R., Gontijo-Lopes, R., Morcos, A. S., Namkoong, H., Farhadi, A., Carmon, Y ., Kornblith, S., et al. Model soups: averaging weights of multiple fine-tuned models improves accuracy with...

    2024

  6. [27]

    Moe- infinity: Activation-aware expert offloading for efficient moe serving

    Xue, L., Fu, Y ., Lu, Z., Mai, L., and Marina, M. Moe- infinity: Activation-aware expert offloading for efficient moe serving. arXiv preprint arXiv:2401.14361,

    arXiv

  7. [28]

    Resolving interference when merging models

    11 QoS-Efficient Serving of Multiple Mixture-of-Expert LLMs Using Partial Runtime Reconfiguration Yadav, P., Tam, D., Choshen, L., Raffel, C., and Bansal, M. Resolving interference when merging models. arXiv preprint arXiv:2306.01708, 1,

    arXiv

  8. [29]

    Moe-i2: Compressing mixture of experts models through inter- expert pruning and intra-expert low-rank decomposition

    Yang, C., Sui, Y ., Xiao, J., Huang, L., Gong, Y ., Duan, Y ., Jia, W., Yin, M., Cheng, Y ., and Yuan, B. Moe-i2: Compressing mixture of experts models through inter- expert pruning and intra-expert low-rank decomposition. arXiv preprint arXiv:2411.01016, 2024a. Yang, E., Wang...

    arXiv

  9. [30]

    Surgeryv2: Bridging the gap be- tween model merging and multi-task learning with deep representation surgery

    Yang, E., Shen, L., Wang, Z., Guo, G., Wang, X., Cao, X., Zhang, J., and Tao, D. Surgeryv2: Bridging the gap be- tween model merging and multi-task learning with deep representation surgery. arXiv preprint arXiv:2410.14389, 2024b. Zellers, R., Holtzman, A., Bisk, Y ., Farhadi,...

    arXiv

  10. [31]

    J., Hassan, H., Zhang, R., Zhao, T., and Gao, J

    Zuo, S., Liu, X., Jiao, J., Kim, Y . J., Hassan, H., Zhang, R., Zhao, T., and Gao, J. Taming sparsely activated transformer with stochastic experts. arXiv preprint arXiv:2110.04260,

    arXiv

  11. [1991]

    Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D

    Jiang, A. Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D. S., Casas, D. d. l., Hanna, E. B., Bressand, F., et al. Mixtral of experts. arXiv preprint arXiv:2401.04088,

    arXiv

  12. [1994]

    Kamahori, K., Gu, Y ., Zhu, K., and Kasikci, B

    doi: 10.1162/neco.1994.6.2.181. Kamahori, K., Gu, Y ., Zhu, K., and Kasikci, B. Fiddler: Cpu-gpu orchestration for fast inference of mixture-of- experts models. arXiv preprint arXiv:2402.07033,

    1994 arXiv

  13. [2009]

    and Mazur, D

    Eliseev, A. and Mazur, D. Fast inference of mixture-of- experts language models with offloading. arXiv preprint arXiv:2312.17238,

    arXiv

  14. [2010]

    Billm: Pushing the limit of post-training quantization for llms

    Huang, W., Liu, Y ., Qin, H., Li, Y ., Zhang, S., Liu, X., Magno, M., and Qi, X. Billm: Pushing the limit of post-training quantization for llms. arXiv preprint arXiv:2402.04291,

    arXiv

  15. [2011]

    M6-10t: A sharing- delinking paradigm for efficient multi-trillion parameter pretraining

    Lin, J., Yang, A., Bai, J., Zhou, C., Jiang, L., Jia, X., Wang, A., Zhang, J., Li, Y ., Lin, W., et al. M6-10t: A sharing- delinking paradigm for efficient multi-trillion parameter pretraining. arXiv preprint arXiv:2110.03888, 2021a. Lin, J., Tang, J., Tang, H., Yang, S., Chen...

    arXiv

  16. [2016]

    Seer-moe: Sparse expert efficiency through regularization for mixture-of-experts

    Muzio, A., Sun, A., and He, C. Seer-moe: Sparse expert efficiency through regularization for mixture-of-experts. arXiv preprint arXiv:2404.05089,

    arXiv

  17. [2018]

    Efficient and effec- tive weight-ensembling mixture of experts for multi-task model merging

    Shen, L., Tang, A., Yang, E., Guo, G., Luo, Y ., Zhang, L., Cao, X., Du, B., and Tao, D. Efficient and effec- tive weight-ensembling mixture of experts for multi-task model merging. arXiv preprint arXiv:2410.21804,

    arXiv

  18. [2020]

    Exploring in-memory accelerators and fp- gas for latency-sensitive dnn inference on edge servers

    Suvizi, A., Subramaniam, S., Lan, T., and Venkatara- mani, G. Exploring in-memory accelerators and fp- gas for latency-sensitive dnn inference on edge servers. In 2024 IEEE Cloud Summit , pp. 1–6,

    2024

  19. [2021]

    Chowdhury, M. N. R., Wang, M., Maghraoui, K. E., Wang, N., Chen, P.-Y ., and Carothers, C. A provably effective method for pruning experts in fine-tuned sparse mixture- of-experts. arXiv preprint arXiv:2405.16646,

    arXiv

  20. [2022]

    Dettmers, T

    URL https://arxiv.org/abs/ 2205.14135. Dettmers, T. and Zettlemoyer, L. The case for 4-bit preci- sion: k-bit inference scaling laws. In International Con- ference on Machine Learning , pp. 7750–7774. PMLR,

    arXiv

  21. [2023]

    K., Tang, H.-K., He, D., et al

    Du, G., Lee, J., Li, J., Jiang, R., Guo, Y ., Yu, S., Liu, H., Goh, S. K., Tang, H.-K., He, D., et al. Parameter competition balancing for model merging. arXiv preprint arXiv:2410.02396,

    arXiv

  22. [2024]

    Measuring mas- sive multitask language understanding

    Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., and Steinhardt, J. Measuring mas- sive multitask language understanding. arXiv preprint arXiv:2009.03300,

    2009 arXiv

  23. [2025]

    Chen, T., Huang, S., Xie, Y ., Jiao, B., Jiang, D., Zhou, H., Li, J., and Wei, F

    Retrieved from https://jax-ml.github.io/scaling-book/. Chen, T., Huang, S., Xie, Y ., Jiao, B., Jiang, D., Zhou, H., Li, J., and Wei, F. Task-specific expert pruning for sparse mixture-of-experts. arXiv preprint arXiv:2206.00277 ,

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.