RepetitionCurse: Measuring and Understanding Router Imbalance in Mixture-of-Experts LLMs under DoS Stress

Dong Chen; Hantao Huang; Qingyue Wang; Ruixuan Huang; Shuai Wang; Wei Wang; Yudong Gao

REVIEW 4 major objections 4 minor 19 references

RepetitionCurse claims that a prompt made of one repeated token forces MoE routers to send nearly all tokens to the same top-k experts, turning expert parallelism into a straggler machine and amplifying prefill latency by 1.5–4.7x across AP

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 13:28 UTC pith:XTIUIAEF

load-bearing objection Useful first measurement of a plausible MoE DoS vector; the core mechanism is real, but the multi-tenant DoS claim is not actually measured. the 4 major comments →

arxiv 2512.23995 v2 pith:XTIUIAEF submitted 2025-12-30 cs.CR cs.LG

RepetitionCurse: Measuring and Understanding Router Imbalance in Mixture-of-Experts LLMs under DoS Stress

Ruixuan Huang , Qingyue Wang , Hantao Huang , Yudong Gao , Dong Chen , Shuai Wang , Wei Wang This is my paper

classification cs.CR cs.LG

keywords mixture-of-expertsexpert parallelismrouter imbalancedenial-of-serviceprompt-based attacklatency amplificationside-channelload balancing

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

RepetitionCurse claims that MoE LLMs, deployed with expert parallelism, can be forced into extreme router imbalance by a prompt made of a single repeated token, with no gradients, search, or model access. The imbalance sends nearly all tokens to the same top-k experts; the GPU hosting those experts becomes a straggler, and the all-reduce barrier makes the whole MoE layer wait on it. The paper measures the effect on 14 open models and commercial APIs: end-to-end latency amplification ranges from 1.514x to 4.728x, and Mixtral-8x7B's prefill latency is amplified 3.063x. It also shows the latency signature is absent in dense models, so a black-box probe can infer whether a hidden backend is MoE. A deployer who believes this result must treat the router as an attack surface, not just an efficiency component.

Core claim

Under expert parallelism, a Mixture-of-Experts model's throughput depends on every GPU finishing its assigned expert computation before an all-reduce. RepetitionCurse identifies a router behavior that breaks this balance by input distribution alone: when a prompt consists of a single token repeated thousands of times, the token embeddings collapse into a homogeneous state, and the top-k router starts choosing the same few experts at every layer. All tokens in the prefill batch then queue on one GPU, which becomes a straggler while the rest idle. The paper's central quantitative claim is that this mechanism is widespread: on the 14 tested open MoE models, vocabulary-wide coverage approaches 1

What carries the argument

The load-bearing object is the top-k router in each MoE layer: a softmax gating over token hidden states that selects the k experts to process each token. RepetitionCurse operates by making every token identical, which yields identical hidden states and drives the gating distribution to a nearly deterministic choice of the same expert subset. The attack is supported by two formal tools: the theoretical maximum imbalance TMI = |D| · min(k, E_d)/k, which expresses the worst-case single-device load relative to the ideal balanced load under a mapping of E_d experts per device, and a coverage metric B(P_t) that averages per-layer max device load over the whole vocabulary. These tools quantify how

Load-bearing premise

The paper's coverage estimates assume that the number of tokens assigned to a GPU determines that GPU's execution time; if memory bandwidth, per-token cost differences, or kernel batching break that link, the simulated bottleneck will not translate into the measured latency amplification.

What would settle it

Profile the fused MoE kernel on a single GPU while feeding it 1x, 2x, and 4x the number of tokens assigned to the same experts; if execution time does not grow roughly in proportion to token count, then the token-count-to-latency proxy that drives the coverage curves fails. Alternatively, run the vocabulary-wide scan on a held-out MoE model not among the 14 tested and check whether a single repeated token still drives at least 90% of tokens to the same top-k experts across most layers.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

A single 20,000-token prompt with one repeated token is enough to double or triple time-to-first-token on major MoE APIs, violating latency SLAs.
The attack grows more effective as expert parallelism scales; providers moving to more GPUs for MoE efficiency are simultaneously increasing the vulnerability.
The pathological routing is a training-time artifact: fine-tuned variants of the same base model show nearly identical coverage, so instruction tuning alone will not fix it.
Routing latency is a reliable architecture-class fingerprint; an attacker can detect MoE backends even when the provider does not disclose the model.
A deployment-time placement defense works for moderately sparse models (coverage dropped by up to 36.7%) by scattering vulnerable experts across GPUs.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

An adversary could plausibly evade simple perplexity filtering by interleaving a few low-frequency tokens or using repeated bigrams/trigrams; the underlying hidden-state collapse is driven by redundancy, not literal character identity, so detection should target router entropy rather than prompt perplexity.
If the mechanism is hidden-state collapse under repetition, models with different normalization or embedding properties should show measurably different vulnerability; this suggests a pretraining-time robustness metric based on router entropy over synthetic repetitive corpora.
The side channel likely extends to estimating expert count and top-k, since TMI depends on k and per-device expert count; fitting amplification across several prompt lengths could fingerprint deployment scale.
The same routing-concentration lens could apply to batched normal requests that share repetitive system prompts or jailbreak templates, meaning the threat is not limited to deliberately adversarial users.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Useful first measurement of a plausible MoE DoS vector; the core mechanism is real, but the multi-tenant DoS claim is not actually measured.

read the letter

RepetitionCurse is worth knowing about. It identifies a simple, black-box way to make MoE routers concentrate all tokens on a few experts, and shows that under expert parallelism this translates into large prefill latency amplification on local vLLM deployments and on commercial APIs. The core mechanism is plausible, and the systematic sweep across 14 models is genuinely new; prior MoE DoS work (buffer overflows, endless loops) is a different class. The coverage metric and TMI bound give a useful vocabulary for thinking about routing imbalance, and the local kernel profiling in Table 3 is real evidence that token-count imbalance moves actual MoE kernel time. I believe the central mechanism holds up.

The soft spots are mostly about how far the evidence reaches. The headline claim in the abstract — 'degrading service availability significantly' — is a DoS against other users, but every measurement is on the attacker's own request. Table 4 is TTFT of attack vs normal prompts, not what happens to a concurrent legitimate request. A 3x slower prefill for the attacker is resource amplification, not automatically a denial of service; continuous batching, prefill/decode separation, admission control, or a scheduler that isolates a long prompt could absorb much of it. The paper asserts 'for all users within its batch' but no multi-tenant experiment supports that. That is load-bearing, because the threat model in Sec 3.1 explicitly targets queuing time for legitimate users.

Second, the API experiments are uncontrolled commercial endpoints. The paper does report a 95% lower bound on R_api, which is better than 'no error bars', but there is no variance or replication information, and provider-side jitter is real. The 1.5–4.7x range is consistent and suggestive, but I would not treat the exact numbers as solid.

Third, the coverage curves in Fig 5 use 'assigned token count' as a latency proxy for arbitrary expert–GPU mappings, an assumption stated in Sec 4.1. It is a fair first-order proxy, and Table 3 gives actual kernel times for the same models, but the two are not directly tied, and no instrumentation is released to check the proxy beyond the tested settings.

No circular reasoning; the TMI formula is a definitional bound, and latency is measured, not fitted. That part is clean.

Who this is for: anyone working on LLM inference security, MoE serving, or scheduler design. It deserves a serious referee, and the main revision should add a real co-tenant experiment (even simulated) and release the profiling code. I'd send it out.

Referee Report

4 major / 4 minor

Summary. The paper identifies and measures a routing-imbalance vulnerability in Mixture-of-Experts (MoE) LLMs served with expert parallelism. The proposed attack, RepetitionCurse, constructs prompts consisting largely of a repeated token; this causes routers to route nearly all tokens to a small fixed set of experts. Under expert parallelism, those experts may be collocated on a single GPU, creating a straggler device and inflating prefill latency / time-to-first-token. The authors provide a theoretical upper bound on load imbalance (TMI, Eq. 3), a vocabulary-wide coverage metric (Eq. 4) computed from router outputs, local vLLM kernel-time measurements for 14 models (Table 3), API TTFT measurements on commercial endpoints and an alleged architecture side-channel (Table 4), and two defenses. The central claim is that this constitutes a low-cost, black-box algorithmic DoS vector.

Significance. If substantiated, the result is significant: it identifies a simple, model-agnostic way to turn an efficiency mechanism of modern LLM serving into a latency amplifier, with direct implications for EP deployment choices and for the robustness of MoE-based APIs. The paper's strengths are the direct kernel-level profiling of R_moe on local vLLM deployments across 14 models, the parameter-light TMI upper bound, the breadth of the Huggingface model survey, and the concrete, falsifiable coverage analysis. No circularity is apparent: TMI is a definitional upper bound, and latency is measured rather than fitted. The main gaps are experimental: the DoS claim for legitimate users is not measured, the coverage proxy is not directly validated against kernel time, and the API results lack basic statistical controls. These gaps are load-bearing for the paper's headline claims.

major comments (4)

[§3.1, §4.2, Eqs. (5)-(6)] The threat model defines the attack objective as inflating queuing time for legitimate user requests, but every reported metric is measured on the attack request alone. R_moe is the MoE kernel time of the attack prompt relative to a normal prompt, and R_api is the attack prompt's own TTFT relative to a normal prompt. No concurrent-request or multi-tenant experiment appears anywhere in the paper. A request that itself becomes slower does not establish denial of service for other users; continuous batching, admission control, prefill/decode separation, or fair queueing can absorb an imbalanced request. The abstract's 'degrading service availability significantly' and the introduction's 'for all users within its batch' are therefore not yet supported. This is load-bearing for the DoS claim and should be tested with a co-tenant latency/queueing experiment, or the claims should be explicitly
[§4.1, Eq. (4), §5.3] The coverage metric B is computed from router outputs under the assumption stated in §4.1 that 'the assigned tokens number serves as a proxy for its latency.' This proxy is not validated against the actual kernel times used in Table 3; no correlation between B and R_moe is reported, and the profiling instrumentation is not released. Fused grouped-GEMM time depends on total token count per device, memory bandwidth, and kernel batching, so a token-count imbalance need not translate linearly into latency. Observations 1-4 of §4.1 and the defense evaluation in Fig. 8 extend this proxy to EP sizes up to 512, while direct latency measurements stop at EP=8. Please validate the proxy on at least a few models across EP sizes, or restrict the conclusions to token-count imbalance.
[Table 4, §4.2] The API LAR results are uncontrolled point estimates. The text mentions a '95% confidence lower bound of R_api,' but no confidence intervals, error bars, or per-repetition statistics are shown; each API uses only 100 attack and 100 normal prompts, with no indication of how normal-prompt content was matched or how network/cache variability was handled. The dense-model control is also ambiguous: Llama-3-8B-Instruct has R_api=1.160, which is not 'close to or less than 1' in any statistical sense, and Qwen3-14B has 0.635; with only two dense models and no CIs, the claimed MoE side-channel is not established. Report matched-pair measurements over multiple time slots with confidence intervals and a statistical test.
[§5.3, Algorithm 1] The vulnerability-aware defense is evaluated only by the same token-count coverage metric (Fig. 8), not by kernel time or end-to-end latency. The algorithm also requires per-expert vulnerability degrees derived with the ad-hoc threshold τ=90% from §5.2, and the greedy assignment's tie-breaking is unspecified. The claim that the defense 'prevents the bottleneck and effectively mitigates latency amplification' should be demonstrated with actual latency measurements on at least one high-sparse model.

minor comments (4)

[§3.4, after Table 1] The paragraph after Table 1 contains corrupted placeholder sequences such as 'uni00000013/uni00000014...' that should be replaced with the intended table text or removed.
[§4.2, Eq. (5)] R_moe is defined for 'the i-th layer' of the bottleneck GPU, but Table 3 presumably reports some aggregation (max, mean, or a particular layer) across the 32-48 layers. Please specify how the layer is selected and how kernel times are aggregated.
[§4.2, Eq. (6)] The phrase 'we calculate the point estimate of R_api' is misleading: with 100 samples one should report a distribution, not a point estimate. Please report medians, quantiles, and confidence intervals.
[Figure 5] In the bottom-right panel, the legend lists 'Qwen3-Next-80B-A3B' while Table 2 lists 'Qwen3-Next-30B-A3B'; the model naming is inconsistent and should be harmonized.

Circularity Check

0 steps flagged

No significant circularity: coverage and LAR are measured quantities; TMI is a definitional bound; cited work is contextual.

full rationale

The derivation chain is self-contained. Eq. (3) is an explicitly labeled theoretical upper bound computed from the stated deployment parameters (|D|, k, E_d), not a fitted prediction. Eq. (4) is a coverage statistic measured from router outputs; it is an empirical description of router behavior, and the paper separately validates the token-count latency proxy with instrumented MoE kernel timings (Eq. 5) and end-to-end TTFT measurements (Eq. 6). The headline 3.063x amplification on Mixtral is a measured ratio of attack-prompt to normal-prompt latency, not a value obtained by fitting a parameter to the same data and then renaming it a prediction. No load-bearing argument reduces to a self-citation: references to prior MoE/DoS work are contextual, and there is no imported uniqueness theorem or ansatz-by-citation. The defense evaluation reuses the vulnerability-degree definition from the attack analysis, but that is an attack-specific mitigation design rather than a circular derivation of the vulnerability. The main limitation — flagged in the Ethical Statement ('DoS effects were assessed via latency rather than direct service disruption') and in the threat model's stated objective of inflating queuing time for legitimate requests — is that co-tenant queuing impact is not directly measured; that is an evidence/validity gap, not circularity.

Axiom & Free-Parameter Ledger

1 free parameters · 5 axioms · 0 invented entities

The attack and its measurements introduce no new physical entities or causal mechanisms beyond the existing MoE routing structure. The only free parameter is a threshold used for analyzing vulnerable experts in the defense section. The central claim depends mostly on empirical domain assumptions about how token-count imbalance translates into real kernel latency and how long expert placement remains static.

free parameters (1)

vulnerable-expert threshold tau = 0.9 (90%)
Used in Sec. 5.2 only to classify experts as vulnerable for the defense evaluation; it is not used to construct the attack prompts or to derive the central latency claim.

axioms (5)

domain assumption MoE execution time on a device is proportional to the total number of tokens assigned to the experts hosted on that device
Explicit in Sec. 4.1 ('the assigned tokens number serves as a proxy for its latency') and underpins the coverage metric B and the universality claims.
domain assumption Expert-to-GPU mapping is static at attack timescales
Sec. 3.3 real-world constraints: EPLB and similar load balancers rebalance only at coarse intervals (minutes), so the bottleneck persists within a relocation window.
domain assumption Router behavior under repeated tokens is set during pretraining and carries over to fine-tuned variants
Inferred from Figure 5 family similarity and used to claim that the vulnerability is universal across post-trained and instructed models.
domain assumption API TTFT differences are attributed to backend compute and routing rather than network jitter, caching, or middleware
Used in Sec. 4.2 and 4.3 for API LAR and the side-channel inference; the API experiments are uncontrolled and report point estimates only.
standard math Softmax top-k routing equations (Eqs. 1 and 2) describe inference-time expert selection
Standard MoE formulation, not contested by the paper.

pith-pipeline@v1.3.0-alltime-deepseek · 14128 in / 9213 out tokens · 91934 ms · 2026-08-03T13:28:16.319519+00:00 · methodology

0 comments

read the original abstract

Mixture-of-Experts architectures have become the standard for scaling large language models due to their superior parameter efficiency. To accommodate the growing number of experts in practice, modern inference systems commonly adopt expert parallelism to distribute experts across devices. However, the absence of explicit load balancing constraints during inference allows adversarial inputs to trigger severe routing concentration. We demonstrate that out-of-distribution prompts can manipulate the routing strategy such that all tokens are consistently routed to the same set of top-$k$ experts, which creates computational bottlenecks on certain devices while forcing others to idle. This converts an efficiency mechanism into a denial-of-service attack vector, leading to violations of service-level agreements for time to first token. We propose RepetitionCurse, a low-cost black-box strategy to exploit this vulnerability. By identifying a universal flaw in MoE router behavior, RepetitionCurse constructs adversarial prompts using simple repetitive token patterns in a model-agnostic manner. On widely deployed MoE models like Mixtral-8x7B, our method increases end-to-end inference latency by 3.063x, degrading service availability significantly.

Figures

Figures reproduced from arXiv: 2512.23995 by Dong Chen, Hantao Huang, Qingyue Wang, Ruixuan Huang, Shuai Wang, Wei Wang, Yudong Gao.

**Figure 1.** Figure 1: RepetitionCurse attack overview. Under attack inputs, all tokens are routed to the same top-k experts, causing a single GPU to become a straggler while other GPUs have to idle and wait. The MoE architecture introduces a fundamentally different model structure compared to dense models. Instead of a single monolithic feed-forward network (FFN) per layer, MoE layers contain multiple independent expert network… view at source ↗

**Figure 3.** Figure 3: Illustration of RepetitionCurse prompt. possible to create a bottleneck. In this paper, we introduce a very simple method to construct the near optimal adversarial prompts called RepetitionCurse, where all tokens are set to be the same except for the instruction template and system prompt (see [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

**Figure 4.** Figure 4: Expert workload comparison between RepetitionCurse and balanced baseline. Each cell represents the expert workload: percentage of tokens routed to the expert. models, which are characterized by a smaller total number of experts (i.e., 8-16). The remaining models with architectures like DeepSeekV3 and Qwen3Moe are characterized by sparser Top-k experts, with typically only 2 to 12 experts activated from … view at source ↗

**Figure 5.** Figure 5: Coverage of different models under different EP size. tokens number serves as a proxy for its latency. This allows us to evaluate bottleneck coverage under arbitrary expertGPU mapping without requiring full system execution. We summarize our observations and suggestions as follows: 1. Models from the same family exhibit similar coverage values (e.g., Mixtral-8x7B and its fine-tuned variants). This suggest… view at source ↗

**Figure 6.** Figure 6: Normalized entropy of the expert selection distribution under different prompt lengths and system prompts. by a prompt, without relying on device-level bottleneck assumptions. Formally, the normalized entropy is defined as H(P) = 1 L X L l=1 − P e∈El ρl,e(P) log ρl,e(P) log E (7) Lower entropy values correspond to more concentrated expert selection, indicating stronger routing imbalance. The results shown… view at source ↗

**Figure 8.** Figure 8: Effect of vulnerability-aware load balance strategy. Solid lines show the bottleneck coverage under the default expert–GPU assignment, while dashed lines correspond to coverage after applying the proposed strategy. olding; however, introducing an additional PPL evaluation stage requires an auxiliary screening model, incurring extra latency and deployment overhead. Notably, across all evaluated commercial… view at source ↗

discussion (0)

Reference graph

Works this paper leans on

19 extracted references · 8 linked inside Pith

[1]

S., Tumanov, A., and Ramjee, R

Agrawal, A., Kedia, N., Panwar, A., Mohan, J., Kwatra, N., Gulavani, B. S., Tumanov, A., and Ramjee, R. Tam- ing throughput-latency tradeoff in LLM inference with sarathi-serve. In Gavrilovska, A. and Terry, D. B. (eds.), 18th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2024, Santa Clara, CA, USA, July 10-12, 2024, pp. 117–134. U...

2024
[5]

DeepSeek-AI, Liu, A., Feng, B., Wang, B., Wang, B., Liu, B., Zhao, C., Dengr, C., Ruan, C., et al

Accessed: 2025-12-14. DeepSeek-AI, Liu, A., Feng, B., Wang, B., Wang, B., Liu, B., Zhao, C., Dengr, C., Ruan, C., et al. Deepseek-v2: A strong, economical, and efficient mixture-of-experts lan- guage model.arXiv preprint arXiv: 2405.04434, 2024a. DeepSeek-AI, Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C., Zhao, C., Deng, C., et al. Deepseek-v3 tech...

Pith/arXiv arXiv 2025
[8]

Buffer overflow in mixture of experts

Hayes, J., Shumailov, I., and Yona, I. Buffer overflow in mixture of experts. InNeurips Safe Generative AI Workshop 2024,

2024
[9]

Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D

Jiang, A. Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D. S., de las Casas, D., Hanna, E. B., et al. Mixtral of experts.arXiv preprint arXiv: 2401.04088,

Pith/arXiv arXiv
[11]

L., Almeida, D., Altenschmidt, J., et al

OpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., et al. Gpt-4 technical report.arXiv preprint arXiv: 2303.08774,

Pith/arXiv arXiv
[12]

V ., Hinton, G

Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q. V ., Hinton, G. E., and Dean, J. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. In5th International Conference on Learning Rep- resentations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net,

2017
[15]

N., Kaiser, L., and Polosukhin, I

9 RepetitionCurse: Measuring and Understanding Router Imbalance in Mixture-of-Experts LLMs under DoS Stress Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. Attention is all you need. In Guyon, I., von Luxburg, U., Bengio, S., Wallach, H. M., Fergus, R., Vishwanathan, S. V . N., and Garnett, R. (...

2017
[16]

I know what you asked: Prompt leakage via kv- cache sharing in multi-tenant llm serving

Wu, G., Zhang, Z., Zhang, Y ., Wang, W., Niu, J., Wu, Y ., and Zhang, Y . I know what you asked: Prompt leakage via kv- cache sharing in multi-tenant llm serving. InProceedings of the 2025 Network and Distributed System Security (NDSS) Symposium. San Diego, CA, USA,

2025
[17]

Leechhijack: Covert computational resource exploitation in intelligent agent systems.arXiv preprint arXiv: 2512.02321,

Zhang, Y ., Wang, W., Zhou, Z., Wang, K., Zhang, J., Sun, L., Liu, Y ., and Su, S. Leechhijack: Covert computational resource exploitation in intelligent agent systems.arXiv preprint arXiv: 2512.02321,

arXiv
[18]

H., Cao, S., Kozyrakis, C., Stoica, I., et al

Zheng, L., Yin, L., Xie, Z., Sun, C., Huang, J., Yu, C. H., Cao, S., Kozyrakis, C., Stoica, I., et al. Sglang: Efficient execution of structured language model programs. In Globersons, A., Mackey, L., Belgrave, D., Fan, A., Pa- quet, U., Tomczak, J. M., and Zhang, C. (eds.),Advances in Neural Information Processing Systems 38: Annual Conference on Neural ...

2024
[19]

Z., and Fredrikson, M

Zou, A., Wang, Z., Carlini, N., Nasr, M., Kolter, J. Z., and Fredrikson, M. Universal and transferable adversarial attacks on aligned language models.arXiv preprint arXiv: 2307.15043,

Pith/arXiv arXiv
[1967]

Longbench v2: Towards deeper understanding and reasoning on realistic long-context multitasks.ArXiv preprint, abs/2412.15204,

Bai, Y ., Tu, S., Zhang, J., Peng, H., Wang, X., Lv, X., Cao, S., Xu, J., Hou, L., et al. Longbench v2: Towards deeper understanding and reasoning on realistic long-context multitasks.ArXiv preprint, abs/2412.15204,

Pith/arXiv arXiv
[2017]

Megatron-lm: Training multi- billion parameter language models using model paral- lelism.arXiv preprint arXiv: 1909.08053,

Shoeybi, M., Patwary, M., Puri, R., LeGresley, P., Casper, J., and Catanzaro, B. Megatron-lm: Training multi- billion parameter language models using model paral- lelism.arXiv preprint arXiv: 1909.08053,

Pith/arXiv arXiv 1909
[2019]

Efficient attention mechanisms for large language models: A survey.arXiv preprint arXiv: 2507.19595,

Sun, Y ., Li, Z., Zhang, Y ., Pan, T., Dong, B., Guo, Y ., and Wang, J. Efficient attention mechanisms for large language models: A survey.arXiv preprint arXiv: 2507.19595,

arXiv
[2021]

Denial-of-service poisoning attacks against large language models.arXiv preprint arXiv: 2410.10760,

Gao, K., Pang, T., Du, C., Yang, Y ., Xia, S.-T., and Lin, M. Denial-of-service poisoning attacks against large language models.arXiv preprint arXiv: 2410.10760,

Pith/arXiv arXiv
[2022]

Switch transformers: Scaling to trillion parameter models with simple and effi- cient sparsity.arXiv preprint arXiv: 2101.03961,

Fedus, W., Zoph, B., and Shazeer, N. Switch transformers: Scaling to trillion parameter models with simple and effi- cient sparsity.arXiv preprint arXiv: 2101.03961,

Pith/arXiv arXiv
[2023]

Fundamental capabilities and applications of large language models: A survey.ACM Comput

Li, J., Gao, Y ., Yang, Y ., Bai, Y ., Zhou, X., Li, Y ., Sun, H., Liu, Y ., Si, X., et al. Fundamental capabilities and applications of large language models: A survey.ACM Comput. Surv., 58(2), 2025a. ISSN 0360-0300. doi: 10.1145/3735632. Li, Y ., Wang, J., Zhu, H., Lin, J., Chang, S., and Guo, M. Thinktrap: Denial-of-service attacks against black-box ll...

arXiv
[2024]

Amdahl, G. M. Validity of the single processor approach to achieving large scale computing capabilities. InPro- ceedings of the April 18-20, 1967, spring joint computer conference, pp. 483–485,

1967
[2025]

DeepSeek-AI

doi: 10.1109/TKDE.2025.3554028. DeepSeek-AI. Eplb: Expert parallelism load bal- ancer. https://github.com/deepseek-ai/ EPLB,

arXiv 2025

[1] [1]

S., Tumanov, A., and Ramjee, R

Agrawal, A., Kedia, N., Panwar, A., Mohan, J., Kwatra, N., Gulavani, B. S., Tumanov, A., and Ramjee, R. Tam- ing throughput-latency tradeoff in LLM inference with sarathi-serve. In Gavrilovska, A. and Terry, D. B. (eds.), 18th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2024, Santa Clara, CA, USA, July 10-12, 2024, pp. 117–134. U...

2024

[2] [5]

DeepSeek-AI, Liu, A., Feng, B., Wang, B., Wang, B., Liu, B., Zhao, C., Dengr, C., Ruan, C., et al

Accessed: 2025-12-14. DeepSeek-AI, Liu, A., Feng, B., Wang, B., Wang, B., Liu, B., Zhao, C., Dengr, C., Ruan, C., et al. Deepseek-v2: A strong, economical, and efficient mixture-of-experts lan- guage model.arXiv preprint arXiv: 2405.04434, 2024a. DeepSeek-AI, Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C., Zhao, C., Deng, C., et al. Deepseek-v3 tech...

Pith/arXiv arXiv 2025

[3] [8]

Buffer overflow in mixture of experts

Hayes, J., Shumailov, I., and Yona, I. Buffer overflow in mixture of experts. InNeurips Safe Generative AI Workshop 2024,

2024

[4] [9]

Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D

Jiang, A. Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D. S., de las Casas, D., Hanna, E. B., et al. Mixtral of experts.arXiv preprint arXiv: 2401.04088,

Pith/arXiv arXiv

[5] [11]

L., Almeida, D., Altenschmidt, J., et al

OpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., et al. Gpt-4 technical report.arXiv preprint arXiv: 2303.08774,

Pith/arXiv arXiv

[6] [12]

V ., Hinton, G

Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q. V ., Hinton, G. E., and Dean, J. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. In5th International Conference on Learning Rep- resentations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net,

2017

[7] [15]

N., Kaiser, L., and Polosukhin, I

9 RepetitionCurse: Measuring and Understanding Router Imbalance in Mixture-of-Experts LLMs under DoS Stress Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. Attention is all you need. In Guyon, I., von Luxburg, U., Bengio, S., Wallach, H. M., Fergus, R., Vishwanathan, S. V . N., and Garnett, R. (...

2017

[8] [16]

I know what you asked: Prompt leakage via kv- cache sharing in multi-tenant llm serving

Wu, G., Zhang, Z., Zhang, Y ., Wang, W., Niu, J., Wu, Y ., and Zhang, Y . I know what you asked: Prompt leakage via kv- cache sharing in multi-tenant llm serving. InProceedings of the 2025 Network and Distributed System Security (NDSS) Symposium. San Diego, CA, USA,

2025

[9] [17]

Leechhijack: Covert computational resource exploitation in intelligent agent systems.arXiv preprint arXiv: 2512.02321,

Zhang, Y ., Wang, W., Zhou, Z., Wang, K., Zhang, J., Sun, L., Liu, Y ., and Su, S. Leechhijack: Covert computational resource exploitation in intelligent agent systems.arXiv preprint arXiv: 2512.02321,

arXiv

[10] [18]

H., Cao, S., Kozyrakis, C., Stoica, I., et al

Zheng, L., Yin, L., Xie, Z., Sun, C., Huang, J., Yu, C. H., Cao, S., Kozyrakis, C., Stoica, I., et al. Sglang: Efficient execution of structured language model programs. In Globersons, A., Mackey, L., Belgrave, D., Fan, A., Pa- quet, U., Tomczak, J. M., and Zhang, C. (eds.),Advances in Neural Information Processing Systems 38: Annual Conference on Neural ...

2024

[11] [19]

Z., and Fredrikson, M

Zou, A., Wang, Z., Carlini, N., Nasr, M., Kolter, J. Z., and Fredrikson, M. Universal and transferable adversarial attacks on aligned language models.arXiv preprint arXiv: 2307.15043,

Pith/arXiv arXiv

[12] [1967]

Longbench v2: Towards deeper understanding and reasoning on realistic long-context multitasks.ArXiv preprint, abs/2412.15204,

Bai, Y ., Tu, S., Zhang, J., Peng, H., Wang, X., Lv, X., Cao, S., Xu, J., Hou, L., et al. Longbench v2: Towards deeper understanding and reasoning on realistic long-context multitasks.ArXiv preprint, abs/2412.15204,

Pith/arXiv arXiv

[13] [2017]

Megatron-lm: Training multi- billion parameter language models using model paral- lelism.arXiv preprint arXiv: 1909.08053,

Shoeybi, M., Patwary, M., Puri, R., LeGresley, P., Casper, J., and Catanzaro, B. Megatron-lm: Training multi- billion parameter language models using model paral- lelism.arXiv preprint arXiv: 1909.08053,

Pith/arXiv arXiv 1909

[14] [2019]

Efficient attention mechanisms for large language models: A survey.arXiv preprint arXiv: 2507.19595,

Sun, Y ., Li, Z., Zhang, Y ., Pan, T., Dong, B., Guo, Y ., and Wang, J. Efficient attention mechanisms for large language models: A survey.arXiv preprint arXiv: 2507.19595,

arXiv

[15] [2021]

Denial-of-service poisoning attacks against large language models.arXiv preprint arXiv: 2410.10760,

Gao, K., Pang, T., Du, C., Yang, Y ., Xia, S.-T., and Lin, M. Denial-of-service poisoning attacks against large language models.arXiv preprint arXiv: 2410.10760,

Pith/arXiv arXiv

[16] [2022]

Switch transformers: Scaling to trillion parameter models with simple and effi- cient sparsity.arXiv preprint arXiv: 2101.03961,

Fedus, W., Zoph, B., and Shazeer, N. Switch transformers: Scaling to trillion parameter models with simple and effi- cient sparsity.arXiv preprint arXiv: 2101.03961,

Pith/arXiv arXiv

[17] [2023]

Fundamental capabilities and applications of large language models: A survey.ACM Comput

Li, J., Gao, Y ., Yang, Y ., Bai, Y ., Zhou, X., Li, Y ., Sun, H., Liu, Y ., Si, X., et al. Fundamental capabilities and applications of large language models: A survey.ACM Comput. Surv., 58(2), 2025a. ISSN 0360-0300. doi: 10.1145/3735632. Li, Y ., Wang, J., Zhu, H., Lin, J., Chang, S., and Guo, M. Thinktrap: Denial-of-service attacks against black-box ll...

arXiv

[18] [2024]

Amdahl, G. M. Validity of the single processor approach to achieving large scale computing capabilities. InPro- ceedings of the April 18-20, 1967, spring joint computer conference, pp. 483–485,

1967

[19] [2025]

DeepSeek-AI

doi: 10.1109/TKDE.2025.3554028. DeepSeek-AI. Eplb: Expert parallelism load bal- ancer. https://github.com/deepseek-ai/ EPLB,

arXiv 2025