Pith. sign in

REVIEW 4 major objections 4 minor 46 references

WaterMoE embeds a watermark by perturbing which experts an MoE model routes tokens through, achieving near-native output quality with about 1% extra inference latency.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 06:30 UTC pith:JAVNRII6

load-bearing objection A genuinely new MoE-routing watermark that shows strong benchmark results, but the detection signal is not proved to accumulate and the evaluation has tuning-calibration gaps. the 4 major comments →

arxiv 2607.13099 v1 pith:JAVNRII6 submitted 2026-07-14 cs.CR cs.AI

WaterMoE: Expert-Routing-based Watermarking for High Fidelity and Efficiency

Z Sun , Q Jiang , S Sheng , L Xiang This is my paper
classification cs.CR cs.AI
keywords LLM watermarkingMixture-of-Expertsrouting biasexpert selectiontext provenanceinference efficiencytoken-sampling watermarkdetection statistic
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that existing LLM watermarks fail in practice because they distort the token-sampling distribution and add per-token hashing overhead, hurting performance on constrained tasks and inflating latency. It proposes instead to embed the watermark inside the routing logic of Mixture-of-Experts (MoE) models, which are increasingly used in large LLMs. The mechanism is a small additive bias applied at inference time to a randomly chosen half of the experts in every MoE layer, shifting which experts are selected without materially changing what they compute. The paper proves this bias changes expected routing probabilities only at second order, so output quality stays close to native, while the accumulated routing shifts produce a statistically detectable drift in token likelihoods. The authors claim detection rates above 90% at 1% false-positive on code and reasoning tasks, a 4x speedup in watermark insertion compared with previous methods, and only about 1% latency overhead over native generation.

Core claim

WaterMoE places the watermark in the MoE router: a precomputed map labels half the experts in each layer as green, and during generation a small positive bias δ is added to their routing scores. Because green experts are chosen from functionally similar alternatives, the routing distribution shifts only to second order (Proposition 1), so output quality stays close to native; but across many layers the bias accumulates into a consistent, statistically detectable preference for watermarked tokens. Detection evaluates the same model with and without the bias and counts tokens where the biased log-likelihood exceeds the base log-likelihood, reporting high detection accuracy and about 1% waterma

What carries the argument

The Green Expert Map Mℓ is a random, deterministic function from the previous layer's top-1 selected expert to a subset of K/2 'green' experts per MoE layer. Adding the scalar bias δ to the routing scores of exactly those green experts is the watermark carrier. The key theoretical result, Proposition 1, shows that the expected routing probability changes only as O(δ²) because the green assignment is permutation-symmetric, which keeps the perturbation nearly invisible to the model's internal representations. The detector uses the statistic S(x;τ), the fraction of tokens where the biased model's log-likelihood exceeds the unbiased model's by τ, calibrated by a z-score Z(x) against a reference

Load-bearing premise

The detection statistic assumes that adding a small bias to router scores reliably makes the biased model assign higher log-likelihood than the base model to the generated tokens; the paper proves only that the expected routing shift is second-order, not that this drift is positive for any given task.

What would settle it

Run WaterMoE on a high-entropy or heavily constrained task (e.g., open-domain dialogue or free-form creative writing) and compute the mean of Δt over a large sample; if the mean is not significantly positive, the detection test cannot distinguish watermarked text, and the central claim fails. Alternatively, measure TPR@1% across tasks with very different routing entropy at the same bias strength and check whether detection collapses.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Watermarking can be integrated into MoE inference with negligible insertion cost, making it deployable in latency-critical serving environments.
  • The method preserves near-native quality on high-complexity tasks such as competitive coding, math reasoning, and instruction following, where token-level watermarks degrade accuracy by over 15 points in the paper's experiments.
  • Detection requires no separate model: the same model is evaluated twice under two routing configurations and a z-test on the log-likelihood difference flags watermarked text.
  • The watermark is stealthy under black-box statistical detectors: the paper reports high p-values (around 0.9–1.0) under R-G and FIXED detection, unlike several token-sampling baselines.
  • Robustness to heavy semantic rewriting is limited: under a full-text paraphrase attack detection drops to 63% TPR@1%, a limitation the paper acknowledges.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The detection pass requires running the model twice (biased and unbiased). The paper reports detection at roughly 150 ms per sample, but for very long sequences the unbiased pass doubles the compute; the 1% latency figure applies to insertion only, not the full detect-on-request workflow.
  • The O(δ²) bound concerns expected routing probabilities, not the sign of the per-token log-likelihood difference Δt. On tasks with very high token or routing entropy, the aggregate drift could vanish or even reverse, making detection unreliable; a task-level experiment measuring the mean of Δt would map where the watermark still works.
  • Because the bias is applied to a random half of experts, watermark detectability likely depends on how many functionally interchangeable experts exist per layer. In models with highly specialized experts, the same δ may produce larger distribution shifts (stronger signal but more quality loss) or weaker shifts—an interaction not quantified in the paper.
  • A natural extension, not explored here, is embedding the watermark during iterative decoding or multi-token generation where routing decisions for future tokens depend on past biased routing; the paper evaluates only standard autoregressive generation, so accumulated bias could entangle with the model's feedback loops in ways not captured by the single-layer analysis.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes WaterMoE, a watermarking scheme for Mixture-of-Experts (MoE) LLMs that embeds a watermark by adding a fixed bias to the routing scores of a precomputed 'green expert' set, without modifying token logits. Detection is performed with the same model under biased versus unbiased routing configurations: for a candidate text, the token-level log-likelihood difference Δ_t = log P_bias(x_t) - log P_base(x_t) is computed, the fraction S(x;τ) of tokens with Δ_t > τ is obtained, and a Z-test is applied using reference calibration statistics µ0, σ0. The authors benchmark WaterMoE against nine watermarking baselines on ten datasets organized into low-, moderate-, and high-complexity tiers, reporting state-of-the-art TPR@1%, near-native task quality, ~1% embedding latency overhead, and up to 4× speedup over token-sampling methods.

Significance. If the empirical results hold, WaterMoE provides a genuinely different watermarking design that moves the watermark signal from the token-sampling layer into the MoE routing mechanism, which is timely given the increasing deployment of MoE models. The benchmark is broad (10 datasets, 9 baselines, 2 MoE models) and the efficiency and stealthiness analyses are useful additions. However, detection power is not justified by the presented theory, and the evaluation lacks calibration details, error bars, and a clear hyperparameter selection protocol. The core idea is sufficiently interesting to merit revision, but the evidence is not yet at the standard needed for acceptance.

major comments (4)
  1. [§3.1, Eq. (4)–(6), Appendix C] Proposition 1 (Eq. (26)) only shows that the expected routing probability at a single layer is first-order invariant over the random green map. The detection statistic S(x;τ) is defined from token-level log-likelihood differences Δ_t; the paper does not prove or quantitatively analyze that E[Δ_t] > 0 under WaterMoE generation, nor how δ, depth, or task entropy affect the aggregate Z-score. The 'Why works?' passage asserts amplification through feed-forward dynamics, but this is an assertion, not a derivation. Because detection power is the central claim, please supply either a formal analysis of Δ_t or an empirical decomposition showing that the positive shift of Δ_t is caused by the routing bias and not by other artifacts.
  2. [§3.2, Eq. (6), Tables 2–3] The calibration of µ0(τ) and σ0(τ) is underspecified. It is not stated whether reference texts are unwatermarked outputs of the same model, how many are used, whether they are task/length-matched, or how the 1% FPR threshold is imposed. All TPR/AUC values are point estimates without confidence intervals or sample sizes. Without these details, the claimed 90–100% TPR values cannot be reliably compared across methods. Please specify the calibration corpus, the sample sizes, and report variance across repeated generations/splits.
  3. [§4.1 vs Table 7] The paper states 'we set as default δ=0.2 and τ=0 across all experiments,' yet Table 7 assigns δ=0.8 to the low-complexity tier. This inconsistency, plus the absence of a stated held-out selection protocol for δ and for baseline strengths, makes the per-tier tuning look post hoc. If δ was chosen on the test data, the detection numbers are optimistic. Please clarify the exact selection protocol and, if possible, include results for a fixed δ across all tiers.
  4. [§4.4 / RQ4] Robustness is evaluated for WaterMoE alone (e.g., 63% TPR@1 under GLM-4.5 paraphrasing), but no per-attack comparison to token-sampling baselines appears in the main text. Since the paper claims consistent outperformance over state-of-the-art methods, the robustness dimension should be compared under identical attack prompts. Please add baseline rows/columns for the attacks tested, including the deletion and paraphrase settings.
minor comments (4)
  1. [Table 2] The SynthID row reports Accuracy=41.1 on IFEval, which is far below its TPR and the accuracy of all other methods; this appears to be a typo.
  2. [Table 4] The 'EXP' rows display garbled values such as '1.001.8×10−6'; use proper p-value formatting.
  3. [Appendix C] The proof assumes δ≪1, but Table 7 uses δ=0.8; the O(δ²) bound is not obviously tight at this value. Please discuss this discrepancy.
  4. [Fig. 3] The '4× speedup' claim is not clearly derived from the plotted insertion-latency overhead percentages; specify the comparator and the formula used.

Circularity Check

0 steps flagged

No significant circularity: WaterMoE's routing-bias watermark is defined independently of its detection statistic; δ, τ, and the green map are fixed before evaluation, and detection performance is measured against external benchmarks with standard Z-test calibration.

full rationale

Walking the claimed derivation chain: (1) Embedding (Sec. 3.1, Eqs. 1–3) defines the green expert map Mℓ and the additive routing bias δgℓ with a fixed green proportion of K/2; nothing in this definition references the detection statistic S(x;τ) or the calibration moments µ0, σ0. (2) Proposition 1 (Sec. 3.1, Appendix C) is a genuine first-principles result: it shows the first-order term of the routing-probability shift vanishes in expectation over the permutation-symmetric random map and bounds the second-order term by (5/8)pᵢδ². Its assumptions (fixed green proportion, Pr(g_{ℓ,i}=1)=1/2, small δ) do not include the target claim of detectability, so it is not a self-fulfilling premise. (3) Detection (Sec. 3.2, Eqs. 4–6) computes a per-token log-likelihood difference between biased and unbiased routing of the same model and calibrates Z(x) = (S−µ0)/σ0. Estimating µ0 and σ0 on reference texts is standard hypothesis-test normalization; the watermarked texts are not used to set those constants, so no fitted parameter is renamed as a prediction. (4) All reported TPR/AUC/PPL values are measured on text generated for external datasets (C4, Booksum, MultiNews, ELI5, APPS, CodeContests, GSM8K, MMLU, IFEval, WritingBench) with fixed defaults (δ=0.2, τ=0) and per-tier operating points (Table 7), compared against nine independent baselines; the benchmark is external, not constructed from the model's own outputs. (5) The reference list contains no self-citations, and the motivating MoE citations ([21], [22]) support only the generic flexibility of expert routing, not the watermark design itself. The paper's genuine weakness — detection power is asserted via the 'Why works?' paragraph and not formally derived from Prop. 1, since that proposition averages over random green maps and does not by itself establish a positive mean of Δₜ for a fixed map — is a completeness/correctness gap (the mean log-likelihood ratio is non-negative by a Gibbs/KL argument for text sampled under the biased model), not a circular reduction. No equation in the paper reduces to its own input by construction, and the Sec. 5 limitation (MoE-only scope) is an acknowledged design boundary, not an admission of circularity.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The central claim rests on five main assumptions: the permutation-symmetric green map, the functional-similarity fidelity premise, the unproven detectability drift, standard calculus, and the representativeness of the calibration null. The free parameters are the routing bias δ, the token threshold τ, and the null calibration statistics. No new physical or conceptual entities are introduced beyond the green-expert map, which is a secret key/artifact rather than a postulated entity.

free parameters (3)
  • routing bias δ = 0.2 default; 0.8 for low-complexity tasks
    Watermark strength added to green expert logits (Eq. 3); tuned per task tier in Table 7 and varied 0.2-1.2 in sensitivity analysis; detection results depend on it.
  • token-level threshold τ = 0
    Threshold on Δ_t in Eq. 5; set to 0 by default with no sensitivity analysis.
  • reference calibration statistics μ0(τ), σ0(τ) = estimated from reference model outputs
    Mean and std of S under null, Eq. 6; estimated on data, so the detector's z-score depends on fitted null parameters.
axioms (5)
  • domain assumption Green expert map is permutation-symmetric: every expert has equal marginal probability 1/2 of being green, independent of routing state.
    Required for Prop. 1's first-order cancellation (Eq. 15 in Appendix C); can be constructed by random maps, but the paper does not prove the map remains independent of e_{l-1} once e_{l-1} is biased by previous layers.
  • domain assumption Adding a small bias to routing scores among functionally similar experts leaves output quality nearly unchanged.
    Core fidelity argument in §3.1 and §3.2; supported by intuition and citations [21,22], not measured directly.
  • ad hoc to paper The biased routing trajectory shifts token log-likelihoods positively enough for the aggregate statistic S(x;τ) to be detectable.
    The detection mechanism in §3.2 assumes consistent positive Δ_t under watermark; no formal bound is given; Prop. 1 only bounds expected routing probability shift.
  • standard math Softmax derivative identity and Taylor expansion truncated at O(δ²).
    Used in Appendix C proof of Prop. 1; standard calculus.
  • domain assumption Reference model likelihoods under base configuration provide a valid null distribution for arbitrary text.
    Detector calibrates μ0, σ0 on reference model (§3.2); false-positive control depends on this null being representative across domains.

pith-pipeline@v1.3.0-alltime-deepseek · 22144 in / 12946 out tokens · 127171 ms · 2026-08-02T06:30:29.675602+00:00 · methodology

0 comments
read the original abstract

Large language models (LLMs) have achieved remarkable success but raise growing concerns about content provenance and misuse, motivating the need for reliable watermarking techniques. However, these techniques have rarely been adopted in practice mainly for two reasons: i) severely degraded model performance, and ii) additional inference overhead. To confirm the problem, we construct a comprehensive benchmark spanning different generation tasks to systematically evaluate 9 representative watermarking methods. We found almost all existing methods are designed for text fluency, but not for restricted and complicated tasks, and their overhead prevents them from deployment in latency-critical systems. To address i) and ii), we propose an LLM watermarking scheme \textit{WaterMoE} for the growingly popular Mixture-of-Experts (MoE) LLMs. WaterMoE embeds watermarking signals through controlled perturbation into the expert selection at each router, which accumulates to token selection shift at the final output. In contrast to watermarking as a post-processing token-sampling approach, WaterMoE embeds watermark within the inference loop incurring negligible quality degradation and computational overhead. Extensive experiments demonstrate that our method achieves a fidelity performance close to the unwatermarked and consistently outperforms state-of-the-art watermarking methods on the benchmark, with up to $4\times$ speedup, incurring merely 1\% additional inference latency compared to native generation. The results demonstrate the capability of WaterMoE to be deployed in real-world tasks.

Figures

Figures reproduced from arXiv: 2607.13099 by L Xiang, Q Jiang, S Sheng, Z Sun.

Figure 1
Figure 1. Figure 1: (a) Efficiency comparison on Mixtral-8x7B between WaterMoE and baseline watermarking algorithms (KGW, SynthID, SIR, and Morphmark). We report per-token insertion latency overhead (%) and average detection time per sample (ms). (b) Performance com￾parison of WaterMoE across different methods. The x-axis denotes text gen￾eration quality measured by ROUGE-L, while the y-axis represents watermark detectability… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the WaterMoE framework. (a) Watermark embedding via routing bias. A [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: (a) We report per-token insertion latency overhead (%) and average detection time per [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Detailed Framework of MoE. A Related Work Semantic-level watermarking. Recent approaches explore embedding watermark signals at the semantic level. SIR [42] employs learned encoders to map semantically similar inputs to similar watermark patterns, improving robustness to paraphrasing. However, this design introduces additional system complexity and incurs non-trivial latency due to sentence-level encoding.… view at source ↗
Figure 5
Figure 5. Figure 5: Trigram entropy and semantic entropy comparison. WaterMoE preserves entropy closest to [PITH_FULL_IMAGE:figures/full_fig_p018_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Detailed Global Prompts used across various datasets to standardize model instructions and [PITH_FULL_IMAGE:figures/full_fig_p018_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Prompt used for the token-level substitution attack. [PITH_FULL_IMAGE:figures/full_fig_p019_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Prompt used for the document-level paraphrasing attack. [PITH_FULL_IMAGE:figures/full_fig_p019_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: ROC-AUC curves of different watermarking methods on MultiNews (Mixtral-8x7B). [PITH_FULL_IMAGE:figures/full_fig_p020_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: An example of the Summarization task from the Multi-News dataset. The texts are [PITH_FULL_IMAGE:figures/full_fig_p020_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: An example of the Coding task from the APPS dataset. The text demonstrates the prompt [PITH_FULL_IMAGE:figures/full_fig_p021_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: An example of the Alignment task from the IFEval dataset. The text illustrates the prompt [PITH_FULL_IMAGE:figures/full_fig_p021_12.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

46 extracted references · 16 linked inside Pith

  1. [1]

    A survey of llm-based agents in medicine: How far are we from baymax? Findings of the Association for Computational Linguistics: ACL 2025, pages 10345–10359, 2025

    Wenxuan Wang, Zizhan Ma, Zheng Wang, Chenghan Wu, Jiaming Ji, Wenting Chen, Xiang Li, and Yixuan Yuan. A survey of llm-based agents in medicine: How far are we from baymax? Findings of the Association for Computational Linguistics: ACL 2025, pages 10345–10359, 2025

    2025

  2. [2]

    A continued pretrained llm approach for automatic medical note generation

    Dong Yuan, Eti Rastogi, Gautam Naik, Sree Prasanna Rajagopal, Sagar Goyal, Fen Zhao, Bharath Chintagunta, and Jeffrey Ward. A continued pretrained llm approach for automatic medical note generation. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Sh...

    2024

  3. [3]

    When to retrieve: Teaching llms to utilize information retrieval effectively

    Tiziano Labruna, Jon Ander Campos, and Gorka Azkune. When to retrieve: Teaching llms to utilize information retrieval effectively. InProceedings of the 15th International Conference on Recent Advances in Natural Language Processing-Natural Language Processing in the Generative AI Era, pages 623–632, 2025

    2025

  4. [4]

    Large language models for information retrieval: A survey.ACM Transactions on Information Systems, 44(1):1–54, 2025

    Yutao Zhu, Huaying Yuan, Shuting Wang, Jiongnan Liu, Wenhan Liu, Chenlong Deng, Haonan Chen, Zheng Liu, Zhicheng Dou, and Ji-Rong Wen. Large language models for information retrieval: A survey.ACM Transactions on Information Systems, 44(1):1–54, 2025

    2025

  5. [5]

    Rethinking driving world model as synthetic data generator for perception tasks.arXiv preprint arXiv:2510.19195, 2025

    Kai Zeng, Zhanqian Wu, Kaixin Xiong, Xiaobao Wei, Xiangyu Guo, Zhenxin Zhu, Kalok Ho, Lijun Zhou, Bohan Zeng, Ming Lu, et al. Rethinking driving world model as synthetic data generator for perception tasks.arXiv preprint arXiv:2510.19195, 2025

    arXiv 2025

  6. [6]

    Adapting llms for efficient, personalized information retrieval: Methods and implications

    Samira Ghodratnama and Mehrdad Zakershahrak. Adapting llms for efficient, personalized information retrieval: Methods and implications. InInternational conference on service-oriented computing, pages 17–26. Springer, 2023

    2023

  7. [7]

    Competition-level code generation with alphacode.Science, 378(6624):1092–1097, 2022

    Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, et al. Competition-level code generation with alphacode.Science, 378(6624):1092–1097, 2022

    2022

  8. [8]

    A review on code generation with llms: Application and evaluation

    Jianxun Wang and Yixiang Chen. A review on code generation with llms: Application and evaluation. In2023 IEEE International Conference on Medical Artificial Intelligence (MedAI), pages 284–289. IEEE, 2023

    2023

  9. [9]

    An empirical study of retrieval-augmented code generation: Challenges and opportunities.ACM Transactions on Software Engineering and Methodology, 34(7):1–28, 2025

    Zezhou Yang, Sirong Chen, Cuiyun Gao, Zhenhao Li, Xing Hu, Kui Liu, and Xin Xia. An empirical study of retrieval-augmented code generation: Challenges and opportunities.ACM Transactions on Software Engineering and Methodology, 34(7):1–28, 2025

    2025

  10. [10]

    A survey on large language models for code generation.ACM Transactions on Software Engineering and Methodology, 35(2):1–72, 2026

    Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. A survey on large language models for code generation.ACM Transactions on Software Engineering and Methodology, 35(2):1–72, 2026

    2026

  11. [11]

    Understanding the planning of llm agents: A survey.arXiv preprint arXiv:2402.02716, 2024

    Xu Huang, Weiwen Liu, Xiaolong Chen, Xingmei Wang, Hao Wang, Defu Lian, Yasheng Wang, Ruiming Tang, and Enhong Chen. Understanding the planning of llm agents: A survey.arXiv preprint arXiv:2402.02716, 2024

    Pith/arXiv arXiv 2024

  12. [12]

    Dynamic llm-agent net- work: An llm-agent collaboration framework with agent team optimization.arXiv preprint arXiv:2310.02170, 2023

    Zijun Liu, Yanzhe Zhang, Peng Li, Yang Liu, and Diyi Yang. Dynamic llm-agent net- work: An llm-agent collaboration framework with agent team optimization.arXiv preprint arXiv:2310.02170, 2023

    Pith/arXiv arXiv 2023

  13. [13]

    Llm agents for education: Advances and applications

    Zhendong Chu, Shen Wang, Jian Xie, Tinghui Zhu, Yibo Yan, Jinheng Ye, Aoxiao Zhong, Xuming Hu, Jing Liang, Philip S Yu, et al. Llm agents for education: Advances and applications. arXiv preprint arXiv:2503.11733, 2, 2025

    arXiv 2025

  14. [14]

    Agentswift: Efficient llm agent design via value-guided hierarchical search

    Yu Li, Lehui Li, Zhihao Wu, Qingmin Liao, Jianye Hao, Kun Shao, and Fengli Xu. Agentswift: Efficient llm agent design via value-guided hierarchical search. InProceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 31843–31851, 2026

    2026

  15. [15]

    Watermarking techniques for large language models: A survey.Artificial Intelligence Review, 59(2):74, 2026

    Yuqing Liang, Jiancheng Xiao, Wensheng Gan, and Philip S Yu. Watermarking techniques for large language models: A survey.Artificial Intelligence Review, 59(2):74, 2026. 10

    2026

  16. [16]

    Rlspoofer: A lightweight evaluator for llm watermark spoofing resilience.arXiv preprint arXiv:2604.11546, 2026

    Hanbo Huang, Xuan Gong, Yiran Zhang, Hao Zheng, and Shiyu Liang. Rlspoofer: A lightweight evaluator for llm watermark spoofing resilience.arXiv preprint arXiv:2604.11546, 2026

    Pith/arXiv arXiv 2026

  17. [17]

    A watermark for large language models

    John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. InInternational conference on machine learning, pages 17061–17084. PMLR, 2023

    2023

  18. [18]

    Scalable watermarking for identifying large language model outputs.Nature, 634(8035):818–823, 2024

    Sumanth Dathathri, Abigail See, Sumedh Ghaisas, Po-Sen Huang, Rob McAdam, Johannes Welbl, Vandana Bachani, Alex Kaskasoli, Robert Stanforth, Tatiana Matejovicova, et al. Scalable watermarking for identifying large language model outputs.Nature, 634(8035):818–823, 2024

    2024

  19. [19]

    A survey on mixture of experts.Authorea Preprints, 2024

    Weilin Cai, Juyong Jiang, Fan Wang, Jing Tang, Sunghun Kim, and Jiayi Huang. A survey on mixture of experts.Authorea Preprints, 2024

    2024

  20. [20]

    Llama-moe: Building mixture-of-experts from llama with continual pre-training

    Tong Zhu, Xiaoye Qu, Daize Dong, Jiacheng Ruan, Jingqi Tong, Conghui He, and Yu Cheng. Llama-moe: Building mixture-of-experts from llama with continual pre-training. InProceedings of the 2024 conference on empirical methods in natural language processing, pages 15913– 15923, 2024

    2024

  21. [21]

    Deepspeed-moe: Advancing mixture-of- experts inference and training to power next-generation ai scale

    Samyam Rajbhandari, Conglong Li, Zhewei Yao, Minjia Zhang, Reza Yazdani Aminabadi, Ammar Ahmad Awan, Jeff Rasley, and Yuxiong He. Deepspeed-moe: Advancing mixture-of- experts inference and training to power next-generation ai scale. InInternational conference on machine learning, pages 18332–18346. PMLR, 2022

    2022

  22. [22]

    Mixture of cache-conditional experts for efficient mobile device inference.arXiv preprint arXiv:2412.00099, 2024

    Andrii Skliar, Ties van Rozendaal, Romain Lepert, Todor Boinovski, Mart Van Baalen, Markus Nagel, Paul Whatmough, and Babak Ehteshami Bejnordi. Mixture of cache-conditional experts for efficient mobile device inference.arXiv preprint arXiv:2412.00099, 2024

    Pith/arXiv arXiv 2024

  23. [23]

    Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    Pith/arXiv arXiv 2024

  24. [24]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    Pith/arXiv arXiv 2025

  25. [25]

    Uni-moe: Scaling unified multimodal llms with mixture of experts.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

    Yunxin Li, Shenyuan Jiang, Baotian Hu, Longyue Wang, Wanqi Zhong, Wenhan Luo, Lin Ma, and Min Zhang. Uni-moe: Scaling unified multimodal llms with mixture of experts.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

    2025

  26. [26]

    An entropy-based text water- marking detection method

    Yijian Lu, Aiwei Liu, Dianzhi Yu, Jingjing Li, and Irwin King. An entropy-based text water- marking detection method. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11724–11735, 2024

    2024

  27. [27]

    Morphmark: Flexible adaptive watermarking for large language models

    Zongqi Wang, Tianle Gu, Baoyuan Wu, and Yujiu Yang. Morphmark: Flexible adaptive watermarking for large language models. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4842–4860, 2025

    2025

  28. [28]

    Markllm: An open-source toolkit for llm watermarking

    Leyi Pan, Aiwei Liu, Zhiwei He, Zitian Gao, Xuandong Zhao, Yijian Lu, Binglin Zhou, Shuliang Liu, Xuming Hu, Lijie Wen, et al. Markllm: An open-source toolkit for llm watermarking. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 61–71, 2024

    2024

  29. [29]

    Mixtral of experts.arXiv preprint arXiv:2401.04088, 2024

    Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Mixtral of experts.arXiv preprint arXiv:2401.04088, 2024

    Pith/arXiv arXiv 2024

  30. [30]

    Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of machine learning research, 21(140):1–67, 2020

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of machine learning research, 21(140):1–67, 2020

    2020

  31. [31]

    Booksum: A collection of datasets for long-form narrative summarization

    Wojciech Kry´sci´nski, Nazneen Rajani, Divyansh Agarwal, Caiming Xiong, and Dragomir Radev. Booksum: A collection of datasets for long-form narrative summarization. InFindings of the association for computational linguistics: EMNLP 2022, pages 6536–6558, 2022. 11

    2022

  32. [32]

    Multi-news: A large-scale multi-document summarization dataset and abstractive hierarchical model

    Alexander Richard Fabbri, Irene Li, Tianwei She, Suyi Li, and Dragomir Radev. Multi-news: A large-scale multi-document summarization dataset and abstractive hierarchical model. In Proceedings of the 57th annual meeting of the association for computational linguistics, pages 1074–1084, 2019

    2019

  33. [33]

    Eli5: Long form question answering

    Angela Fan, Yacine Jernite, Ethan Perez, David Grangier, Jason Weston, and Michael Auli. Eli5: Long form question answering. InProceedings of the 57th annual meeting of the association for computational linguistics, pages 3558–3567, 2019

    2019

  34. [34]

    Measuring coding challenge competence with apps.arXiv preprint arXiv:2105.09938, 2021

    Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, et al. Measuring coding challenge competence with apps.arXiv preprint arXiv:2105.09938, 2021

    Pith/arXiv arXiv 2021

  35. [35]

    Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Pith/arXiv arXiv 2021

  36. [36]

    Measuring massive multitask language understanding.arXiv preprint arXiv:2009.03300, 2020

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding.arXiv preprint arXiv:2009.03300, 2020

    Pith/arXiv arXiv 2009

  37. [37]

    Instruction-following evaluation for large language models.arXiv preprint arXiv:2311.07911, 2023

    Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-following evaluation for large language models.arXiv preprint arXiv:2311.07911, 2023

    Pith/arXiv arXiv 2023

  38. [38]

    Writingbench: A comprehensive benchmark for generative writing.arXiv preprint arXiv:2503.05244, 2025

    Yuning Wu, Jiahao Mei, Ming Yan, Chenliang Li, Shaopeng Lai, Yuran Ren, Zijia Wang, Ji Zhang, Mengyue Wu, Qin Jin, et al. Writingbench: A comprehensive benchmark for generative writing.arXiv preprint arXiv:2503.05244, 2025

    arXiv 2025

  39. [39]

    Robust distortion- free watermarks for language models.arXiv preprint arXiv:2307.15593, 2023

    Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. Robust distortion- free watermarks for language models.arXiv preprint arXiv:2307.15593, 2023

    Pith/arXiv arXiv 2023

  40. [40]

    Unbiased watermark for large language models.arXiv preprint arXiv:2310.10669, 2023

    Zhengmian Hu, Lichang Chen, Xidong Wu, Yihan Wu, Hongyang Zhang, and Heng Huang. Unbiased watermark for large language models.arXiv preprint arXiv:2310.10669, 2023

    Pith/arXiv arXiv 2023

  41. [41]

    An unforgeable publicly verifiable watermark for large language models.arXiv preprint arXiv:2307.16230, 2023

    Aiwei Liu, Leyi Pan, Xuming Hu, Shu’ang Li, Lijie Wen, Irwin King, and Philip S Yu. An unforgeable publicly verifiable watermark for large language models.arXiv preprint arXiv:2307.16230, 2023

    Pith/arXiv arXiv 2023

  42. [42]

    A semantic invariant robust watermark for large language models.arXiv preprint arXiv:2310.06356, 2023

    Aiwei Liu, Leyi Pan, Xuming Hu, Shiao Meng, and Lijie Wen. A semantic invariant robust watermark for large language models.arXiv preprint arXiv:2310.06356, 2023

    Pith/arXiv arXiv 2023

  43. [43]

    Can watermarks survive translation? on the cross-lingual consistency of text watermark for large language models

    Zhiwei He, Binglin Zhou, Hongkun Hao, Aiwei Liu, Xing Wang, Zhaopeng Tu, Zhuosheng Zhang, and Rui Wang. Can watermarks survive translation? on the cross-lingual consistency of text watermark for large language models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4115–4129, 2024

    2024

  44. [44]

    Black-box detection of language model watermarks.arXiv preprint arXiv:2405.20777, 2024

    Thibaud Gloaguen, Nikola Jovanovi´c, Robin Staab, and Martin Vechev. Black-box detection of language model watermarks.arXiv preprint arXiv:2405.20777, 2024

    Pith/arXiv arXiv 2024

  45. [45]

    Semstamp: A semantic watermark with paraphrastic robustness for text generation

    Abe Hou, Jingyu Zhang, Tianxing He, Yichen Wang, Yung-Sung Chuang, Hongwei Wang, Lingfeng Shen, Benjamin Van Durme, Daniel Khashabi, and Yulia Tsvetkov. Semstamp: A semantic watermark with paraphrastic robustness for text generation. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human ...

    2024

  46. [46]

    Global Prompt

    Abe Hou, Jingyu Zhang, Yichen Wang, Daniel Khashabi, and Tianxing He. k-semstamp: A clustering-based semantic watermark for detection of machine-generated text. InFindings of the Association for Computational Linguistics: ACL 2024, pages 1706–1715, 2024. 12 Layer Norm MoE Layer Attention Layer Norm Router FFN 1 FFN 2 FFN N Topk Sum Weighted Layer 1 Layer ...

    2024