REVIEW 2 major objections 7 minor 38 references
A disentangled MoE with RL routing generalizes AI-text detection to unseen domains.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-04 00:22 UTC pith:56CQKGNV
load-bearing objection Promising architecture, but the zero-shot gains are inflated by test-set hyperparameter tuning; needs revision before the headline claim can be trusted. the 2 major comments →
DEER: Disentangled Mixture of Experts with Instance-Adaptive Routing for Generalizable Machine-Generated Text Detection
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
DEER's central claim is that explicitly decoupling domain-local and domain-invariant knowledge—rather than pooling all training domains into one model—preserves the transferable signatures of machine generation, and that a task-reward-driven router can exploit that structure without domain labels at test time. Concretely, training instantiates one expert per source domain plus a shared expert, with a domain-aware gate; inference uses a policy network to output a distribution over source domains, selects the top-m, and aggregates their logits weighted by selection probability. The reported result is that this beats state-of-the-art detectors on both in-domain and five unseen out-of-domain dat
What carries the argument
The load-bearing mechanism is the two-stage DEER architecture: (1) a Disentangled Mixture-of-Experts with per-domain expert MLPs and a shared expert, trained with domain labels so each expert owns a domain's local patterns while the shared expert learns cross-domain generation signatures; (2) an RL policy network, trained with REINFORCE on detection rewards (accuracy minus a domain-average baseline), that maps each input's encoded state to a soft distribution over source domains. At inference, top-m domain experts plus the shared expert are fused by policy probabilities. The RL reward is what aligns routing with the actual detection objective instead of domain similarity.
Load-bearing premise
The load-bearing premise is that the RL policy trained on source-domain classification rewards will pick experts that generalize to unseen domains, rather than exploiting shortcuts to the binary label (such as always choosing the expert whose bias matches the majority class); the paper does not analyze the policy's action distribution on OOD inputs to rule this out.
What would settle it
A concrete check: take a trained DEER model, feed out-of-domain inputs, record the policy's top-1 domain choice per input, and compare that distribution against a uniform or majority-class baseline. If the policy collapses (e.g., always picks the same domain or the domain whose label prior dominates), and OOD F1 drops to the level of that single expert, the central claim of instance-adaptive routing is falsified. Equivalently, training the same architecture with a randomly initialized policy frozen at uniform weights should match the reported OOD gains if routing were not doing the work.
If this is right
- Domain-generalizable MGT detection improves without target-domain labels, as shown by average F1 gains of 5.32% out-of-domain and 1.39% in-domain over the strongest baseline.
- Detectors built this way tolerate input perturbations; DEER keeps higher F1 than metric-based and model-based baselines under repetition, deletion, and replacement attacks.
- Modular expansion works: adapting to a new domain by adding one expert and fine-tuning only shared experts preserves old-domain performance while cutting trainable parameters by roughly 90% versus full fine-tuning.
- Because the router is optimized for detection reward rather than style match, aggregating multiple expert groups yields further gains, plateauing near m=3.
Where Pith is reading between the lines
- A natural next test is whether the learned policy's selection probabilities correlate with actual domain similarity when labels are available on a held-out domain; if they do not, the RL objective is selecting for something subtler than stylistic match.
- The reward baseline (average over all domain experts) implies the policy is trained to beat the average expert; a direct implication is that the approach may be most effective when source domains are diverse enough that the average expert is weak, which could be probed by varying domain diversity.
- The architecture suggests a recipe for other open-set detection tasks where domain labels exist during training but vanish at test time: disentangle per-domain experts, then learn a reward-driven router.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes DEER, a two-stage framework for domain-generalizable machine-generated text (MGT) detection. In the first stage, a disentangled mixture-of-experts (DMoE) module is trained with source-domain supervision: each source domain has dedicated experts, and shared experts capture cross-domain features. In the second stage, the DMoE is frozen and a reinforcement-learning policy is trained to route each input instance to the most useful top-m expert groups at inference, without requiring domain labels. The authors evaluate on five in-domain and five out-of-domain MAGE datasets and report state-of-the-art accuracy and F1, with headline OOD gains of about 5.32% F1 and 3.61% accuracy. Ablations, robustness tests, and an incremental-adaptation experiment are also reported.
Significance. If the out-of-domain result is substantiated, the contribution is meaningful: decoupling domain-local and domain-invariant expertise is a plausible route to more generalizable MGT detection, and the RL-based instance-level router addresses a real deployment gap when domain labels are absent. The paper's Table 2 ablations support the value of both expert types and of the learned routing over oracle/random/classifier alternatives, and the robustness and incremental-adaptation analyses are useful additions. However, the central OOD claim is currently weakened by a selection-protocol problem: key architectural hyperparameters appear to be chosen using the target-domain test sets. The significance of the paper therefore depends on a clean re-evaluation with hyperparameters fixed on source validation data only.
major comments (2)
- [Appendix C.1 / Table 1] The headline zero-shot DG-MGT result is compromised by test-set hyperparameter selection. Appendix C.1 states that the numbers of domain-specific experts m1, shared experts m2, and the routing parameter top-m were chosen using the DG-MGT F1 scores shown in Figure 3. Since DG-MGT is the five unseen target domains (CMV, ELI5, WP, TLDR, ROCT), these target-test labels influence architectural choices before the final numbers in Table 1 are produced. The baselines are not given the same target-data privilege. Consequently, the reported average OOD improvements of 5.32% F1 and 3.61% accuracy cannot be interpreted as clean zero-shot generalization; they may partly reflect oracle hyperparameter fitting. The source-domain validation split described in Appendix A.1 should be used for all model selection. Please rerun the comparison with hyperparameters fixed on source validation only, and report b
- [§3.2, Eq. (5)-(6), Table 2] The routing policy is trained with rewards derived from the frozen detector's own binary classification outcome on the training instance. This objective rewards selecting any expert that makes the detector's prediction correct, not necessarily the expert that captures domain-relevant structure. The paper does not analyze the learned policy's action distribution, nor does it show that the selected top-m expert groups are sensible on out-of-domain inputs. Please add diagnostics: the policy's entropy and action frequencies on source vs. OOD data, a comparison to a uniform top-m ensemble, and a comparison to a policy trained with an independent domain-classification reward. Without these, the claim that DEER achieves 'task-aligned, domain-agnostic' routing rather than a label shortcut is not directly evidenced.
minor comments (7)
- [Abstract / §4.3] The abstract at the beginning of the submission reports average F1 improvements of 1.28% and 2.92%, and accuracy gains of 1.35% and 2.26%, whereas the full-text abstract and Section 4.3 report 1.39% and 5.32% F1 and 1.35% and 3.61% accuracy. These headline numbers must be reconciled.
- [§2 Related Work] There are missing citation placeholders in the Mixture-of-Experts paragraph: '(Jacobs et al. 1991;?)' and '(Li et al. 2022; ?;?;?)'. Please complete these references.
- [Table 1] Many cells in Table 1 have malformed formatting, e.g., the RoBERTa row reads '88.77 0.64 95.241.07' with no clear separation between accuracy and standard deviation. Please reformat the table so each value is readable.
- [Algorithm 1] In Algorithm 1, Step 8 says the predicted label is calculated using Eq. (4), but Eq. (4) is the inference-time expert-group aggregation formula; the training-stage prediction is described by Eq. (3). Please correct the cross-reference.
- [Eq. (4)] The notation p(ŷ|d_j, x) is described in the text as a 'prediction logit' but written as a conditional probability. This is confusing: if it is a logit, the expression inside the softmax is a weighted sum of logits; if it is a probability, the operation is not a standard mixture. Please clarify the notation.
- [§3.1] The phrase 'enhancing the detector’s DG-MGT detection ceapability' contains a typo ('ceapability' should be 'capability').
- [§4.2 / Appendix B] The two domain-generalization baselines MSCL and TACIT are said to be 'adapted to the detection task,' but the adaptation procedure is not specified. Please provide enough detail for reproducibility.
Circularity Check
DG-MGT gains in Table 1 are partly fitted to the five 'unseen' target domains via hyperparameter selection (Appendix C.1), so the zero-shot OOD claim is not cleanly out-of-sample.
specific steps
-
fitted input called prediction
[Appendix C.1 (Hyperparameter Analysis), Figure 3; Appendix A.1; §4.1; §4.3/Table 1]
"we explore various configurations ofm1 andm2 and adopt the best-performing setting as reported in the right heatmap of Figure 3. ... Hyperparameter analysis on DG-MGT.Left: performance variation with different values of top-m expert group selection. Right: F1-score heatmap under different configurations of shared and domain-specific experts. ... DG-MGT measures zero-shot generalization to five unseen target domains (CMV, ELI5, WP, TLDR, and ROCT)."
The architectural hyperparameters m1, m2, and top-m are selected by maximizing F1 on the five DG-MGT target test domains (CMV, ELI5, WP, TLDR, ROCT). Table 1/§4.3 then reports DEER's zero-shot OOD improvements (e.g., +5.32% F1) on exactly those same domains. The headline OOD number is therefore not a prediction of a fixed model applied to unseen domains; it is partly an optimized fit to the target test distribution. Since no equivalent target-domain hyperparameter tuning is reported for the baselines, the comparison is not cleanly zero-shot and the claimed generalization gain is partially built into the experimental setup.
full rationale
The paper contains no mathematical derivation chain whose outputs equal its inputs, and the RL routing is evaluated against independent external benchmarks, so most of the work is not circular in the derivation sense. The one load-bearing circular/contamination issue is the target-domain hyperparameter selection: Appendix C.1 explicitly states that m1/m2 (and top-m) were chosen by DG-MGT F1, i.e., using the same five 'unseen' domains on which the zero-shot claim is later reported. That makes the central OOD improvement at least partly a fitted value rather than a clean prediction. I also flag two non-circular weaknesses for the record: the Related Work contains malformed citation placeholders ('Jacobs et al. 1991;?'), and the paper does not analyze the learned policy's action distribution to rule out reward hacking; these affect evidence quality but are not circularity. Self-citations to the authors' prior detectors are present but are not load-bearing for the central empirical comparison.
Axiom & Free-Parameter Ledger
free parameters (4)
- m1 (number of domain-specific sub-experts per domain) =
5
- m2 (number of shared experts) =
6
- top-m (number of expert groups used at inference) =
3
- Policy network hidden dimension and optimizer hyperparameters =
768x512; lr=1e-3; weight decay 1e-5
axioms (3)
- domain assumption The five chosen source domains (Xsum, HellaSwag, SQuAD, Yelp, Sci) are representative of the MGT-detection domain shift in MAGE.
- domain assumption The RL policy reward, computed as source-domain classification accuracy of the frozen detector, is a valid proxy for routing quality on unseen domains.
- domain assumption The MAGE benchmark target datasets do not leak into the source training set.
read the original abstract
Detecting machine-generated text has become a critical challenge amid the rapid advancement of LLMs, yet existing detectors degrade severely under domain shift. Through systematic pilot studies, we trace this vulnerability to two fundamental flaws in current generalization strategies, namely the incomplete preservation of domain-specific knowledge during multi-domain training and the misalignment between knowledge retrieval and the detection objective at inference. To address these gaps, we propose DEER, a Disentangled mixturE-of-ExpeRts framework that explicitly decouples domain-local and domain-invariant knowledge into specialized expert modules. Instead of static domain matching, DEER employs a reinforcement learning-driven router that selects expert pathways based on instance-level detection rewards. This task-aligned, domain-agnostic mechanism ensures robust adaptation to unseen distributions by prioritizing detection utility over stylistic resemblance. Extensive experiments demonstrate that DEER consistently outperforms state-of-the-art detectors, achieving average F1 improvements of 1.28% and 2.92%, and accuracy gains of 1.35% and 2.26% on in-domain and out-of-domain datasets, offering reliable generalization for open-world deployment.
Figures
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
AI@Meta. 2024. Llama 3 Model Card
2024
-
[4]
Anthropic. 2024. Claude3. Website
2024
-
[5]
Bao, G.; Zhao, Y.; Teng, Z.; Yang, L.; and Zhang, Y. 2024. Fast-Detect GPT : Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature. In The Twelfth International Conference on Learning Representations
2024
-
[6]
Bhattacharjee, A.; Kumarage, T.; Moraffah, R.; and Liu, H. 2023. Conda: Contrastive domain adaptation for ai-generated text detection. arXiv preprint arXiv:2309.03992
Pith/arXiv arXiv 2023
-
[7]
Bhattacharjee, A.; Moraffah, R.; Garland, J.; and Liu, H. 2024. Eagle: A domain generalization framework for ai-generated text detection. arXiv preprint arXiv:2403.15690
Pith/arXiv arXiv 2024
-
[8]
T.; Li, Z.; Tang, L.; Zhang, L.; et al
Chen, J.; Zhu, X.; Liu, T.; Chen, Y.; Xinhui, C.; Yuan, Y.; Leong, C. T.; Li, Z.; Tang, L.; Zhang, L.; et al. 2025. Imitate Before Detect: Aligning Machine Stylistic Preference for Machine-Revised Text Detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, 23559--23567
2025
-
[9]
Dai, Y.; Li, X.; Liu, J.; Tong, Z.; and Duan, L.-Y. 2021. Generalizable person re-identification with relevance-aware mixture of experts. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 16145--16154
2021
-
[10]
Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), 4171--4186
2019
-
[11]
Gehrmann, S.; Strobelt, H.; and Rush, A. M. 2019. GLTR: Statistical Detection and Visualization of Generated Text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, 111--116
2019
-
[12]
Hans, A.; Schwarzschild, A.; Cherepanova, V.; Kazemi, H.; Saha, A.; Goldblum, M.; Geiping, J.; and Goldstein, T. 2024. Spotting llms with binoculars: Zero-shot detection of machine-generated text. arXiv preprint arXiv:2401.12070
Pith/arXiv arXiv 2024
-
[13]
Hu, X.; Chen, P.-Y.; and Ho, T.-Y. 2023. Radar: Robust ai-text detection via adversarial learning. arXiv preprint arXiv:2307.03838
Pith/arXiv arXiv 2023
-
[14]
A.; Jordan, M
Jacobs, R. A.; Jordan, M. I.; Nowlan, S. J.; and Hinton, G. E. 1991. Adaptive mixtures of local experts. Neural computation, 3(1): 79--87
1991
-
[15]
Li, B.; Shen, Y.; Yang, J.; Wang, Y.; Ren, J.; Che, T.; Zhang, J.; and Liu, Z. 2022. Sparse mixture-of-experts are domain generalizable learners. arXiv preprint arXiv:2206.04046
Pith/arXiv arXiv 2022
-
[16]
Li, Y.; Li, Q.; Cui, L.; Bi, W.; Wang, Z.; Wang, L.; Yang, L.; Shi, S.; and Zhang, Y. 2024. MAGE : Machine-generated Text Detection in the Wild. In Ku, L.-W.; Martins, A.; and Srikumar, V., eds., Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 36--53. Bangkok, Thailand: Association for Compu...
2024
-
[17]
Li, Y.; Zhang, Z.; Li, C.; Shen, C.; and Liu, X. 2025. Iron Sharpens Iron: Defending Against Attacks in Machine-Generated Text Detection with Adversarial Training. arXiv:2502.12734
Pith/arXiv arXiv 2025
-
[18]
Liu, S.; Liu, X.; Wang, Y.; Cheng, Z.; Li, C.; Zhang, Z.; Lan, Y.; and Shen, C. 2024. Does detectgpt fully utilize perturbation? bridging selective perturbation to fine-tuned contrastive learning detector would be better. arXiv preprint arXiv:2402.00263
Pith/arXiv arXiv 2024
-
[19]
Liu, X.; and Kong, L. 2024. AI text detection method based on perplexity features with strided sliding window. Working notes of clef
2024
-
[20]
Liu, X.; Zhang, Z.; Wang, Y.; Pu, H.; Lan, Y.; and Shen, C. 2023. CoCo: Coherence-Enhanced Machine-Generated Text Detection Under Low Resource With Contrastive Learning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 16167--16188
2023
-
[21]
Liu, Y.; Ott, M.; Goyal, N.; Du, J.; Joshi, M.; Chen, D.; Levy, O.; Lewis, M.; Zettlemoyer, L.; and Stoyanov, V. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692
Pith/arXiv arXiv 2019
-
[22]
Ma, J.; Zhao, Z.; Yi, X.; Chen, J.; Hong, L.; and Chi, E. H. 2018. Modeling task relationships in multi-task learning with multi-gate mixture-of-experts. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, 1930--1939
2018
-
[23]
Mireshghallah, F.; Mattern, J.; Gao, S.; Shokri, R.; and Berg-Kirkpatrick, T. 2023. Smaller Language Models are Better Black-box Machine-Generated Text Detectors. arXiv preprint arXiv:2305.09859
Pith/arXiv arXiv 2023
-
[24]
D.; and Finn, C
Mitchell, E.; Lee, Y.; Khazatsky, A.; Manning, C. D.; and Finn, C. 2023. DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature. ICML 2023
2023
-
[25]
OpenAI. 2024. GPT-4o System Card
2024
-
[26]
Pudasaini, S.; Miralles, L.; Lillis, D.; and Salvador, M. L. 2025. Benchmarking AI Text Detection: Assessing Detectors Against New Datasets, Evasion Tactics, and Enhanced LLM s. In Alam, F.; Nakov, P.; Habash, N.; Gurevych, I.; Chowdhury, S.; Shelmanov, A.; Wang, Y.; Artemova, E.; Kutlu, M.; and Mikros, G., eds., Proceedings of the 1stWorkshop on GenAI Co...
2025
-
[27]
Qu, J.; Faney, T.; Wang, Z.; Gallinari, P.; Yousef, S.; and de Hemptinne, J.-C. 2022. Hmoe: Hypernetwork-based mixture of experts for domain generalization. arXiv preprint arXiv:2211.08253
Pith/arXiv arXiv 2022
-
[28]
Ren, X.; Zhou, P.; Meng, X.; Huang, X.; Wang, Y.; Wang, W.; Li, P.; Zhang, X.; Podolskiy, A.; Arshinov, G.; et al. 2023. Pangu- \ Sigma \ : Towards trillion parameter language model with sparse heterogeneous computing. arXiv preprint arXiv:2303.10845
Pith/arXiv arXiv 2023
-
[29]
Shum, K.; Diao, S.; and Zhang, T. 2023. Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Data. arXiv preprint arXiv:2302.12822
Pith/arXiv arXiv 2023
-
[30]
Song, R.; Giunchiglia, F.; Li, Y.; Tian, M.; and Xu, H. 2024. Tacit: A target-agnostic feature disentanglement framework for cross-domain text classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 18999--19007
2024
-
[31]
Sutton, R. S. 1988. Learning to predict by the methods of temporal differences. Machine learning, 3: 9--44
1988
-
[32]
Tan, Q.; He, R.; Bing, L.; and Ng, H. T. 2022. Domain generalization for text classification with memory-based supervised contrastive learning. In Proceedings of the 29th International Conference on Computational Linguistics, 6916--6926
2022
-
[33]
Tang, H.; Liu, J.; Zhao, M.; and Gong, X. 2020. Progressive layered extraction (ple): A novel multi-task learning (mtl) model for personalized recommendations. In Proceedings of the 14th ACM conference on recommender systems, 269--278
2020
-
[34]
Verma, V.; Fleisig, E.; Tomlin, N.; and Klein, D. 2024. Ghostbuster: Detecting Text Ghostwritten by Large Language Models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), 1702--1717
2024
-
[35]
Wang, P.; Li, L.; Ren, K.; Jiang, B.; Zhang, D.; and Qiu, X. 2023. Seq XGPT : Sentence-Level AI -Generated Text Detection. In The 2023 Conference on Empirical Methods in Natural Language Processing
2023
-
[36]
Wang, Y.; Feng, S.; Hou, A.; Pu, X.; Shen, C.; Liu, X.; Tsvetkov, Y.; and He, T. 2024. Stumbling Blocks: Stress Testing the Robustness of Machine-Generated Text Detectors Under Attacks. In Ku, L.-W.; Martins, A.; and Srikumar, V., eds., Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2894--2...
2024
-
[37]
Zellers, R.; Holtzman, A.; Rashkin, H.; Bisk, Y.; Farhadi, A.; Roesner, F.; and Choi, Y. 2019. Defending against neural fake news. Advances in neural information processing systems, 32
2019
-
[38]
Zhong, T.; Chi, Z.; Gu, L.; Wang, Y.; Yu, Y.; and Tang, J. 2022. Meta-dmoe: Adapting to domain shift by meta-distillation from mixture-of-experts. Advances in Neural Information Processing Systems, 35: 22243--22257
2022
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.