{"id":"e1f78d1c-bb28-4c6d-9fc5-f3719159e01f","arxiv_id":"2511.01192","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"DEER, a disentangled mixture-of-experts detector with RL-based instance routing, reports F1 gains of about 1.4 in-domain and 5.3 points out-of-domain over prior MGT detectors.","lead":"This paper introduces DEER, a machine-generated-text detector that separates domain-specific and shared knowledge with mixture-of-experts modules and uses a reinforcement-learned router to pick experts per input. It reports accuracy and F1 gains over existing detectors, with the largest improvements on unseen domains.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DG-MGT gains may reflect test-set hyperparameter selection: m1, m2, and top-m are chosen using the target-domain test sets, so the zero-shot claim is not cleanly supported.","rationale":"The reader's verdict was CONDITIONAL, and I agree with that overall. The reader's stated weakest assumption was that the RL policy might learn a shortcut to binary labels rather than domain-appropriate routing. That is a plausible mechanism but remains speculative without policy analysis. A more concrete and textually supported threat is the evaluation protocol: Appendix C.1 explicitly selects m1, m2, and top-m using DG-MGT test-domain performance. This directly undermines the zero-shot claim, because the final architecture's hyperparameters are influenced by the very target domains on which generalization is measured. The abstract/main-text numeric discrepancy (+1.28/+2.92 vs +1.39/+5.32) and missing code are additional credibility concerns, but the test-set tuning is the single load-bearing issue that can be settled by re-running the evaluation with a clean validation split. My proposed test does exactly that. I mark agreement as partial because the reader mentioned the hyperparameter issue in their rationale but did not make it the primary weakest assumption; I elevate it to the primary concern.","tokens_in":16212,"tokens_out":4297,"duration_ms":46674,"concrete_test":"Re-run the DEER pipeline with m1, m2, and top-m selected using only the held-out source-domain validation set (e.g., grid search maximizing IND-MGT F1 on the validation split), then evaluate on the five DG-MGT test sets with no further selection. If the average F1 gain over the strongest baseline drops below the reported ~5.32% or becomes non-significant, the zero-shot claim is not supported. Report the validation-selected hyperparameters; if they differ from the test-selected ones in Appendix C.1, the evaluation leak is confirmed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that DEER outperforms SOTA under domain shift rests on the DG-MGT evaluation in Table 1. Appendix C.1 states that the number of domain-specific experts m1, shared experts m2, and the routing parameter top-m were selected by evaluating performance on the DG-MGT test domains: 'we explore various configurations of m1 and m2 and adopt the best-performing setting as reported in the right heatmap of Figure 3' (Figure 3 shows DG-MGT F1 scores across configurations). This is test-set hyperparameter tuning: the five unseen target domains (CMV, ELI5, WP, TLDR, ROCT) were used to choose architectural hyperparameters before reporting final results. Baselines are not given the same privilege; they use standard configurations. Consequently, the reported average OOD improvements of 5.32% F1 and 3.61% accuracy may partly encode information from the target distributions, making the zero-shot generalization claim unfalsifiable in its current form. This is a more direct threat to the abstract's headline numbers than the RL-shortcut risk: even if the policy is perfectly trained, the comparison is unfair if DEER's hyperparameters are tuned on the test set. The paper's own validation set (Appendix A.1) exists but is not used for this purpose.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DEER, a two-stage framework for domain-generalizable machine-generated text (MGT) detection. In the first stage, a disentangled mixture-of-experts (DMoE) module is trained with source-domain supervision: each source domain has dedicated experts, and shared experts capture cross-domain features. In the second stage, the DMoE is frozen and a reinforcement-learning policy is trained to route each input instance to the most useful top-m expert groups at inference, without requiring domain labels. The authors evaluate on five in-domain and five out-of-domain MAGE datasets and report state-of-the-art accuracy and F1, with headline OOD gains of about 5.32% F1 and 3.61% accuracy. Ablations, robustness tests, and an incremental-adaptation experiment are also reported.","tokens_in":16462,"tokens_out":7483,"duration_ms":84524,"significance":"If the out-of-domain result is substantiated, the contribution is meaningful: decoupling domain-local and domain-invariant expertise is a plausible route to more generalizable MGT detection, and the RL-based instance-level router addresses a real deployment gap when domain labels are absent. The paper's Table 2 ablations support the value of both expert types and of the learned routing over oracle/random/classifier alternatives, and the robustness and incremental-adaptation analyses are useful additions. However, the central OOD claim is currently weakened by a selection-protocol problem: key architectural hyperparameters appear to be chosen using the target-domain test sets. The significance of the paper therefore depends on a clean re-evaluation with hyperparameters fixed on source validation data only.","major_comments":[{"comment":"The headline zero-shot DG-MGT result is compromised by test-set hyperparameter selection. Appendix C.1 states that the numbers of domain-specific experts m1, shared experts m2, and the routing parameter top-m were chosen using the DG-MGT F1 scores shown in Figure 3. Since DG-MGT is the five unseen target domains (CMV, ELI5, WP, TLDR, ROCT), these target-test labels influence architectural choices before the final numbers in Table 1 are produced. The baselines are not given the same target-data privilege. Consequently, the reported average OOD improvements of 5.32% F1 and 3.61% accuracy cannot be interpreted as clean zero-shot generalization; they may partly reflect oracle hyperparameter fitting. The source-domain validation split described in Appendix A.1 should be used for all model selection. Please rerun the comparison with hyperparameters fixed on source validation only, and report b","section":"Appendix C.1 / Table 1"},{"comment":"The routing policy is trained with rewards derived from the frozen detector's own binary classification outcome on the training instance. This objective rewards selecting any expert that makes the detector's prediction correct, not necessarily the expert that captures domain-relevant structure. The paper does not analyze the learned policy's action distribution, nor does it show that the selected top-m expert groups are sensible on out-of-domain inputs. Please add diagnostics: the policy's entropy and action frequencies on source vs. OOD data, a comparison to a uniform top-m ensemble, and a comparison to a policy trained with an independent domain-classification reward. Without these, the claim that DEER achieves 'task-aligned, domain-agnostic' routing rather than a label shortcut is not directly evidenced.","section":"§3.2, Eq. (5)-(6), Table 2"}],"minor_comments":[{"comment":"The abstract at the beginning of the submission reports average F1 improvements of 1.28% and 2.92%, and accuracy gains of 1.35% and 2.26%, whereas the full-text abstract and Section 4.3 report 1.39% and 5.32% F1 and 1.35% and 3.61% accuracy. These headline numbers must be reconciled.","section":"Abstract / §4.3"},{"comment":"There are missing citation placeholders in the Mixture-of-Experts paragraph: '(Jacobs et al. 1991;?)' and '(Li et al. 2022; ?;?;?)'. Please complete these references.","section":"§2 Related Work"},{"comment":"Many cells in Table 1 have malformed formatting, e.g., the RoBERTa row reads '88.77 0.64 95.241.07' with no clear separation between accuracy and standard deviation. Please reformat the table so each value is readable.","section":"Table 1"},{"comment":"In Algorithm 1, Step 8 says the predicted label is calculated using Eq. (4), but Eq. (4) is the inference-time expert-group aggregation formula; the training-stage prediction is described by Eq. (3). Please correct the cross-reference.","section":"Algorithm 1"},{"comment":"The notation p(ŷ|d_j, x) is described in the text as a 'prediction logit' but written as a conditional probability. This is confusing: if it is a logit, the expression inside the softmax is a weighted sum of logits; if it is a probability, the operation is not a standard mixture. Please clarify the notation.","section":"Eq. (4)"},{"comment":"The phrase 'enhancing the detector’s DG-MGT detection ceapability' contains a typo ('ceapability' should be 'capability').","section":"§3.1"},{"comment":"The two domain-generalization baselines MSCL and TACIT are said to be 'adapted to the detection task,' but the adaptation procedure is not specified. Please provide enough detail for reproducibility.","section":"§4.2 / Appendix B"}],"recommendation":"major_revision","confidential_remarks":"The most serious issue is the selection of m1, m2, and top-m using DG-MGT test-domain F1. If the authors cannot produce a clean source-validation-based selection, the zero-shot claim should be withdrawn or substantially weakened. I recommend that the editor require this re-evaluation before further consideration."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a genuinely new architecture for DG-MGT detection, and the ablation study shows the two ingredients each do work. But as submitted, the out-of-domain numbers are not a clean zero-shot evaluation, because the architecture hyperparameters m1, m2, and top-m were selected using the DG-MGT test domains (Appendix C.1). That alone prevents me from treating the 5.32% F1 gain as reliable. There is also an unresolved inconsistency between the abstract (+1.28/+2.92) and Section 4.3 (+1.39/+5.32); such a discrepancy in the flagship claim should have been caught. No code or data is released, and there are literal '?' placeholders in Section 2, signaling the paper was rushed. The RL-shortcut concern raised internally strikes me as a secondary issue: the reward is the frozen detector's accuracy on the training instance, so the policy could in principle memorize a label shortcut, but the paper's own oracle-label ablation suggests the routing is doing something meaningful, and there is no evidence of collapse. Still, some analysis of the policy's action distribution on unseen domains would help.\n\nThe disentangled MoE with instance-adaptive RL routing is a clean idea, the writing is generally clear, and the ablations (Table 2) show that removing either expert type hurts, and that RL routing beats random/classifier alternatives. That is real evidence the architecture is doing something. The comparison to baselines uses standard settings, so the in-domain gains are less suspect.\n\nFor a reader: if you work on MGT detection or domain generalization, this is worth reading as a proposal, but I wouldn't yet cite the claimed OOD numbers. It deserves peer review, not because the numbers are credible in this form, but because the idea is promising and the flaws are fixable. A serious referee should ask for a version where hyperparameters are chosen on the validation split (which exists), the abstract/full-text numbers are reconciled, and code is released.","headline":"Promising architecture, but the zero-shot gains are inflated by test-set hyperparameter tuning; needs revision before the headline claim can be trusted.","tokens_in":17026,"tokens_out":1808,"would_cite":false,"duration_ms":21185,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A disentangled MoE with RL routing generalizes AI-text detection to unseen domains.","keywords":["machine-generated text detection","domain generalization","mixture of experts","reinforcement learning routing","disentangled representation","zero-shot detection","out-of-distribution generalization","LLM text attribution"],"falsifier":"A concrete check: take a trained DEER model, feed out-of-domain inputs, record the policy's top-1 domain choice per input, and compare that distribution against a uniform or majority-class baseline. If the policy collapses (e.g., always picks the same domain or the domain whose label prior dominates), and OOD F1 drops to the level of that single expert, the central claim of instance-adaptive routing is falsified. Equivalently, training the same architecture with a randomly initialized policy frozen at uniform weights should match the reported OOD gains if routing were not doing the work.","tokens_in":16043,"feed_emoji":"🤖","tokens_out":2817,"duration_ms":29729,"temperature":0.7,"pith_summary":"The paper argues that existing machine-generated text detectors fail under domain shift because they entangle domain-specific artifacts with universal generation signals, and because expert routing at inference is misaligned with the detection objective. DEER counters this with a two-stage design: domain-supervised training of separate domain-specific and shared experts, then a reinforcement-learned policy that picks which experts to consult for each input based on detection reward, not stylistic similarity. The authors report consistent F1 gains over strong baselines, largest on out-of-domain benchmarks. If right, it offers a modular path to detectors that keep working as new writing domains appear.","feed_headline":"Disentangled experts + RL routing lift unseen-domain AI-text F1 by 5.32%","feed_subtitle":"DEER splits domain-specific and shared knowledge, then learns to pick the right experts per input without domain labels.","key_machinery":"The load-bearing mechanism is the two-stage DEER architecture: (1) a Disentangled Mixture-of-Experts with per-domain expert MLPs and a shared expert, trained with domain labels so each expert owns a domain's local patterns while the shared expert learns cross-domain generation signatures; (2) an RL policy network, trained with REINFORCE on detection rewards (accuracy minus a domain-average baseline), that maps each input's encoded state to a soft distribution over source domains. At inference, top-m domain experts plus the shared expert are fused by policy probabilities. The RL reward is what aligns routing with the actual detection objective instead of domain similarity.","core_discovery":"DEER's central claim is that explicitly decoupling domain-local and domain-invariant knowledge—rather than pooling all training domains into one model—preserves the transferable signatures of machine generation, and that a task-reward-driven router can exploit that structure without domain labels at test time. Concretely, training instantiates one expert per source domain plus a shared expert, with a domain-aware gate; inference uses a policy network to output a distribution over source domains, selects the top-m, and aggregates their logits weighted by selection probability. The reported result is that this beats state-of-the-art detectors on both in-domain and five unseen out-of-domain dat","pith_inferences":["A natural next test is whether the learned policy's selection probabilities correlate with actual domain similarity when labels are available on a held-out domain; if they do not, the RL objective is selecting for something subtler than stylistic match.","The reward baseline (average over all domain experts) implies the policy is trained to beat the average expert; a direct implication is that the approach may be most effective when source domains are diverse enough that the average expert is weak, which could be probed by varying domain diversity.","The architecture suggests a recipe for other open-set detection tasks where domain labels exist during training but vanish at test time: disentangle per-domain experts, then learn a reward-driven router."],"forward_implications":["Domain-generalizable MGT detection improves without target-domain labels, as shown by average F1 gains of 5.32% out-of-domain and 1.39% in-domain over the strongest baseline.","Detectors built this way tolerate input perturbations; DEER keeps higher F1 than metric-based and model-based baselines under repetition, deletion, and replacement attacks.","Modular expansion works: adapting to a new domain by adding one expert and fine-tuning only shared experts preserves old-domain performance while cutting trainable parameters by roughly 90% versus full fine-tuning.","Because the router is optimized for detection reward rather than style match, aggregating multiple expert groups yields further gains, plateauing near m=3."],"fun_headline_variants":["Disentangled experts + RL routing boost AI-text detection on unseen domains","DEER: RL router picks experts per input, no domain labels needed","Unseen-domain AI-text F1 up 2.92% via disentangled experts","RL-driven expert selection adapts AI-text detection to new domains","Disentangling shared and domain-specific knowledge beats AI-text domain shift"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the RL policy trained on source-domain classification rewards will pick experts that generalize to unseen domains, rather than exploiting shortcuts to the binary label (such as always choosing the expert whose bias matches the majority class); the paper does not analyze the policy's action distribution on OOD inputs to rule this out.","fun_headline_variants_meta":{"raw":{"variants":["Disentangled experts + RL routing boost AI-text detection on unseen domains","DEER: RL router picks experts per input, no domain labels needed","Unseen-domain AI-text F1 up 2.92% via disentangled experts","RL-driven expert selection adapts AI-text detection to new domains","Disentangling shared and domain-specific knowledge beats AI-text domain shift"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000763,"raw_usage":{"total_tokens":3219,"prompt_tokens":734,"completion_tokens":2485,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":478,"completion_tokens_details":{"reasoning_tokens":2403}},"tokens_in":478,"tokens_out":2485,"duration_ms":18935,"temperature":1.0,"reasoning_tokens":2403,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T00:22:59.613847+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete check: take a trained DEER model, feed out-of-domain inputs, record the policy's top-1 domain choice per input, and compare that distribution against a uniform or majority-class baseline. If the policy collapses (e.g., always picks the same domain or the domain whose label prior dominates), and OOD F1 drops to the level of that single expert, the central claim of instance-adaptive routing is falsified. Equivalently, training the same architecture with a randomly initialized policy frozen at uniform weights should match the reported OOD gains if routing were not doing the work.","supporting_citations":[],"review_version":1}