{"id":"a34b6750-1ebc-44ba-8e04-8a6ecfab4405","arxiv_id":"2505.13855","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"DoGEN routes each input to the two most likely domain-specialist detectors and blends their scores, beating internally trained single models on in-domain MAGE and out-of-domain RAID AUROC.","lead":"This paper introduces DoGEN, a detector that combines ten domain-specialist AI-text detectors using weights from a domain classifier. A smart generalist should read it because it is a modular recipe for adapting AI-text detection to new domains without retraining the whole system.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"OOD generalization claim is contradicted by DoGEN's own MAGE held-out results; the apparent RAID win rests on near-domain overlap, and no external baselines support the SOTA claim.","rationale":"I read the paper in good faith. DoGEN is a modular idea, code is released, and the authors disclose limitations. My concern is not that the method fails internally; it is that the strongest abstract claim is broader than the evidence. Table 1's held-out columns are the most direct evidence about unseen domains, and they cut the other way. The RAID benchmark is a reasonable evaluation, but Appendix B itself documents overlap, so 'outperforms models twice its size on out-of-domain detection' is only supported for a partial near-domain reading. The absence of any published baseline means 'state-of-the-art' is unverified; conditional acceptance with revision of scope and external comparisons is the right outcome. This largely matches the reader's weakest assumption, though I would emphasize that the problem is the scope of the central claim rather than the router objective itself.","tokens_in":15695,"tokens_out":7600,"duration_ms":75571,"concrete_test":"Re-run the OOD comparison using only the four MAGE held-out domains, reporting unweighted mean AUROC for DoGEN and Qwen32B; if DoGEN stays below Qwen32B (as Table 1 suggests), the paper must weaken the OOD claim to 'RAID-like near-domain settings' or add external baselines before acceptance.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract claims state-of-the-art in-domain detection and that DoGEN outperforms larger models out-of-domain. The paper's own MAGE table is the cleanest test of the OOD claim: on all four held-out domains that were never seen during training (cnn, dialog_sum, imdb, pubmed), DoGEN is below the Qwen1.5-32B baseline by 14.5, 44.9, 9.0, and 11.9 AUROC points, respectively (Table 1). The reported aggregate 'all' column averages these failures with the ten in-domain columns, so it does not license a broad 'out-of-domain' statement. RAID, the benchmark where DoGEN wins, is not a clean OOD test: Appendix B.3 concedes that abstracts, books, news, reddit, wiki, and reviews share sources with MAGE, and Appendix B.4 lists text-davinci-002 and gpt-3.5-turbo as generators in both datasets. So the only benchmark with a positive OOD result is the one with maximal domain/generator overlap. Appendix C's supporting correlation (router probability vs expert AUROC, rho=0.64) is a between-expert aggregate and does not establish per-example transfer. Independently, no comparison to published detectors (Fast-DetectGPT, RADAR, Binoculars, etc.) appears anywhere, so the 'state-of-the-art' half of the central claim is unsupported as written.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DoGEN (Domain Gating Ensemble Networks), a method for AI-generated text detection that combines N domain-specific expert detectors with a learned router that classifies each input document into one of N training domains. The router's softmax probabilities are used to weight the top-k expert scores, with k=2 at inference. The authors fine-tune 10 Qwen1.5-1.8B experts on the 10 MAGE training domains plus a Qwen1.5-1.8B router, and compare against Qwen1.5 dense baselines (1.8B and 32B), a Qwen1.5-MoE model, equal-vote and logistic-regression ensembles, and joint-training variants. Evaluation is on MAGE (10 in-domain and 4 held-out domains) and the RAID benchmark's non-adversarial subset, with AUROC as the primary metric. The abstract claims state-of-the-art in-domain detection and that DoGEN outperforms models twice its size on out-of-domain detection; the results tables show DoGEN besting the Qwen32B baseline on the aggregate MAGE 'all' column and on RAID, while being below Qwen32B on all four MAGE held-out domains individually.","tokens_in":15938,"tokens_out":3303,"duration_ms":29676,"significance":"The core idea of using a domain classifier to gate expert detectors is a plausible and modular approach to domain adaptation for machine-text detection, and the paper makes a positive practical contribution by releasing code and trained models. The internal comparisons among the in-house baselines are coherent, and the experiment design (per-domain balancing, same hyperparameters, controlled parameter counts) is careful. However, the headline claims go beyond the evidence: no published third-party detectors are included, so 'state-of-the-art' is ungrounded; and the paper's own cleanest out-of-domain test (MAGE held-out domains) contradicts the claimed superiority over larger models. The RAID results, while positive, rest on benchmarks with substantial domain and generator overlap with MAGE. If the claims are carefully narrowed and external baselines are added, the method would be a useful contribution.","major_comments":[{"comment":"The abstract's claim that DoGEN outperforms models twice its size on out-of-domain detection is contradicted by the paper's own MAGE held-out results. In Table 1, DoGEN is below Qwen32B on all four unseen domains: cnn 78.98 vs 93.52, dialog_sum 45.64 vs 90.50, imdb 84.93 vs 93.89, and pubmed 79.10 vs 90.97. The aggregate 'all' column averages these losses with the ten in-domain columns and therefore does not support a broad out-of-domain statement. The OOD claim should be either restricted to RAID or revised to acknowledge the failure on MAGE held-out domains.","section":"§5, Table 1"},{"comment":"The 'state-of-the-art' claim in the abstract is not supported because every baseline in the comparison is trained or constructed in-house (Qwen dense variants, Qwen-MoE, equal/weighted votes, joint-training variants). No published detectors such as Fast-DetectGPT, RADAR, Binoculars, or Ghostbuster are evaluated. A state-of-the-art claim requires comparison with established detectors on the same benchmarks; otherwise the paper can only claim superiority over its own baselines.","section":"§4.4, Tables 1 and 2"},{"comment":"The RAID benchmark is not a clean out-of-domain test for this method. Appendix B.3 states that six of the eight RAID domains (abstracts, books, news, reddit, wiki, reviews) share sources with MAGE, and Appendix B.4 notes that text-davinci-002 and gpt-3.5-turbo appear as generators in both datasets. The positive RAID result therefore largely reflects near-domain and near-generator overlap, not robust generalization to genuinely unseen distributions. The paper should either evaluate on a truly disjoint benchmark or explicitly temper the out-of-domain generalization claim.","section":"§5 and Appendices B.3-B.4"},{"comment":"The supporting evidence for the routing mechanism is a between-expert correlation (Pearson ρ=0.64 between average gate weight and expert AUROC on RAID). This aggregate correlation does not demonstrate per-example routing quality, which is the load-bearing mechanism behind the method's supposed transfer benefit. A more direct test would compare top-2 routing accuracy against random or uniform expert selection, or report whether the router assigns higher probability to experts that are individually correct on each example.","section":"Appendix C"}],"minor_comments":[{"comment":"In the Qwen1.5-MoE description, 'chose it model' should read 'chose this model'.","section":"§4.4"},{"comment":"The cross-reference is wrong: 'In Table 9 we list the domains present in the RAID dataset' should point to Table 10, which actually contains domain descriptions; Table 9 lists generators.","section":"Appendix B.3"},{"comment":"The source for Reviews is listed as 'imbd.com'; the correct domain is IMDb, so the spelling should be 'imdb.com'.","section":"Table 10"},{"comment":"No error bars, confidence intervals, or significance tests are reported for any AUROC differences. Given that several aggregate differences are small (e.g., 95.81 vs 94.73 on RAID), the paper should at least report whether the differences are stable across seeds or subsets.","section":"Tables 1 and 2"},{"comment":"The discussion of paraphrased test settings (gpt4_para and human_para) is valuable, but the reader is left without a table or metric showing the full per-setting breakdown for all baselines; Table 14 covers only four domains and not the full comparison. Adding the full setting-level results would make the analysis more complete.","section":"Appendix D.1"}],"recommendation":"major_revision","confidential_remarks":"The evaluation is mostly in-house, and several of the authors are also involved in the creation of RAID, which creates a mild risk of benchmark familiarity even though the training does not fit to RAID. The central result is not circular, but the abstract's state-of-the-art and out-of-domain claims are substantially overstated relative to the evidence. The paper could become acceptable after narrowing the claims and adding external baselines. I would encourage the editor to require that the revised version clearly separates 'best among our baselines' from 'state-of-the-art'."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: DoGEN is a real, well-scoped engineering contribution, but the headline claims go beyond what its own tables show. The core idea—train per-domain specialist detectors, then use a frozen domain-classifier router to pick and weight the top two at test time—is clean and, as far as the cited work goes, new as a combination. The internal experiments are coherent: on the ten MAGE training domains DoGEN beats a Qwen1.5-32B dense baseline by about 2.4 AUROC points on average, and on RAID it beats the same baseline by about 1.1 points while activating far fewer parameters. The authors release code and models, and they are unusually candid about known weak spots (the dialog_sum failure, the paraphrased-human-label noise, the narrow set of base models). That honesty counts.\n\nThe soft spots are real and concentrated in the claims, not the engineering. First, \"state-of-the-art\" is unsupported: the comparisons are all against models trained by the same authors with the same recipe. Fast-DetectGPT, RADAR, Binoculars, or any published third-party detector never appears. You can't call SOTA without running at least one of them. Second, the out-of-domain generalization claim is contradicted by the paper's own MAGE held-out columns: DoGEN trails Qwen32B by 14.5, 44.9, 9.0, and 11.9 AUROC points on cnn, dialog_sum, imdb, and pubmed. The aggregate \"all\" column hides those losses behind ten in-domain columns. Third, RAID is not clean OOD evidence here. The authors themselves note the domain and generator overlap with MAGE (Appendix B.3/B.4), so the positive RAID result is closer to near-domain transfer than to broad generalization. The Appendix C correlation (rho=0.64) is between experts, not per-example, so it doesn't rescue the transfer story. I also note no error bars anywhere, and the per-domain balancing strategy was selected after peeking at MAGE-test Table 4; that's test-set selection, albeit for a preprocessing choice.\n\nNone of this sinks the method. For someone who wants a domain-adaptive detector and has the compute to train ten small experts, DoGEN is a plausible, modular recipe, and the release makes it reproducible. The right fix is to rescope the abstract claims, add external baselines, report variance, and move the balancing-strategy choice onto validation data. As is, I would send it to review—the architecture and writeup are solid enough to warrant referee time, and the weaknesses are fixable in revision. I'd want the authors pushed on the OOD claim and the missing baselines before trusting the headline.","headline":"DoGEN is a clean, modular domain-gating ensemble with honest internal experiments, but the SOTA and OOD claims outrun the evidence in the paper's own tables.","tokens_in":16497,"tokens_out":2330,"would_cite":true,"duration_ms":22230,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A domain-gating ensemble of ten 1.8B detectors, routed by a domain classifier, reports higher average AUROC than a 32B dense model on both MAGE and RAID while activating 5.4B parameters at inference.","keywords":["AI-generated text detection","domain gating","ensemble networks","out-of-domain generalization","router network","MAGE benchmark","RAID benchmark"],"falsifier":"A reader could check whether, on dialog_sum documents, the router assigns the highest weights to the hswag and eli5 experts, which score 0.962 and 0.956 standalone AUROC on that domain; if the router does not, the failure is routing, and if it does, the weighted combination is dragging performance down. Either observation would settle whether the transfer assumption holds for structurally novel domains.","tokens_in":15471,"feed_emoji":"🤖","tokens_out":8525,"duration_ms":71669,"temperature":0.7,"pith_summary":"AI-text detectors often fail when asked to flag text from a genre or generator they were not trained on. The paper proposes DoGEN, an ensemble of ten domain-specialist detectors plus a domain router that picks the two most relevant experts for each document. On the MAGE benchmark it reports 97.60 AUROC across all fourteen test domains, ahead of a 32B dense model at 95.20. On RAID, an out-of-domain benchmark of unseen genres and newer generators, it reports 95.81 versus 94.73 for the 32B model, while activating 5.4B parameters instead of 32B. The authors' central claim is that domain-based gating transfers: a router trained only to name the training domain still selects experts that work on related unseen domains.","feed_headline":"Ten gated experts beat a 32B model at AI-text detection","feed_subtitle":"DoGEN routes each document to two of ten domain specialists, beating a 32B dense baseline on unseen domains.","key_machinery":"The central object is the Domain Gating Ensemble Network (DoGEN): ten expert detectors $E_i$, each a Qwen1.5-1.8B with a binary classification head fine-tuned on a single MAGE domain, and a router $G$ that maps a document to a probability distribution over the ten domains via $p_i(x) = \\exp(w_i^\\top \\phi(x)) / \\sum_j \\exp(w_j^\\top \\phi(x))$. At inference the router keeps the top $k=2$ experts, renormalizes their probabilities, and computes the final score $s^{(k)}(x) = \\sum_{i \\in I_k} w_i E_i(x)$ with $w_i = p_i / \\sum_{j \\in I_k} p_j$. The mechanism works because experts specialize: the router learns to trust a specialist when the input matches its domain, and on unseen domains it selects the experts whose training domains are closest. The paper reports a Pearson correlation of 0.64 between an expert's standalone AUROC on RAID and the average gate weight assigned to it, supporting the claim that routing weight tracks expert usefulness.","core_discovery":"The paper claims that ensembling domain specialists with a domain-classification router yields a detector that is both highly accurate in-domain and unusually robust out-of-domain. Each expert is a Qwen1.5-1.8B model fine-tuned on one of MAGE's ten training domains; the router is another Qwen1.5-1.8B model fine-tuned to classify documents into those ten domains. At inference the router's softmax probabilities select the top two experts, and the final score is the weighted sum of their outputs. DoGEN's reported AUROC on the MAGE test set is 97.60 on average, and on RAID's non-adversarial set it is 95.81, compared with 95.20 and 94.73 respectively for the 32B dense baseline. The paper also reports that on MAGE's four unseen domains DoGEN trails the 32B model, attributing the gap to structurally novel inputs such as dialogue in dialog_sum and to GPT-4 paraphrased data.","pith_inferences":["One natural extension is to replace the router's domain-identity objective with a reliability objective that predicts each expert's expected AUROC on the input; this might improve domains like dialog_sum where the domain label is uninformative but some experts still perform well.","The fixed top-$k=2$ strategy is a design choice; on inputs where the router is uncertain, forcing two experts can include a poor specialist, so an adaptive $k$ based on router entropy is a testable extension the paper does not explore.","The modularity result has a deployment reading: a production detector could be extended incrementally as new generators and genres appear, with the cost of one expert plus a router retrain per new domain, rather than a full retrain of a dense model."],"forward_implications":["A detector can match or beat a 32B dense model on both in-domain and out-of-domain benchmarks while activating only two 1.8B experts plus a 1.8B router, about 5.4B parameters in total.","Because experts are trained independently, adding a new domain such as dialog_sum only requires training one new expert and retraining the router, not retraining the entire ensemble.","Router quality drives generalization: on RAID, experts with higher standalone AUROC receive higher average gate weights, so the ensemble's out-of-domain strength depends on the router recognizing which specialists are trustworthy.","Per-domain class balancing during expert training matters; the paper reports per-domain balancing outperforms global or unbalanced training for both in-domain and out-of-domain detection.","The state-of-the-art claim is qualified to average performance: on MAGE's unseen domains DoGEN trails the 32B model, notably on dialog_sum (45.64 versus 90.50), so the advantage is not uniform across every domain."],"supporting_citations":[{"why":"Supplies the MAGE benchmark: the ten training domains on which the experts and router are fine-tuned, plus the four held-out domains and generator lists used for evaluation.","marker":"(Li et al., 2024)"},{"why":"Supplies the RAID benchmark and its non-adversarial subset, the out-of-domain evaluation where DoGEN reports 95.81 AUROC.","marker":"(Dugan et al., 2024)"},{"why":"Defines the Weighted Vote baseline that learns static logistic-regression ensemble weights, which DoGEN's input-dependent routing is compared against.","marker":"(Abburi et al., 2023)"},{"why":"Provides the Qwen1.5-MoE-A2.7B architecture used as the Mixture-of-Experts baseline, isolating the effect of domain-aware gating versus generic MoE routing.","marker":"(Qwen Team, 2024)"}],"fun_headline_variants":["Gated experts route around 32B model on AI-text detection","Ten-domain router lifts AI detector past 32B baseline","Small gated ensemble beats 32B on unseen AI text","Adaptive expert gating outdoes 32B on out-of-domain","Ensemble router wins on unseen AI-text detection tasks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that a router trained only to name the ten training domains will assign useful weights for documents from domains it has never seen, because experts trained on similar-looking domains will still detect AI text in those new domains.","fun_headline_variants_meta":{"raw":{"variants":["Gated experts route around 32B model on AI-text detection","Ten-domain router lifts AI detector past 32B baseline","Small gated ensemble beats 32B on unseen AI text","Adaptive expert gating outdoes 32B on out-of-domain","Ensemble router wins on unseen AI-text detection tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00066,"raw_usage":{"total_tokens":2981,"prompt_tokens":870,"completion_tokens":2111,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":486,"completion_tokens_details":{"reasoning_tokens":2025}},"tokens_in":486,"tokens_out":2111,"duration_ms":12964,"temperature":1.0,"reasoning_tokens":2025,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:08:35.348410+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A reader could check whether, on dialog_sum documents, the router assigns the highest weights to the hswag and eli5 experts, which score 0.962 and 0.956 standalone AUROC on that domain; if the router does not, the failure is routing, and if it does, the weighted combination is dragging performance down. Either observation would settle whether the transfer assumption holds for structurally novel domains.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Weighted Vote baseline that learns static logistic-regression ensemble weights, which DoGEN's input-dependent routing is compared against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Qwen1.5-MoE-A2.7B architecture used as the Mixture-of-Experts baseline, isolating the effect of domain-aware gating versus generic MoE routing."}],"review_version":1}