{"id":"123c889b-a153-44b8-9ffc-69c7a14f69c0","arxiv_id":"2504.14171","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A new active domain adaptation framework for multimodal fake news detection that fuses text, image, and cross-modal classifiers and selects the most uncertain and diverse target samples for annotation.","lead":"This paper presents ADOSE, a system that adapts fake news detectors to new topic domains by actively choosing a small set of target examples for human labeling. It combines three specialized classifiers (text, image, text-image) with an uncertainty-and-diversity sample selector, and reports accuracy gains of 2.7 to 14.0 percent over existing active domain adaptation methods on two public datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed gains over existing ADA baselines are not yet established: most baselines lack ADOSE's MEFN backbone and no variance is reported, so the 2.72–14.02% margins may reflect architecture or noise rather than the LUS+MDC selection strategy.","rationale":"The reader's weakest_assumption focused on the LUS uncertainty approximation inherited from prior work; that is a genuine reproducibility and validity gap because K, J, and the variance schedule are not reported. However, the most load-bearing risk to the paper's central claim is that the headline comparison in Table 1 does not control for the MEFN architecture, which the paper's own ablations show is worth several points. The only backbone-controlled ADA comparison is against Entropy, and even that margin lacks error bars. A controlled re-run with shared features and multiple seeds would directly determine whether the proposed active-selection mechanism, rather than the new backbone or run-to-run noise, explains the reported improvements. The verdict remains CONDITIONAL: the method is plausible and internally coherent, but the central empirical claim needs stronger experimental support before acceptance as stated.","tokens_in":15802,"tokens_out":11239,"duration_ms":112706,"concrete_test":"Re-run every row of Table 1 with Detective, CLUE, EADA, Entropy, and ADOSE sharing the same pretrained TextCNN+ResNet50 feature extractor and the same MEFN-style fusion head for all methods (keeping only the selection criterion different), using 10 random seeds and the authors' exact train/test splits. Report mean±std and a paired significance test. If ADOSE's accuracy advantage over the best same-backbone baseline is within one standard deviation or is not significant on either dataset, the claimed improvement over existing ADA methods is not established.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is the empirical superiority in Table 1 (Section 5.2). This comparison is confounded by architecture and lacks statistical control. In Section 5.1.4, only Entropy is explicitly run on the MEFN backbone; Detective, CLUE, and EADA are used as their original methods, so their features and fusion mechanisms differ from ADOSE. The ablations in Table 2 show MEFN itself contributes about 4 accuracy points on Pheme (83.23→79.23 without MEFN) and 3 points on Weibo (86.66→83.65), so a baseline without MEFN cannot isolate the value of the active-selection components. The one backbone-controlled comparison, ADOSE vs. Entropy, gives the smallest margins (3.24% on Pheme, 2.72% on Weibo). No standard deviations, number of seeds, or significance tests are reported, and CLUE's Pheme Avg. F1(fake)=20.16 is anomalously low, suggesting high variance. Therefore the headline 'outperforms existing ADA methods by 2.72%–14.02%' may be driven by implementation or architecture differences rather than by LUS+MDC.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ADOSE, an Active Domain Adaptation (ADA) framework for multimodal fake news detection, combining a Modal-dependency Expertise Fusion Network (MEFN) with three expert classifiers (text, image, cross-modal), a Least-disagree Uncertainty Selector (LUS), and a Multi-view Diversity Calculator (MDC). The MEFN uses adversarial training and contrastive learning to obtain domain-invariant and semantically aligned unimodal and cross-modal features; LUS selects uncertain samples by approximating a least-disagree metric via Gaussian weight perturbations; MDC then diversifies the candidate set using multi-view similarity. Experiments on Pheme and Weibo report that ADOSE outperforms existing ADA methods by 2.72%-14.02% in average accuracy, with ablations showing each module contributes to the final result.","tokens_in":16056,"tokens_out":7939,"duration_ms":67345,"significance":"If the claimed improvements are reliable, this would be a meaningful step for active domain adaptation in multimodal fake news detection, where fine-grained modality dependencies and target-domain informativeness both matter. The paper also demonstrates a sensible decomposition into intra-modal and inter-modal expertise and an active selection criterion tailored to domain shift. However, the current empirical evidence is weakened by backbone mismatches in the baseline comparison, absent statistical significance reporting, and per-dataset tuning of a key hyperparameter; the contribution is therefore promising but not yet fully established.","major_comments":[{"comment":"The central claim of outperforming existing ADA methods is confounded by backbone mismatch. Only the Entropy baseline is explicitly run on the MEFN backbone; Detective, CLUE, and EADA are used as their original methods. Since the ablation in Table 2 shows MEFN itself contributes 4.00 accuracy points on Pheme (83.23 vs 79.23 without MEFN) and 3.01 points on Weibo (86.66 vs 83.65), the large margins over those baselines cannot be attributed solely to the LUS+MDC selection strategy. The backbone-matched comparison against Entropy yields much smaller margins (3.24% on Pheme, 2.72% on Weibo). Please rerun all ADA baselines on the same MEFN backbone, or provide an explicit decomposition of the contribution of the backbone versus the selection strategy.","section":"Section 5.1.4, Table 1"},{"comment":"No variance or statistical significance measures are reported. Tables 1 and 2 list single-point estimates without standard deviations, number of seeds, or significance tests. Given that the smallest backbone-controlled margin is only about 2.7-3.2 percentage points, the reported improvements cannot be judged as reliable without such statistics. In addition, the CLUE result on Pheme (Avg. Acc=69.21, Avg. F1(fake)=20.16) is an extreme outlier relative to other methods and suggests an unstable or failed run; the paper should examine and explain this anomaly.","section":"Tables 1 and 2"},{"comment":"The multiplier m in the selection process is tuned per dataset: the paper states m=2 for Pheme and m=5 for Weibo. Figure 3 shows that accuracy is sensitive to m, and the tuning is done on the same datasets used for the final evaluation without a held-out validation split. This constitutes a form of test-set selection and inflates the reported gains. Please either fix m a priori, use a validation-based selection procedure, or discuss the risk of per-dataset overfitting.","section":"Section 5.1.2, Figure 3"},{"comment":"The LUS component, which the ablation identifies as the most impactful, does not report its core hyperparameters. The number of perturbation rounds K, the number of weight samplings J, and the variance schedule {sigma_k^2} are defined in Equations (11)-(15) but never specified in the experimental setup. This prevents reproduction and leaves open the question of how sensitive the method is to these choices. Provide the concrete values and include an ablation over K and J to demonstrate stability of the LDM approximation for the multimodal fusion network.","section":"Section 4.2"},{"comment":"Equation (8) as printed is not a valid probability normalization: P_mefn = softmax(score_y / log_{y' in Y} exp(score_y')) divides the logits by the log-sum-exp of the same logits, which is not the standard product-of-experts softmax. I suspect the intended formula is P_mefn = softmax(score_y), with score_y defined as the sum of log-probabilities from the three classifiers. Please correct the equation and verify that the implemented model matches the corrected formula.","section":"Equation (8)"},{"comment":"The ablation setup for 'w/o LUS' is underspecified. Removing the Least-disagree Uncertainty Selector could mean random target sample selection, diversity-only selection, or another replacement strategy. Since the paper argues that LUS is the core selection mechanism, it is essential to state exactly what selection strategy replaces it in this ablation so the reader can interpret the large drop in accuracy (77.67 on Pheme, 80.45 on Weibo).","section":"Section 5.3, Table 2"}],"minor_comments":[{"comment":"In the definition of the adversarial loss, the sentence reads 'L_adv = L_abv_t + L_abv_t'; the second term should be L_abv_v.","section":"Section 4.1.1"},{"comment":"There is a typo in the perturbation notation: 'eW_cla_v' should be 'eW_cls_v' for consistency with the other classifier weight symbols.","section":"Equation (12)"},{"comment":"The word 'hypermeter' should be 'hyperparameter', and in Section 6 'experrt' should be 'expert'.","section":"Section 4.4"},{"comment":"The lower limit of the summation over negative samples is typeset incorrectly ('b˝ i≠j'); please clarify the index range for the contrastive loss.","section":"Equation (7)"},{"comment":"The total number of target unlabeled samples N_tu for each dataset is not reported; providing these numbers would help readers interpret the active annotation budget k = B/5.","section":"Section 5.1.1"},{"comment":"The case study uses subjective visual descriptions such as 'brighter' and 'darker' without quantitative evidence; consider showing the corresponding feature distances or model confidence values to make the illustrations more rigorous.","section":"Section 5.4"}],"recommendation":"major_revision","confidential_remarks":"The core idea is sound and timely, but the evaluation currently overstates the contribution because the main baseline comparison is not architecture-controlled and lacks statistical grounding. The per-dataset tuning of m further weakens the headline numbers. I would be open to accepting after the authors address the major comments, especially the backbone-matched baselines, significance testing, and LUS hyperparameter reporting. Also, the reference list is dominated by papers from the authors' own institution/group; while not disqualifying, it would be good to see broader coverage of the active-learning and multimodal-domain-adaptation literature."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper. First, the core idea is reasonable: it combines three expert classifiers (text, image, cross-modal) with active selection that mixes least-disagree uncertainty and multi-view diversity, and it targets a real operational problem—reducing annotation cost for cross-domain multimodal fake news detection. Second, the headline performance claims are not yet established. The evaluation has no error bars or significance tests, and most baselines do not share the paper's MEFN backbone, so the reported 2.72–14.02% margins likely conflate architecture with selection strategy. The one controlled comparison, ADOSE versus Entropy (which does use the MEFN backbone), gives margins of only about 3% on each dataset—plausible but not confirmed without repeated runs.\n\nThe paper does some things well. The ablation study is genuinely informative: removing LUS hurts most, then MEFN, then MDC, which is consistent with the design logic. The comparison of ADOSE to its own no-active-selection variant (ADOSE-UDA) shows that active selection adds value. The problem formulation is clear, and the method is described at a conceptual level well enough for someone to reimplement the main pieces. The authors also cite prior work on the least-disagree metric explicitly, which is honest.\n\nThe soft spots are in proportion to the claims. The biggest issue is statistical: for deep learning experiments, 2–4% differences can easily be noise, and without multiple seeds we cannot tell. The headline range is misleading because the upper bound (14.02%) comes almost entirely from CLUE's anomalously low performance on Pheme (Avg F1(fake)=20.16), which suggests an implementation mismatch. A more honest claim would focus on the ~3% margin over the same-backbone Entropy baseline. The LUS module, which the ablation says is the most important, is missing key hyperparameters: the paper never reports K (perturbation rounds), J (weight samples), or the variance schedule. The method is not fully reproducible as written. Finally, the multiplier m is tuned per dataset (2 for Pheme, 5 for Weibo) with no validation protocol described, which raises a mild overfitting concern.\n\nNone of these flaws are fatal—the method is not circular, the writing is honest, and the ablations support the internal logic. This paper is for people working on fake news detection or active domain adaptation specifically. A reader in that area gets a useful baseline and a concrete example of how evaluation pitfalls can inflate performance claims. I would send it to peer review rather than desk-reject it, but I would ask the authors to release code, report variance across seeds, run backbone-controlled baselines, and specify the LUS details before publishing any specific margin as fact.","headline":"A sensible incremental combination of active learning and domain adaptation for multimodal fake news, but the headline gains are inflated by a weak baseline and the evaluation lacks variance; worth a real review, not a desk reject.","tokens_in":16594,"tokens_out":3385,"would_cite":false,"duration_ms":30113,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ADOSE is an active domain adaptation framework for multimodal fake news detection; the paper claims it outperforms existing methods by 2.72% to 14.02% average accuracy.","keywords":["active domain adaptation","multimodal fake news detection","least-disagree uncertainty","diversity-based sampling","cross-modal dependency learning","Gaussian perturbation","domain shift","text-image mismatch"],"falsifier":"Run the ADOSE pipeline on Pheme and Weibo with the LUS selection step replaced by random selection of the same number of target samples, keeping MEFN and MDC fixed. If average accuracy stays within the reported 3.24–14.02% range over baseline methods, the least-disagree uncertainty estimate is not what drives the gains.","tokens_in":15579,"feed_emoji":"📰","tokens_out":5643,"duration_ms":46748,"temperature":0.7,"pith_summary":"ADOSE is an active domain adaptation framework for multimodal fake news detection. The paper claims that by training three expert classifiers—one on text, one on images, and one on cross-modal pairs—and choosing which target-domain samples to annotate through a least-disagree uncertainty score plus a multi-view diversity score, a model can adapt to a new news topic with only 10% of target labels. On two standard datasets the paper reports average accuracy gains of 3.24%–14.02% (Pheme) and 2.72%–7.04% (Weibo) over existing active and unsupervised domain adaptation baselines. If correct, this gives practitioners a working recipe for cross-topic fake news detection without waiting for large amounts of newly labeled data.","feed_headline":"Active domain adaptation lifts fake-news detection by up to 14%","feed_subtitle":"Three expert classifiers and smart target sampling adapt detection to new topics with 10% of labels.","key_machinery":"The load-bearing object is the least-disagree metric (LDM): the smallest probability, over hypotheses that flip a sample's predicted label, that two hypotheses disagree on a random target sample. Because the true LDM cannot be computed, the paper replaces it with an estimator that perturbs the last-layer weights of each expert classifier with Gaussian noise across several variance scales and counts how often the fused prediction flips; smaller estimated values mark samples close to the decision boundary. Around this estimator the framework builds a three-classifier fusion network (text, image, and cross-modal experts) and a diversity scorer that averages cosine similarities over the three modality views to pick the most representative subset.","core_discovery":"The paper's central claim is that domain shift in multimodal fake news detection can be handled by separating three kinds of deception evidence—within-text errors, within-image errors, and text–image inconsistencies—and actively annotating the target samples that sit closest to the current decision boundary while also covering the target feature space. The proposed pipeline, ADOSE, fuses the predictions of two unimodal classifiers and one cross-modal classifier (MEFN), ranks unlabeled target samples by an estimated least-disagree metric obtained from Gaussian weight perturbations (LUS), then re-ranks the top candidates by diversity across text, image, and cross-modal feature views (MDC). The experiments on Pheme and Weibo report average accuracy of 83.23% and 86.66%, respectively, exceeding all compared unsupervised and active domain adaptation methods on the average metrics.","pith_inferences":["One testable extension is to calibrate the perturbation schedule (number of rounds K, samples per round J, variance range) on a held-out domain pair; the paper does not report these values, so its headline gains may be sensitive to them.","The same uncertainty-plus-diversity selection could be applied to other multimodal verification tasks such as rumor detection in social media posts or AI-generated image–text pairs, since the selection logic does not depend on news-specific features.","If domain divergence between source and target is small, the ablation suggests uncertainty selection matters more than diversity; a production system could tune the m multiplier per domain pair rather than per dataset.","The paper's comparison against CLUE's low Pheme F1(fake) score hints that clustering-based ADA methods may be poorly suited to binary fake news data, though the paper does not test this directly."],"forward_implications":["A fixed small annotation budget—10% of the target domain—can be enough to transfer a fake news detector from one event or topic domain to another.","Decomposing deception patterns into intra-modal and inter-modal dependencies and scoring them separately gives a measurable edge over a single fused classifier.","Uncertainty-based selection alone leaves redundant samples in the annotated set; adding a multi-view diversity filter improves average accuracy, especially when source and target topics differ sharply.","The LUS ranking can identify target samples whose deception patterns deviate most from the source domain, which is exactly the subset that helps the model adapt."],"supporting_citations":[{"why":"Supplies the least-disagree metric approximation, based on Gaussian perturbations, that the LUS selector adopts.","marker":"[2]"},{"why":"Provides the multi-modal feature alignment baseline (RDCM) and the domain-invariant feature extraction plus negative-sample filtering strategy the framework builds on.","marker":"[27]"},{"why":"Clustering-based active domain adaptation baseline (CLUE) that the experiments must outperform.","marker":"[40]"},{"why":"Energy-based active domain adaptation baseline (EADA) defining the standard 10% budget setup the paper follows.","marker":"[49]"},{"why":"Multi-source active domain transfer baseline (Detective) compared against in the main table.","marker":"[55]"},{"why":"Source of the multi-domain Weibo topic splits used for evaluation.","marker":"[42]"},{"why":"Cross-modal contrastive learning approach (COOLANT) whose alignment idea motivates the contrastive component of MEFN.","marker":"[45]"},{"why":"Source of the Weibo multimodal fake news dataset used in experiments.","marker":"[46]"},{"why":"Source of the Pheme breaking-news dataset used in experiments.","marker":"[60]"}],"fun_headline_variants":["ADOSE: Active domain adaptation for fake news beats baselines by up to 14%","Smart target sampling and three experts cut fake news domain shift by 14%","Three experts and smart sampling adapt fake news detection by up to 14%","Active learning picks the right fake news samples, boosting detection by 14%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's gains rest on the assumption that adding Gaussian noise to the final classifier weights creates a family of plausible models whose smallest disagreement rate measures a sample's true proximity to the decision boundary, so that the samples most affected by such noise are the most informative ones to label.","fun_headline_variants_meta":{"raw":{"variants":["ADOSE: Active domain adaptation for fake news beats baselines by up to 14%","Smart target sampling and three experts cut fake news domain shift by 14%","Three experts and smart sampling adapt fake news detection by up to 14%","Active learning picks the right fake news samples, boosting detection by 14%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001122,"raw_usage":{"total_tokens":4661,"prompt_tokens":933,"completion_tokens":3728,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":549,"completion_tokens_details":{"reasoning_tokens":3642}},"tokens_in":549,"tokens_out":3728,"duration_ms":23185,"temperature":1.0,"reasoning_tokens":3642,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:54:18.347082+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the ADOSE pipeline on Pheme and Weibo with the LUS selection step replaced by random selection of the same number of target samples, keeping MEFN and MDC fixed. If average accuracy stays within the reported 3.24–14.02% range over baseline methods, the least-disagree uncertainty estimate is not what drives the gains.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the least-disagree metric approximation, based on Gaussian perturbations, that the LUS selector adopts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the multi-modal feature alignment baseline (RDCM) and the domain-invariant feature extraction plus negative-sample filtering strategy the framework builds on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Clustering-based active domain adaptation baseline (CLUE) that the experiments must outperform."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Energy-based active domain adaptation baseline (EADA) defining the standard 10% budget setup the paper follows."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Multi-source active domain transfer baseline (Detective) compared against in the main table."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Source of the multi-domain Weibo topic splits used for evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Cross-modal contrastive learning approach (COOLANT) whose alignment idea motivates the contrastive component of MEFN."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Source of the Weibo multimodal fake news dataset used in experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Source of the Pheme breaking-news dataset used in experiments."}],"review_version":1}