{"id":"eb8c9536-ac23-4581-9a9a-4b49da339b6e","arxiv_id":"2505.19010","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":11,"one_line_summary":"A new multimodal fusion architecture reports small state-of-the-art gains on two offensive content benchmarks using co-attention, dimension-wise gating, and expert fusion.","lead":"The paper introduces Co-AttenDWG, a neural architecture that fuses text and image features with co-attention, channel-wise gating, and expert fusion to detect offensive social media content. On the MIMIC and SemEval Memotion benchmarks it reports small accuracy gains over generic vision-language models, but the result is not independently reproducible from the paper alone.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Internal contradiction between Table III and Table VI: the same XLM-RoBERTa/ResNet50 configuration is reported as 87.19/87.16 and 87.79/87.83 on Misogyny, so the headline SOTA claim is not reproducible from the manuscript.","rationale":"The reader's weakest_assumption was the split/upsampling order and possible train/test leakage. That is a fair concern, and I partially agree: the manuscript never states whether upsampling occurs before or after the 80/20 split, and Table I's balanced-count column describes whole datasets. However, the most load-bearing checkable problem is the internal inconsistency of the reported headline numbers: Table VI contradicts Table III on the exact same architecture and dataset. This is immediately verifiable from the manuscript, does not depend on external code availability, and directly affects the claimed improvement margin. I retain CONDITIONAL rather than REJECT because the inconsistency is addressable: a code release with a fixed seed and a single consistent run would settle it. This does not change the reader's verdict, but it sharpens the condition: before accepting the SOTA claim, the authors must reconcile Tables III, V, and VI and provide one reproducible set of numbers.","tokens_in":19030,"tokens_out":5532,"duration_ms":47348,"concrete_test":"Ask the authors to provide the exact training/evaluation script and fixed seed, and to re-run the best configuration from Table V (XLM-RoBERTa + ResNet50, 8 experts, 8 cross-attention heads, 4 co-attention heads, MambaFormer kernel 3/depth 2, dropout 0.1, images 200x160, LR 2e-5) with the 80/20 split and upsampling applied only to the training split. Report the Misogyny accuracy/F1 and compare against Table III (87.19/87.16), Table V (87.16 F1), and Table VI (87.79/87.83). If the re-run matches Table VI, Table III's +0.80 improvement row is wrong and must be recomputed against a single consistent run; if it matches Table III, Table VI's backbone-ablation conclusion is invalid. Either outcome requires a corrected, single-consistent-number version of the central claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that Co-AttenDWG establishes state-of-the-art results with e.g. 87.19% accuracy and 87.16% F1 on Misogyny in Table III. The paper's own backbone ablation (Table VI, Section IV-E) reports Co-AttenDWG with XLM-RoBERTa + ResNet50, the configuration identified as best on MIMIC, at 87.79% accuracy and 87.83% F1 on the same task. These two numbers cannot both describe the same final model under the same protocol; the gap (0.60 accuracy, 0.67 F1) is larger than the claimed +0.80 improvement over the previous best baseline (VisualBERT, 86.39). Table V's best row reports yet another Misogyny F1 value (87.16), so the manuscript contains conflicting values for what should be the same reported run. Without code, random seeds, or error bars, a reader cannot determine which number is trustworthy, and the claimed margin over baselines depends on which internally inconsistent value is used. This is not a stylistic issue: it directly undermines the reproducibility of the central state-of-the-art claim and the '+0.80' improvement narrative.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Co-AttenDWG, a multimodal fusion architecture for offensive content detection that combines co-attention, dimension-wise gating, dual-path MambaFormer encoders, additional cross-attention, and an expert fusion module with learned gating and self-attention. The method is evaluated on the MIMIC and SemEval Memotion 1.0 datasets, with reported state-of-the-art results such as 87.19% accuracy and 87.16% F1 on Misogyny, 94.80% on Objectification, 97.15% on Prejudice, 98.80% on Humiliation, and 84.29% accuracy / 84.26% F1 on Memotion. The paper includes ablations over architectural components, hyperparameters, and backbone choices, plus qualitative interpretability analyses.","tokens_in":19238,"tokens_out":4636,"duration_ms":42917,"significance":"If the empirical claims hold, Co-AttenDWG would provide a competitive recipe for fine-grained vision–language fusion, with a useful set of ablations showing the contribution of each component. The paper also ships detailed per-task error counts and interpretability heatmaps, which are strengths. However, the central claim is an empirical benchmark result, and the current manuscript does not yet support the claimed margins: the reported numbers are internally inconsistent, the preprocessing order is ambiguous, and no code, seeds, or error bars are provided to make the single-run comparisons verifiable.","major_comments":[{"comment":"The headline Misogyny result is internally inconsistent. Table III reports Co-AttenDWG at 87.19% accuracy and 87.16% F1, while Table VI reports the same configuration (XLM-RoBERTa + ResNet50, identified as the best backbone) at 87.79% and 87.83% on the same task. The gap (0.60 accuracy, 0.67 F1) is larger than the claimed +0.80 improvement over VisualBERT in Table III. The authors must explain which number is correct, ensure that all tables report the same final run, and clarify whether Table V numbers are validation or test results. As written, the central state-of-the-art claim is not reproducible from the manuscript.","section":"Table III vs. Table VI, Section IV-E"},{"comment":"The order of upsampling relative to the train/test split is ambiguous and potentially leak-prone. Section IV-B states that the authors 'partition both datasets into 80% training and 20% testing splits' and then apply upsampling, but Table I reports balanced counts for the entire MIMIC and Memotion datasets. If upsampling is performed before the split, the test set can contain exact copies of training samples, which would inflate all reported accuracies. The authors must explicitly state that upsampling is applied only to the training split after partitioning, or re-run the experiments if the current order does not guarantee separation.","section":"Section IV-B and Table I"},{"comment":"The reported margins over baselines are small (e.g., +0.07 accuracy on Objectification, +0.13 on Prejudice, and a negative margin on Humiliation), yet all results appear to come from a single run with no error bars, significance tests, or released code or seeds. Without such information, a reader cannot distinguish genuine improvement from random variation, especially for differences below one percentage point. The authors should provide multiple-seed statistics, confidence intervals or permutation tests, and ideally release code and seeds to make the comparisons auditable.","section":"Section IV-C and Tables III-VI"},{"comment":"The 'MambaFormer' encoder is not adequately defined or cited. The text references [40] and [41], which are Vaswani et al. (Transformer) and Hu et al. (Squeeze-and-Excitation), neither of which introduces a MambaFormer block. No equation or architectural detail specifies what MambaFormer consists of, despite it being a named component in Equations (7) and (9) and in the ablations of Table IV (MF). The authors need to provide a precise description and proper references, or the architecture cannot be reproduced.","section":"Section III-C, Equations (7)-(10)"}],"minor_comments":[{"comment":"The dataset name is misspelled as 'MMIC' in the captions of Tables III and IV; it should be 'MIMIC'.","section":"Tables III and IV captions"},{"comment":"The abbreviation 'FF' in Table IV is defined as 'Fine-grained Fusion' but no module with that name appears in the architecture description. The fusion network in Equations (15)-(16) is the likely referent, but the mapping should be made explicit.","section":"Section III-D and Table IV"},{"comment":"The phrase 'static fusion technique such as simple concatenation' is grammatically incomplete; it should be 'static fusion techniques such as simple concatenation'.","section":"Section II, paragraph 2"},{"comment":"The 'Pixel Value' column is hard to parse, especially the entry '2242×10−5' which appears to be a rendering artifact combining image size '224' and learning rate '2×10−5'. Please ensure each hyperparameter is in its own clearly separated column.","section":"Table V"},{"comment":"The statement that Co-AttenDWG 'consistently outperforms all baseline models across the majority of categories' is imprecise because the Humiliation category is below VisualBERT and mCLIP by 0.11 and 0.10 points, respectively; the wording should be adjusted to match the reported table.","section":"Section IV-C"}],"recommendation":"major_revision","confidential_remarks":"The core co-attention with dimension-wise gating is taken from the authors' own prior work [16], and the present contribution is an incremental combination with MambaFormer refinement and expert fusion. This is not fatal by itself, but the manuscript should state clearly what is new relative to [16]. The more urgent concern is verification: the internal table inconsistency and the ambiguous upsampling order mean that the central benchmark claims cannot currently be trusted. If the authors can correct the numbers, clarify the split order, and add statistical support, the paper may become acceptable; otherwise the empirical case remains unsupported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Co-AttenDWG is a coherent, incremental architecture for multimodal offensive content detection. The building blocks—cross-attention, dimension-wise gating, MambaFormer refinement, expert fusion—are all previously published, but the specific combination is new and the ablations suggest each piece contributes. The interpretability section (Grad-CAM, gating weights) is a plus. That is the good news.\n\nThe problem is the central empirical claim. The paper asserts state-of-the-art results, with 87.19% accuracy and 87.16% F1 on Misogyny in Table III. But Table VI, which reports backbone ablations, lists the XLM-RoBERTa+ResNet50 configuration at 87.79% and 87.83% on the same task. The paper identifies that backbone combination as the best on MIMIC and it is what should be used in Table III. The numbers should match. They do not, and the gap (0.60 accuracy, 0.67 F1) is larger than the claimed margin over the previous best baseline. The other four metrics (Objectification, Prejudice, Humiliation, Memotion) match between Table III and Table VI, which suggests this is a typo rather than a different run. But a typo in the headline result is not something the reader can fix without code.\n\nBeyond that, the evaluation protocol is underspecified. Section IV-B says the data is split 80/20 and then upsampling is applied, but the order is not stated. Table I shows balanced counts for entire datasets, so the test set may also be upsampled, which would be leakage. There are no error bars or significance tests, and the hyperparameters in Table V are selected from a validation sweep, so the reported test numbers are likely optimistic. Code is not released. The dimension-wise gating is close to the authors' prior work [16]; they cite it, but the paper does not quantify the delta over that method.\n\nNone of this makes the architecture worthless. The equations are clear, the ablations are internally consistent, and the limitations section is honest. It is a reasonable incremental engineering contribution. But right now, the reproducibility of the SOTA claim is not established. If the Misogyny discrepancy is a typo, it is fixable; if not, the improvement narrative needs revisiting.\n\nWho should read this? Anyone working on multimodal fusion or content moderation will find the architectural ideas usable, but they should treat the numbers as unverified. It deserves a serious referee, because the topic is timely and the work is substantial enough to warrant revision. I would not cite it as is, but if the authors release code and fix the inconsistencies, it could be worth citing.","headline":"Coherent incremental architecture for multimodal offensive detection, but the headline SOTA claim is undercut by an internal inconsistency in the Misogyny numbers and missing reproducibility details.","tokens_in":19869,"tokens_out":3540,"would_cite":false,"duration_ms":40865,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that Co-AttenDWG, by combining bidirectional co-attention with dimension-wise gating and expert fusion, sets new state-of-the-art results on the MIMIC and SemEval Memotion benchmarks for multimodal offensive content…","keywords":["multimodal offensive content detection","co-attention","dimension-wise gating","expert fusion","MambaFormer","MIMIC dataset","Memotion dataset","cross-modal alignment"],"falsifier":"Re-run the comparison with a pre-registered protocol in which the MIMIC and Memotion datasets are split into 80 percent training and 20 percent test before any upsampling, apply upsampling only to the training portion, freeze this split across all models, and train each baseline with the same hyperparameter budget; if Co-AttenDWG's margins over mCLIP and VisualBERT shrink to within one standard error or disappear, the state-of-the-art claim rests on protocol rather than architecture.","tokens_in":18759,"feed_emoji":"🛡️","tokens_out":5929,"duration_ms":46996,"temperature":0.7,"pith_summary":"The paper proposes Co-AttenDWG, an architecture for detecting offensive content in memes that pairs text and image encoders with a fusion stack designed to let the two modalities influence each other before they are combined. The central claim is that this stack—bidirectional co-attention, channel-level gating, MambaFormer refinement, and expert fusion—outperforms established vision-language models such as mCLIP, VisualBERT, ALBEF, and BLIP on the MIMIC and SemEval Memotion benchmarks. On the reported test splits it achieves state-of-the-art results on three MIMIC subtasks (Misogyny, Objectification, Prejudice) and on Memotion, while trailing VisualBERT by 0.11 percentage points on Humiliation. A combinatorial ablation shows every major module contributes, with the largest drops when multiple modules are removed. If the result holds under a leak-free evaluation protocol, it argues that fine-grained cross-modal gating plus mixture-of-experts fusion is a better recipe than static concatenation or standard cross-attention for this task.","feed_headline":"Fusion model Co-AttenDWG tops offensive-meme benchmarks","feed_subtitle":"Channel-wise gating plus expert fusion beats mCLIP, VisualBERT, ALBEF, BLIP on MIMIC and Memotion tests.","key_machinery":"The load-bearing object is the gated co-attention plus expert-fusion stack assembled inside Co-AttenDWG. Co-attention computes multi-head attention in both directions between text and image; dimension-wise gating applies a sigmoid gate per channel to the attention outputs, which is what lets the model emphasize or suppress individual feature dimensions rather than whole modalities. MambaFormer encoders, which combine selective-state-space processing with convolutional and self-attention operations, refine the gated cross-modal features, and an additional cross-attention layer realigns the two paths. The expert fusion module combines a feed-forward network, a softmax-gated weighted sum of the two modality experts, and a self-attention refinement, all under layer normalization. These pieces carry the claim because the ablations show that removing any one of them lowers accuracy on every benchmark.","core_discovery":"Co-AttenDWG is a dual-path fusion model. Text and image features are projected into a shared space; a co-attention layer lets text attend to image and image to text, and a dimension-wise gating network multiplies the attention outputs channel-by-channel by sigmoid weights so each feature dimension is strengthened or suppressed. The gated features pass through two MambaFormer encoders, each refining the other modality's signal before element-wise addition with the original projection, and an extra cross-attention layer aligns the two paths again. A fusion module then concatenates the paths, computes a softmax gating weight for each modality, and sums the weighted experts with the output of a feed-forward network and a self-attention layer under layer normalization. With XLM-RoBERTa and ResNet50 as backbones, the paper reports 87.19 accuracy and 87.16 F1 on Misogyny, 94.80 on Objectification, 97.15 on Prejudice, 98.80 on Humiliation, and 84.29 accuracy with 84.26 F1 on Memotion, and the ablation results attribute the gains to every component of the stack.","pith_inferences":["A natural next test is to freeze the backbones and train only the fusion stack; if the gains mostly vanish, the improvement may come from fine-tuning XLM-RoBERTa rather than from the co-attention and gating machinery itself.","The same fusion recipe could be evaluated on other multimodal tasks such as hate-speech detection, sarcasm detection, and sentiment analysis, where the per-channel gating logic should transfer if the mechanism is general.","The reported gains over strong baselines are small (0.07 to 1.69 percentage points), so repeated runs with multiple seeds and a pre-registered train/test split would tell whether the improvement is real or within noise.","Since the error table shows most confusion is between adjacent offensiveness levels, framing the Memotion task as ordinal regression might capture more signal than the current four-way softmax."],"forward_implications":["If the reported results are correct, Co-AttenDWG sets new state-of-the-art numbers on the MIMIC Misogyny, Objectification, and Prejudice tasks and on Memotion offense classification.","The ablation study implies that each architectural choice—co-attention, dimension-wise gating, dual-path MambaFormer refinement, additional cross-attention, and expert fusion—contributes to the final score, with the largest degradation when several are removed together.","The 31.1 ms per-sample inference time on a single consumer GPU suggests the extra fusion machinery does not make the model impractical for content-moderation pipelines.","The per-sample gating and attention visualizations indicate the model adapts its fusion strategy, leaning on text when visual cues are ambiguous and on images when they are decisive."],"supporting_citations":[{"why":"Supplies the BERT text encoder that produces the [CLS] features used in the Co-AttenDWG text branch.","marker":"[6]"},{"why":"Supplies the ResNet50 image encoder that extracts the visual features projected into the shared embedding space.","marker":"[10]"},{"why":"VisualBERT is the strongest baseline on the Humiliation task and a primary comparison point for the claimed state-of-the-art results.","marker":"[14]"},{"why":"mCLIP is a key vision-language baseline that Co-AttenDWG claims to outperform on MIMIC and Memotion.","marker":"[48]"},{"why":"Supplies the SemEval Memotion 1.0 benchmark with four offensive-intensity classes.","marker":"[51]"},{"why":"Supplies the MIMIC benchmark with four misogyny-related binary tasks in code-mixed Hindi-English.","marker":"[52]"},{"why":"The sparse mixture-of-experts gating idea that the expert fusion module adapts.","marker":"[42]"},{"why":"The source of the dimension-wise gating idea, applied here channel-wise after co-attention.","marker":"[41]"},{"why":"Supplies the multi-head attention and cross-attention formulations used throughout the architecture.","marker":"[40]"}],"fun_headline_variants":["Co-AttenDWG: channel gating + expert fusion tops meme benchmarks","Dimension-wise gating and co-attention propel offensive-meme detection","Co-AttenDWG fuses text and images, beats SOTA on MIMIC and Memotion","New model Co-AttenDWG wins on multi-modal offensive content detection","Dual-path fusion with channel gating sets new high on meme tasks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the evaluation protocol is fair and leak-free: the class-balancing upsampling is applied only to the training split, and every baseline is trained under the same split, tuning, and metric as Co-AttenDWG; the paper does not explicitly state that upsampling was applied after the train/test split.","fun_headline_variants_meta":{"raw":{"variants":["Co-AttenDWG: channel gating + expert fusion tops meme benchmarks","Dimension-wise gating and co-attention propel offensive-meme detection","Co-AttenDWG fuses text and images, beats SOTA on MIMIC and Memotion","New model Co-AttenDWG wins on multi-modal offensive content detection","Dual-path fusion with channel gating sets new high on meme tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000213,"raw_usage":{"total_tokens":1462,"prompt_tokens":1026,"completion_tokens":436,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":642,"completion_tokens_details":{"reasoning_tokens":330}},"tokens_in":642,"tokens_out":436,"duration_ms":5514,"temperature":1.0,"reasoning_tokens":330,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:21:48.268882+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the comparison with a pre-registered protocol in which the MIMIC and Memotion datasets are split into 80 percent training and 20 percent test before any upsampling, apply upsampling only to the training portion, freeze this split across all models, and train each baseline with the same hyperparameter budget; if Co-AttenDWG's margins over mCLIP and VisualBERT shrink to within one standard error or disappear, the state-of-the-art claim rests on protocol rather than architecture.","supporting_citations":[{"cited_title":"Task report: Memotion analysis 1.0@ semeval 2020: The visuo-lingual metaphor,","cited_arxiv_id":null,"evidence_quote":"Supplies the SemEval Memotion 1.0 benchmark with four offensive-intensity classes."},{"cited_title":"Mimic: misogyny identification in multimodal internet content in hindi-english code-mixed language,","cited_arxiv_id":null,"evidence_quote":"Supplies the MIMIC benchmark with four misogyny-related binary tasks in code-mixed Hindi-English."}],"review_version":1}