{"id":"639ee6ef-1392-4082-a4cc-2414f1a8f87f","arxiv_id":"2411.14823","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"One jointly trained image-manipulation-localization model attains top average scores on natural, document, face, and scene-text forensics benchmarks and gains artifact-description ability from a new GPT-4o-annotated dataset.","lead":"Researchers built a single AI model that can find and localize edited or fake regions in photos, documents, faces, and scene text using one set of weights instead of separate models. They also created a 273,776-image dataset with structured, automatically generated text explanations of the tampering artifacts.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Claim of state-of-the-art across all four tasks is contradicted by the paper's own per-dataset numbers; Omni-IML loses on several datasets (e.g., CASIAv1, IMD20, T-IC13, DocTamper-SCD), so 'SOTA' only holds as an average, not per benchmark.","rationale":"I agree with the reader's conditional assessment, but the single most load-bearing concern is not primarily the training-data comparability of Tables 2–4 (the reader's stated weakest assumption). It is that the paper's own reported numbers contradict the headline claim of 'state-of-the-art performance across all four major IML tasks' when interpreted in the usual per-benchmark sense. This is internally checkable and does not depend on external data or on whether the training data are matched. The loss on CASIAv1, NIST16, IMD20 (natural), DocTamper-SCD (document), and T-IC13 (scene text) is directly visible in the tables. The paper's Section 5.2 says Omni-IML can 'outperform existing specialized methods on each individual task,' but this is only true if 'task' means the average of multiple datasets, not the individual benchmark. The reader's rationale does mention that 'Omni-IML loses on several individual datasets,' so there is partial agreement, but the weakest_assumption field focuses on unfair comparison due to different training data. My concern is orthogonal: even if training data were perfectly matched, the overclaim would persist unless the per-dataset results are presented and the claim qualified. The controlled Table 5 actually strengthens the paper's case for joint-training robustness, but it does not rescue the unqualified SOTA statement because it uses aggregate numbers and an incomplete baseline set. The interpretation circularity, missing appendix, and unreleased code/dataset are additional validity/reproducibility issues, but the per-dataset contradiction is the most direct threat to the abstract's central promise. A CONDITIONAL acceptance with a required rephrasing and per-dataset analysis is appropriate, which matches the reader's verdict; hence I recommend UNCHANGED.","tokens_in":17404,"tokens_out":7822,"duration_ms":72755,"concrete_test":"Compute, from Tables 2–4, for each dataset and metric (IoU and F1), whether Omni-IML is strictly the highest-scoring method among all listed baselines. Tally wins/losses per task. If Omni-IML is not the best on a majority of datasets in every task—it currently loses on CASIAv1, NIST16, IMD20, DocTamper-SCD, and T-IC13—then the phrase 'state-of-the-art across all four tasks' is inaccurate and should be revised to 'competitive average performance' or 'average state-of-the-art across datasets.' Additionally, re-run Table 5 with the full set of baselines from Tables 2–4 (including APSC-Net, DTD, SegFormer, Swin-UPer) using the same training data and pipeline, and report per-dataset IoU to verify whether the joint-training advantage persists on each dataset rather than only in aggregate.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract and Section 5.2 claim that a single Omni-IML model 'achieves state-of-the-art performance across all four major IML tasks' and 'can simultaneously outperform existing specialized methods on each individual task.' This strong claim is not supported by the paper's own Tables 2–4. In Table 2, APSC-Net achieves higher IoU than Omni-IML on CASIAv1 (0.810 vs 0.796), NIST16 (0.525 vs 0.522), and IMD20 (0.679 vs 0.662). In Table 3, DTD beats Omni-IML on DocTamper-SCD (0.691 vs 0.664). In Table 4, SegFormer (0.778) and Swin-UPer (0.773) beat Omni-IML (0.741) on T-IC13 IoU. Thus, 'state-of-the-art' is only true if one averages across datasets within each task, a qualification the paper never states. This matters because the paper's central value proposition is that one generalist model can replace per-task specialists; if task-specific specialists still win on specific widely used benchmarks, the replacement claim is overstated. The controlled comparison in Table 5 does not resolve this: it reports only aggregate per-task IoU, includes only a subset of baselines (no SegFormer, no DiffForensics, no UnionFormer), and does not provide per-dataset breakdowns. Therefore, the central claim as worded is not merely under-supported by external training-data differences; it is internally contradicted by the reported evaluation numbers.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Omni-IML, a single image-manipulation-localization model intended to work across natural, document, face, and scene-text images without task-specific fine-tuning. The method combines a Modal Gate Encoder that chooses between RGB and RGB+frequency features, an Anomaly Enhancement module trained with box supervision, and a Dynamic Weight Decoder that selects decoder filters per sample. The authors also introduce Omni-273k, a large dataset of GPT-4o-generated artifact descriptions, and an interpretation module that feeds the input image together with a highlight overlay of the predicted mask into a multimodal LLM. Experiments evaluate localization on four tasks and interpretation on the new dataset, with ablations showing that each proposed module contributes to the reported averages and that the proposed model degrades less than several baselines under joint training.","tokens_in":17711,"tokens_out":8676,"duration_ms":92152,"significance":"If the claims are supported, the paper would be a valuable step toward a generalist IML system: it addresses a real practical limitation of task-specific forgery detectors, provides a large new interpretability dataset, and introduces a controlled joint-training comparison that is rarely reported in this area. The localized ablation study and the consistency of the per-module gains are strengths. However, the paper's central value proposition is currently overstated: the reported per-dataset numbers do not support the abstract's and Section 5.2's claim that a single model achieves state-of-the-art performance across all four tasks. In addition, the interpretation experiments evaluate models against the same GPT-4o-generated references used for training, so the quantitative interpretation results are partly circular. The contribution is still publishable after the claims are qualified and the evaluation is strengthened.","major_comments":[{"comment":"The claim that Omni-IML 'achieves state-of-the-art performance across all four major IML tasks' and 'can simultaneously outperform existing specialized methods on each individual task' is contradicted by the paper's own per-dataset results. In Table 2, APSC-Net has higher IoU on CASIAv1 (0.810 vs 0.796), NIST16 (0.525 vs 0.522), and IMD20 (0.679 vs 0.662). In Table 3, DTD has higher IoU on DocTamper-SCD (0.691 vs 0.664). In Table 4, SegFormer and Swin-UPer have higher IoU on T-IC13 (0.778 and 0.773 vs 0.741) and also higher F1 (0.875 and 0.872 vs 0.851). The 'state-of-the-art' statement therefore holds only if one averages over datasets within each task, a qualification that is never stated. Since the paper's central value proposition is that a generalist can replace per-task specialists, this overstatement is load-bearing and should be corrected in the abstract, introduction, and Section 5.2.","section":"Abstract and §5.2, Tables 2–4"},{"comment":"The controlled comparison in Table 5 does not fully resolve the SOTA concern. It reports only per-task aggregate IoU, includes only a subset of the strongest baselines from Tables 2–4 (notably omitting SegFormer, DiffForensics, UnionFormer, and CAT-Netv2), and provides no per-dataset breakdown. Furthermore, the comparisons in Tables 2–4 use baselines with different training data, iteration counts, and backbone initializations; Omni-IML trains for 400k iterations with an ADE20k-pretrained ConvNeXt-Base and additional COCO data. The paper should either restrict the SOTA claim to the controlled setting, or extend Table 5 with per-dataset results and the missing strong baselines so that the reader can separate the effect of the proposed modules from the effect of more training data and a stronger backbone. The lack of error bars or significance tests is also a concern, since several reported margins are smaller than 0.01 IoU.","section":"§5.2, Table 5"},{"comment":"The interpretation evaluation is partly circular. The Omni-273k annotations are generated by GPT-4o, Step 3 of the chain-of-thought pipeline uses GPT-4o to correct and rewrite GPT-4o's own previous responses, and Tables 7 and 8 then measure the fine-tuned models against those same GPT-4o-generated references. High ROUGE/BLEU/cosine-similarity scores in this setting can reflect successful mimicry of GPT-4o rather than accurate forensic interpretation. The paper also states that the final annotations are 'very closed to human annotation' without reporting any human agreement study. I recommend adding a human evaluation on a sampled subset, or at least an independent set of human-written references, before claiming high-quality interpretation.","section":"§4.1 and §5.4, Tables 7–8"},{"comment":"The training signal for the Modal Gate is not specified in the submitted text. The gate is described as a binary classifier that decides whether to use the fused RGB+frequency features or the RGB-only features, but there is no statement of what supervises this decision or how a hard selection is made differentiable during end-to-end training. The main text refers to the appendix for details, but no appendix is included in the version under review. Since the Modal Gate is one of the three core contributions, the paper needs to specify the gate supervision and the differentiable relaxation (or provide the appendix). This is a reproducibility issue, not merely a presentation issue.","section":"§3.1"}],"minor_comments":[{"comment":"The caption of Figure 5 reads 'The proposed Chain-of-Thoughts Pipeline,' but the figure appears to show qualitative localization results; the caption should match the figure content.","section":"Figure 5"},{"comment":"There are several typos and inconsistencies: 'OSFT' should be 'OSTF' in Section 4.3, 'datatset' and 'isthe' appear in Section 4.4, 'Singe target' appears in Table 1, and the text alternately uses 'DWF' and 'DW' in Table 6 and the associated ablation discussion.","section":"§4.3 and §5.2"},{"comment":"The DWF dilation schedule and the number of dynamic filters are not reported in the main text; since the decoder's behavior depends on these hyperparameters, they should be stated explicitly or the appendix should be made available.","section":"§3.3"},{"comment":"The interpretation experiments compare fine-tuned models with and without the reference visual prompt, but they do not compare against prior interpretable-IML systems such as SIDA or FakeShield. Adding such a comparison would clarify the contribution relative to existing methods.","section":"§5.4"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid and useful contribution to IML, but the main claims need to be brought in line with the reported numbers. I would ask for a revised version that qualifies the SOTA claim, strengthens the controlled comparison, and adds human evaluation for the interpretation component. Please also ensure that the appendix is included with the resubmission, since several core implementation details are currently deferred to it."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a well-built generalist IML system with a genuinely useful new dataset, but the headline claim does not survive contact with its own tables. Omni-IML is not state-of-the-art on each benchmark; it wins on average. That matters because the abstract says \"across all four major IML tasks\" and Section 5.2 says it beats specialists on \"each individual task.\" Its own Table 2 shows APSC-Net higher on CASIAv1, NIST16, and IMD20; Table 3 shows DTD higher on DocTamper-SCD; Table 4 shows SegFormer and Swin-UPer higher on T-IC13. So SOTA only holds as an average, and the paper never says that qualification.\n\nWhat is genuinely new: per-sample modality gating, sample-adaptive decoder weights, and box-supervised anomaly enhancement in one model jointly trained across four IML domains. That combination is not in the cited prior work. The controlled comparison in Table 5 is the strongest evidence: when prior specialists are retrained on all tasks with the same data and pipeline, they collapse off-domain (TruFor drops below 0.1 IoU), while Omni-IML degrades only modestly and keeps the best average. That is a real result and it supports the core engineering claim. The Omni-273k dataset is also a concrete contribution: 273k images, multi-target, structured JSON annotations, and a chain-of-thought pipeline that plausibly improves over single-query GPT-4o descriptions. If the dataset ships, it will be useful regardless of the localization story.\n\nSoft spots are real but not fatal. First, the SOTA wording should be corrected to \"best average\" with per-dataset breakdowns shown. Second, Table 5 only reports aggregate per-task IoU and omits several strong baselines (SegFormer, DiffForensics, UnionFormer), so the controlled claim is narrower than stated. Third, there are no error bars, seeds, or significance tests; differences of 0.01-0.02 may be noise. Fourth, the interpretation evaluation is partly circular: GPT-4o both generates the ground truth and performs the self-examination, then Table 8 scores models against those same GPT-4o-derived labels. A human-validated subset would fix this. Fifth, the appendix with prompts and the promised code/data are not available, so as submitted some claims cannot be checked.\n\nThe stress-test note is right, and I do not think it is overreading. This paper deserves a serious referee because the architecture and dataset are substantive, but the authors need to revise the claims and release the artifacts. My recommendation: send to review with an expectation of major revision, not desk rejection.","headline":"Solid generalist IML system and a useful new dataset, but the state-of-the-art claim is overstated: it is average SOTA, not per-benchmark SOTA, and the interpretation evaluation is partly circular.","tokens_in":18288,"tokens_out":2408,"would_cite":true,"duration_ms":25455,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Omni-IML claims that one generalist model can localize tampering in natural images, documents, faces, and scene text at state-of-the-art level, without task-specific fine-tuning, and can describe what looks wrong.","keywords":["image manipulation localization","generalist forensic model","image forensics","multimodal LLM","document forgery","face forgery","scene text tampering","chain-of-thoughts annotation"],"falsifier":"Run the strongest per-task baselines and Omni-IML on all four tasks under identical training data, backbone weights, optimizer, input size, and iteration count, as Table 5 does but for the full baseline set; if any baseline matches or exceeds Omni-IML's average IoU of 0.714 in this controlled setup, the central claim of a generalist advantage is refuted rather than confirmed.","tokens_in":17159,"feed_emoji":"🕵️","tokens_out":8335,"duration_ms":76570,"temperature":0.7,"pith_summary":"Omni-IML is a proposed generalist model for image manipulation localization: one set of weights claims to localize tampered regions across natural photos, documents, faces, and scene text, without per-task or per-benchmark fine-tuning. The authors argue this matters because current forensic models are built for single image types and joint training makes them degrade, so practical forensic systems must maintain several separate specialists. The paper's three localization modules, a per-sample modality gate, a per-sample dynamic decoder filter, and a training-only box-supervision enhancement, are what supposedly let the model adapt instead of being confused by diverse tampering cues. The paper also contributes a 273,776-image interpretability dataset with structured natural-language artifact descriptions and an interpretation module that highlights the predicted mask for a multimodal LLM. If the claims hold, one generalist forensic model could replace four task-specific pipelines while also explaining its decisions.","feed_headline":"One model tops four forgery-localization tasks at once","feed_subtitle":"The model picks the right encoding per image, so one checkpoint beats per-task specialists on all four domains.","key_machinery":"The argument is carried by three sample-adaptive components plus an annotation pipeline. The Modal Gate Encoder computes both RGB and fused RGB-frequency features, produces two coarse predictions, and runs a small binary classifier that picks the more reliable encoding modality for each image, so noisy frequency features can be discarded when they hurt. The Dynamic Weight Decoder replaces a fixed filter stack with Dynamic Weight Filters: each filter averages the input feature into a global representation, combines it with an image-level vector through a fully connected layer, and computes a weighted sum of four base filters to produce sample-specific depthwise convolution kernels. The Anomaly Enhancement module adds a Faster R-CNN-style detection head, consisting of two feature pyramid networks, a region proposal network, and a box head, on top of two encoder feature levels, and these detection branches exist only during training; after training their enhanced features are added back to the segmentation features, so inference cost is nearly unchanged. For interpretation, the paper builds Omni-273k through a chain-of-thoughts pipeline that recognizes tampered instances one by one, generates focused artifact descriptions per instance, and then has the annotator self-examine the response; the annotations are stored as structured JSON with content, absolute position, relative position, and artifact fields. The Interpretation Module forms a reference prompt $I_{\\mathrm{ref}} = (I_{\\mathrm{input}} + I_{\\mathrm{mask}})/2$ by averaging the input image with the predicted mask, concatenates it along the longest side with the original image, and feeds both to a multimodal LLM, which draws the LLM's attention to suspect regions without changing the LLM architecture.","core_discovery":"The paper's central claim is that a single model, Omni-IML, can localize manipulated regions in natural images, document images, face images, and scene text images at once, with no task-specific or benchmark-specific fine-tuning, and that it reaches state-of-the-art results on all four in the reported comparisons. The authors attribute prior failures to unify these tasks to task-dependent architecture choices, such as edge enhancement for natural objects, frequency fusion for documents, and texture filters for faces, and to the absence of per-sample adaptation. Omni-IML instead routes each sample: a Modal Gate decides whether to feed the encoder frequency-plus-vision features or pure RGB features, a Dynamic Weight Decoder generates sample-specific convolutional filters, and an Anomaly Enhancement module, supervised with bounding boxes that exist only during training, sharpens tampered-region features. The paper also introduces Omni-273k, a 273,776-image dataset with structured natural-language artifact descriptions produced by a chain-of-thoughts annotation pipeline, and an interpretation module that feeds the LLM a reference image with the predicted mask highlighted, improving artifact description accuracy. On the paper's own controlled comparison, joint training degrades Omni-IML's document IoU by only 1.6 points, versus 6.9 points for a document specialist, supporting the claim that the model suffers less from multi-task confusion.","pith_inferences":["Because the headline comparisons use published numbers from differently trained baselines, the cleanest test of the claim is a full four-way matched comparison; Table 5 is only a step in that direction.","The sample-adaptive gate could be read as a learned routing policy; a testable prediction is that the gate's binary choices will cluster by image type, choosing fused features on documents and pure RGB on most natural images, and that these choices correlate with where frequency features help.","The chain-of-thoughts annotation format, with per-instance recognition and self-examination, could be reused to build explanation datasets for other dense prediction tasks with many small targets, such as medical or satellite imagery, and its structured JSON keys enable typed evaluation.","The interpretation module's simple averaging of mask and image creates a visual reference that could be applied to any segmentation-augmented vision-language model; it is not specific to image manipulation localization."],"forward_implications":["A single Omni-IML checkpoint can be deployed in place of four separate specialists on natural, document, face, and scene-text forgery localization, with no task-specific fine-tuning.","Joint training, which currently makes specialized forensic models unreliable, no longer needs to be avoided if the model adapts per sample; Omni-IML's degradation under joint training is reported as much smaller than that of prior models.","Frequency-domain evidence can be used selectively: the gate lets document images keep the benefit of frequency fusion while natural images are protected from the noise that always-on frequency features introduce.","The Omni-273k dataset, with structured JSON artifact annotations across four image types, allows fine-grained evaluation of exactly what a forensic explanation model gets right, such as content, position, and artifact type, rather than a coarse string match.","The reference visual prompt improves artifact description on multi-target and low-obviousness cases such as documents, and the method transfers across several multimodal LLMs."],"supporting_citations":[{"why":"It supplies the frequency-vision fusion design and the DocTamper dataset, and its always-on frequency features motivate the Modal Gate.","marker":"[32]"},{"why":"It is the strongest natural-IML baseline and the source of the MIML training data, and Omni-IML's natural-image results are compared against it.","marker":"[34]"},{"why":"It is the face-IML specialist cited to show joint-training degradation, and it provides the face localization baseline Omni-IML must beat.","marker":"[11]"},{"why":"It is the face-IML baseline with the strongest prior IoU and F1 scores, and Table 4 compares Omni-IML's face numbers against it.","marker":"[25]"},{"why":"It is a natural-IML baseline with noise-filter and transformer features, and it is also re-trained in Table 5 as a joint-training comparison.","marker":"[10]"},{"why":"It is a prior interpretable IML method using a single-query LLM annotation pipeline, and Omni-273k's chain-of-thoughts pipeline is contrasted with it.","marker":"[44]"},{"why":"It is a prior interpretable IML dataset and annotation pipeline that works only for single-target natural images, and it motivates the structured multi-target annotations.","marker":"[14]"},{"why":"It provides the region proposal network and box head that the Anomaly Enhancement module borrows for training-only box supervision.","marker":"[35]"},{"why":"Feature pyramid networks are used both in the anomaly enhancement detection branch and in the dynamic weight decoder's multi-scale fusion.","marker":"[21]"},{"why":"The pyramid pooling module supplies the global feature vector used to condition the dynamic filters.","marker":"[48]"}],"fun_headline_variants":["Single model tops all four forgery localization benchmarks","One model, four tasks: a universal tamper spotter","Omni-IML: one generalist for every image manipulation","Unified image manipulation localization with dynamic routing","Four tasks, no retraining: Omni-IML sets the bar"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that Omni-IML outperforms the specialized baselines because of its proposed modules, not because it is trained on more data with a stronger initialization; if the advantage comes mainly from the extra training data or the ADE20k-pretrained backbone, the 'state of the art across all four tasks' claim would be overstated.","fun_headline_variants_meta":{"raw":{"variants":["Single model tops all four forgery localization benchmarks","One model, four tasks: a universal tamper spotter","Omni-IML: one generalist for every image manipulation","Unified image manipulation localization with dynamic routing","Four tasks, no retraining: Omni-IML sets the bar"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00074,"raw_usage":{"total_tokens":3374,"prompt_tokens":1084,"completion_tokens":2290,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":700,"completion_tokens_details":{"reasoning_tokens":2207}},"tokens_in":700,"tokens_out":2290,"duration_ms":15335,"temperature":1.0,"reasoning_tokens":2207,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:51:24.135748+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the strongest per-task baselines and Omni-IML on all four tasks under identical training data, backbone weights, optimizer, input size, and iteration count, as Table 5 does but for the full baseline set; if any baseline matches or exceeds Omni-IML's average IoU of 0.714 in this controlled setup, the central claim of a generalist advantage is refuted rather than confirmed.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It supplies the frequency-vision fusion design and the DocTamper dataset, and its always-on frequency features motivate the Modal Gate."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It is the strongest natural-IML baseline and the source of the MIML training data, and Omni-IML's natural-image results are compared against it."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It is the face-IML specialist cited to show joint-training degradation, and it provides the face localization baseline Omni-IML must beat."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It is a natural-IML baseline with noise-filter and transformer features, and it is also re-trained in Table 5 as a joint-training comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It provides the region proposal network and box head that the Anomaly Enhancement module borrows for training-only box supervision."}],"review_version":1}