REVIEW 4 major objections 5 minor 26 references
Weakly-supervised Localization of Manipulated Image Regions Using Multi-resolution Learned Features
T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Weakly-supervised localization of manipulated image regions is feasible without pixel-level labels, using fused activation and segmentation maps.
desk verdict A sensible segmentation-refinement idea is sunk by evaluation on a manually selected 40-image subset and a Bayesian fusion step that is never actually defined. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The method's engine is a three-part fusion. First, the image-wise detector WCBnet, built on ResNet-50 with Cross-block Attention Modules, weights and fuses hierarchical features; Grad-CAM over three CBAM configurations yields activation maps $A_i$ that are aggregated by geometric mean $A = (\prod_{i\in\{2,3,4\}} A_i)^{1/3}$. Second, pre-trained segmentation models (DeepLab, SAM, PSPnet) partition the image into masks $M_i$. Third, a distance-transform similarity $S(M_i,A)$ selects the best mask $M_{i^*}$, and a Bayesian expression $P(A|M_{i^*}) = P(M_{i^*}|A)P(A) / P(M_{i^*})$ is used to refine the heatmap. The distance transform ties the coarse activation to object boundaries; the Bayesian step is the formal mechanism by which the selected mask sharpens the map, though the paper does not specify how the probability terms are estimated.
What would settle it
Evaluate the described pipeline (with any fixed instantiation of the undefined probabilities) on all tampered CASIA2.0 images rather than the manually selected 40, and check whether the AUC/F1 advantage over ManTraNet survives; if it vanishes on the unselected set, the reported superiority is an artifact of hand-picking images where the activation maps already point at the tampered region.
Extended reading notes
Core claim
The central claim is that fusing activation maps from multiple receptive fields of an image-level manipulation classifier with a segmentation mask selected by boundary proximity, then applying Bayes' rule, produces accurate pixel-level localization despite having no pixel labels. The paper argues that the geometric mean of Grad-CAM maps from CBAM variants at scales 2, 3, and 4 captures both fine detail and global context, and that the pre-trained segmentation mask (especially DeepLab) supplies the precise object boundaries the activation maps lack. The reported improvement over both the ResNet backbone and the fully-supervised ManTraNet is presented as evidence that this weak-supervision route is viable.
Load-bearing premise
The load-bearing premise is that applying Bayes' rule to activation values and a segmentation mask genuinely improves localization, even though the probability terms in the fusion equation are never defined or estimated from data.
Editorial extensions
If this is right
- If the reported results hold, forensic tools can locate spliced or copy-moved regions in images that carry only an image-level 'manipulated' flag, removing the need for expensive pixel-wise annotation.
- The geometric-mean fusion of activation maps across receptive fields is a generally applicable ingredient: any image-level classifier with multi-branch feature weighting can be turned into a coarse localizer this way.
- Combining a coarse activation map with a pre-trained segmentation mask yields better localization than either source alone, with DeepLab's larger regions giving the best trade-off in this paper.
- A fully-supervised model such as ManTraNet is not an insurmountable upper bound in localization quality when the evidence is weak supervision plus generic segmentation.
Reading between the lines
- One extension the authors do not pursue is a full-dataset evaluation: if the AUC/F1 advantage over ManTraNet persists when the 40-image hand-picked subset is replaced by all tampered CASIA2.0 images, the method's practical reach is much wider than the paper demonstrates.
- The Bayesian fusion in equation (7) is formally an identity until the probabilities are given operational definitions; specifying them empirically (e.g., activation value histograms over training images) would turn the refinement step into a concrete, reproducible algorithm, and its benefit could be measured by ablation.
- The distance-transform similarity could serve as a general selection rule for choosing which segmentation mask to trust for a given image, a component that other weakly-supervised localization pipelines might reuse.
- If this recipe transfers to video or deepfake localization, it would inherit the same advantage of needing only clip-level labels.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a weakly-supervised method for localizing manipulated image regions without pixel-level labels. The pipeline uses the authors' image-level detector WCBnet to compute Grad-CAM activation maps at multiple feature scales, aggregates them by geometric mean, and then combines the resulting coarse map with masks from pre-trained segmentation models (DeepLab, SAM, PSPnet) through a step described as Bayesian inference. The method is evaluated on 40 manually selected images from CASIA2.0; Table 1 reports an AUC of 0.704 and an F1-score of 0.682 for WCBnet combined with DeepLab, which the authors compare favorably against the fully-supervised ManTraNet (AUC 0.653, F1 0.238). The paper's central claim is that accurate manipulation localization is feasible without pixel-level annotations.
Significance. The motivation is relevant and the direction of combining class activation maps with off-the-shelf segmentation models is a sensible one for weakly-supervised localization. The paper also provides qualitative evidence that multi-scale activation maps can improve localization over a single backbone, and it honestly acknowledges the limitation of small-region manipulation. However, the quantitative evidence is not currently convincing: the 40-image test set is selected on the outcome, the Bayesian refinement is not an operational algorithm, and the conclusion's stated F1 improvement does not match the numbers in Table 1. These issues directly affect the paper's central feasibility claim. If the authors replace the biased evaluation with a random or full-dataset protocol and specify an implementable refinement step, the approach could become publishable; as it stands, the empirical contribution is unverified and the method is not reproducible.
major comments (4)
- [Section 3.5, Table 1] The quantitative evaluation is invalid for the paper's central claim because Section 3.5 states that the 40 test images were "manually selected ... for which the activation maps approximately highlight the target region." This is selection on the outcome: only images whose coarse activation maps already localize the manipulation enter the test set, so the reported AUC of 0.704 and F1 of 0.682 are not estimates for any definable population. The comparison to ManTraNet on the same cherry-picked subset is therefore not a meaningful benchmark; the authors should report results on the full CASIA2.0 test split or a random subset, and if the selection is retained, it should be presented as an illustrative case study rather than as evidence for the feasibility claim.
- [Section 2.4, Eq. (7)] Equation (7) is Bayes' rule, not an algorithm. The quantities P(A), P(Mi*) and P(Mi*|A) are never defined in terms of the available data, never estimated, and no procedure is given for computing the "enhanced activation heatmap" A* from Eq. (7). Without an operational definition of these probabilities, the refinement step cannot be implemented, reproduced, or validated, and the claimed Bayesian inference is a mathematical identity that does no computational work. The authors need to specify how these probabilities are computed from A and Mi*, or replace Eq. (7) with a concrete algorithmic step.
- [Section 2.4, Eqs. (5)-(6)] Because the selected mask is the one that maximizes the similarity to A (Eq. (6)) and P(Mi*|A) is never estimated, the final output A* is essentially a reweighting or smoothing of A by the most similar segmentation mask. The paper does not demonstrate that the segmentation maps contribute information beyond a monotone transform of the activation map; a simple test would be to compare A* against the original A on a random set using the same metrics. As written, the combination step risks being circular and does not substantiate the claim of adding independent region information.
- [Conclusion and Table 1] The conclusion's claim of a "5% to 11%" F1 improvement over the backbone model is not supported by Table 1. The table shows WCBnet+DeepLab at 0.682 F1 versus ResNet+DeepLab at 0.296 (a much larger relative improvement), WCBnet+SAM at 0.310 versus ResNet+SAM at 0.257, and WCBnet+PSPnet at 0.365 versus ResNet+PSPnet at 0.259; no column in the table reports the backbone model alone without segmentation, and no value in the table falls in the claimed 5-11% range. The authors should specify the baseline and the computation used for this claim and report the corresponding numbers.
minor comments (5)
- [Section 3.2] The sentence "To claim, we only use the segmentation masks from DeepLab although it could be a semantic segmentation model" is incomplete and inconsistent with Table 1, which lists SAM and PSPnet; please clarify which segmentation models are used for quantitative evaluation and which are used only for visualization.
- [Section 3.1] The paper reports training on approximately 1,800 images per class but does not state how the 40 test images are split from or related to the training set; please describe the validation/test protocol and the manipulation types (splicing versus copy-move) in the 40-image subset.
- [Equation (5)] The notation D(Mi) is used without a precise definition of the distance transform (for example, Euclidean distance to the mask boundary) and its normalization; please define it explicitly so that Eq. (5) is reproducible.
- [Figure 2] The caption uses "The features maps" (typo) and the symbols WCBnet2, WCBnet3, WCBnet4, and WCBnetm are not defined in the caption; please add a clear legend or explanation.
- [Section 3.5] The paper reports AUC and F1 with a "fixed threshold" but never states the threshold value or how it was chosen; this should be specified for reproducibility.
Circularity Check
The headline AUC/F1 numbers come from a 40-image subset selected because activation maps already highlight the target region, and Eq. (7)'s 'Bayesian inference' is Bayes' rule with undefined terms; the central feasibility claim is therefore only partially supported.
-
self definitional
[Section 3.5, Table 1 evaluation protocol]
"For this evaluation, we manually selected 40 images that are correctly identified as manipulated and for which the activation maps approximately highlight the target region."
The test set is defined by the outcome it is meant to measure: only images whose activation maps already approximately highlight the target region are scored. Table 1's AUC/F1 and the 'outperforms fully-supervised ManTraNet' comparison therefore describe an outcome-filtered subset, not a defined population. The feasibility claim is built into the selection criterion, so the headline numbers are not independent evidence.
-
self definitional
[Section 2.4, Eq. (7)]
"Here, P(A) represents the manipulation probability of the activation map A, P(Mi∗) is the prior probability of the mask Mi∗, and P(A | Mi∗) denotes the conditional probability of A given the mask Mi∗. The enhanced activation heatmap A∗, is computed as follows: P(A | Mi∗) =P(Mi∗ | A) · P(A) P(Mi∗) . (7)"
None of P(A), P(Mi*), or P(Mi*|A) is defined or estimated from data, so Eq. (7) is Bayes' theorem written as an identity; any A* can satisfy it by choosing P(Mi*|A) appropriately. Since Mi* is selected as the segmentation mask maximizing S(Mi,A), the 'refinement' only picks the mask most similar to the activation map and then rewrites that map through an undefined probability identity. The enhancement step therefore adds no independently derived information; it reduces to a smoothing of the input activation map.
full rationale
Most of the pipeline is concrete: ResNet50 plus CBAM feature weighting, multi-view Grad-CAMs, geometric mean, and similarity scoring against pre-trained segmentation masks. Those steps are not circular in themselves. The circularity enters at two points. First, the only quantitative validation of the central feasibility claim is Table 1, whose 40-image test set is selected because activation maps 'approximately highlight the target region'; this is selection on the dependent variable, so the reported AUC/F1 and the ManTraNet comparison do not estimate performance on any defined population. Second, the claimed Bayesian refinement is not an operational algorithm: Eq. (7) is Bayes' rule with unspecified, unestimated probabilities, and the mask used is the one most similar to the activation map, so the output is determined by the input activation map. The self-citation to the authors' WCBnet is real but not the load-bearing circularity; WCBnet is used as a trained feature extractor rather than as evidence for the fusion step. Overall, the feasibility claim is partially circular because its headline evidence reduces to the selection criterion and the refinement reduces to an identity.
Assumptions & free parameters
free parameters (1)
- Activation scale set / n=3 =
{2, 3, 4}
assumptions (4)
- domain assumption Pre-trained segmentation models segment natural objects that correspond to manipulated regions
- domain assumption Activation maps from image-level classifier highlight manipulated regions
- ad hoc to paper Bayes' rule applies as an enhancement without defined probability distributions
- domain assumption Distance-transform similarity scores correctly rank segmentation masks
Cite this review
Pith. "Pith review of Weakly-supervised Localization of Manipulated Image Regions Using Multi-resolution Learned Features." pith.science (2026). https://pith.science/paper/QCLZKLX6
@misc{pith2026250523586,
author = {Pith},
title = {Pith review of: Weakly-supervised Localization of Manipulated Image Regions Using Multi-resolution Learned Features},
year = {2026},
howpublished = {\url{https://pith.science/paper/QCLZKLX6}},
note = {Machine review of arXiv:2505.23586}
}
read the original abstract
The explosive growth of digital images and the widespread availability of image editing tools have made image manipulation detection an increasingly critical challenge. Current deep learning-based manipulation detection methods excel in achieving high image-level classification accuracy, they often fall short in terms of interpretability and localization of manipulated regions. Additionally, the absence of pixel-wise annotations in real-world scenarios limits the existing fully-supervised manipulation localization techniques. To address these challenges, we propose a novel weakly-supervised approach that integrates activation maps generated by image-level manipulation detection networks with segmentation maps from pre-trained models. Specifically, we build on our previous image-level work named WCBnet to produce multi-view feature maps which are subsequently fused for coarse localization. These coarse maps are then refined using detailed segmented regional information provided by pre-trained segmentation models (such as DeepLab, SegmentAnything and PSPnet), with Bayesian inference employed to enhance the manipulation localization. Experimental results demonstrate the effectiveness of our approach, highlighting the feasibility to localize image manipulations without relying on pixel-level labels.
Figures
Reference graph
Works this paper leans on
-
[1]
Detecting deepfake images using deep learning techniques and explainable AI methods
Wahidul Hasan Abir, Faria Rahman Khanam, Kazi Nabiul Alam, Myriam Hadjouni, Hela Elmannai, Sami Bourouis, Rajesh Dey, and Mohammad Monirujjaman Khan. Detecting deepfake images using deep learning techniques and explainable AI methods. Intelligent Automation & Soft Computing, 35(2):2151–2169, 2023
work page 2023
-
[2]
Copy move source-target disam- biguation through multi-branch CNNs
Mauro Barni, Quoc-Tin Phan, and Benedetta Tondi. Copy move source-target disam- biguation through multi-branch CNNs. IEEE Transactions on Information Forensics and Security, 16:1825–1840, 2021. doi: 10.1109/TIFS.2020.3045903
arXiv 2021
-
[3]
Convolutional neural network initialization approaches for image manipulation detection
Ivan Castillo Camacho and Kai Wang. Convolutional neural network initialization approaches for image manipulation detection. Digital Signal Processing, 122:103376, 2022
work page 2022
-
[4]
H. R. Chennamma and B. Madhushree. A comprehensive survey on image authentica- tion for tamper detection with localization. Multimedia Tools and Applications, 82(2): 1873–1904, 01 2023
work page 1904
-
[5]
CASIA image tampering detection evaluation database
Jing Dong, Wei Wang, and Tieniu Tan. CASIA image tampering detection evaluation database. In 2013 IEEE China Summit and International Conference on Signal and Information Processing, pages 422–426, 2013. doi: 10.1109/ChinaSIP.2013.6625374
arXiv 2013
-
[6]
Teddy Surya Gunawan, Siti Amalina Mohammad Hanafiah, Mira Kartiwi, Nanang Is- mail, Nor Farahidah Za’bah, and Anis Nurashikin Nordin. Development of photo forensics algorithm by detecting photoshop manipulation using error level analysis. Indonesian Journal of Electrical Engineering and Computer Science , 7(1):131–137, 2017
work page 2017
-
[7]
Hierarchical fine-grained image forgery detection and localization
Xiao Guo, Xiaohong Liu, Zhiyuan Ren, Steven Grosz, Iacopo Masi, and Xiaoming Liu. Hierarchical fine-grained image forgery detection and localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3155–3165, June 2023
work page 2023
-
[8]
Image Segmentation Keras : Implementation of Segnet, FCN, UNet, PSPNet and other models in Keras
Divam Gupta. Image segmentation Keras: Implementation of Segnet, FCN, Unet, PSPnet and other models in Keras. arXiv preprint arXiv:2307.13215, 2023
work page Pith review arXiv 2023
Show all 26 references
-
[9]
Automatic landslide segmentation using a combination of grad-CAM visualization and K-means clustering techniques
Kemal Hacıefendio ˘glu, Süleyman Adanur, and Gökhan Demir. Automatic landslide segmentation using a combination of grad-CAM visualization and K-means clustering techniques. Iranian Journal of Science and Technology, Transactions of Civil Engi- neering, 48(2):943–959, 2024
2024
-
[10]
An improvised CNN model for fake image detec- tion
Yasir Hamid, Sanaa Elyassami, Yonis Gulzar, Veeran Ranganathan Balasaraswathi, Tetiana Habuza, and Sharyar Wani. An improvised CNN model for fake image detec- tion. International Journal of Information Technology, 15(1):5–15, 2023
2023
-
[11]
Seg-XRes-CAM: Explaining spatially local regions in image segmentation
Syed Nouman Hasany, Caroline Petitjean, and Fabrice Mériaudeau. Seg-XRes-CAM: Explaining spatially local regions in image segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Work- shops, pages 3733–3738, June 2023. W ANG, A...
2023
-
[12]
Berg, Wan-Yen Lo, Piotr Dol- lár, and Ross Girshick
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dol- lár, and Ross Girshick. Segment anything. arXiv:2304.02643, 2023
2023 arXiv
-
[13]
Weakly supervised semantic segmentation via adversarial learning of classifier and reconstructor
Hyeokjun Kweon, Sung-Hoon Yoon, and Kuk-Jin Yoon. Weakly supervised semantic segmentation via adversarial learning of classifier and reconstructor. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11329– 11339, 2023
2023
-
[14]
Grad-CAM: visual explanations from deep networks via gradient-based localization
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-CAM: visual explanations from deep networks via gradient-based localization. International journal of computer vision , 128:336– 359, 2020
2020
-
[15]
Recent advances in digital image manipulation de- tection techniques: A brief review
Rahul Thakur and Rajesh Rohilla. Recent advances in digital image manipulation de- tection techniques: A brief review. Forensic science international, 312:110311, 2020
2020
-
[16]
A detailed analysis of image and video forgery detection techniques
Shobhit Tyagi and Divakar Yadav. A detailed analysis of image and video forgery detection techniques. The Visual Computer, 39(3):813–833, 2023
2023
-
[17]
Using XAI for deep learning-based image manipulation detection with shapley additive explanation
Savita Walia, Krishan Kumar, Saurabh Agarwal, and Hyunsung Kim. Using XAI for deep learning-based image manipulation detection with shapley additive explanation. Symmetry, 14(8):1611, 2022
2022
-
[18]
WCBnet: Weighted convolutional block mod- elling of signed-value error levels for image-wise copy-move and splicing detection
Ziyong Wang and Charith Abhayaratne. WCBnet: Weighted convolutional block mod- elling of signed-value error levels for image-wise copy-move and splicing detection. In 2023 IEEE 25th International Workshop on Multimedia Signal Processing (MMSP), pages 1–6. IEEE, 2023
2023
-
[19]
Collins, Yukun Zhu, Liangzhe Yuan, Dahun Kim, Qihang Yu, Daniel Cremers, Laura Leal-Taixe, Alan L
Mark Weber, Huiyu Wang, Siyuan Qiao, Jun Xie, Maxwell D. Collins, Yukun Zhu, Liangzhe Yuan, Dahun Kim, Qihang Yu, Daniel Cremers, Laura Leal-Taixe, Alan L. Yuille, Florian Schroff, Hartwig Adam, and Liang-Chieh Chen. DeepLab2: A Tensor- Flow Library for Deep Labeling. arXiv: 2...
2021 arXiv
-
[20]
ManTra-Net: Manipulation tracing network for detection and localization of image forgeries with anomalous fea- tures
Yue Wu, Wael AbdAlmageed, and Premkumar Natarajan. ManTra-Net: Manipulation tracing network for detection and localization of image forgeries with anomalous fea- tures. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9543–9552, 2019
2019
-
[21]
Image forgery detection: a survey of recent deep-learning approaches
Marcello Zanardelli, Fabrizio Guerrini, Riccardo Leonardi, and Nicola Adami. Image forgery detection: a survey of recent deep-learning approaches. Multimedia Tools and Applications, 82(12):17521–17566, 2023
2023
-
[22]
Semi- supervised image manipulation localization with residual enhancement.Expert Systems with Applications, 252:124171, 2024
Qiang Zeng, Hongxia Wang, Yang Zhou, Rui Zhang, and Sijiang Meng. Semi- supervised image manipulation localization with residual enhancement.Expert Systems with Applications, 252:124171, 2024
2024
-
[23]
Towards generic image manipulation detection with weakly-supervised self-consistency learning
Yuanhao Zhai, Tianyu Luan, David Doermann, and Junsong Yuan. Towards generic image manipulation detection with weakly-supervised self-consistency learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 22390–22400, 2023. 12 W ANG, ABHA Y AR...
2023
-
[24]
A survey of semi-and weakly supervised semantic segmentation of images
Man Zhang, Yong Zhou, Jiaqi Zhao, Yiyun Man, Bing Liu, and Rui Yao. A survey of semi-and weakly supervised semantic segmentation of images. Artificial Intelligence Review, 53:4259–4288, 2020
2020
-
[25]
Exploring weakly-supervised image manipulation localization with tampering edge-based class activation map
Yang Zhou, Hongxia Wang, Qiang Zeng, Rui Zhang, and Sijiang Meng. Exploring weakly-supervised image manipulation localization with tampering edge-based class activation map. Expert Systems with Applications, 249:123501, 2024
2024
-
[26]
Weakly-supervised deepfake localization in diffusion-generated images
Dragos,-Constantin T, ânt,aru, Elisabeta Oneat,˘a, and Dan Oneat ,˘a. Weakly-supervised deepfake localization in diffusion-generated images. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 6258–6268, 2024
2024
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.