{"id":"959d46cd-bac0-4882-8b28-be7a7d09ccbd","arxiv_id":"2412.08197","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"SAFIRE uses point prompting and feature clustering to partition forged images into multiple source regions, and reports state-of-the-art results on both binary forgery localization and a new multi-source partitioning task.","lead":"SAFIRE is an image forensics method that finds forged regions by asking a SAM-based model to segment the source region around each of many points, rather than labeling pixels as simply forged or original. It is the first system reported to split a multiply edited image into three or more source regions, and it also reports higher binary forgery-localization scores than prior methods.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Multi-source evaluation may be compromised by overlap between SafireMS-Expert and the DPReview pretraining pool; no overlap analysis is provided.","rationale":"The reader's weakest assumption focuses on the connected-component training signal in Eq. (3), which is a real theoretical limitation: adjacent different-source regions are merged into one component, and disconnected same-source regions are ignored, so the training cannot express arbitrary source partitions. This concern is substantive and worth testing, but it is about the method's generality rather than the validity of the reported evidence. The overlap risk is more load-bearing because it directly threatens the empirical basis for the central new-task claim. If the 238 Expert images are not disjoint from the DPReview pretraining pool, Table 3 may overstate the model's true partitioning ability, and the 'first' claim becomes unsupported. The paper never states a split, and the shared use of DPReview makes overlap plausible. I partially agree with the reader because they did flag the missing overlap check in their rationale, but I consider it the primary concern rather than the connected-component issue. The verdict should remain CONDITIONAL: the paper can be accepted only if code and data are released and the overlap analysis shows disjointness; if overlaps are found, the multi-source contribution would need to be re-evaluated on a disjoint subset, potentially moving to UNVERDICTED or REJECT until new evidence is provided.","tokens_in":17061,"tokens_out":9658,"duration_ms":106307,"concrete_test":"Obtain the 238 SafireMS-Expert images and the 30,244 DPReview source images used for SafireMS-Auto pretraining. Compute a perceptual hash (e.g., pHash) for each Expert image and each DPReview image, and also for each source region used in Expert composites if available; flag exact or near-duplicate matches. Recompute the multi-source metrics (p mIoU, ARI) from Table 3 restricted to the subset of Expert images that have no hash match to any pretraining image. If no image overlaps, the concern does not land and the multi-source claim stands; if overlaps exist, compare restricted-subset scores to the reported values. A drop in ARI of more than, say, 5 points on the disjoint subset would indicate leakage contributed materially to the claimed capability.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that SAFIRE is the first method for multi-source source-region partitioning is supported almost entirely by Table 3 on the 238-image SafireMS-Expert dataset. The Appendix (SafireMS section) states that SafireMS-Auto pretraining collects 30,244 photographs from DPReview, and that SafireMS-Expert 'original images are sourced from DPReview and COCO 2017.' The paper never reports an overlap check between the 238 Expert images and the DPReview images used to build the pretraining corpus. If any of the 238 images (or their source layers) also appear among the 30,244 DPReview photographs, then the image encoder has been contrastively pretrained on exactly those photographs, and the multi-source scores in Table 3 (e.g., 2src p mIoU 90.3, ARI 80.7; 4src p mIoU 54.0) could reflect memorization or domain familiarity rather than a generalizable partitioning capability. Because the paper's novelty claim of being 'the first method capable of distinguishing each source' is supported only by this dataset, the absence of an explicit disjointness statement is a load-bearing gap. The reproducibility checklist confirms no statistical tests or distributional analysis, but that is secondary; the primary risk is unaddressed overlap.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces SAFIRE, a point-prompt-based image forgery localization method built on the Segment Anything Model (SAM). The method has three phases: region-to-region contrastive pretraining on a large synthetic dataset (SafireMS-Auto), training on binary forgery masks converted into point masks via Eq. (3), and inference by aggregating grid-prompt predictions through clustering of representative embedding features. The paper reports state-of-the-art binary IFL results on five public datasets and introduces a new multi-source partitioning task evaluated on a 238-image author-created dataset (SafireMS-Expert), claiming to be the first method that can distinguish multiple source regions in a forged image.","tokens_in":17332,"tokens_out":7461,"duration_ms":77202,"significance":"If the results hold, SAFIRE makes a useful contribution by reframing IFL as source-region partitioning and showing that a SAM-based point-prompting model can be trained on binary labels to produce multi-source partitions. The binary IFL results are strong and consistent across five external datasets, and the authors provide a clear pipeline with a released-code commitment and a large synthetic pretraining dataset. The main caveat is that the multi-source novelty claim rests on a single author-created dataset with a potential overlap with the pretraining data, and the entire empirical evaluation lacks error bars.","major_comments":[{"comment":"The pretraining corpus for the image encoder is built from 30,244 DPReview photographs, and the multi-source test set SafireMS-Expert uses original images from DPReview and COCO 2017. The paper does not report any check for overlap between the DPReview images used in SafireMS-Expert and those used to create SafireMS-Auto. Since the pretraining phase performs contrastive learning on these images, any overlap would give the encoder direct exposure to the test images, and the multi-source scores in Table 3 (e.g., 2-source pmIoU 90.3, ARI 80.7) could partly reflect memorization rather than generalizable partitioning. This is load-bearing because Table 3 is the sole quantitative support for the paper's central claim of being \"the first method capable of distinguishing each source.\" Please include an overlap analysis (exact-match or perceptual-hash) and, if overlap exists, retrain with a disjoint pretraining pool or exclude overlapping test images.","section":"Appendix, SafireMS-Auto and SafireMS-Expert"},{"comment":"All results are measured once, with no error bars or statistical significance tests, as stated in the appendix (\"All results of the proposed method and comparison techniques were measured once identically\") and confirmed by the reproducibility checklist entries. The paper's claims of \"superior performance\" and \"top performance\" in Tables 1 and 3 are therefore unsupported by any measure of variance. Given that the improvements over prior methods are large in most cases this is not disqualifying, but the absence of distributional information makes it impossible to assess the reliability of the smaller differences and the stability of the multi-source results. Please report multiple runs with means and standard deviations (or confidence intervals) and apply appropriate significance tests for the key comparisons.","section":"Appendix, Experimental Environment; Reproducibility Checklist"},{"comment":"The point-mask construction assumes that a source region is exactly a connected component of the binary forgery mask. If a single source region appears as two disconnected patches (e.g., two separate pasted objects from the same donor image, or a copy-move where the original and the duplicate belong to the same source), the training label in Eq. (3) cannot express the true source partition and will actively teach the model to split the source. The paper does not discuss this assumption or scope the method accordingly. Please clarify the definition of \"source region,\" state the connectedness assumption explicitly, and discuss or evaluate how the method behaves when source regions are disconnected. If the SafireMS-Expert benchmark contains only connected source regions, the claim \"capable of distinguishing each source\" should be limited to connected-source scenarios.","section":"Eq. (3), Point Mask Creation"}],"minor_comments":[{"comment":"The affiliation \"NA VER WEBTOON AI\" appears to be a typo for \"NAVER WEBTOON AI.\"","section":"Title page, affiliations"},{"comment":"Eq. (1) contains \"Inf oN CE\" with unusual spacing, and the description in Eq. (2) that \"· returns the average over all elements\" is unclear since no averaging symbol is visible in the equation.","section":"Eq. (1)-(2)"},{"comment":"The generalized pmIoU in Eq. (13) is not fully specified for the case where the predicted number of clusters N_pred is less than the ground-truth number of sources N; the text only explains the case N_pred > N.","section":"Appendix, Metrics for Multi-source Partitioning"},{"comment":"The checklist states that \"All novel datasets introduced in this paper are included in a data appendix\" with answer \"partial,\" but the appendix contains only statistics and examples, not the actual dataset; please clarify how this item is satisfied.","section":"Reproducibility Checklist"},{"comment":"Training hyperparameters such as learning rate, number of epochs, optimizer, batch size, and adapter dimensions are not reported in the main text or in the appendix excerpt; please include them for reproducibility.","section":"Experimental Settings, Implementation Details"}],"recommendation":"major_revision","confidential_remarks":"The main concern is the potential overlap between the pretraining pool and the multi-source test set; this should be addressed before publication. The single-run evaluation also weakens the empirical claims. I do not see evidence of intentional data leakage, but the manuscript must verify disjointness. The binary IFL results are strong and the multi-source idea is promising, so the issues are fixable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"SAFIRE is worth reading: it reframes IFL as source-region partitioning using SAM-style point prompting, and it is the first in the cited literature to do multi-source (3+) partitioning. The binary results on five external datasets are consistently ahead of prior work—F1 fixed 62.6 avg vs TruFor 56.3, F1 best 73.0 vs 68.7—and the ablation shows the prompting mechanism, not just the pretraining or loss, is what drives the gain. That is a real contribution.\n\nThe soft spots are all in the evaluation of the new capability, and they are addressable. First, Table 3 rests entirely on 238 author-created images, with no error bars or significance tests; every number is a single run. Second, and more important, the appendix says SafireMS-Expert originals come from DPReview and COCO 2017, while the pretraining corpus SafireMS-Auto is built from 30,244 DPReview photographs. There is no stated disjointness check between the 238 Expert images and the pretraining pool. If any Expert image (or its source layer) was contrastively pretrained on, the multi-source scores could reflect domain familiarity or even memorization rather than a generalizable ability. That is not a demonstrated fatal flaw—the binary results on external datasets are independent evidence that the method has real signal—but it does mean the headline 'first to partition 3+ sources' is not yet backed by a clean evaluation. Third, the training point-mask construction in Eq. (3) assumes connected components correspond to source regions; that is a heuristic the paper acknowledges implicitly but never tests, and it could hurt when a single source appears as multiple disconnected patches. That is a minor concern given the strong results, but worth noting.\n\nThis paper deserves a serious referee. The task formulation is new, the method is clearly described, and the binary evidence is strong. The multi-source evaluation needs an overlap analysis, multiple runs with variance, and ideally an independent third-party dataset. I would engage with it and cite it, but ask for those fixes before trusting Table 3.","headline":"A genuinely new task formulation with strong binary results, but the multi-source evaluation has an unaddressed train/test overlap question.","tokens_in":17844,"tokens_out":2860,"would_cite":true,"duration_ms":29449,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SAFIRE claims point-prompt source segmentation can partition a forged image into every originating source region, a first, while also setting new binary forgery localization results.","keywords":["image forensics","forgery localization","multi-source partitioning","point prompting","segment anything","contrastive learning","source segmentation"],"falsifier":"Take an image from one camera, paste the same donor patch into two separate non-adjacent locations, and present it to SAFIRE with binary ground truth; if the two pasted patches are placed in different clusters or only one is localized, the connected-component source assumption fails.","tokens_in":16841,"feed_emoji":"🔍","tokens_out":3026,"duration_ms":27984,"temperature":0.7,"pith_summary":"SAFIRE reframes image forgery localization as a source-partitioning problem: instead of labeling pixels forged or original, the model asks which originating source each point belongs to. The paper claims this is the first method that can distinguish each source when an image has been forged two or more times, producing a three-or-more-source partition, and that the same model also outperforms existing binary forgery localizers on five public datasets. A sympathetic reader would care because real forgeries are often composites of several donors, and knowing how many sources compose an image and where each one sits is richer forensic evidence than a single forged mask. The method trains on ordinary binary masks yet gains the multi-source ability.","feed_headline":"Point prompts split forged images into every source region","feed_subtitle":"SAFIRE partitions a manipulated image by originating source, then beats binary forgery localizers, too.","key_machinery":"The central object is the point prompt as a source reference. Given a point, the mask decoder is trained to segment exactly the connected source region containing it; the training signal is an area-adaptive source segmentation loss on point masks built from binary ground truths by labeling the prompted connected component 1, neighboring components 0, and ignoring the rest. Around this, region-to-region contrastive pretraining pulls image embeddings from the same source region together, and grid-based inference aggregates per-point masks by clustering representative features, with the most confident mask per cluster forming the final partition.","core_discovery":"The central claim is that point-prompt source segmentation is a better formulation of forgery localization than binary segmentation. SAFIRE feeds an image plus a grid of point prompts into a Segment Anything-style image encoder, mask decoder, and prompt encoder; each prompt returns the region sharing the source of that point, and the per-point prediction maps are clustered by their representative image embeddings to partition the image into source regions. Trained with a connected-component-derived point mask that labels only the prompted region and its immediate neighbors, the model learns label-agnostic source consistency rather than memorized forgery traces. The paper reports that this achieves state-of-the-art binary F1 scores and, uniquely, multi-source partitioning with up to four sources in its constructed expert dataset.","pith_inferences":["The connected-component premise implies that a source split into two disconnected patches, or two sources touching in a way that merges components, will be mis-partitioned; a direct stress test would forge an image by pasting the same donor twice in two places and check whether both patches land in one cluster.","The clustering stage suggests that the method's upper bound is set by how well embedding averages separate sources, so unusually small donor regions may be lost even when the mask decoder is correct.","Because partition quality depends on grid density, adaptive point placement near region boundaries could improve results relative to a fixed grid with the same compute."],"forward_implications":["Forensic analysis can answer 'how many sources and where' without retraining on multi-source labels.","Binary localization gains from the relational formulation, with the paper reporting the highest average F1 fixed and F1 best among compared methods.","Training on binary masks transfers to multi-source inference, so existing labeled datasets remain usable.","The approach opens a route to provenance filtering, since each partitioned region can be matched to candidate donor images."],"supporting_citations":[{"why":"Supplies the Segment Anything Model architecture and the point-prompting mechanism SAFIRE adapts.","marker":"(Kirillov et al. 2023)"},{"why":"Supplies the InfoNCE loss used in region-to-region contrastive pretraining.","marker":"(Oord, Li, and Vinyals 2018)"},{"why":"Supplies the TruFor baseline, the evaluation protocol, and the robustness-test setup.","marker":"(Guillaro et al. 2023)"},{"why":"Supplies the self-consistency idea and the permuted metric that SAFIRE generalizes to multi-source.","marker":"(Huh et al. 2018)"},{"why":"Supplies the IF-OSN baseline and motivates robustness to social-network post-processing.","marker":"(Wu et al. 2022)"},{"why":"Supplies the CAT-Net training setting and a JPEG-artifact forensic baseline.","marker":"(Kwon et al. 2022)"}],"fun_headline_variants":["Point prompts reveal every source in a forged image","SAFIRE splits forged images into source regions","Segment forged images by source with point prompts","Point-prompt forgery localization beats binary methods","Multi-source forgery segmentation via point prompting"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that in a binary forgery mask, each connected component corresponds to exactly one source region, so the training signal cannot express a source that appears as two disconnected patches or two sources fused into one component.","fun_headline_variants_meta":{"raw":{"variants":["Point prompts reveal every source in a forged image","SAFIRE splits forged images into source regions","Segment forged images by source with point prompts","Point-prompt forgery localization beats binary methods","Multi-source forgery segmentation via point prompting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000138,"raw_usage":{"total_tokens":1090,"prompt_tokens":817,"completion_tokens":273,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":433,"completion_tokens_details":{"reasoning_tokens":204}},"tokens_in":433,"tokens_out":273,"duration_ms":3251,"temperature":1.0,"reasoning_tokens":204,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:05:57.401403+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take an image from one camera, paste the same donor patch into two separate non-adjacent locations, and present it to SAFIRE with binary ground truth; if the two pasted patches are placed in different clusters or only one is localized, the connected-component source assumption fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the self-consistency idea and the permuted metric that SAFIRE generalizes to multi-source."}],"review_version":1}