REVIEW 5 major objections 5 minor 42 references
Modelship Attribution: Tracing Multi-Stage Manipulations Across Generative Models
T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read This paper introduces Modelship Attribution and a transformer, MAT, that reconstructs the sequence of generative models behind a multiply edited face image.
desk verdict New task and dataset for multi-stage model attribution, but the evaluation is self-contained and missing key baselines; the input-assumption concern is real but not fatal. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing device is the Modelship Attribution Transformer (MAT), which converts the editing history into a set of slice tasks. Its Sequence Slice module takes the output of stage $t$ and the image from an earlier stage $t-n$ as an input pair, so the network learns the transition rather than the whole chain at once. Self Spatial Extraction passes each slice through CNN and FPN backbones and a Transformer encoder to produce spatially rich features. Modelship Trace Modeling then runs cross-attention between the two feature sets, combines them with tokenized operation sequences (with SOS, EOS, and No-Manipulation tokens), and injects a Gaussian-weighted Spatial Trace Map built from the normalized difference of the two images as an additive log-prior in the attention logits. The module decodes the model sequence autoregressively, which lets it emit variable-length Modelship Tokens representing the models and their order.
What would settle it
Feed MAT only the final image from a four-stage edit, with no intermediate stage, and measure sequence reconstruction; if accuracy falls to near chance, the central claim does not extend to attribution from the final image alone, and the 87.57% Ada-ACC result is specific to settings with edit-history snapshots.
Extended reading notes
Core claim
The central claim is that the evolution of a multiply edited image can be traced by learning the editing paradigms of the generative models involved, even when later edits obscure the fingerprints of earlier ones. The paper supports this by constructing a dataset in which four facial regions are edited in random order by StyleMapGAN, DiffSwap, and FacePartsSwap, and by showing that per-model spectra differ even when the specified editing regions are identical, while StyleMapGAN's full-image regeneration produces near-identical spectra across stages that cover prior traces. MAT therefore predicts the manipulation sequence (for example, StyleMapGAN then DiffSwap then StyleMapGAN then FacePartsSwap) directly, treating attribution as autoregressive sequence generation over model tokens rather than single-label classification. The reported results are 87.57% Ada-ACC and 76.01% Strict-Match on ResNet-50, with ablation experiments indicating that the sequence-slice input pairs, the spatial trace map, and the autoregressive decoding each contribute to that performance.
Load-bearing premise
The method assumes access to at least one earlier intermediate edited version of the image alongside the current stage, so a query that offers only the final image cannot form the input pairs on which the reported accuracy depends.
Editorial extensions
If this is right
- Attribution becomes a sequence-reconstruction problem, so the output of a forensic tool can be a full editing history rather than a single source label.
- Intermediate editing stages are treated as signal, not noise, which means retaining version history in editing pipelines could make provenance tracing substantially more accurate.
- A benchmark for multi-stage deepfake attribution now exists with a fixed protocol: three architectures, four face regions, and 19 labeled sequences, giving subsequent work a common testbed.
- Per-model editing patterns can survive full-image regeneration edits, suggesting that later manipulations do not necessarily destroy all evidence of earlier ones.
- Autoregressive decoding with padding tokens yields both a graded accuracy measure (Ada-ACC) and an exact-match measure, which is useful when partial recovery of the sequence still matters.
Reading between the lines
- If the paradigm-learning idea transfers, the same slice-and-autoregress design could be applied to mixed toolchains beyond face swaps, such as inpainting, super-resolution, or text-to-image revision, where each tool leaves a different operational trace.
- The method's reliance on paired inputs ($I_t$ and $I_{t-n}$) implies that a practical deployment would need access to edit-history snapshots, for example from versioned editing software or platform logs; forensic analysis of a lone final image is a different, harder problem that the current results do not cover.
- A straightforward stress test of the generalization claim is to train on a subset of model combinations and test on held-out combinations or a fourth unseen generator; strong performance there would confirm paradigm learning rather than memorization of the 19 benchmark labels.
- The Gaussian Spatial Trace Map could double as a localization output: the same weights that guide attention identify where each edit occurred, potentially giving free region-level explanations alongside the sequence prediction.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces a new task, Modelship Attribution, which aims to identify the sequence of generative models that manipulated an image over multiple editing stages. The authors construct a dataset by sequentially applying StyleMapGAN, DiffSwap, and FacePartsSwap to four facial regions in randomized order, and propose the Modelship Attribution Transformer (MAT), comprising Sequence Slice, Self Spatial Extraction, and Modelship Trace Modeling modules. On a held-out split of this dataset, MAT is reported to achieve 87.57% Ada-ACC and 76.01% Strict-Match with ResNet-50, outperforming the provided baselines. Extensive ablations show that each component contributes to the reported performance.
Significance. If the claimed results hold, this is a useful step beyond single-stage model attribution: it provides a new benchmark for multi-stage manipulation tracing and demonstrates that sequence-level attribution is feasible under a closed set of three models. Strengths of the work include the construction of a new dataset with randomized operation orders and regions, clearly defined evaluation metrics, and a component-level ablation study. The task is framed as supervised learning with ground-truth labels, so there is no circularity in the main empirical claim. However, the significance is currently limited by a serious mismatch between the motivating real-world scenario and the method's input requirements, by unresolved inconsistencies in dataset statistics and label space, and by the absence of comparisons to the sequential deepfake detection methods cited in the related work.
major comments (5)
- [§3.2–3.3, Figure 5] The method's input protocol is inconsistent with the motivating scenario. Section 3.3 states that the model selects the image at stage t and the image at stage t−n as inputs, and the evaluation in Section 4 uses all intermediate images from the constructed dataset. The introduction and Figure 1(a) motivate the task by real-world multi-stage manipulation on social media, where a detector would typically receive only the final shared image. Under that protocol, the required input pairs (I_t, I_{t-n}) are unavailable, so the reported 87.57% Ada-ACC and 76.01% Strict-Match do not substantiate the advertised capability of tracing the complete evolution of an image from the final output alone. Please clarify the intended query protocol; if only the final image is available, the architecture and evaluation need to be adapted (e.g., a final-image-only variant) before this claim can be supported.
- [§3.1, §4.1, Abstract] The dataset statistics are internally inconsistent. The abstract and §3.1 report 83,700 images (16,740 images × 5), but §3.1 also states "16,740 face part-swapping images across four stages," which is incompatible with 16,740 five-image sets, and the introduction reports 17,440 image sets. Moreover, with four stages, three models, and the requirement that every model appear at least once, there are 36 possible model sequences, not the "19 permutations" stated in §4.1. Please correct these numbers and specify the exact label space used for both dataset construction and evaluation.
- [§2.2 vs §4.2, Table 1] The related-work section identifies sequential deepfake detection methods (Shao et al., MMNet, and the Spectral Transformer) as the closest line of work, but Table 1 does not compare against any of them. The selected baselines are a multi-label classifier, a Photoshop-detection method (DRN), an object detector (DETR), and a high-frequency two-stream detector; none is designed for sequential manipulation. Consequently, the claim that MAT outperforms existing sequential approaches is not established. Please adapt and evaluate at least one or two sequential deepfake detectors, or explain concretely why they are not applicable to the modelship-attribution setting.
- [§4.2, Results paragraph] The Strict-Match comparison is self-contradictory. The text states that MC-Cls, DRN, DETR, and Two-Stream "do not have Strict-Match accuracy scores," yet two sentences later it reports that MAT's Strict-Match accuracy "exceeds other methods by up to 16.73%." Please report the actual Strict-Match values for the baselines, or remove the comparative claim.
- [Table 1, Table 2] No measure of variability is reported. Single-run accuracies without error bars or significance tests make it difficult to assess whether the margins over baselines are stable. Please report the mean and standard deviation over at least three seeds, or a paired significance test, for the main results and the ablations.
minor comments (5)
- [Eq. (Spatial Trace Map), §4.1] In the Spatial Trace Map equation, λ is introduced as a hyperparameter controlling the Gaussian spread, but §4.1 describes λ = 4 as "balancing the training objectives." Please clarify the role of λ and whether it is tuned or fixed.
- [§3.1, Figure 3 caption] Figure 3 is described as showing "two examples" of the generation process, but the caption and the surrounding text describe the full dataset generation with four stages and randomized regions; please align the caption with the content.
- [§4.2, Table 1] The text refers to "ML-Cls" in one place while the table and earlier text use "MC-Cls"; please standardize the naming.
- [§3.1, quality filtering] The CLIB-FIQA quality threshold is mentioned but no threshold value or number of discarded samples is reported. Please provide these details for reproducibility.
- [§1, §5] The paper does not explicitly state the closed-set assumption that the manipulator must be one of the three known models. This is a significant practical limitation and should be acknowledged in the introduction and conclusion.
Circularity Check
No significant circularity: MAT's reported accuracy is an empirical hold-out result on randomly assigned stage labels, not a quantity forced by construction; the intermediate-image input caveat is an evaluation-validity issue, not a circular reduction.
full rationale
The paper's derivation chain is a standard supervised-learning pipeline. Ground-truth labels (the 19 orderings of StyleMapGAN, DiffSwap, and FacePartsSwap over four randomly shuffled facial regions) are assigned stochastically during dataset construction, with Section 3.1 stating that 'the manipulation order of the four facial regions—nose, eyes, mouth, and eyebrows—was randomly shuffled for each image. Each of these four stages was assigned to a randomly selected deepfake model.' The target sequence is therefore not derivable from the input images by any closed-form rule, and the reported 87.57% Ada-ACC and 76.01% Strict-Match are genuinely empirical held-out results rather than quantities determined by construction. The reference list contains no prior work by the present authors, so there is no load-bearing self-citation chain and no invented uniqueness theorem invoked to force a design choice. The 'Modelship' task is a genuine extension of the sequential-deepfake-detection literature acknowledged in Section 2.2 rather than a pure renaming of it. The one caveat that borders on circularity is that the method and its evaluation consume intermediate edited images as inputs (Section 3.3: 'we select the output image I_t at stage t and the image I_{t-n} from stage t-n as the input pair'), so the test protocol partially supplies the very stage history whose reconstruction is the advertised contribution; a real-world query containing only the final image lies outside the measured regime. This is an evaluation-validity and external-consistency limitation, however, not a reduction of the output to the input by construction, because the pair (I_{t-n}, I_t) does not analytically determine the labels and the mapping must be learned. Minor template artifacts (a 'Trovato et al.' running head and 2018 copyright on a 2025 arXiv paper) indicate an unpolished resubmission but have no bearing on circularity. The score is 1 rather than 0 only to register that the benchmark, metrics, and protocol are entirely author-defined with no external anchor; the central claim still rests on independent, empirically measured content.
Assumptions & free parameters
free parameters (6)
- Spatial Trace Map Gaussian spread lambda =
4
- Transformer depth =
2 encoder layers, 2 decoder layers, 4 attention heads
- Learning rates =
1e-3 transformer, 1e-4 CNN
- Training epochs =
150
- Batch size =
64
- CLIB-FIQA quality threshold =
not specified
assumptions (6)
- domain assumption Intermediate-variant availability: inputs I_t and I_{t-n} are available for every stage t at test time.
- domain assumption Per-model editing paradigm consistency: each model leaves a stable, learnable pattern when editing a facial region, and that pattern survives later edits well enough to be recognized.
- ad hoc to paper Closed-set model universe: the only possible manipulators are StyleMapGAN, DiffSwap, and FacePartsSwap.
- domain assumption Ground-truth completeness: every edit in the dataset is one of the four facial region swaps and the annotated model order is correct and complete.
- domain assumption No external post-processing: the images are used directly at 256x256 with no additional compression, resizing, or platform artifacts beyond the described pipeline.
- domain assumption Quality filtering is harmless: CLIB-FIQA filtering removes only distorted outliers and does not bias the attribution task.
Cite this review
Pith. "Pith review of Modelship Attribution: Tracing Multi-Stage Manipulations Across Generative Models." pith.science (2026). https://pith.science/paper/6YGDLDYI
@misc{pith2026250602405,
author = {Pith},
title = {Pith review of: Modelship Attribution: Tracing Multi-Stage Manipulations Across Generative Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/6YGDLDYI}},
note = {Machine review of arXiv:2506.02405}
}
read the original abstract
As generative techniques become increasingly accessible, authentic visuals are frequently subjected to iterative alterations by various individuals employing a variety of tools. Currently, to avoid misinformation and ensure accountability, a lot of research on detection and attribution is emerging. Although these methods demonstrate promise in single-stage manipulation scenarios, they fall short when addressing complex real-world iterative manipulation. In this paper, we are the first, to the best of our knowledge, to systematically model this real-world challenge and introduce a novel method to solve it. We define a task called "Modelship Attribution", which aims to trace the evolution of manipulated images by identifying the generative models involved and reconstructing the sequence of edits they performed. To realistically simulate this scenario, we utilize three generative models, StyleMapGAN, DiffSwap, and FacePartsSwap, that sequentially modify distinct regions of the same image. This process leads to the creation of the first modelship dataset, comprising 83,700 images (16,740 images*5). Given that later edits often overwrite the fingerprints of earlier models, the focus shifts from extracting blended fingerprints to characterizing each model's distinctive editing patterns. To tackle this challenge, we introduce the modelship attribution transformer (MAT), a purpose-built framework designed to effectively recognize and attribute the contributions of various models within complex, multi-stage manipulation workflows. Through extensive experiments and comparative analysis with other related methods, our results, including comprehensive ablation studies, demonstrate that the proposed approach is a highly effective solution for modelship attribution.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Akin Caliskan, Berkay Kicanaoglu, and Hyeongwoo Kim. 2025. PAV: Personalized Head Avatar from Unstructured Video Collection. In European Conference on Computer Vision. Springer, 109–125
work page 2025
-
[2]
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexan- der Kirillov, and Sergey Zagoruyko. 2020. End-to-end object detection with transformers. In European conference on computer vision . Springer, 213–229
2020
-
[3]
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition . Ieee, 248–255
2009
-
[4]
Bernhard Egger, William A. P. Smith, Ayush Tewari, Stefanie Wuhrer, Michael Zollhoefer, Thabo Beeler, Florian Bernard, Timo Bolkart, Adam Kortylewski, Sami Romdhani, Christian Theobalt, Volker Blanz, and Thomas Vetter. 2020. 3D Morphable Face Models – Past, Present and Future. arXiv:1909.01815 [cs.CV] https://arxiv.org/abs/1909.01815
work page Pith review arXiv 2020
-
[5]
Claudio Ferrari, Matteo Serpentoni, Stefano Berretti, and Alberto Del Bimbo
-
[6]
Joel Frank, Thorsten Eisenhofer, Lea Schönherr, Asja Fischer, Dorothea Kolossa, and Thorsten Holz. 2020. Leveraging Frequency Analysis for Deep Fake Image Recognition. arXiv:2003.08685 [cs.CV] https://arxiv.org/abs/2003.08685
arXiv 2020
-
[7]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2020. Generative adversarial networks. Commun. ACM 63, 11 (2020), 139–144
2020
-
[8]
Luca Guarnera, Oliver Giudice, and Sebastiano Battiato. 2024. Level up the deepfake detection: a method to effectively discriminate images generated by gan architectures and diffusion models. In Intelligent Systems Conference. Springer, 615–625
2024
Show all 42 references
-
[9]
Junfeng Guo, Yiming Li, Lixu Wang, Shu-Tao Xia, Heng Huang, Cong Liu, and Bo Li. 2024. Domain watermark: Effective and harmless dataset copyright protection is closed at hand. Advances in Neural Information Processing Systems 36 (2024)
2024
-
[10]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition . 770–778
2016
-
[11]
Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. Advances in neural information processing systems 33 (2020), 6840–6851
2020
-
[12]
Cheng-Yao Hong, Yen-Chi Hsu, and Tyng-Luh Liu. 2024. Contrastive Learning for DeepFake Classification and Localization via Multi-Label Ranking. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 17627– 17637
2024
-
[13]
Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. 2020. Training Generative Adversarial Networks with Limited Data. In Advances in Neural Information Processing Systems
2020
-
[14]
Tero Karras, Samuli Laine, and Timo Aila. 2019. A Style-Based Generator Architecture for Generative Adversarial Networks. arXiv:1812.04948 [cs.NE] https://arxiv.org/abs/1812.04948
2019 arXiv
-
[15]
Hyunsu Kim, Yunjey Choi, Junho Kim, Sungjoo Yoo, and Youngjung Uh. 2021. Exploiting spatial dimensions of latent in gan for real-time image editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 852–861
2021
-
[16]
Diederik P Kingma. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)
2013 arXiv
-
[17]
Cheng-Han Lee, Ziwei Liu, Lingyun Wu, and Ping Luo. 2020. MaskGAN: Towards Diverse and Interactive Facial Image Manipulation. arXiv:1907.11922 [cs.CV] https://arxiv.org/abs/1907.11922
2020 arXiv
-
[18]
Bowen Li, Xiaojuan Qi, Thomas Lukasiewicz, and Philip Torr. 2019. Controllable text-to-image generation. Advances in neural information processing systems 32 (2019)
2019
-
[19]
Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. 2017. Feature pyramid networks for object detection. In Proceed- ings of the IEEE conference on computer vision and pattern recognition . 2117–2125
2017
-
[20]
Yuchen Luo, Yong Zhang, Junchi Yan, and Wei Liu. 2021. Generalizing face forgery detection with high-frequency features. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 16317–16326
2021
-
[21]
Guangyu Nie, Changhoon Kim, Yezhou Yang, and Yi Ren. 2023. Attributing image generative models using latent fingerprints. In International Conference on Machine Learning. PMLR, 26150–26165
2023
-
[22]
Fu-Zhao Ou, Chongyi Li, Shiqi Wang, and Sam Kwong. 2024. CLIB-FIQA: Face Image Quality Assessment with Confidence Calibration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 1694– 1704
2024
-
[23]
Ziqiao Peng, Wentao Hu, Yue Shi, Xiangyu Zhu, Xiaomei Zhang, Hao Zhao, Jun He, Hongyan Liu, and Zhaoxin Fan. 2024. Synctalk: The devil is in the synchro- nization for talking head synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 666–676
2024
-
[24]
Alec Radford. 2015. Unsupervised representation learning with deep convolu- tional generative adversarial networks. arXiv preprint arXiv:1511.06434 (2015)
2015 arXiv
-
[25]
Zeyang Sha, Zheng Li, Ning Yu, and Yang Zhang. 2023. De-fake: Detection and attribution of fake images generated by text-to-image generation models. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security. 3418–3432
2023
-
[26]
Sowdagar Mahammad Shahid, Sudev Kumar Padhi, Umesh Kashyap, and Sk Subidh Ali. 2024. Generalized deepfake attribution. arXiv preprint arXiv:2406.18278 (2024)
2024 arXiv
-
[27]
Rui Shao, Tianxing Wu, and Ziwei Liu. 2022. Detecting and recovering sequential deepfake manipulation. In European Conference on Computer Vision . Springer, 712–728
2022
-
[28]
Rui Shao, Tianxing Wu, and Ziwei Liu. 2023. Robust sequential deepfake detection. arXiv preprint arXiv:2309.14991 (2023)
2023 arXiv
-
[29]
Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. 2023. Consistency Models. arXiv preprint arXiv:2303.01469 (2023)
2023 arXiv
-
[30]
Sheng-Yu Wang, Oliver Wang, Andrew Owens, Richard Zhang, and Alexei A Efros. 2019. Detecting photoshopped faces by scripting photoshop. InProceedings of the IEEE/CVF International Conference on Computer Vision . 10072–10081
2019
-
[31]
Zhenting Wang, Chen Chen, Yi Zeng, Lingjuan Lyu, and Shiqing Ma. 2024. Where did i come from? origin attribution of ai-generated images. Advances in neural information processing systems 36 (2024)
2024
-
[32]
Zhenting Wang, Vikash Sehwag, Chen Chen, Lingjuan Lyu, Dimitris N Metaxas, and Shiqing Ma. 2024. How to Trace Latent Generative Model Generated Images without Artificial Watermark? arXiv preprint arXiv:2405.13360 (2024)
2024 arXiv
-
[33]
Ruiyang Xia, Decheng Liu, Jie Li, Lin Yuan, Nannan Wang, and Xinbo Gao
-
[34]
Tianyun Yang, Ziyao Huang, Juan Cao, Lei Li, and Xirong Li. 2022. Deepfake net- work architecture attribution. In Proceedings of the AAAI Conference on Artificial Intelligence
2022
-
[35]
Ning Yu, Larry S Davis, and Mario Fritz. 2019. Attributing fake images to gans: Learning and analyzing gan fingerprints. In Proceedings of the IEEE/CVF interna- tional conference on computer vision . 7556–7566
2019
-
[36]
Ning Yu, Vladislav Skripniuk, Sahar Abdelnabi, and Mario Fritz. 2021. Artificial fingerprinting for generative models: Rooting deepfake attribution in training data. In Proceedings of the IEEE/CVF International conference on computer vision . 14448–14457
2021
-
[37]
Ning Yu, Vladislav Skripniuk, Dingfan Chen, Larry Davis, and Mario Fritz. 2020. Responsible disclosure of generative models using scalable fingerprinting. arXiv preprint arXiv:2012.08726 (2020)
2020 arXiv
-
[38]
Guisheng Zhang, Mingliang Gao, Qilei Li, Siyou Guo, and Gwanggil Jeon. 2024. Detecting Sequential Deepfake Manipulation via Spectral Transformer With Pyramid Attention in Consumer IoT. IEEE Transactions on Consumer Electronics (2024)
2024
-
[39]
Wenliang Zhao, Yongming Rao, Weikang Shi, Zuyan Liu, Jie Zhou, and Jiwen Lu. 2023. Diffswap: High-fidelity and controllable face swapping via 3d-aware masked diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8568–8577
2023
-
[40]
Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043 (2023)
2023 arXiv
-
[2022]
In 2022 26th International Conference on Pattern Recognition (ICPR)
What makes you, you? Analyzing recognition by swapping face parts. In 2022 26th International Conference on Pattern Recognition (ICPR) . IEEE, 945–951
2022
-
[2024]
IEEE Transactions on Information Forensics and Security (2024)
Mmnet: multi-collaboration and multi-supervision network for sequential deepfake detection. IEEE Transactions on Information Forensics and Security (2024)
2024
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.