REVIEW 4 major objections 43 references
RelayFormer: A Unified Local-Global Attention Framework for Scalable Image and Video Manipulation Localization
T0 review · 4 major / 0 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read RelayFormer claims that one transformer, using fixed-size sub-images and Global Local Relay tokens, can localize tampering in both images and video without resizing or padding.
desk verdict The uploaded text is a different paper entirely, so RelayFormer is currently unverifiable and the right move is to send it back, not to peer review. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Global Local Relay (GLR) tokens. These are learned tokens inserted between fixed-size sub-images; a relay-based attention mechanism lets them carry structured context, such as semantic and temporal consistency, from one sub-image to the next while the local attention inside each sub-image preserves fine-grained manipulation artifacts. The tokens are the bridge that makes a single architecture work for both still frames and video.
What would settle it
A benchmark in which tampered regions are deliberately cropped so that edited boundaries cross sub-image partition lines; if localization quality drops sharply compared with identical regions lying entirely inside one sub-image, the relayed context is not preserving boundary-level forensic cues.
Extended reading notes
Core claim
The central claim is that resolution diversity and image-video unification can be handled by the same mechanism: partition inputs into fixed-size sub-images, run local attention within each, and exchange global context through learned GLR tokens. RelayFormer thereby avoids uniform resizing and excessive padding, which distort subtle forensic traces, and avoids sparse attention, which can lose cues. The paper reports superior performance and strong efficiency across image and video manipulation localization benchmarks, with no interpolation and a favorable accuracy-computation balance.
Load-bearing premise
Slicing inputs into fixed-size sub-images and connecting them only through relay tokens preserves the fine-grained, boundary-level clues that reveal tampering.
Editorial extensions
If this is right
- High-resolution forensic images can be processed without downsampling, preserving the edit-boundary and resampling traces that resizing erases.
- A single model, rather than separate image and video branches, handles both static and temporal inputs.
- Computational cost grows with the number of sub-images and relay tokens rather than with a forced uniform resolution, so cost scales with content.
- Because no sparse attention is needed, the method keeps dense local context inside each sub-image.
- The same architecture could be applied to variable-resolution inputs without retraining on a fixed input size.
Reading between the lines
- A natural stress test is to place a tampered region exactly on a sub-image boundary: if the relay tokens cannot carry the boundary-level forensic signal across partitions, accuracy should drop measurably. The paper does not report such an experiment.
- The sub-image design is a form of local attention with learned global memory; the same token-relay idea could transfer to other dense prediction tasks where high resolution and global context both matter, such as medical image segmentation or document forensics.
- For video, the relay mechanism implies that temporal consistency is propagated through tokens rather than through 3D convolutions or optical flow; this suggests the method's video performance depends on whether token-based temporal context is as precise as explicit motion cues.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The submission, identified as arXiv:2508.09459, presents an abstract for 'RelayFormer: A Unified Local-Global Attention Framework for Scalable Image and Video Manipulation Localization.' The abstract claims that RelayFormer partitions inputs into fixed-size sub-images, uses Global Local Relay (GLR) tokens to propagate structured context, preserves fine-grained manipulation artifacts, and achieves superior performance and efficiency across image and video manipulation localization benchmarks. However, the full text supplied with the submission is an unrelated manuscript titled 'Towards Self-cognitive Exploration: Metacognitive Knowledge Graph Retrieval Augmented Generation' (MetaKGRAG). The body contains no description of RelayFormer, no architecture details, no equations defining GLR attention, no experimental setup, no results tables, and no ablation studies. Consequently, none of the abstract's claims can be verified from the submitted manuscript.
Significance. If the claims in the abstract were fully supported, RelayFormer would potentially offer a useful unified architecture for image and video manipulation localization, addressing resolution diversity and modality unification. The idea of fixed-size sub-image partitioning with relay tokens for global context is a plausible design direction. However, the supplied manuscript provides no evidence that such a system exists, works, or preserves the forensic cues required for manipulation localization. There are no machine-checked proofs, no reproducible code or experiments, and no parameter-free derivations. The significance of the contribution therefore cannot be assessed from this submission; the only verifiable content is the unrelated MetaKGRAG paper, which is outside the scope of the claimed contribution.
major comments (4)
- [Full text (entire manuscript)] The full text is not the paper described in the abstract. It is 'Towards Self-cognitive Exploration: Metacognitive Knowledge Graph Retrieval Augmented Generation,' a completely different manuscript about KG-RAG question answering. There is no RelayFormer architecture, no definition of Global Local Relay tokens, no attention formulation, no manipulation localization datasets, and no experiments. This is a load-bearing failure: the central claim of the abstract is unsupported by any content in the submission.
- [Abstract, 'preserving fine-grained manipulation artifacts'] The abstract asserts that partitioning inputs into fixed-size sub-images and relaying global context through GLR tokens preserves fine-grained manipulation artifacts. This is the key technical premise: if sub-image boundaries cut through tampered regions or relay attention dilutes high-frequency forensic cues, the claimed accuracy advantage collapses. The manuscript provides no architecture description or experiments that test this premise, so the claim is not merely unproven; it is unauditable.
- [Abstract, 'Experiments across diverse benchmarks demonstrate superior performance'] No experiments, benchmarks, tables, or numerical results appear anywhere in the supplied text. The claimed 'superior performance and strong efficiency' cannot be checked. There are no baselines, no protocol descriptions, no ablations, and no statistical comparisons. This absence directly contradicts the abstract's empirical assertion.
- [Abstract, 'Code is available at ...'] A GitHub link is provided, but no code, model checkpoints, or evaluation scripts are included or described in the manuscript. In the absence of any implementation details, the link does not support reproducibility. The manuscript should at least specify the repository state or provide a hash, but no such information is present.
Circularity Check
No circular derivation present; the submitted body is an unrelated RAG paper, leaving the RelayFormer abstract without any chain to audit.
full rationale
The abstract claims RelayFormer partitions inputs into fixed-size sub-images and uses Global Local Relay tokens, but the supplied full text is entirely a different manuscript: 'Towards Self-cognitive Exploration: Metacognitive Knowledge Graph Retrieval Augmented Generation'. There are no equations defining RelayFormer, no architecture section, no experiments for manipulation localization, and no ablation studies. The only formula in the text is PathSimilarity, which belongs to the RAG method and is irrelevant to the claimed visual manipulation localization derivation. Consequently, there is no derivation chain that could be shown to reduce to its own inputs, and no fitted parameter is renamed as a prediction. The abstract's assertions about preserving fine-grained manipulation artifacts and achieving superior efficiency are unsupported and unauditable, but missing support is an integrity/completeness concern, not a circularity. Under the hard rules, circularity requires quoting a specific reduction (e.g., Eq. X = Eq. Y by construction), and none exists in the supplied text. The honest non-finding is therefore a circularity score of 0.
Assumptions & free parameters
free parameters (1)
- Sub-image partition size (fixed-size input granularity)
assumptions (3)
- domain assumption Manipulation artifacts (boundary inconsistencies, resampling traces, frame inconsistencies) survive at the chosen sub-image granularity and are not destroyed by partitioning or by attention pooling.
- domain assumption The comparison against prior approaches is fair and comprehensive.
- ad hoc to paper The supplied full text is the paper under review (documentary coherence).
invented entities (1)
-
Global Local Relay (GLR) tokens
Cite this review
Pith. "Pith review of RelayFormer: A Unified Local-Global Attention Framework for Scalable Image and Video Manipulation Localization." pith.science (2026). https://pith.science/paper/IPL5XYEX
@misc{pith2026250809459,
author = {Pith},
title = {Pith review of: RelayFormer: A Unified Local-Global Attention Framework for Scalable Image and Video Manipulation Localization},
year = {2026},
howpublished = {\url{https://pith.science/paper/IPL5XYEX}},
note = {Machine review of arXiv:2508.09459}
}
read the original abstract
Visual manipulation localization (VML) aims to identify tampered regions in images and videos, a task that has become increasingly challenging with the rise of advanced editing tools. Existing methods face two central issues. The first is resolution diversity. Resizing or padding can distort subtle forensic cues and introduce unnecessary computational cost. The second is the difficulty of extending spatial models for images to spatio-temporal inputs in videos, which often results in maintaining separate architectures for the two data types. To address these challenges, we propose RelayFormer, a unified framework that adapts to varying resolutions and naturally handles both static and temporal visual data. RelayFormer partitions inputs into fixed-size sub-images and introduces Global Local Relay (GLR) tokens that propagate structured context through a relay-based attention mechanism. This design enables efficient exchange of global cues, such as semantic or temporal consistency, while preserving fine-grained manipulation artifacts. Unlike prior approaches that depend on uniform resizing or sparse attention, RelayFormer scales to variable resolutions and video sequences with minimal overhead. Experiments across diverse benchmarks demonstrate superior performance and strong efficiency, combining resolution adaptivity without interpolation or excessive padding, unified processing for images and videos, and a favorable balance between accuracy and computational cost. Code is available at~\href{https://github.com/WenOOI/RelayFormer}{https://github.com/WenOOI/RelayFormer}.
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
Chen, X.; Dong, C.; Ji, J.; Cao, J.; and Li, X. 2021. Image Manipulation Detection by Multi-View Multi-Scale Supervision. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 14165–14173. Montreal, QC, Canada: IEEE. ISBN 978-1-66542-812-5
work page 2021
-
[4]
G.; Kirillov, A.; and Girdhar, R
Cheng, B.; Misra, I.; Schwing, A. G.; Kirillov, A.; and Girdhar, R. 2022. Masked-attention mask transformer for universal image segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 1290--1299
2022
-
[5]
Ding, H.; Liu, C.; He, S.; Jiang, X.; Torr, P. H.; and Bai, S. 2023. MOSE: A new dataset for video object segmentation in complex scenes. In Proceedings of the IEEE/CVF international conference on computer vision, 20224--20234
work page 2023
-
[6]
Dong, J.; Wang, W.; and Tan, T. 2013. CASIA Image Tampering Detection Evaluation Database. In 2013 IEEE China Summit and International Conference on Signal and Information Processing, 422–426. Beijing, China: IEEE. ISBN 978-1-4799-1043-4
work page 2013
-
[7]
Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; Uszkoreit, J.; and Houlsby, N. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. (arXiv:2010.11929). ArXiv:2010.11929 [cs]
arXiv 2021
-
[8]
N.; Delgado, A.; Zhou, D.; Kheyrkhah, T.; Smith, J.; and Fiscus, J
Guan, H.; Kozak, M.; Robertson, E.; Lee, Y.; Yates, A. N.; Delgado, A.; Zhou, D.; Kheyrkhah, T.; Smith, J.; and Fiscus, J. 2019. MFC Datasets: Large-Scale Benchmark Datasets for Media Forensic Challenge Evaluation. In 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW), 63–72. Waikoloa Village, HI, USA: IEEE. ISBN 978-1-72811-392-0
work page 2019
Show all 43 references
-
[9]
Guillaro, F.; Cozzolino, D.; Sud, A.; Dufour, N.; and Verdoliva, L. 2023. TruFor: Leveraging all-round clues for trustworthy image forgery detection and localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 20606--20615
2023
-
[10]
Hsu, Y.-f.; and Chang, S.-f. 2006. Detecting Image Splicing using Geometry Invariants and Camera Characteristics Consistency. In 2006 IEEE International Conference on Multimedia and Expo, 549–552. Toronto, ON, Canada: IEEE. ISBN 978-1-4244-0367-7
2006
-
[11]
J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; Chen, W.; et al
Hu, E. J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; Chen, W.; et al. 2022. Lora: Low-rank adaptation of large language models. ICLR, 1(2): 3
2022
-
[12]
Kim, D.; Woo, S.; Lee, J.-Y.; and Kweon, I. S. 2019. Recurrent temporal aggregation framework for deep video inpainting. IEEE transactions on pattern analysis and machine intelligence, 42(5): 1038--1052
2019
-
[13]
Kwon, M.-J.; Nam, S.-H.; Yu, I.-J.; Lee, H.-K.; and Kim, C. 2022. Learning JPEG compression artifacts for image manipulation detection and localization. International Journal of Computer Vision, 130(8): 1875--1895
2022
-
[14]
Li, Z.; Lu, C.-Z.; Qin, J.; Guo, C.-L.; and Cheng, M.-M. 2022. Towards an end-to-end framework for flow-guided video inpainting. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 17562--17571
2022
-
[15]
Liu, R.; Deng, H.; Huang, Y.; Shi, X.; Lu, L.; Sun, W.; Wang, X.; Dai, J.; and Li, H. 2021. Fuseformer: Fusing fine-grained information in transformers for video inpainting. In Proceedings of the IEEE/CVF international conference on computer vision, 14040--14049
2021
-
[16]
Liu, X.; Liu, Y.; Chen, J.; and Liu, X. 2022. Pscc-net: Progressive spatio-channel correlation network for image manipulation detection and localization. IEEE Transactions on Circuits and Systems for Video Technology, 32(11): 7505--7517
2022
-
[17]
Loshchilov, I.; and Hutter, F. 2017. SGDR: Stochastic Gradient Descent with Warm Restarts. (arXiv:1608.03983). ArXiv:1608.03983 [cs, math]
2017 arXiv
-
[18]
Loshchilov, I.; and Hutter, F. 2019. Decoupled Weight Decay Regularization. (arXiv:1711.05101). ArXiv:1711.05101 [cs, math]
2019 arXiv
-
[19]
Lou, Z.; Cao, G.; and Lin, M. 2024. Trusted video inpainting localization via deep attentive noise learning. arXiv preprint arXiv:2406.13576
2024 arXiv
-
[20]
Lou, Z.; Cao, G.; and Lin, M. 2025. Video inpainting localization with contrastive learning. IEEE Signal Processing Letters
2025
-
[21]
Ma, X.; Du, B.; Liu, X.; Hammadi, A. Y. A.; and Zhou, J. 2023. Iml-vit: Image manipulation localization by vision transformer
2023
-
[22]
Ma, X.; Zhu, X.; Su, L.; Du, B.; Jiang, Z.; Tong, B.; Lei, Z.; Yang, X.; Pun, C.-M.; Lv, J.; et al. 2024. IMDL-BenCo: A Comprehensive Benchmark and Codebase for Image Manipulation Detection & Localization
2024
-
[23]
D.; Fang, S.; and Stamm, M
Nguyen, T. D.; Fang, S.; and Stamm, M. C. 2024. Videofact: detecting video forgeries using attention, scene context, and forensic traces. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, 8563--8573
2024
-
[24]
Novozamsky, A.; Mahdian, B.; and Saic, S. 2020. IMD2020: A Large-Scale Annotated Dataset Tailored for Detecting Manipulated Images. In 2020 IEEE Winter Applications of Computer Vision Workshops (WACVW), 71–80. Snowmass Village, CO, USA: IEEE. ISBN 978-1-72817-162-3
2020
-
[25]
W.; Lee, S.; Lee, J.-Y.; and Kim, S
Oh, S. W.; Lee, S.; Lee, J.-Y.; and Kim, S. J. 2019. Onion-peel networks for deep video completion. In Proceedings of the IEEE/CVF international conference on computer vision, 4403--4412
2019
-
[26]
Pei, P. 2023. UVL2: A Unified Framework for Video Tampering Localization. arXiv preprint arXiv:2309.16126
2023 arXiv
-
[27]
Perazzi, F.; Pont-Tuset, J.; McWilliams, B.; Van Gool, L.; Gross, M.; and Sorkine-Hornung, A. 2016. A benchmark dataset and evaluation methodology for video object segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, 724--732
2016
-
[28]
Su, J.; Ahmed, M.; Lu, Y.; Pan, S.; Bo, W.; and Liu, Y. 2024. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568: 127063
2024
-
[29]
Su, L.; Ma, X.; Zhu, X.; Niu, C.; Lei, Z.; and Zhou, J.-Z. 2025. Can we get rid of handcrafted feature extractors? sparsevit: Nonsemantics-centered, parameter-efficient image manipulation localization through spare-coding transformer. In Proceedings of the AAAI Conference on A...
2025
-
[30]
N.; Kaiser, .; and Polosukhin, I
Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. Advances in neural information processing systems, 30
2017
-
[31]
Wang, J.; Wu, Z.; Chen, J.; Han, X.; Shrivastava, A.; Lim, S.-N.; and Jiang, Y.-G. 2022. ObjectFormer for Image Manipulation Detection and Localization. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2354–2363. New Orleans, LA, USA: IEEE. ISBN 9...
2022
-
[32]
Wang, P.; Bai, S.; Tan, S.; Wang, S.; Fan, Z.; Bai, J.; Chen, K.; Liu, X.; Wang, J.; Ge, W.; et al. 2024. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12191
2024 arXiv
-
[33]
Wen, B.; Zhu, Y.; Subramanian, R.; Ng, T.-T.; Shen, X.; and Winkler, S. 2016. COVERAGE — A novel database for copy-move forgery detection. In 2016 IEEE International Conference on Image Processing (ICIP), 161–165. Phoenix, AZ, USA: IEEE. ISBN 978-1-4673-9961-6
2016
-
[34]
Wu, H.; Chen, Y.; and Zhou, J. 2023. Rethinking image forgery detection via contrastive learning and unsupervised clustering. arXiv preprint arXiv:2308.09307
2023 arXiv
-
[35]
Wu, H.; and Zhou, J. 2021. IID-Net: Image inpainting detection network via neural architecture search and attention. IEEE Transactions on Circuits and Systems for Video Technology, 32(3): 1172--1185
2021
-
[36]
Wu, H.; Zhou, J.; Tian, J.; Liu, J.; and Qiao, Y. 2022. Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security, 17: 443--456
2022
-
[37]
Wu, Y.; et al. 2019. ManTra-Net: Manipulation Tracing Network for Detection and Localization of Image Forgeries With Anomalous Features. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 9535–9544. Long Beach, CA, USA: IEEE. ISBN 978-1-72813-293-8
2019
-
[38]
M.; and Luo, P
Xie, E.; Wang, W.; Yu, Z.; Anandkumar, A.; Alvarez, J. M.; and Luo, P. 2021. SegFormer: Simple and efficient design for semantic segmentation with transformers. Advances in neural information processing systems, 34: 12077--12090
2021
-
[39]
Yang, J.; Li, C.; Zhang, P.; Dai, X.; Xiao, B.; Yuan, L.; and Gao, J. 2021. Focal self-attention for local-global interactions in vision transformers. arXiv preprint arXiv:2107.00641
2021 arXiv
-
[40]
Zeng, Y.; Fu, J.; and Chao, H. 2020. Learning joint spatial-temporal transformations for video inpainting. In European conference on computer vision, 528--543. Springer
2020
-
[41]
Y.; and Feng, W
Zhou, J.; Ma, X.; Du, X.; Alhammadi, A. Y.; and Feng, W. 2023. Pre-training-free Image Manipulation Localization through Non-Mutually Exclusive Contrastive Learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 22346--22356
2023
-
[42]
S.; Shrivastava, A.; and Lim, S.-N
Zhou, P.; Yu, N.; Wu, Z.; Davis, L. S.; Shrivastava, A.; and Lim, S.-N. 2021. Deep video inpainting detection. arXiv preprint arXiv:2101.11080
2021 arXiv
-
[43]
Zhu, X.; Ma, X.; Su, L.; Jiang, Z.; Du, B.; Wang, X.; Lei, Z.; Feng, W.; Pun, C.-M.; and Zhou, J.-Z. 2025. Mesoscopic insights: orchestrating multi-scale & hybrid architecture for image manipulation localization. In Proceedings of the AAAI Conference on Artificial Intelligence...
2025
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.