REVIEW 3 major objections 5 minor 20 references
Unmasking Deep Fakes: Leveraging Deep Learning for Video Authenticity Detection
T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read The paper claims that MTCNN face detection plus an EfficientNet-B5 frame classifier detects deepfake videos on the DFDC dataset with 0.4278 log loss, 0.938 AUC, and 0.8682 F1, beating one published ensemble baseline on log loss.
desk verdict Routine MTCNN+EfficientNet-B5 pipeline with plausible DFDC numbers, but the missing untouched test split means the headline result is unverified. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanism is a two-stage pipeline. MTCNN (Multi-task Cascaded Convolutional Networks) detects faces and five landmarks through a P-Net, R-Net, O-Net cascade, producing cropped face regions at original resolution. EfficientNet-B5, a compound-scaled CNN, classifies each cropped frame; training adds a 30 percent margin around the crop, uses balanced batches, SGD with momentum, a polynomial learning-rate scheduler, and label smoothing. Video-level predictions come from confidence-weighted aggregation over 32 frames per video, where high-confidence frames weigh more and low-confidence frames are discarded, and SSIM masks computed during preprocessing are stored as auxiliary PNG files.
What would settle it
Re-run the same MTCNN-EfficientNet-B5 training recipe on the DFDC data with a pre-registered held-out test set, one whose folders are untouched by early stopping and threshold selection, and recompute the three metrics; if the log loss rises materially above 0.4278 or the AUC falls materially below 0.938, the paper's central performance claim is not reproducible.
Extended reading notes
Core claim
The central claim is that the MTCNN-EfficientNet-B5 pipeline distinguishes fake from real videos on the DFDC dataset with a video-level log loss of 0.4278, an AUC of 0.938, and an F1 score of 0.8682, using only face crops, SSIM-based difference masks, and confidence-weighted averaging of 32 frame predictions per video. The authors present this as competitive with published DFDC results: it beats an ensemble CNN on log loss and trails an EfficientNet-Vision Transformer hybrid by 0.013 in AUC and 0.012 in F1. They attribute the result to MTCNN's robust face localization, EfficientNet-B5's compound scaling, and a confidence-weighted pooling scheme that downweights uncertain frames instead of using simple averaging.
Load-bearing premise
The load-bearing premise is that the reported log loss, AUC, and F1 come from a test split that was never used to choose the model, tune the confidence thresholds, or stop training; the paper only describes an evolving validation setup and never states that such a final untouched split exists.
Editorial extensions
If this is right
- Deployable single-model detection: a pipeline with one CNN and a face detector can reach a log loss of 0.4278 on DFDC, so high-accuracy screening does not require large ensembles.
- Calibrated confidence: the low log loss implies the model's probabilities are well-calibrated enough to rank and filter videos, not just classify them.
- Rejected uncertain frames: confidence-weighted averaging means frame-level uncertainty is explicitly discounted, which should improve robustness on blurry or occluded clips.
- Architecture verdict: EfficientNet-B5 beats its smaller and larger siblings in this task, giving a concrete cost-accuracy sweet spot for the detection pipeline.
Reading between the lines
- If the numbers reproduce on a true holdout, the log-loss advantage over the ensemble baseline suggests single-model pipelines can be better calibrated, and confidence-weighted pooling may transfer to other video-classification tasks.
- Because the pipeline depends on face detection, its accuracy on videos with occluded, tiny, off-center, or multiple faces is an open question the paper does not address; those conditions are a likely failure mode in the wild.
- The method's components are standard and the DFDC benchmark is public, so the natural next check is a direct, matched-split comparison with the two baselines cited in the paper, plus evaluation on other deepfake datasets, which would show whether the margins generalize.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper describes a deepfake video detection pipeline that combines MTCNN face detection, an EfficientNet-B5 frame-level classifier, and a confidence-weighted aggregation of 32 sampled frames per video. The model is trained and evaluated on the Kaggle DFDC dataset, and the paper reports a log loss of 0.4278, an AUC of 0.9380, and an F1 score of 0.8682. The authors compare these numbers with an EfficientNet-Vision Transformer baseline and an ensemble CNN baseline from the literature, and they argue that the proposed pipeline is competitive without extensive preprocessing or ensembling.
Significance. If the performance numbers survive a properly defined untouched test split, this is a modest and useful empirical contribution: the pipeline uses established components (MTCNN, EfficientNet) plus a simple confidence-weighted aggregation rule, and it is described transparently enough to be reproduced from public data and pretrained models. The authors are also candid about the remaining gap to the ViT hybrid and about face-detection vulnerabilities. The significance is, however, conditional: the evaluation protocol in Section 4.5 is ambiguous, and the state-of-the-art comparison in Table 1 is not controlled. The paper's central claim of competitive performance therefore cannot be accepted as stated without a clarified and, ideally, untouched evaluation split.
major comments (3)
- [Section 4.5, Section 5.2, Table 1] The evaluation protocol is not sufficiently defined to support the headline numbers. Section 4.5 states that 'the validation approach evolved throughout the development process' and only describes an initial holdout of folders 0-2; no final, untouched test split is defined. Because Section 4.3 reports selecting EfficientNet-B5 through experiments and Section 4.4 introduces confidence thresholds for aggregation, the reader cannot rule out that the log loss 0.4278, AUC 0.938, and F1 0.8682 were produced by the same validation loop used for model selection and threshold tuning. Please specify a final test split (e.g., a set of DFDC folders or video IDs) that was never used for early stopping, variant selection, or threshold tuning, and report all headline metrics on that split; if the reported numbers are validation numbers, say so explicitly and temper the claims accordingly.
- [Section 5.2, Table 1] The comparison with the two baselines is uncontrolled. The values for EfficientNet-Vision Transformer [9] and Ensemble CNN [7] are taken from the literature, and it is not established that they were computed on the same test split, same frame sampling, same face detector, or same aggregation method. The claim in the contributions that the proposed model 'outperformed many state-of-the-art models' is therefore not supported by the evidence presented. Please either run the baselines under the same protocol on the final test split or state clearly that Table 1 is a literature comparison with no claim of a controlled head-to-head.
- [Sections 4.3 and 4.4] The central design choices are not quantitatively justified. The paper states that EfficientNet-B5 provided the best trade-off and that confidence-weighted aggregation was beneficial, but it reports no ablation comparing variants (B0-B7) or simple averaging versus the weighted aggregation. Without these ablations, the contribution of the proposed fusion and heuristic cannot be separated from the choice of the backbone. Please add ablation results on the validation set, and confirm that the final configuration was fixed before computing metrics on the untouched test split.
minor comments (5)
- [Abstract, Section 5.2] The abstract and Section 5.2 report '42.78% log loss' and '93.80% AUC'; log loss and AUC are not percentages and should be reported as 0.4278 and 0.9380.
- [Section 4.5] Section 4.5 refers to 'folders 0 through 2' but the folder partition of DFDC is never defined; clarify how the data were split into folders and how the final test split relates to them.
- [Section 4.1] Section 4.1 mentions computing SSIM masks and storing them as PNG files, but the rest of the paper never states whether or how these masks are used by the model; either integrate them into the pipeline description or remove them.
- [Equation (1)] Equation (1) is typeset incorrectly in the submitted text; the summation and variables should be formatted properly, and the definitions of y-hat and y should be stated in the text rather than only in a parenthetical.
- [Throughout] The manuscript contains numerous language and typographical issues (e.g., 'now a days', 'intent to enhance', 'Architechture', 'EfficinetNet', 'maneuver', 'contribute important perceptions'); a careful copyedit is needed.
Circularity Check
Reported DFDC performance is not tied to an untouched test split after the model variant and thresholds were selected using the evolving validation set, making the headline metrics partially fitted rather than independently predicted.
-
fitted input called prediction
[Sections 4.3-4.5 and 5.2/Table 1]
"Experiments with different variants indicated that EfficientNet B5 provided the best trade-off... The validation approach evolved throughout the development process. Initially, folders 0 through 2 were held out for validation... Specifically, the model achieved a log loss of 0.4278, an AUC of 0.938, and an F1 score of 0.8682"
The model variant (EfficientNet-B5) was selected through experiments, and the confidence-weighted aggregation uses thresholds that were evidently tuned during development. Section 4.5 describes the validation set as evolving and never states that a final, untouched test split exists. Section 5.2 then presents the resulting log loss, AUC, and F1 as the model's achieved performance. If these metrics come from the same validation loop used for model and threshold selection, the reported performance is the fitted outcome of that selection process, not an out-of-sample prediction. The central performance claim therefore reduces, by the paper's own description of the evaluation protocol, to a quantity that was used to choose the model, making it partially circular by construction.
full rationale
The paper contains no mathematical derivation and no self-citation chain; MTCNN and EfficientNet-B5 are standard components and the DFDC dataset is a public benchmark. The only load-bearing circularity is evaluation-based. Section 4.3 states that experiments with variants led to choosing B5, and Section 4.4 describes a confidence-weighted aggregation with thresholds. Section 4.5 says the validation approach evolved and initially held out folders 0-2, but it never defines a separate final test set. Section 5.2 then reports the headline log loss, AUC, and F1. If those numbers were produced by the same evolving validation process used to select the architecture and thresholds, the reported performance is a fitted quantity rather than a genuine prediction. This is partial circularity, not a fully forced derivation, because the underlying data and model components are external and the numerical values remain empirical; hence score 6 rather than higher. No other circular steps, self-citation load-bearing arguments, or uniqueness-imported-from-authors issues were identified.
Assumptions & free parameters
free parameters (6)
- EfficientNet variant =
B5
- Number of sampled frames per video =
32
- Face crop margin =
30%
- Label smoothing coefficient =
not reported
- Confidence thresholds for weighted averaging =
not reported
- SGD momentum, learning rate, polynomial schedule parameters, batch size =
not reported
assumptions (3)
- domain assumption The DFDC dataset's fake/real distribution and generation techniques are representative of the deepfake detection problem the model will face in practice.
- domain assumption MTCNN reliably detects faces with sufficient accuracy so that face crops retain deepfake artifacts.
- domain assumption The confidence-weighted averaging heuristic improves over simple averaging.
Cite this review
Pith. "Pith review of Unmasking Deep Fakes: Leveraging Deep Learning for Video Authenticity Detection." pith.science (2026). https://pith.science/paper/HUNZKUEP
@misc{pith2026250506528,
author = {Pith},
title = {Pith review of: Unmasking Deep Fakes: Leveraging Deep Learning for Video Authenticity Detection},
year = {2026},
howpublished = {\url{https://pith.science/paper/HUNZKUEP}},
note = {Machine review of arXiv:2505.06528}
}
read the original abstract
Deepfake videos, produced through advanced artificial intelligence methods now a days, pose a new challenge to the truthfulness of the digital media. As Deepfake becomes more convincing day by day, detecting them requires advanced methods capable of identifying subtle inconsistencies. The primary motivation of this paper is to recognize deepfake videos using deep learning techniques, specifically by using convolutional neural networks. Deep learning excels in pattern recognition, hence, makes it an ideal approach for detecting the intricate manipulations in deepfakes. In this paper, we consider using MTCNN as a face detector and EfficientNet-B5 as encoder model to predict if a video is deepfake or not. We utilize training and evaluation dataset from Kaggle DFDC. The results shows that our deepfake detection model acquired 42.78% log loss, 93.80% AUC and 86.82% F1 score on kaggle's DFDC dataset.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[9]
Combining efficientnet and vision transformers for video deepfake detec- tion
Davide Alessandro Coccomini, Nicola Messina, Claudio Gennaro, and Fabrizio Falchi. Combining efficientnet and vision transformers for video deepfake detec- tion. In International conference on image analysis and processing , pages 219–229. Springer, 2022
work page 2022
-
[7]
Video face manipulation detection through ensem- ble of cnns
Nicolo Bonettini, Edoardo Daniele Cannas, Sara Mandelli, Luca Bondi, Paolo Bestagini, and Stefano Tubaro. Video face manipulation detection through ensem- ble of cnns. In2020 25th international conference on pattern recognition (ICPR) , pages 5012–5019. IEEE, 2021
work page 2021
-
[1]
Fakhar Abbas and Araz Taeihagh. Unmasking deepfakes: A systematic review of deepfake detection and generation techniques using artificial intelligence.Expert Systems With Applications , page 124260, 2024
work page 2024
-
[2]
Detecting deep- fake videos from appearance and behavior
Shruti Agarwal, Hany Farid, Tarek El-Gaaly, and Ser-Nam Lim. Detecting deep- fake videos from appearance and behavior. In2020 IEEE international workshop on information forensics and security (WIFS) , pages 1–6. IEEE, 2020
work page 2020
-
[3]
Tashin Ahmed and Noor Hossain Nuri Sabab. Classification and understanding of cloud structures via satellite images with efficientunet.SN Computer Science , 3(1):99, 2022
work page 2022
-
[4]
Video and audio deepfake datasets and open issues in deepfake technology: being ahead of the curve
Zahid Akhtar, Thanvi Lahari Pendyala, and Virinchi Sai Athmakuri. Video and audio deepfake datasets and open issues in deepfake technology: being ahead of the curve. Forensic Sciences, 4(3):289–377, 2024
work page 2024
-
[5]
Unmasking deception: Empowering deepfake detection with vision transformer network
Muhammad Asad Arshed, Ayed Alwadain, Rao Faizan Ali, Shahzad Mumtaz, Muhammad Ibrahim, and Amgad Muneer. Unmasking deception: Empowering deepfake detection with vision transformer network. Mathematics, 11(17):3710, 2023. 12 M. Hasan et al
work page 2023
-
[6]
Nency Bansal, Turki Aljrees, Dhirendra Prasad Yadav, Kamred Udham Singh, Ankit Kumar, Gyanendra Kumar Verma, and Teekam Singh. Real-time ad- vanced computational intelligence for deep fake video detection.Applied Sciences, 13(5):3095, 2023
work page 2023
Show all 20 references
-
[8]
Protecting world leader using facial speaking pattern against deepfakes.IEEE Signal Process- ing Letters, 29:2078–2082, 2022
Beilin Chu, Weike You, Zhen Yang, Linna Zhou, and Renying Wang. Protecting world leader using facial speaking pattern against deepfakes.IEEE Signal Process- ing Letters, 29:2078–2082, 2022
2022
-
[10]
Unmasking deepfakes: Masked autoencoding spatiotemporal transformers for en- hanced video forgery detection
SayantanDas,MojtabaKolahdouzi,LeventÖzparlak,WillHickie,andAliEtemad. Unmasking deepfakes: Masked autoencoding spatiotemporal transformers for en- hanced video forgery detection. In2023 IEEE International Joint Conference on Biometrics (IJCB), pages 1–11. IEEE, 2023
2023
-
[11]
Deepfake video detection based on efficientnet-v2 network
Liwei Deng, Hongfei Suo, and Dongjie Li. Deepfake video detection based on efficientnet-v2 network. Computational Intelligence and Neuroscience , 2022(1):3441549, 2022
2022
-
[12]
Unmask- ing deepfakes with simple features.arXiv preprint arXiv:1911.00686 , 2019
Ricard Durall, Margret Keuper, Franz-Josef Pfreundt, and Janis Keuper. Unmask- ing deepfakes with simple features.arXiv preprint arXiv:1911.00686 , 2019
1911 arXiv
-
[13]
Deep learning methods to detect image falsification.Applied Sciences, 13(13):7694, 2023
Laimonas Janut˙ enas, J¯ urat˙ e Janut˙ enait˙ e-Bogdanien˙ e, and Dmitrij Šešok. Deep learning methods to detect image falsification.Applied Sciences, 13(13):7694, 2023
2023
-
[14]
Emergence of deepfakes and video tampering detection approaches: A survey.Multimedia Tools and Applica- tions, 82(7):10165–10209, 2023
Staffy Kingra, Naveen Aggarwal, and Nirmal Kaur. Emergence of deepfakes and video tampering detection approaches: A survey.Multimedia Tools and Applica- tions, 82(7):10165–10209, 2023
2023
-
[15]
Deepfake detection: A systematic literature review.IEEE access, 10:25494–25513, 2022
Md Shohel Rana, Mohammad Nur Nobi, Beddhu Murali, and Andrew H Sung. Deepfake detection: A systematic literature review.IEEE access, 10:25494–25513, 2022
2022
-
[16]
Blockchain technology for combating deepfake and protect video/image integrity.Journal of Multimedia Society, 24(8):1044–1058, 2021
Md Mamunur Rashid, Suk-Hwan Lee, and Ki-Ryong Kwon. Blockchain technology for combating deepfake and protect video/image integrity.Journal of Multimedia Society, 24(8):1044–1058, 2021
2021
-
[17]
Deep fake video detection using transfer learning approach.Arabian Journal for Science and Engineering , 48(8):9727–9737, 2023
Shraddha Suratkar and Faruk Kazi. Deep fake video detection using transfer learning approach.Arabian Journal for Science and Engineering , 48(8):9727–9737, 2023
2023
-
[18]
Comparison of deepfake detection techniques through deep learning
Maryam Taeb and Hongmei Chi. Comparison of deepfake detection techniques through deep learning. Journal of Cybersecurity and Privacy , 2(1):89–106, 2022
2022
-
[19]
Efficientnet: Rethinking model scaling for convolutional neural networks
Mingxing Tan. Efficientnet: Rethinking model scaling for convolutional neural networks. arXiv preprint arXiv:1905.11946 , 2019
1905 arXiv
-
[20]
Minor privacy protection through real-time video processing at the edge
Meng Yuan, Seyed Yahya Nikouei, Alem Fitwi, Yu Chen, and Yunxi Dong. Minor privacy protection through real-time video processing at the edge. In2020 29th International Conference on Computer Communications and Networks (ICCCN) , pages 1–6. IEEE, 2020
2020
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.