FRIDAY: Mitigating Unintentional Facial Identity in Deepfake Detectors Guided by Facial Recognizers

Changick Kim; Myung-Joon Kwon; Wonjun Lee; Younhun Kim

REVIEW 5 major objections 6 minor 18 references

FRIDAY: Mitigating Unintentional Facial Identity in Deepfake Detectors Guided by Facial Recognizers

T0 review · 5 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that freeing a deepfake detector's embedding from the direction a frozen face recognizer learns removes unintended identity shortcuts and improves detection on both seen and unseen forgery domains.

desk verdict A plausible training-time trick that makes a modest empirical dent, but the identity-attenuation claim is not actually pinned down by the experiments. read the letter →

arxiv 2412.14623 v1 pith:LKFTGOVF submitted 2024-12-19 cs.CV cs.CR

Younhun Kim , Myung-Joon Kwon , Wonjun Lee , Changick Kim This is my paper · ORCID

classification cs.CVcs.CR

keywords deepfakedetectionfacialidentityattenuationfacerecognitioncross-domaingeneralizationcosinesimilaritylossleakageforensicembeddingorthogonalization

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that deepfake detectors often learn who a face is rather than whether it was manipulated, and that this hidden identity shortcut is a major reason they fail on new forgery methods. FRIDAY is a training method that removes this shortcut by pairing the detector with a frozen face recognizer during training. The two networks see the same image, and the loss drives their feature embeddings to be orthogonal, so the detector cannot rely on the identity direction the recognizer has learned. The authors report that this improves accuracy on the training domain and raises cross-domain performance over the compared baselines. If true, it offers a reusable regularizer for making forensic detectors generalize beyond the artifacts they were trained on.

What carries the argument

The key object is the Facial Identity Attenuating loss, the absolute cosine similarity between the frozen face recognizer's embedding and the deepfake detector's embedding. The recognizer, trained on real faces only, acts as a stable reference direction for facial identity, and driving the detector's embedding orthogonal to that direction is intended to strip identity information while leaving artifact features intact. The full machinery is the two-phase training scheme: first train the recognizer with cross-entropy over 720 FF++ identities, then freeze it and train the detector with a total loss comprising the classification loss plus a weighting factor times the identity-attenuating loss, with the weighting factor set to 10 after a sweep.

What would settle it

Train the Phase-1 recognizer on a strict subset of identities, then evaluate FRIDAY on fake videos of held-out identities; if cross-domain AUC does not change or the face-identification probe still converges quickly on those held-out identities, the single recognizer direction was not actually erasing identity information and the reported gains must come from a different mechanism such as general regularization.

Watch

Extended reading notes

Core claim

The central claim is that unintentional facial identity learning is a removable cause of cross-domain generalization failure. In Phase 1 the authors train a face recognizer with the same backbone as the detector, using a closed-set cross-entropy loss over 720 real identities. In Phase 2 they freeze that recognizer and add the Facial Identity Attenuating loss, defined as the absolute value of the cosine similarity between the two embeddings, to the binary cross-entropy classification loss. Minimizing this term makes the detector's embedding orthogonal to the recognizer's identity-oriented embedding, attenuating identity features while still training the detector to classify real versus fake. The paper demonstrates the reduction in identity content with a probe that retrains the detector's classifier for face identification, and reports that FRIDAY improves both in-domain FF++ accuracy and cross-domain AUC on Celeb-DF v1, Celeb-DF v2, and DFD.

Load-bearing premise

The method assumes that a face recognizer trained on 720 identities from the training set captures an identity direction general enough that making the detector's embedding orthogonal to it removes identity information for unseen faces, without removing the artifact features the detector needs.

Editorial extensions

If this is right

In-domain detection on FF++ reaches 95.18% ACC and 99.18% AUC with FRIDAY, exceeding the compared baseline models in Table I.
Cross-domain average AUC reaches 84.37% across Celeb-DF v1, Celeb-DF v2, and DFD, with DFD ACC at 90.12%.
The identity-attenuation effect is measurable: after FRIDAY training, a face-identification probe converges less, indicating reduced facial identity content in the detector's embedding.
Because the recognizer and detector use the same backbone, the method can be applied to other deepfake detector architectures by adding the frozen recognizer and the identity-attenuating loss.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper constrains the detector against a single recognizer vector, but facial identity is likely a subspace; a natural extension the paper leaves implicit is projecting out multiple identity directions or the top-k recognizer directions, which may preserve artifact features more cleanly.
The same two-phase recipe could transfer to other shortcut-prone forensic tasks, such as synthetic text or synthetic audio detection, where a content recognizer supplies the direction to erase.
The lambda sweep suggests a tradeoff between identity removal and artifact preservation, so the optimal strength likely varies with the diversity of identities in the target domain; per-dataset tuning is a plausible next step.
If orthogonality to the recognizer genuinely removes identity information, per-identity accuracy should become more uniform, which is a checkable side prediction the paper does not report.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

FRIDAY is a simple training-time addition: freeze a face recognizer trained on the same backbone, and add a term that pushes the detector embedding's absolute cosine similarity toward zero. The combination is genuinely new to me, and the paper is easy to read. The reported gains on FF++ and cross-domain sets are modest but consistent, and the lambda sweep actually shows that the identity check (Fig. 4c) tracks performance, which is an attempt to close the loop.

The soft spots are in the mechanism claim, not in the basic reproducibility. The biggest missing experiment is a same-backbone detector trained without L_fia. Without that ablation, the 1-2 point gains could come from any regularizer. The comparison table omits the most relevant prior work—Dong et al.'s implicit identity leakage paper [13]—so we don't know whether this is better than an established way of attacking the same problem. There are no error bars, and lambda is selected on the test sets, which inflates the headline numbers. No code is released, which makes the single-run results harder to trust. The stress-test note makes a fair additional point: L_fia imposes one orthogonality constraint per sample, while identity lives in a high-dimensional subspace, and the recognizer was trained only on 720 FF++ identities. The paper does not show that identities unseen in that training set are actually neutralized. That is a real gap, but it weakens the explanation, not necessarily the empirical result. The gains might come from a useful regularizer and the identity story be partly wrong; the method can still work.

This paper is for anyone working on deepfake detector generalization. It is a short, readable paper that deserves a serious referee. I would send it out with a request for the missing ablation, error bars, and a comparison against [13]. Those are addressable and the core idea is worth testing.

Recommendation: accept for review.

Referee Report

5 major / 6 minor

Summary. The paper proposes FRIDAY, a two-phase training method for deepfake detectors. In Phase 1, a face recognizer with the same backbone (EfficientNet-B3) is trained on 720 FF++ identities using cross-entropy. In Phase 2, the recognizer is frozen and the detector is trained with the usual binary cross-entropy plus a Facial Identity Attenuating loss, L_fia = |cos(z_f, z_d)| (Eq. 6), which encourages the detector embedding to be orthogonal to the recognizer embedding. The authors claim this reduces the detector's reliance on facial identity and improves in-domain and cross-domain performance. Experiments on FF++, Celeb-DF v1/v2, and DFD (Table I) report gains over CapsuleNet, Xception, CViT, and UIA-ViT, with FF++ ACC 95.18% and average cross-domain AUC 84.37%. The paper includes an analysis of the loss weight lambda (Fig. 4) and a qualitative convergence check for identity content (Fig. 2).

Significance. If the central claim is validated, FRIDAY offers a simple and potentially general training regularizer that could be attached to many deepfake detectors, and it directly targets a known generalization problem. The proposed loss is new, the method is straightforward to implement, and the reported numbers are internally consistent. However, the current evidence does not yet establish the claimed mechanism: the identity-reduction effect is measured with the same convergence check used to motivate the problem, the loss weight is tuned on the evaluation sets, and no same-backbone ablation is provided. These gaps currently limit the significance of the contribution.

major comments (5)

[Section III-D, Fig. 4] The optimal lambda is selected by measuring ACC and AUC on the in-domain FF++ test set, and the same test set is then used to report the in-domain results in Table I. This makes the reported FF++ numbers optimistic and does not allow a fair comparison with baselines whose hyperparameters are not tuned on this split. The authors should specify a validation split and report results for a lambda chosen from that split, or at least report the sensitivity of all datasets to lambda over a small grid.
[Table I] There is no ablation of the proposed method with the same EfficientNet-B3 backbone and lambda = 0. All baseline models use different architectures or training schemes, so the observed gains over Xception, CViT, and UIA-ViT could be due to backbone capacity, augmentation, or other training details rather than the Facial Identity Attenuating loss. A same-backbone baseline with and without L_fia is necessary to attribute the improvement to the proposed loss.
[Section II-C, Eq. (6)] L_fia imposes only a single orthogonality constraint (one per sample) between the detector embedding and the frozen recognizer embedding. Facial identity is generally believed to span a high-dimensional subspace, and the paper does not demonstrate that the single recognizer output direction is a sufficient statistic for identity, nor that this direction remains meaningful for unseen identities or for fake images outside the recognizer's training distribution. The identity check in Fig. 2a/4c reports convergence on the datasets used rather than a quantitative identity-leakage measurement on unseen identities. The claimed mechanism therefore is not established; the accuracy gains could come from a generic regularizing effect. Please provide quantitative identity-reduction measurements on unseen identities and compare L_fia against a control orthogonality constraint to an unrelated auxiliary embedding.
[Section III-C and References [13]] The paper's motivation is explicitly based on Dong et al., 'Implicit Identity Leakage' [13], which directly addresses identity leakage in deepfake detectors, but no comparison with that method or its follow-up works is included in Table I. Since this is the most relevant prior method, omitting it leaves the claimed superiority incomplete. A comparison should be added, or the omission should be justified.
[Section III-A and Table I] All results are reported from a single training run with no error bars, multiple seeds, or statistical tests. Given that the reported cross-domain differences are around 1-2% AUC, and the model uses stochastic augmentation and optimization, it is not clear that the improvements are statistically reliable. Please report mean and standard deviation over at least three random seeds.

minor comments (6)

[Section III-D] The sentence 'we concluded that λ = 10is the optimal value' is missing a space; it should read 'λ = 10 is the optimal value'.
[Section III-D, Fig. 4 caption] The text refers to 'as depicted in Fig. 2a' when describing the identity-content check across lambda values, but Fig. 2a is a schematic of the check method; the actual identity-content results appear in Fig. 4c. Please correct the cross-reference.
[Section III-B] The face recognizer architecture is not fully specified: the text mentions a backbone and an MLP classifier, but does not give the hidden layer sizes, embedding dimension, or whether batch normalization is used. This information is needed for reproducibility.
[Section II-B] The recognizer is trained on N = 720 identities 'based on the FF++ dataset', but it is not stated how these are split from the FF++ training set and whether the same identities appear in the detector training set or in the identity check. Please clarify the identity split.
[Abstract and Section I] The claim that this is 'the first time' a face recognizer is used in deepfake detector training is stronger than the evidence provided, given the prior identity-based methods cited in [13] and [14]. Consider softening the novelty claim.
[Section II-A, Fig. 2] Fig. 2b is labeled 'Epoch Loss' but does not state which loss is plotted, on which dataset, or over how many epochs. Without this context, the convergence plot is hard to interpret.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: FRIDAY's objective and cross-domain benchmarks are independent of the paper's inputs.

full rationale

The paper's derivation chain does not reduce to its own inputs. Phase 1 trains a face recognizer with a standard closed-set identity classification loss (Eq. 3) on 720 FF++ identities. Phase 2 freezes that recognizer and trains the detector with BCE plus a Facial Identity Attenuating loss L_fia = |cos(z_f,z_d)| (Eq. 6). The claim that this attenuates facial identity is a causal hypothesis, and the evaluation in Table I uses independent held-out FF++, Celeb-DF, and DFD splits with standard ACC/AUC metrics. No equation or fitted parameter is renamed as a prediction: L_fia is not the evaluation metric, and the recognizer is a fixed input rather than a fitted prediction of cross-domain performance. The identity-convergence probe of Fig. 2a is used both to motivate the problem and, in Fig. 4c, to validate the method, but that probe is a mediating variable; the paper's central result is detection performance on external benchmarks. Reference [13], which supports the 'unintentional identity leakage' premise, is by different authors and is not a load-bearing self-citation. The choice of lambda=10 is a hyperparameter selection on in-domain data and may raise correctness concerns about test-set peeking, but it is not a circular derivation. No specific equation, definition, or citation can be exhibited that makes the claimed result equivalent to its input by construction.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

No new physical or architectural entities are introduced. The method relies on three domain assumptions about identity information and its measurability via cosine similarity, plus a tuned hyperparameter lambda. The single detector training is standard.

free parameters (1)

lambda (Facial Identity Attenuating loss weight) = 10
Selected by sweeping lambda and picking the value with the best ACC/AUC on the same evaluation datasets (FF++, Celeb-DF v1/v2, DFD), as shown in Fig. 4. This makes the reported cross-domain gains partially fitted rather than independent predictions.

assumptions (3)

domain assumption Deepfake detectors unintentionally learn facial identity features during training.
Motivates the whole method; based on citation [13], restated in Section II-A.
domain assumption Absolute cosine similarity between the detector and recognizer embeddings is a valid proxy for the amount of facial identity information in the detector.
Used to define L_fia in Eq. 6; the paper does not prove that orthogonality to the recognizer embedding removes identity information.
domain assumption A face recognizer trained on 720 identities from FF++ generalizes its identity representation to unseen identities in Celeb-DF and DFD.
Phase 1 training uses only FF++ real identities; cross-domain evaluation relies on this generalization.

how reviews work

0 comments

Cite this review

Pith. "Pith review of FRIDAY: Mitigating Unintentional Facial Identity in Deepfake Detectors Guided by Facial Recognizers." pith.science (2026). https://pith.science/paper/LKFTGOVF

@misc{pith2026241214623,
  author       = {Pith},
  title        = {Pith review of: FRIDAY: Mitigating Unintentional Facial Identity in Deepfake Detectors Guided by Facial Recognizers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LKFTGOVF}},
  note         = {Machine review of arXiv:2412.14623}
}

read the original abstract

Previous Deepfake detection methods perform well within their training domains, but their effectiveness diminishes significantly with new synthesis techniques. Recent studies have revealed that detection models often create decision boundaries based on facial identity rather than synthetic artifacts, resulting in poor performance on cross-domain datasets. To address this limitation, we propose Facial Recognition Identity Attenuation (FRIDAY), a novel training method that mitigates facial identity influence using a face recognizer. Specifically, we first train a face recognizer using the same backbone as the Deepfake detector. The recognizer is then frozen and employed during the detector's training to reduce facial identity information. This is achieved by feeding input images into both the recognizer and the detector, and minimizing the similarity of their feature embeddings through our Facial Identity Attenuating loss. This process encourages the detector to generate embeddings distinct from the recognizer, effectively reducing the impact of facial identity. Extensive experiments demonstrate that our approach significantly enhances detection performance on both in-domain and cross-domain datasets.

Figures

Figures reproduced from arXiv: 2412.14623 by the authors.

**Figure 1.** Overview of FRIDAY. During the Deepfake detector training, unintended facial identity learning occurs, causing biased predictions. We address this by pushing the Deepfake detector’s decision boundary away from the face recognizer’s, leveraging its identity extraction capability. in cross-domain datasets where the already limited effective features are further diminished, leading to a performance drop. Therefore, ens… view at source ↗

**Figure 2.** Demonstrating how much FRIDAY reduces face identity information. • We propose a novel training method called FRIDAY, which can be applied to many Deepfake detectors to improve their generalization performances. • For the first time, we use a face recognizer in the training process of Deepfake detectors to attenuate unintentional facial identity embedding. • We demonstrate the superior performance of our method throu… view at source ↗

**Figure 4.** Effect of λ. These figures illustrate the effect of varying λ on FRIDAY. (a) and (b) depict the accuracy and AUC, both of which are highest at λ = 10. (c) shows the amount of face identity using the unintentional face identity check method referenced in Fig. s2a. According to the result, λ = 10 exhibits the lowest inclusion of face identity. fake frames, we used the same augmentations and added an additional Gaussia… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 5 canonical work pages

[13]

Implicit identity leakage: The stumbling block to improving deepfake detection gener- alization,

S. Dong, J. Wang, R. Ji, J. Liang, H. Fan, and Z. Ge, “Implicit identity leakage: The stumbling block to improving deepfake detection gener- alization,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 3994–4004

work page 2023
[1]

Deepfakes,

FaceSwapDevs, “Deepfakes,” https://github.com/deepfakes/faceswap, 2019, accessed: 2024-07-02

work page 2019
[2]

Face2face: Real-time face capture and reenactment of rgb videos,

J. Thies, M. Zollhofer, M. Stamminger, C. Theobalt, and M. Nießner, “Face2face: Real-time face capture and reenactment of rgb videos,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2387–2395

2016
[3]

Faceswap,

M. Kowalski, “Faceswap,” https://github.com/MarekKowalski/Faceswap, 2018, accessed: 2024-07-02

work page 2018
[4]

Faceshifter: To- wards high fidelity and occlusion aware face swapping,

L. Li, J. Bao, H. Yang, D. Chen, and F. Wen, “Faceshifter: To- wards high fidelity and occlusion aware face swapping,” arXiv preprint arXiv:1912.13457, 2019

arXiv 1912
[5]

Faceforensics++: Learning to detect manipulated facial images,

A. Rossler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies, and M. Nießner, “Faceforensics++: Learning to detect manipulated facial images,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 1–11

2019
[6]

Mesonet: a compact facial video forgery detection network,

D. Afchar, V . Nozick, J. Yamagishi, and I. Echizen, “Mesonet: a compact facial video forgery detection network,” in 2018 IEEE international workshop on information forensics and security (WIFS) . IEEE, 2018, pp. 1–7

2018
[7]

Use of a capsule network to detect fake images and videos,

H. H. Nguyen, J. Yamagishi, and I. Echizen, “Use of a capsule network to detect fake images and videos,” arXiv preprint arXiv:1910.12467 , 2019

arXiv 1910

Show all 18 references

[8]

Noise based deepfake detection via multi- head relative-interaction,

T. Wang and K. P. Chow, “Noise based deepfake detection via multi- head relative-interaction,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 12, 2023, pp. 14 548–14 556

2023
[9]

Uia-vit: Unsupervised inconsistency-aware method based on vision transformer for face forgery detection,

W. Zhuang, Q. Chu, Z. Tan, Q. Liu, H. Yuan, C. Miao, Z. Luo, and N. Yu, “Uia-vit: Unsupervised inconsistency-aware method based on vision transformer for face forgery detection,” in European conference on computer vision . Springer, 2022, pp. 391–407

2022
[10]

Face x-ray for more general face forgery detection,

L. Li, J. Bao, T. Zhang, H. Yang, D. Chen, F. Wen, and B. Guo, “Face x-ray for more general face forgery detection,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 5001–5010

2020
[11]

Exposing deepfake videos by detecting face warping artifacts,

Y . Li and S. Lyu, “Exposing deepfake videos by detecting face warping artifacts,” arXiv preprint arXiv:1811.00656 , 2018

2018 arXiv
[12]

Detecting deepfakes with self-blended images,

K. Shiohara and T. Yamasaki, “Detecting deepfakes with self-blended images,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 18 720–18 729

2022
[14]

Protecting celebrities from deepfake with identity consistency transformer,

X. Dong, J. Bao, D. Chen, T. Zhang, W. Zhang, N. Yu, D. Chen, F. Wen, and B. Guo, “Protecting celebrities from deepfake with identity consistency transformer,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 9468–9478

2022
[15]

Deepfake video detection using convolutional vision transformer,

D. Wodajo and S. Atnafu, “Deepfake video detection using convolutional vision transformer,” arXiv preprint arXiv:2102.11126 , 2021

2021 arXiv
[16]

Celeb-df: A large- scale challenging dataset for deepfake forensics,

Y . Li, X. Yang, P. Sun, H. Qi, and S. Lyu, “Celeb-df: A large- scale challenging dataset for deepfake forensics,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 3207–3216

2020
[17]

Deepfakes detection dataset by google & jigsaw,

D. Nicholas, G. Andrew, K. Per, V . V . Alexey, L. Thomas, C. Jeremiah, and B. Christoph, “Deepfakes detection dataset by google & jigsaw,” 2019, available at: https://ai.googleblog.com/2019/ 09/contributing-data-to-deepfake-detection.html

2019
[18]

Dlib-ml: A machine learning toolkit,

D. E. King, “Dlib-ml: A machine learning toolkit,” The Journal of Machine Learning Research , vol. 10, pp. 1755–1758, 2009

2009

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [13]

Implicit identity leakage: The stumbling block to improving deepfake detection gener- alization,

S. Dong, J. Wang, R. Ji, J. Liang, H. Fan, and Z. Ge, “Implicit identity leakage: The stumbling block to improving deepfake detection gener- alization,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 3994–4004

work page 2023

[2] [1]

Deepfakes,

FaceSwapDevs, “Deepfakes,” https://github.com/deepfakes/faceswap, 2019, accessed: 2024-07-02

work page 2019

[3] [2]

Face2face: Real-time face capture and reenactment of rgb videos,

J. Thies, M. Zollhofer, M. Stamminger, C. Theobalt, and M. Nießner, “Face2face: Real-time face capture and reenactment of rgb videos,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2387–2395

2016

[4] [3]

Faceswap,

M. Kowalski, “Faceswap,” https://github.com/MarekKowalski/Faceswap, 2018, accessed: 2024-07-02

work page 2018

[5] [4]

Faceshifter: To- wards high fidelity and occlusion aware face swapping,

L. Li, J. Bao, H. Yang, D. Chen, and F. Wen, “Faceshifter: To- wards high fidelity and occlusion aware face swapping,” arXiv preprint arXiv:1912.13457, 2019

arXiv 1912

[6] [5]

Faceforensics++: Learning to detect manipulated facial images,

A. Rossler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies, and M. Nießner, “Faceforensics++: Learning to detect manipulated facial images,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 1–11

2019

[7] [6]

Mesonet: a compact facial video forgery detection network,

D. Afchar, V . Nozick, J. Yamagishi, and I. Echizen, “Mesonet: a compact facial video forgery detection network,” in 2018 IEEE international workshop on information forensics and security (WIFS) . IEEE, 2018, pp. 1–7

2018

[8] [7]

Use of a capsule network to detect fake images and videos,

H. H. Nguyen, J. Yamagishi, and I. Echizen, “Use of a capsule network to detect fake images and videos,” arXiv preprint arXiv:1910.12467 , 2019

arXiv 1910