Pith. sign in

REVIEW 4 major objections 5 minor 21 references

Is it the model or the metric -- On robustness measures of deeplearning models

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read Robust accuracy alone hides how deepfake detectors differ

desk verdict A fair point about robust accuracy's blind spot, but the empirical RR doesn't match the defined RR (attacks vs. universal quantifier, missing b), so the headline claim isn't established. read the letter →

arxiv 2412.09795 v1 pith:G364AQYB submitted 2024-12-13 cs.LG cs.CV

classification cs.LGcs.CV
keywords robustaccuracyratiodeepfakedetectionadversarialrobustnessmodelevaluationlocalperturbationtolerancedeeplearningsafety
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that robust accuracy (RA), the standard metric for adversarial robustness, is insufficient to compare deep-learning models for deepfake detection. The authors introduce robust ratio (RR), which measures the fraction of samples for which an input perturbation with bounded L2 norm leaves the model's probability output within a fixed tolerance. Experiments across three deepfake detectors and three attack types show that models with almost identical RA can have clearly different RR under the same perturbation tolerances. The conclusion is that RA alone hides information about how stably a model's output probability behaves, and RR should be used as a complementary metric in model selection.

What carries the argument

The robust ratio, defined in Eqs. 2 and 3, is the fraction of test samples satisfying a local-robustness property borrowed from formal verification: the classifier's probability for its predicted class moves by at most b over the entire L2 ball of radius ε around the input. This object carries the argument because it converts robustness into a per-sample stability check on probability outputs rather than a per-sample label check, which is why it can differ between models with the same RA.

What would settle it

Compute RR exactly for a small set of samples by verifying the local-robustness condition across the full L2 ball (for example through formal verification or dense sampling) and compare it with the attack-based RR. If two models have different attack-based RR but identical verified RR, or vice versa, the empirical core of the paper's claim would be undercut.

Watch

Extended reading notes

Core claim

The central claim is that RA measures only whether the predicted label survives a perturbation, while ignoring how much the continuous probability output shifts. RR captures that shift: a sample counts as robust when every perturbation δ with ∥δ∥2 ≤ ε changes the probability of the predicted class by no more than a bound b. Comparing three models (Meso4, Meso4Inception, ResNet34) under FGSM, PGD, and CW attacks on three deepfake datasets, the paper finds RA nearly identical across models while RR varies substantially and depends on the tolerance b. This is presented as evidence that RA is not sufficient and that RR reveals model-specific regions of stable operation.

Load-bearing premise

Eq. 2 defines RR over all perturbations within an L2 ball, but the experiments evaluate only FGSM, PGD, and CW attacks, so the claim that the measured RR represents the defined RR rests on the unstated assumption that these three attacks stand in for every bounded perturbation.

Editorial extensions

If this is right

  • Model selection for deepfake detection should report RR alongside RA to avoid choosing between models that appear equally robust by accuracy only.
  • RR's dependence on tolerance b gives a way to define an application-specific safe operating region, such as the perturbation level up to which a model's output probability stays trustworthy.
  • Existing robustness comparisons that rely solely on RA may be incomplete, since they miss probability-level instability that RR detects.
  • For video inputs, the observed roughly linear RR-versus-tolerance relationship suggests that temporal attack accumulation could be predictable from RR curves.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial: The same RA-versus-RR divergence should appear in any classification task where decisions are made from probability outputs, not just deepfake detection, because the mechanism is generic to bounded perturbations of continuous outputs.
  • Editorial: A natural testable extension is whether models with higher RR also show better output calibration or lower disagreement under noise in deployment; this would connect RR to practical reliability.
  • Editorial: The paper's attack-based evaluation could be validated or challenged by computing RR exactly on a small subset using formal verification of the L2 ball; if attack-based RR and verified RR diverge, the empirical claims would need to be reinterpreted.
  • Editorial: The video linearity hypothesis could be isolated by ablating how attacks are applied across frames; confirming it would let RR be predicted from frame-level perturbation schedules.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper argues that robust accuracy (RA) alone is insufficient to characterize the robustness of deepfake-detection models, and proposes a complementary metric, robust ratio (RR), defined via a formal-verification-style universal condition on the stability of the model's probability output under L2-bounded perturbations. Experiments on image and video data with Meso4, Meso4Inception, and ResNet34 under FGSM, PGD, and CW attacks are reported, and the authors claim that models with similar RA can nevertheless show different RR values as a function of tolerance.

Significance. If the proposed metric were rigorously validated, RR would be a useful addition to the robustness-evaluation toolbox, particularly for high-stakes applications where probability shifts below the misclassification threshold still matter. The link to formal verification (Eq. 2) is conceptually sound and gives the metric a clean definition. The observation that RA can miss output-distribution instability is worth making, and the paper explicitly frames RR as complementary rather than replacement, which is appropriate. However, as presented, the empirical demonstration does not measure the defined quantity, and the missing output-bound parameter prevents reproducibility. The central claim therefore is not yet established, though it is plausibly fixable.

major comments (4)
  1. [Section 3, Adversarial Attacks; Eq. (2)-(3)] Eq. (2) defines robustness with a universal quantifier over all δ with ‖δ‖₂ ≤ ε, and Eq. (3) defines RR as the fraction of samples satisfying that universal condition. Section 3 computes this fraction only for perturbations produced by FGSM, PGD, and CW. These attacks generate a sparse set of adversarial directions and provide no certificate that the property holds for every δ in the L2 ball, so the plotted RR values are not the RR defined in Eq. (3) unless an explicit approximation statement is added. The manuscript never states that the reported quantity is an attack-based estimate or a lower bound. Please either provide a certified verification (e.g., Lipschitz-based bounds) or explicitly rename the measured quantity to 'attack-based RR' and discuss the gap to Eq. (2).
  2. [Section 3, Tolerance Settings; Eq. (2)] Eq. (2) requires two tolerance parameters: the input perturbation radius ε and the output deviation bound b. The Tolerance Settings paragraph lists only δ = {0, .01, .02, ..., 0.2} and describes it as 'tolerance (proportional to the bound)', but the value of b is never specified anywhere in the paper. Without b, the condition |P(Y|x_i+δ,w_i)(ŷ) − P(Y|x_i,w_i)(ŷ)| ≤ b is undefined, and the RR values in Fig. 1 cannot be reproduced, interpreted, or compared across models. Please specify both ε and b for every experiment, or provide the exact mapping from the listed δ values to (ε, b).
  3. [Section 2, Eq. (1)] Eq. (1) defines RA as (1/N) Σ 𝟙(ŷ_i = y_i), which is ordinary accuracy and contains no dependence on adversarial perturbation δ. The surrounding text, however, describes RA as the proportion of predictions that 'remain correct under an adversarial perturbation δ'. As written, Eq. (1) does not measure robustness at all; RA must be defined with respect to perturbed inputs (e.g., ŷ_i = f(x_i + δ)) and the attack/ε used must be stated. This error affects the main comparison framework, since the claim that RA shows similar performance across models is currently a claim about standard accuracy, not adversarial robustness.
  4. [Section 3, Datasets; Section 4] The video dataset consists of only 5 videos, and no error bars, confidence intervals, or repeated runs are reported for any of the RA or RR results. With n=5, the observed 'more linear relationship' of RR with tolerance for video data (Section 4) and the image-vs-video comparison are not statistically supported. At minimum, add error bars or confidence intervals, and either substantially increase the video sample size or temper the conclusions drawn from the video data.
minor comments (5)
  1. [Section 3, Tolerance Settings] The notation δ is used both for the adversarial perturbation in Eq. (2) and for the tolerance level in Section 3; please use distinct symbols (e.g., τ for tolerance) to avoid confusion.
  2. [Section 4, first paragraph] The phrase 'RA, function of only the attack type' is inaccurate: RA also depends on the model, the dataset, and the perturbation bound; please rephrase to reflect the empirical observation that RA is similar across the tested models.
  3. [Figure 1] The caption does not explain how RA is aggregated across the three attack methods (FGSM, PGD, CW); please state whether RA is averaged, shown separately, or computed with a specific attack, and describe the curves and markers clearly.
  4. [Section 3, Adversarial Attacks] The attack parameters (e.g., number of PGD iterations, step size, CW confidence, number of restarts) are not reported; without these, the RR computation is not reproducible. Please add the full attack configurations.
  5. [Introduction, contributions] The paper is described as an extended abstract, but it presents a new metric and empirical results; a short 'limitations' paragraph acknowledging the attack-based approximation and the small video dataset would improve clarity and honesty.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: RR is a newly defined metric with an externally cited robustness condition, and the RA-versus-RR comparison is an empirical measurement, not a derivation that reduces to its inputs.

full rationale

The paper does not present a derivation chain that reduces to its own inputs. Robust ratio (RR) is explicitly defined in Eqs. (2) and (3) as the fraction of samples satisfying a local robustness condition borrowed from formal verification and attributed to reference [13], which is not authored by the present authors. No parameter is fitted to data and then renamed as a prediction; the tolerance levels are chosen evaluation settings, not fitted values. The central claim that models with similar robust accuracy can differ in RR is an empirical observation about two distinct quantities, and the difference is by construction only in the sense that RA and RR are defined to measure different aspects of model behavior. The formal robustness definition is imported from an external citation, which creates a dependency but not circularity. The gap between the universal quantifier in Eq. (2) and the attack-based evaluation in Section 3 is a validity and reproducibility concern, not a circularity concern, because the reported RR is not claimed to be derived from the attacks by definition; it is measured on them. No self-citation is load-bearing, no uniqueness theorem is imported from the authors' own prior work, and no known result is merely renamed. The paper is an empirical metric-proposal and comparison, and its reasoning does not collapse into its definitions or into a self-citation chain.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim rests on two hand-chosen threshold parameters and on the unstated assumption that a few attack algorithms can stand in for the universal quantifier in the robustness definition. No new physical or conceptual entities are introduced.

free parameters (2)
  • Perturbation bound (epsilon) = Set to 0, 0.01, ..., 0.2
    The tolerance levels at which robust ratio is evaluated are chosen by hand; the paper does not explain how these values map to the epsilon in Eq. 2 and whether they are attack-specific.
  • Output deviation bound (b) = Not specified
    The bound on the allowed change in predicted-class probability appears as b in Eq. 2, but the experimental section only mentions 'tolerance (proportional to the bound)' without giving concrete values for b.
assumptions (3)
  • domain assumption Robustness is defined as a bounded change in the probability of the predicted class under any L2-bounded input perturbation (Eq. 2, after [13]).
    The paper adopts this formal-verification definition as the basis for RR without questioning its appropriateness for measuring model behavior in deployment.
  • domain assumption Model output probabilities are meaningful and comparable across samples and models.
    RR relies on absolute differences in P(Y|x)(y_hat); if probabilities are poorly calibrated, the ratio loses interpretability.
  • ad hoc to paper Adversarial examples from FGSM, PGD, and CW represent all perturbations within the L2 ball in Eq. 2.
    Eq. 2 requires universal quantification, but the experiments only evaluate on attack-generated samples; this approximation is not stated in the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Is it the model or the metric -- On robustness measures of deeplearning models." pith.science (2026). https://pith.science/paper/G364AQYB

@misc{pith2026241209795,
  author       = {Pith},
  title        = {Pith review of: Is it the model or the metric -- On robustness measures of deeplearning models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G364AQYB}},
  note         = {Machine review of arXiv:2412.09795}
}
read the original abstract

Determining the robustness of deep learning models is an established and ongoing challenge within automated decision-making systems. With the advent and success of techniques that enable advanced deep learning (DL), these models are being used in widespread applications, including high-stake ones like healthcare, education, border-control. Therefore, it is critical to understand the limitations of these models and predict their regions of failures, in order to create the necessary guardrails for their successful and safe deployment. In this work, we revisit robustness, specifically investigating the sufficiency of robust accuracy (RA), within the context of deepfake detection. We present robust ratio (RR) as a complementary metric, that can quantify the changes to the normalized or probability outcomes under input perturbation. We present a comparison of RA and RR and demonstrate that despite similar RA between models, the models show varying RR under different tolerance (perturbation) levels.

Figures

Figures reproduced from arXiv: 2412.09795 by the authors.

Figure 1
Figure 1. Comparison of a) Robust ratio vs. Tolerance on left-y-axis, and b) robust accuracy on right-y-axis. obtain the numerator for RR (Eq. 3). It captures the proportion of samples where small perturbations in the input, δ, result in a bounded change in the normalized output or the probability. The robust￾ness of a model ϕ i rob, with weight matrix wi over input xi , holds when: ϕ i rob(wi , xi) ≜ ∀δ : ∥δ∥2 ≤ ϵ ⇒ |P(Y | x… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 19 canonical work pages

  1. [1]

    Explaining and harnessing adversarial ex- amples

    I. J. Goodfellow, J. Shlens, and C. Szegedy. “Explaining and harnessing adversarial ex- amples”. In: arXiv preprint arXiv:1412.6572 (2014)

  2. [2]

    Adversarial Robustness of Convolutional Models Learned in the Frequency Domain

    S. Chaudhury and T. Yamasaki. “Adversarial Robustness of Convolutional Models Learned in the Frequency Domain”. In: ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE. 2024, pp. 7455–7459

  3. [3]

    Noise-Resistant Graph Neural Network for Node Classification

    Z. Deng and H. Yu. “Noise-Resistant Graph Neural Network for Node Classification”. In: ICASSP 2024-2024 IEEE International Con- ference on Acoustics, Speech and Signal Pro- cessing (ICASSP). IEEE. 2024, pp. 7560–7564

  4. [4]

    Un- derstanding Data Augmentation From A Ro- bustness Perspective

    Z. Liu, J. Zhang, Q. He, and C. Wang. “Un- derstanding Data Augmentation From A Ro- bustness Perspective”. In: ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE. 2024, pp. 6760–6764

  5. [5]

    Fixed Inter-Neuron Covariability Induces Adversarial Robustness

    M. A. Shah and B. Raj. “Fixed Inter-Neuron Covariability Induces Adversarial Robustness”. In: ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE. 2024, pp. 7005– 7009

  6. [6]

    Federated Learning under Restricted user Availability

    P. Theodoropoulos, K. E. Nikolakakis, and D. Kalogerias. “Federated Learning under Restricted user Availability”. In: ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE. 2024, pp. 7055–7059

  7. [7]

    Noise-BERT: A Unified Perturbation-Robust Framework with Noise Alignment Pre-Training for Noisy Slot Fill- ing Task

    J. Zhao, G. Dong, Y. Qiu, T. Hui, X. Song, D. Guo, and W. Xu. “Noise-BERT: A Unified Perturbation-Robust Framework with Noise Alignment Pre-Training for Noisy Slot Fill- ing Task”. In: ICASSP 2024-2024 IEEE In- ternational Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE. 2024, pp. 6150–6154

  8. [8]

    Video and audio deepfake datasets and open issues in deepfake technology: being ahead of the curve

    Z. Akhtar, T. L. Pendyala, and V. S. Ath- makuri. “Video and audio deepfake datasets and open issues in deepfake technology: being ahead of the curve”. In: Forensic Sciences 4.3 (2024), pp. 289–377

Show all 21 references
  1. [9]

    Ahern, I

    D. Ahern, I. Bratko, S. Das, S. Delacroix, L. Kastner, L. Fetic, T. Klein, D. Lewis, A. Mazumder, P. Molnar, L. Nieper, and A. Faisal. The Speinshart Recommendations on Generative AI and the EU AI Act . Tech. rep. Imperial College London, Sept. 2023. url: https://doi.org/10.25...

  2. [10]

    E. Union. Regulation (EU) 2024/1689 of the European Parliament and of the Council on harmonised rules on Artificial Intelligence (AI Act) . https://eur-lex.europa.eu/legal- content/EN/TXT/?uri=CELEX:32024R1689. [Online; accessed September-2024]. 2024

  3. [11]

    Deepfake detection: A systematic lit- erature review

    M. S. Rana, M. N. Nobi, B. Murali, and A. H. Sung. “Deepfake detection: A systematic lit- erature review”. In: IEEE access 10 (2022), pp. 25494–25513

  4. [12]

    A comprehensive evaluation framework for deep model robustness

    J. Guo, W. Bao, J. Wang, Y. Ma, X. Gao, G. Xiao, A. Liu, J. Dong, X. Liu, and W. Wu. “A comprehensive evaluation framework for deep model robustness”. In: Pattern Recognition 137 (2023), p. 109308

  5. [13]

    Bridging formal methods and machine learning with model checking and global optimisation

    S. Bensalem, X. Huang, W. Ruan, Q. Tang, C. Wu, and X. Zhao. “Bridging formal methods and machine learning with model checking and global optimisation”. In: Journal of Logical and Algebraic Methods in Programming 137 (2024), p. 100941

  6. [14]

    Are adversarial robustness and common perturba- tion robustness independant attributes?

    A. Laugros, A. Caplier, and M. Ospici. “Are adversarial robustness and common perturba- tion robustness independant attributes?” In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops . 2019

  7. [15]

    Exposing deep fakes using inconsistent head poses

    X. Yang, Y. Li, and S. Lyu. “Exposing deep fakes using inconsistent head poses”. In: ICASSP 2019-2019 IEEE International Con- ference on Acoustics, Speech and Signal Pro- cessing (ICASSP). IEEE. 2019, pp. 8261–8265

  8. [16]

    Faceforen- sics++: Learning to detect manipulated facial images

    A. Rossler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies, and M. Nießner. “Faceforen- sics++: Learning to detect manipulated facial images”. In: Proceedings of the IEEE/CVF international conference on computer vision . 2019, pp. 1–11

  9. [17]

    Celeb-df (v2): a new dataset for deepfake foren- sics [j]

    Y. Li, X. Yang, P. Sun, H. Qi, and S. Lyu. “Celeb-df (v2): a new dataset for deepfake foren- sics [j]”. In: arXiv preprint arXiv (2019). 3

  10. [18]

    Mesonet: a compact facial video forgery detection network

    D. Afchar, V. Nozick, J. Yamagishi, and I. Echizen. “Mesonet: a compact facial video forgery detection network”. In: 2018 IEEE international workshop on information foren- sics and security (WIFS) . IEEE. 2018, pp. 1– 7

  11. [19]

    ResNet 34

    B. Koonce and B. Koonce. “ResNet 34”. In: Convolutional neural networks with swift for tensorflow: image recognition and dataset cat- egorization (2021), pp. 51–61

  12. [20]

    Towards deep learning models resistant to adversarial attacks

    A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. “Towards deep learning models resistant to adversarial attacks”. In: arXiv preprint arXiv:1706.06083 (2017)

  13. [21]

    Towards evalu- ating the robustness of neural networks

    N. Carlini and D. Wagner. “Towards evalu- ating the robustness of neural networks”. In: 2017 ieee symposium on security and privacy (sp). Ieee. 2017, pp. 39–57. 4

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.