Pith. sign in

REVIEW 5 major objections 5 minor 96 references

A Test Suite for Efficient Robustness Evaluation of Face Recognition Systems

T0 review · 5 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read ROBFACE estimates the robustness of unseen face recognition systems from one fixed set of transferable adversarial face images, matching PGD-based rankings at about 200x lower evaluation time.

desk verdict A useful and reproducible test suite for face recognition robustness, but the system-agnostic generalization claim rests on one thin split and needs stronger validation. read the letter →

arxiv 2504.21420 v1 pith:AFTRRIOZ submitted 2025-04-30 cs.SE

Ruihan Zhang , Jun Sun This is my paper · ORCID
classification cs.SE
keywords facerecognitionrobustnessevaluationadversarialexamplestransferabilitytestsuitesystem-agnostictestingperturbationblack-box
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the robustness of a face recognition system can be estimated without running any attack at evaluation time: by simply measuring how many pre-crafted, transferable adversarial face images the system classifies correctly. It introduces ROBFACE, a test suite in which each perturbation type has an optimised subset of adversarial samples, and the fraction of samples that flip a system's prediction becomes the robustness score. Across nine face recognition systems and eight perturbation types, these scores correlate with PGD-based robust accuracy (Pearson 0.90 to 0.99) and, on p-norm perturbations, with CLEVER, while taking about 200 times less runtime than either reference. If this holds, developers gain a fast, black-box, architecture-agnostic way to rank candidate systems before deployment and to re-check robustness after every model update. The load-bearing assumption is that adversarial perturbations transfer across face recognition systems.

What carries the argument

The central object is the test suite $\mathcal{T}=\{T_\gamma\}$, one subset $T_\gamma$ per perturbation scheme $(d,\epsilon)$, where each element is an ordered triple $(x,x',y)$ containing the original input, a carefully perturbed input, and the correct label. The suite is built in two phases: ROBFACE-GEN generates, by gradient descent starting from random perturbation sizes, a large pool of candidates that span a range of prediction losses; then a constrained binary optimisation selects a small subset whose per-system misclassification counts maximise the Pearson correlation with known PGD robustness on a tuning group, regularised to keep the selected examples strong and spread out. At evaluation time, the robustness estimate is simply the fraction of triples in $T_\gamma$ on which the system predicts the same label as $y$.

What would settle it

Run ROBFACE-01 on a face recognition system that is architecturally distant from the nine systems studied, for example a vision-transformer-based encoder not seen during tuning, and compare its predicted robustness ranking to PGD robust accuracy on the same eight perturbations. If the Pearson or Spearman correlation falls substantially below 0.90, or the ranking reverses on several perturbation dimensions, the system-agnostic generality claim is refuted.

Watch

Extended reading notes

Core claim

The central discovery is that robustness evaluation for face recognition can be reduced to a single forward pass over a pre-optimised test suite. The paper constructs ROBFACE-01, covering eight perturbation spaces, by generating candidate adversarial examples with a gradient-descent process and then solving a binary subset-selection problem that maximises the Pearson correlation between the suite's per-system score and PGD-based robust accuracy on a tuning group. Across nine face recognition systems, with five randomly selected for tuning and the rest held out, the resulting scores correlate with PGD robust accuracy between 0.90 and 0.99 per perturbation dimension, and runtime drops from thousands of seconds to tens of seconds. ROBFACE is presented as the first search-free, system-agnostic robustness test suite, applicable also to perturbations such as glasses, masks, illumination, age, pose, and radial distortion that formal Lipschitz-based methods cannot handle.

Load-bearing premise

The suite's estimates are trustworthy only if adversarial examples that fool a handful of tuning systems also, on average, fool unseen face recognition systems with different architectures, training sets, and loss functions; the paper assumes this transferability rather than proving it for the whole population of face recognition systems.

Editorial extensions

If this is right

  • A developer can re-evaluate robustness after every model update in minutes by running the model once on ROBFACE-01, without configuring an attack or accessing gradients.
  • Because the suite is system-agnostic, the same pre-optimised samples can rank robustness across heterogeneous face recognition systems, making fair comparison between versions and vendors straightforward.
  • The suite covers eight perturbation dimensions, with four of them outside the reach of the formal reference method, so robustness under natural transformations such as glasses, masks, illumination, age, pose, and radial distortion can be measured where theoretical bounds are unavailable.
  • Rotation of a secret seed lets an evaluator generate alternative suite instances, so a developer who overfits to the public ROBFACE-01 sees its score collapse from 0.98 to 0.52 on a re-seeded suite.
  • The same subset-optimisation methodology can be applied to build robustness test suites for tasks other than face recognition, since the procedure only requires pair inputs, a binary label, and a reference attack.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the transferability premise suggests that the suite's validity is population-dependent, so a prudent deployment would periodically regenerate the suite against a fresh random sample of contemporary systems rather than relying on one fixed instance forever.
  • Editorial inference: because ROBFACE correlates strongly with PGD but less strongly with CLEVER on p-norm perturbations, practitioners should treat ROBFACE scores as an empirical robustness proxy rather than a formal certificate; worst-case guarantees would still require verification.
  • Editorial inference: a natural next experiment is to extend the validation well beyond nine systems, including architecturally distant encoders such as vision-transformer-based models, to test whether the reported 0.90 to 0.99 correlations persist when the face-recognition architecture distribution shifts.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces ROBFACE, a pre-optimized test suite of transferable adversarial face images intended to estimate the robustness of face recognition systems without per-system iterative search. The suite covers eight perturbation types; each category is generated by a gradient-based algorithm (ROBFACE-GEN) and then subsampled by a genetic algorithm whose objective maximizes the Pearson correlation with PGD-based robust accuracy on a tuning group of face recognition systems. The authors claim that the resulting fixed suite is system-agnostic, consistent with empirical and formal references, and more than 200 times faster. Experiments on nine face recognition systems report correlations of 0.90 to 0.99, an ablation, an efficiency comparison, and a randomization scheme against adaptive overfitting.

Significance. The artifact is valuable: ROBFACE-01, the code, and raw results are publicly released; the evaluation procedure is simple to apply; and the speedup is substantial. The perturbation coverage beyond p-norm is a genuine practical advantage. However, the central generalization claim currently rests on weak evidence: the tuning-group correlations are partly constructed by the optimization objective, and the held-out validation uses a single split with only three systems. With stronger validation, such as multiple splits and confidence intervals, the contribution would be a solid engineering benchmark; as written, the headline claims outrun the evidence.

major comments (5)
  1. [Section III.B, Eq. (12)] The optimization objective as written is inconsistent with the prose. Equation (12) minimizes phi1, but phi1 is defined as the Pearson correlation coefficient; minimizing it drives the correlation toward -1, whereas the text states the objective 'maximises the Pearson correlation coefficient'. Similarly, phi3 = Mean - lambda2*Std is added with a positive lambda1, but the text says the regularization should maximize the averaged value and the standard deviation, which would require subtracting the term. This is a load-bearing correctness issue because the entire construction of the suite depends on this objective. Please correct the sign (likely minimize -phi1 - lambda1*phi3, or define phi1 as negative correlation), or clarify the intended formulation.
  2. [Section IV.B, RQ1 and RQ2] The RQ1 correlations of 0.95 to 0.99 are computed on the tuning systems after the subset has been optimized to maximize correlation with the reference on exactly those systems; these numbers should be presented as fitting quality, not as independent evidence of consistency. The independence of the claim rests entirely on RQ2, but RQ2 uses one random split with only three held-out systems, and no confidence intervals or significance tests are reported. With n=3, a Pearson correlation is extremely noisy, and the paper's own ablation (Figure 6) shows that without regularization the tuning correlation is 1.0 while the testing correlation drops to -0.83, demonstrating that the optimization can strongly overfit. A single favorable regularized split is not sufficient to support a population-level 'system-agnostic' claim. Please provide repeated random splits (or leave-one-out evaluation), report confidence intervals, and consider tempering the generalization claim to the systems studied.
  3. [Section IV.A and Section IV.B] The number of held-out systems is internally inconsistent. Section IV.A states that a tuning group of five systems is randomly selected and 'the test group with the remaining systems' is used in RQ2. Section IV.B (RQ2) then says 'a testing group consists of the remaining three systems'. Since Table I lists nine systems, five tuning systems should leave four testing systems. Please state the exact composition of the tuning and testing groups and resolve this discrepancy, because the reported testing-group correlation depends on which three (or four) systems are held out.
  4. [Section IV.A, Table III] The computation of the PGD reference for non-p-norm perturbations is not specified. Table III marks Reference-1 as applicable to glasses, mask, illumination, radial, age, and pose perturbations, but the PGD algorithm described in Section II is defined for norm-constrained perturbations. The necessary extension of PGD to these transformation-based perturbations is not given. Because all correlations are calibrated against this reference, the missing description threatens both reproducibility and the validity of the robustness values being predicted. Please specify the reference algorithm used for each non-p-norm perturbation, or state where this is defined.
  5. [Section IV.B, RQ1] The reported CLEVER correlations of 0.48 and 0.63 are far below the 0.90 to 0.99 range reported for Reference-1, yet the paper's central claim includes consistency with 'formal analysis' (Section I). The text does not state over which systems, which perturbation types, or which tuning/testing groups these CLEVER correlations are computed, nor does it explain why they are not considered to contradict the consistency claim. Please clarify these numbers and, if appropriate, limit the consistency claim to the empirical PGD reference or qualify it accordingly.
minor comments (5)
  1. [Section IV.C] The phrase 'subjective to adaptive attacks' should be 'subject to adaptive attacks'.
  2. [Section I and RQ3] The contribution statement says 'more than 200 times faster' while RQ3 says 'around 200 times'; please report a single aggregate speedup ratio computed from Table IV so the claim is reproducible.
  3. [Section IV.B, RQ1] The sentence 'the correlation with each reference was higher than the within-reference correlation' is never defined or supported; please define the within-reference correlation and report it, or remove the sentence.
  4. [Figure 5] The caption says testing systems are underlined, but the group assignment is not verifiable from the text given the 5/3 split ambiguity; please make the tuning/testing membership explicit, for example by listing the system indices in a table.
  5. [Example 2, Section III.A] The phrase 'Here, n = 6.254' appears to be a typo; the input dimension should be an integer (75,264 as stated later), so please correct or remove the value.

Circularity Check

1 steps flagged · score 4.0 of 10

RQ1 consistency is partly constructed by the Eq. (12) Pearson-correlation objective; held-out RQ2 gives independent support, so circularity is partial.

  1. fitted input called prediction [Section III-B, Eq. (12), 'Test Set from Subset Optimisation'; Section IV-B, RQ1.]
    "This set selection maximises the Pearson correlation coefficient between the reference robustness and the robustness obtained from evaluating the face recognition systems on the selected set. ... In this research question, we consider that we can tune our test suite (Section III-B, optimisation 12) according to reference."

    The reference robustness r_gamma(h) in Eq. (12) is the PGD robust accuracy of the tuning systems, and phi2(z,h) is the suite robustness on the same systems. The genetic algorithm selects z to maximize exactly their Pearson correlation. RQ1 then reports 'for the training group the correlation between ROBFACE result and reference-1 ranges from 0.95 to 0.99' as evidence that ROBFACE is consistent with existing approaches. That number is the fitted objective value on the training systems, so it is an artifact of the optimization rather than an independent confirmation. The genuinely independent evidence is RQ2's held-out testing group (three systems, 0.95 +/- 0.05), which prevents the central system-agnostic claim from being fully circular.

full rationale

The only load-bearing circularity is the training-group consistency reported in RQ1: the suite is explicitly optimized, via Eq. (12), to maximize Pearson correlation with the reference robustness on the tuning systems, so high correlation on those same systems is a fitted outcome. The paper itself labels RQ1 'Goodness of Fit', which concedes this point. The central 'system-agnostic' claim, however, is supported by RQ2, which evaluates the pre-optimized suite on held-out systems and reports independent correlations of 0.90-0.99; this is a genuine generalization test, albeit with only three held-out systems. The transferability premise is cited from external prior work [17]-[19], not from a self-citation chain. The only self-citation ([30], for an equivalent definition of robustness) is definitional and not load-bearing. Efficiency, diversity, and adaptive-attack experiments are likewise independent of the fitting objective. Thus the paper has partial circularity in one validation step, but the central system-agnostic claim retains independent content.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests on four untested domain assumptions: transferability of adversarial perturbations, PGD as a valid robustness reference, representativeness of the nine selected systems, and the realism of the chosen perturbation models. The construction also uses several unreported free parameters, especially the regularization weights and subset-size bounds in the optimization objective, plus GA settings. No new physical or conceptual entities are introduced.

free parameters (6)
  • lambda1, lambda2
    Positive regularization coefficients in Eq. (12) that balance correlation against mean and standard deviation; values are not reported.
  • kmin, kmax
    Bounds on the number of selected test cases in Eq. (12); values are not reported.
  • N (generation steps) = 100
    Number of iterations in Algorithm 1, ROBFACE-GEN; chosen by the authors.
  • eta (step size)
    Step size in Algorithm 1; the paper says it depends on the perturbation setting, but exact values are not given.
  • Ktune = 5
    Number of tuning face recognition systems used in the subset optimization; a hyperparameter.
  • GA settings = 1000 iterations
    Genetic algorithm settings via pymoo, including 1000 iterations; population size and mutation parameters are not reported.
assumptions (5)
  • domain assumption Adversarial perturbations transfer across face recognition systems.
    The method is explicitly based on transferability (Section I, Figure 1); prior work is cited, but no proof or broad empirical survey is provided.
  • domain assumption PGD robust accuracy is a valid approximation of true robustness.
    PGD is used as the reference in Eq. (12) and in RQ1; the threats-to-validity section admits that adversarial attacks are only an approximation of true robustness.
  • domain assumption The nine face recognition systems in Table I are representative of the space of face recognition systems.
    The generalization claim relies on this; only three or four held-out systems are used to validate, all drawn from the same table.
  • domain assumption The perturbation parameterizations (glasses, mask, pose, age, illumination, radial) capture real-world perturbations.
    The perturbation spaces in Figure 4 and Table III are taken from the literature, but no validation shows these bounds match real-world deployment conditions.
  • domain assumption A single random tuning-test split is sufficient to estimate generalization.
    The paper uses one random split of five tuning systems and a small number of testing systems; no repeated or cross-validated splits are reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Test Suite for Efficient Robustness Evaluation of Face Recognition Systems." pith.science (2026). https://pith.science/paper/AFTRRIOZ

@misc{pith2026250421420,
  author       = {Pith},
  title        = {Pith review of: A Test Suite for Efficient Robustness Evaluation of Face Recognition Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AFTRRIOZ}},
  note         = {Machine review of arXiv:2504.21420}
}
read the original abstract

Face recognition is a widely used authentication technology in practice, where robustness is required. It is thus essential to have an efficient and easy-to-use method for evaluating the robustness of (possibly third-party) trained face recognition systems. Existing approaches to evaluating the robustness of face recognition systems are either based on empirical evaluation (e.g., measuring attacking success rate using state-of-the-art attacking methods) or formal analysis (e.g., measuring the Lipschitz constant). While the former demands significant user efforts and expertise, the latter is extremely time-consuming. In pursuit of a comprehensive, efficient, easy-to-use and scalable estimation of the robustness of face recognition systems, we take an old-school alternative approach and introduce RobFace, i.e., evaluation using an optimised test suite. It contains transferable adversarial face images that are designed to comprehensively evaluate a face recognition system's robustness along a variety of dimensions. RobFace is system-agnostic and still consistent with system-specific empirical evaluation or formal analysis. We support this claim through extensive experimental results with various perturbations on multiple face recognition systems. To our knowledge, RobFace is the first system-agnostic robustness estimation test suite.

Figures

Figures reproduced from arXiv: 2504.21420 by the authors.

Figure 2
Figure 2. Displayed above are three face images, each with [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Differences in robustness estimating space. The left shows the process of evaluating robustness using a test suite ( [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Illustration of various face image perturbations, [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: We show that the proposed approach, ROBFACE, correlates well with the reference robust accuracies, with Pearson correlation ranging from 0.9 to 0.99. A random group of face systems is used for tuning, and the rest are for testing. Indices of the testing systems are und…
Figure 6
Figure 6. Figure 6: Ablation on the proposed methods. We take glasses [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: The tested robustness of face recognition system [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

96 extracted references · 52 canonical work pages

  1. [1]

    S. Z. Li and A. K. Jain, Eds., Handbook of Face Recognition, 2nd Edition. Springer, 2011. [Online]. Available: https://doi.org/10.1007/ 978-0-85729-932-1

    2011

  2. [2]

    Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition,

    M. Sharif, S. Bhagavatula, L. Bauer, and M. K. Reiter, “Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition,” in Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security , ser. CCS ’16. New York, NY , USA: Association for Computing Machinery, 2016, pp. 1528–1540. [Online]. Available: https://...

    arXiv 2016

  3. [3]

    Presentation attack detection methods for face recognition systems: A comprehensive survey,

    R. Ramachandra and C. Busch, “Presentation attack detection methods for face recognition systems: A comprehensive survey,” ACM Comput. Surv. , vol. 50, no. 1, mar 2017. [Online]. Available: https://doi.org/10.1145/3038924

    doi:10.1145/3038924 2017

  4. [4]

    Light can hack your face! black-box backdoor attack on face recognition systems,

    H. Li, Y . Wang, X. Xie, Y . Liu, S. Wang, R. Wan, L. Chau, and A. C. Kot, “Light can hack your face! black-box backdoor attack on face recognition systems,” CoRR, vol. abs/2009.06996, 2020. [Online]. Available: https://arxiv.org/abs/2009.06996

    arXiv 2009

  5. [5]

    Robust heterogeneous discriminative analysis for face recognition with single sample per person,

    M. Pang, Y . ming Cheung, B. Wang, and R. Liu, “Robust heterogeneous discriminative analysis for face recognition with single sample per person,” Pattern Recognition , vol. 89, pp. 91–107, 2019. [Online]. Available: https://www.sciencedirect.com/science/article/pii/ S0031320319300111

    2019

  6. [6]

    Master face attacks on face recognition systems,

    H. H. Nguyen, S. Marcel, J. Yamagishi, and I. Echizen, “Master face attacks on face recognition systems,” IEEE Transactions on Biometrics, Behavior, and Identity Science , vol. 4, no. 3, pp. 398–411, 2022

    2022

  7. [7]

    Custom silicone face masks: Vulnerability of commercial face recognition systems & presentation attack detection,

    R. Ramachandra, S. Venkatesh, K. B. Raja, S. Bhattacharjee, P. Wasnik, S. Marcel, and C. Busch, “Custom silicone face masks: Vulnerability of commercial face recognition systems & presentation attack detection,” in 2019 7th International Workshop on Biometrics and Forensics (IWBF) , 2019, pp. 1–6

    2019

  8. [8]

    Unravelling robustness of deep learning based face recognition against adversarial attacks,

    G. Goswami, N. Ratha, A. Agarwal, R. Singh, and M. Vatsa, “Unravelling robustness of deep learning based face recognition against adversarial attacks,” Proceedings of the AAAI Conference on Artificial Intelligence , vol. 32, no. 1, Apr. 2018. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/view/12341

    2018

Show all 96 references
  1. [9]

    Towards deep learning models resistant to adversarial attacks,

    A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” in 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings . Op...

    2018

  2. [10]

    Foolbox native: Fast adversarial attacks to benchmark the robustness of machine learning models in pytorch, tensorflow, and jax,

    J. Rauber, R. Zimmermann, M. Bethge, and W. Brendel, “Foolbox native: Fast adversarial attacks to benchmark the robustness of machine learning models in pytorch, tensorflow, and jax,” Journal of Open Source Software , vol. 5, no. 53, p. 2607, 2020. [Online]. Available: https:/...

    2020 doi

  3. [11]

    Formal guarantees on the robustness of a classifier against adversarial manipulation,

    M. Hein and M. Andriushchenko, “Formal guarantees on the robustness of a classifier against adversarial manipulation,” in Advances in Neural Information Processing Systems , I. Guyon, U. V . Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds., vol....

  4. [12]

    Limitations of the lipschitz constant as a defense against adversarial examples,

    T. Huster, C.-Y . J. Chiang, and R. Chadha, “Limitations of the lipschitz constant as a defense against adversarial examples,” in ECML PKDD 2018 Workshops, C. Alzate, A. Monreale, H. Assem, A. Bifet, T. S. Buda, B. Caglayan, B. Drury, E. Garc ´ıa-Mart´ın, R. Gavald `a, I. Kopr...

    2018

  5. [13]

    Lipschitz regularity of deep neural networks: analysis and efficient estimation,

    A. Virmaux and K. Scaman, “Lipschitz regularity of deep neural networks: analysis and efficient estimation,” in Advances in Neural Information Processing Systems , S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, Eds., vol. 31. Curran Associat...

    2018

  6. [14]

    Advbox: a toolbox to generate adversarial examples that fool neural networks,

    D. Goodman, X. Hao, Y . Wang, Y . Wu, J. Xiong, and H. Zhang, “Advbox: a toolbox to generate adversarial examples that fool neural networks,” CoRR, vol. abs/2001.05574, 2020. [Online]. Available: https://arxiv.org/abs/2001.05574

    2001 arXiv

  7. [15]

    Explaining and harnessing adversarial examples,

    I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” in 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings , Y . Bengio and Y . LeCun, Eds., 2015. [Onlin...

    2015 arXiv

  8. [16]

    Rdcface: Radial distortion correction for face recognition,

    H. Zhao, X. Ying, Y . Shi, X. Tong, J. Wen, and H. Zha, “Rdcface: Radial distortion correction for face recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020

    2020

  9. [17]

    Why do adversarial attacks transfer? ex- plaining transferability of evasion and poisoning attacks,

    A. Demontis, M. Melis, M. Pintor, M. Jagielski, B. Biggio, A. Oprea, C. Nita-Rotaru, and F. Roli, “Why do adversarial attacks transfer? ex- plaining transferability of evasion and poisoning attacks,” inProceedings of the 28th USENIX Conference on Security Symposium , ser. SEC’...

    2019

  10. [18]

    Enhancing the transferability of adversarial attacks through variance tuning,

    X. Wang and K. He, “Enhancing the transferability of adversarial attacks through variance tuning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2021, pp. 1924–1933

    2021

  11. [19]

    Transferability in machine learning: from phenomena to black-box attacks using adversarial samples,

    N. Papernot, P. D. McDaniel, and I. J. Goodfellow, “Transferability in machine learning: from phenomena to black-box attacks using adversarial samples,” CoRR, vol. abs/1605.07277, 2016. [Online]. Available: http://arxiv.org/abs/1605.07277

    2016 arXiv

  12. [20]

    Evaluating the robustness of neural networks: An extreme value theory approach,

    T. Weng, H. Zhang, P. Chen, J. Yi, D. Su, Y . Gao, C. Hsieh, and L. Daniel, “Evaluating the robustness of neural networks: An extreme value theory approach,” in 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, ...

    2018

  13. [21]

    K. E. Iverson, A Programming Language . USA: John Wiley & Sons, Inc., 1962

    1962

  14. [22]

    R. N. Bracewell and R. N. Bracewell, The Fourier transform and its applications. McGraw-Hill New York, 1986, vol. 31999

    1986

  15. [23]

    Chicco, Siamese Neural Networks: An Overview

    D. Chicco, Siamese Neural Networks: An Overview . New York, NY: Springer US, 2021, pp. 73–94. [Online]. Available: https: //doi.org/10.1007/978-1-0716-0826-5 3

    2021 doi

  16. [24]

    A light cnn for deep face representation with noisy labels,

    X. Wu, R. He, Z. Sun, and T. Tan, “A light cnn for deep face representation with noisy labels,” IEEE Transactions on Information Forensics and Security, vol. 13, no. 11, pp. 2884–2896, 2018

    2018

  17. [25]

    Vision transformer with deformable attention,

    Z. Xia, X. Pan, S. Song, L. E. Li, and G. Huang, “Vision transformer with deformable attention,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 4794–4803

    2022

  18. [26]

    Sok: Certified robustness for deep neural networks,

    L. Li, T. Xie, and B. Li, “Sok: Certified robustness for deep neural networks,” in 44th IEEE Symposium on Security and Privacy, SP 2023, San Francisco, CA, USA, 22-26 May 2023 . IEEE, 2023. [Online]. Available: https://arxiv.org/abs/2009.04131

    2023 arXiv

  19. [27]

    Robustness and accuracy could be reconcilable by (proper) definition,

    T. Pang, M. Lin, X. Yang, J. Zhu, and S. Yan, “Robustness and accuracy could be reconcilable by (proper) definition,” in International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA , ser. Proceedings of Machine Learning Research, K. Chaud...

    2022

  20. [28]

    Characterizing adversarial subspaces using local intrinsic dimensionality,

    X. Ma, B. Li, Y . Wang, S. M. Erfani, S. N. R. Wijewickrema, G. Schoenebeck, D. Song, M. E. Houle, and J. Bailey, “Characterizing adversarial subspaces using local intrinsic dimensionality,” in 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC,...

    2018

  21. [29]

    A survey on: Facial emotion recognition invariant to pose, illumination and age,

    S. Bhattacharya and M. Gupta, “A survey on: Facial emotion recognition invariant to pose, illumination and age,” in 2019 Second International Conference on Advanced Computational and Communication Paradigms (ICACCP). IEEE, 2019, pp. 1–6

    2019

  22. [30]

    Certified robust accuracy of neural networks are bounded due to bayes errors,

    R. Zhang and J. Sun, “Certified robust accuracy of neural networks are bounded due to bayes errors,” in Computer Aided Verification , A. Gurfinkel and V . Ganesh, Eds. Cham: Springer Nature Switzerland, 2024, pp. 352–376

    2024

  23. [32]

    Searching for a search method: Benchmarking search algorithms for generating NLP adversarial examples,

    J. Y . Yoo, J. Morris, E. Lifland, and Y . Qi, “Searching for a search method: Benchmarking search algorithms for generating NLP adversarial examples,” in Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP , A. Alishahi, Y . Bel...

    2020

  24. [33]

    Efficient decision-based black-box adversarial attacks on face recognition,

    Y . Dong, H. Su, B. Wu, Z. Li, W. Liu, T. Zhang, and J. Zhu, “Efficient decision-based black-box adversarial attacks on face recognition,” in proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 7714–7722

    2019

  25. [34]

    On evaluating adversarial robustness,

    N. Carlini, A. Athalye, N. Papernot, W. Brendel, J. Rauber, D. Tsipras, I. J. Goodfellow, A. Madry, and A. Kurakin, “On evaluating adversarial robustness,” CoRR, vol. abs/1902.06705, 2019. [Online]. Available: http://arxiv.org/abs/1902.06705

    1902 arXiv

  26. [35]

    Adversarial machine learning at scale,

    A. Kurakin, I. J. Goodfellow, and S. Bengio, “Adversarial machine learning at scale,” in 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings . OpenReview.net, 2017. [Online]. Available: https://op...

    2017

  27. [36]

    Scaling up the randomized gradient-free adversarial attack reveals overestimation of robustness using established attacks,

    F. Croce, J. Rauber, and M. Hein, “Scaling up the randomized gradient-free adversarial attack reveals overestimation of robustness using established attacks,” International Journal of Computer Vision , vol. 128, no. 4, pp. 1028–1046, Apr 2020. [Online]. Available: https://doi....

    2020 doi

  28. [37]

    Intriguing properties of neural networks,

    C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. J. Goodfellow, and R. Fergus, “Intriguing properties of neural networks,” in 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, Y ...

    2014 arXiv

  29. [38]

    Towards evaluating the robustness of neural networks,

    N. Carlini and D. Wagner, “Towards evaluating the robustness of neural networks,” in 2017 IEEE Symposium on Security and Privacy (SP) , 2017, pp. 39–57

    2017

  30. [39]

    Probabilistically robust learning: Balancing average and worst-case performance,

    A. Robey, L. Chamon, G. J. Pappas, and H. Hassani, “Probabilistically robust learning: Balancing average and worst-case performance,” in Proceedings of the 39th International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, K. Chaudhuri, S. Jegel...

  31. [40]

    Practical black-box attacks against machine learning,

    N. Papernot, P. McDaniel, I. Goodfellow, S. Jha, Z. B. Celik, and A. Swami, “Practical black-box attacks against machine learning,” in Proceedings of the 2017 ACM on Asia conference on computer and communications security, 2017, pp. 506–519

    2017

  32. [41]

    Decision-based adversarial attacks: Reliable attacks against black-box machine learning models,

    W. Brendel, J. Rauber, and M. Bethge, “Decision-based adversarial attacks: Reliable attacks against black-box machine learning models,” in 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proce...

    2018

  33. [42]

    One pixel attack for fooling deep neural networks,

    J. Su, D. V . Vargas, and K. Sakurai, “One pixel attack for fooling deep neural networks,” IEEE Transactions on Evolutionary Computation , vol. 23, no. 5, pp. 828–841, 2019

    2019

  34. [43]

    Genattack: Practical black-box attacks with gradient- free optimization,

    M. Alzantot, Y . Sharma, S. Chakraborty, H. Zhang, C.-J. Hsieh, and M. B. Srivastava, “Genattack: Practical black-box attacks with gradient- free optimization,” in Proceedings of the Genetic and Evolutionary Computation Conference , ser. GECCO ’19. New York, NY , USA: Associat...

    2019

  35. [44]

    An abstract domain for certifying neural networks,

    G. Singh, T. Gehr, M. P ¨uschel, and M. Vechev, “An abstract domain for certifying neural networks,” Proc. ACM Program. Lang., vol. 3, no. POPL, jan 2019. [Online]. Available: https://doi.org/10.1145/3290354

    2019 doi

  36. [45]

    Reluplex: An efficient smt solver for verifying deep neural networks,

    G. Katz, C. Barrett, D. L. Dill, K. Julian, and M. J. Kochenderfer, “Reluplex: An efficient smt solver for verifying deep neural networks,” in Computer Aided Verification , R. Majumdar and V . Kun ˇcak, Eds. Cham: Springer International Publishing, 2017, pp. 97–117

    2017

  37. [46]

    Unleash the black magic in age: A multi-task deep neural network approach for cross-age face verification,

    X. Wang, Y . Zhou, D. Kong, J. Currey, D. Li, and J. Zhou, “Unleash the black magic in age: A multi-task deep neural network approach for cross-age face verification,” in 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017) , 2017, pp. 596– 603

    2017

  38. [47]

    Advhat: Real-world adversarial attack on arcface face id system,

    S. Komkov and A. Petiushko, “Advhat: Real-world adversarial attack on arcface face id system,” in 2020 25th International Conference on Pattern Recognition (ICPR), 2021, pp. 819–826

    2020

  39. [48]

    Cross-pose lfw: A database for studying cross- pose face recognition in unconstrained environments,

    T. Zheng and W. Deng, “Cross-pose lfw: A database for studying cross- pose face recognition in unconstrained environments,”Beijing University of Posts and Telecommunications, Tech. Rep , vol. 5, p. 7, 2018

    2018

  40. [49]

    Illumination invariant face recog- nition: A survey,

    X. Zou, J. Kittler, and K. Messer, “Illumination invariant face recog- nition: A survey,” in 2007 First IEEE International Conference on Biometrics: Theory, Applications, and Systems , 2007, pp. 1–8

    2007

  41. [50]

    Mlfw: A database for face recognition on masked faces,

    C. Wang, H. Fang, Y . Zhong, and W. Deng, “Mlfw: A database for face recognition on masked faces,” in Biometric Recognition , W. Deng, J. Feng, D. Huang, M. Kan, Z. Sun, F. Zheng, W. Wang, and Z. He, Eds. Cham: Springer Nature Switzerland, 2022, pp. 180–188

    2022

  42. [51]

    Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments,

    G. B. Huang, M. Mattar, T. Berg, and E. Learned-Miller, “Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments,” in Workshop on Faces in ’Real-Life’ JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 14 Images: Detection, Align...

    2021

  43. [52]

    Bridging the performance gap between FGSM and PGD adversarial training,

    T. Huang, V . Menkovski, Y . Pei, and M. Pechenizkiy, “Bridging the performance gap between FGSM and PGD adversarial training,” CoRR, vol. abs/2011.05157, 2020. [Online]. Available: https://arxiv.org/abs/ 2011.05157

    2011 arXiv

  44. [53]

    Facenet: A unified embed- ding for face recognition and clustering,

    F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified embed- ding for face recognition and clustering,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2015, pp. 815–823

    2015

  45. [54]

    Distance metric learning for large margin nearest neighbor classification,

    K. Q. Weinberger, J. Blitzer, and L. Saul, “Distance metric learning for large margin nearest neighbor classification,” in Advances in Neural Information Processing Systems , Y . Weiss, B. Sch ¨olkopf, and J. Platt, Eds., vol. 18. MIT Press,

  46. [55]

    Arcface: Additive angular margin loss for deep face recognition,

    J. Deng, J. Guo, N. Xue, and S. Zafeiriou, “Arcface: Additive angular margin loss for deep face recognition,” in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2019, pp. 4685– 4694

    2019

  47. [56]

    EfficientNet: Rethinking model scaling for convolutional neural networks,

    M. Tan and Q. Le, “EfficientNet: Rethinking model scaling for convolutional neural networks,” in Proceedings of the 36th International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, K. Chaudhuri and R. Salakhutdinov, Eds., vol. 97. PMLR, 09–15 ...

    2019

  48. [57]

    Mis-classified vector guided softmax loss for face recognition,

    X. Wang, S. Zhang, S. Wang, T. Fu, H. Shi, and T. Mei, “Mis-classified vector guided softmax loss for face recognition,” Proceedings of the AAAI Conference on Artificial Intelligence , vol. 34, no. 07, pp. 12 241–12 248, Apr. 2020. [Online]. Available: https://ojs.aaai.org/ind...

    2020

  49. [58]

    Rethinking channel dimensions for efficient model design,

    D. Han, S. Yun, B. Heo, and Y . Yoo, “Rethinking channel dimensions for efficient model design,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2021, pp. 732–741

    2021

  50. [59]

    Curricularface: Adaptive curriculum learning loss for deep face recog- nition,

    Y . Huang, Y . Wang, Y . Tai, X. Liu, P. Shen, S. Li, J. Li, and F. Huang, “Curricularface: Adaptive curriculum learning loss for deep face recog- nition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2020

    2020

  51. [60]

    Residual attention network for image classification,

    F. Wang, M. Jiang, C. Qian, S. Yang, C. Li, H. Zhang, X. Wang, and X. Tang, “Residual attention network for image classification,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017

    2017

  52. [61]

    Adacos: Adaptively scaling cosine logits for effectively learning deep face representations,

    X. Zhang, R. Zhao, Y . Qiao, X. Wang, and H. Li, “Adacos: Adaptively scaling cosine logits for effectively learning deep face representations,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019, pp. 10 823–10 832

    2019

  53. [62]

    Ghostnet: More features from cheap operations,

    K. Han, Y . Wang, Q. Tian, J. Guo, C. Xu, and C. Xu, “Ghostnet: More features from cheap operations,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2020

    2020

  54. [63]

    Additive margin softmax for face verification,

    F. Wang, J. Cheng, W. Liu, and H. Liu, “Additive margin softmax for face verification,” IEEE Signal Processing Letters , vol. 25, no. 7, pp. 926–930, 2018

    2018

  55. [64]

    Repvgg: Mak- ing vgg-style convnets great again,

    X. Ding, X. Zhang, N. Ma, J. Han, G. Ding, and J. Sun, “Repvgg: Mak- ing vgg-style convnets great again,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2021, pp. 13 733–13 742

    2021

  56. [65]

    Tf-nas: Rethinking three search freedoms of latency-constrained differentiable neural architecture search,

    Y . Hu, X. Wu, and R. He, “Tf-nas: Rethinking three search freedoms of latency-constrained differentiable neural architecture search,” in Com- puter Vision – ECCV 2020 , A. Vedaldi, H. Bischof, T. Brox, and J.-M. Frahm, Eds. Cham: Springer International Publishing, 2020, pp. 123– 139

    2020

  57. [66]

    Frontal to profile face verification in the wild,

    S. Sengupta, J.-C. Chen, C. Castillo, V . M. Patel, R. Chellappa, and D. W. Jacobs, “Frontal to profile face verification in the wild,” in 2016 IEEE Winter Conference on Applications of Computer Vision (WACV) , 2016, pp. 1–9

    2016

  58. [67]

    Agedb: The first manually collected, in-the-wild age database,

    S. Moschoglou, A. Papaioannou, C. Sagonas, J. Deng, I. Kotsia, and S. Zafeiriou, “Agedb: The first manually collected, in-the-wild age database,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops , July 2017, pp. 51–59

    2017

  59. [68]

    Mirjalili, Genetic Algorithm

    S. Mirjalili, Genetic Algorithm . Cham: Springer International Publishing, 2019, pp. 43–55. [Online]. Available: https://doi.org/10. 1007/978-3-319-93025-1 4

    2019

  60. [69]

    Pymoo: Multi-objective optimization in python,

    J. Blank and K. Deb, “Pymoo: Multi-objective optimization in python,” IEEE Access, vol. 8, pp. 89 497–89 509, 2020

    2020

  61. [70]

    Proa: A probabilistic robustness assessment against functional perturbations,

    T. Zhang, W. Ruan, and J. E. Fieldsend, “Proa: A probabilistic robustness assessment against functional perturbations,” in Machine Learning and Knowledge Discovery in Databases , M.-R. Amini, S. Canu, A. Fischer, T. Guns, P. Kralj Novak, and G. Tsoumakas, Eds. Cham: Springer N...

    2023

  62. [71]

    Deep face recognition: A survey,

    M. Wang and W. Deng, “Deep face recognition: A survey,” Neurocomputing, vol. 429, pp. 215–244, 2021. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0925231220316945

    2021

  63. [72]

    Deep metric learning using triplet network,

    E. Hoffer and N. Ailon, “Deep metric learning using triplet network,” in Similarity-Based Pattern Recognition , A. Feragen, M. Pelillo, and M. Loog, Eds. Cham: Springer International Publishing, 2015, pp. 84–92

    2015

  64. [73]

    Adaptiveface: Adaptive margin and sampling for face recognition,

    H. Liu, X. Zhu, Z. Lei, and S. Z. Li, “Adaptiveface: Adaptive margin and sampling for face recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2019

    2019

  65. [74]

    Improved performance of face recognition using cnn with constrained triplet loss layer,

    H. W. F. Yeung, J. Li, and Y . Y . Chung, “Improved performance of face recognition using cnn with constrained triplet loss layer,” in 2017 International Joint Conference on Neural Networks (IJCNN) , 2017, pp. 1948–1955

    2017

  66. [75]

    Enhancing convolutional neural networks for face recognition with occlusion maps and batch triplet loss,

    D. S ´aez Trigueros, L. Meng, and M. Hartnett, “Enhancing convolutional neural networks for face recognition with occlusion maps and batch triplet loss,” Image and Vision Computing , vol. 79, pp. 99–108, 2018. [Online]. Available: https://www.sciencedirect.com/science/article/...

    2018

  67. [76]

    Deep metric learning with hierarchical triplet loss,

    W. Ge, “Deep metric learning with hierarchical triplet loss,” in Pro- ceedings of the European Conference on Computer Vision (ECCV) , September 2018

    2018

  68. [77]

    Cosface: Large margin cosine loss for deep face recognition,

    H. Wang, Y . Wang, Z. Zhou, X. Ji, D. Gong, J. Zhou, Z. Li, and W. Liu, “Cosface: Large margin cosine loss for deep face recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018

    2018

  69. [78]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2016

    2016

  70. [79]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , October 2021, pp. 10 012–10 022

    2021

  71. [80]

    Sphereface: Deep hypersphere embedding for face recognition,

    W. Liu, Y . Wen, Z. Yu, M. Li, B. Raj, and L. Song, “Sphereface: Deep hypersphere embedding for face recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , July 2017, pp. 212–220

    2017

  72. [81]

    Universal pertur- bation attack against image retrieval,

    J. Li, R. Ji, H. Liu, X. Hong, Y . Gao, and Q. Tian, “Universal pertur- bation attack against image retrieval,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , October 2019, pp. 4899–4908

    2019

  73. [82]

    How benign is benign overfitting ?

    A. Sanyal, P. K. Dokania, V . Kanade, and P. H. S. Torr, “How benign is benign overfitting ?” in 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 . OpenReview.net, 2021. [Online]. Available: https: //openreview.net/foru...

    2021

  74. [83]

    Causality-based neural network repair,

    B. Sun, J. Sun, L. H. Pham, and J. Shi, “Causality-based neural network repair,” in Proceedings of the 44th International Conference on Software Engineering , ser. ICSE ’22. New York, NY , USA: Association for Computing Machinery, 2022, pp. 338–349. [Online]. Available: https:...

    2022

  75. [84]

    Code smells in machine learning systems,

    J. Gesi, S. Liu, J. Li, I. Ahmed, N. Nagappan, D. Lo, E. S. de Almeida, P. S. Kochhar, and L. Bao, “Code smells in machine learning systems,” CoRR, vol. abs/2203.00803, 2022. [Online]. Available: https://doi.org/10.48550/arXiv.2203.00803

    2022 doi arXiv

  76. [85]

    Combining experts’ causal judgments,

    D. Alrajeh, H. Chockler, and J. Y . Halpern, “Combining experts’ causal judgments,” Artificial Intelligence , vol. 288, p. 103355, 2020. [Online]. Available: https://www.sciencedirect.com/science/article/pii/ S0004370220301065

    2020

  77. [86]

    Benchmarking neural network robustness to common corruptions and perturbations,

    D. Hendrycks and T. G. Dietterich, “Benchmarking neural network robustness to common corruptions and perturbations,” in 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 . OpenReview.net, 2019. [Online]. Available: https:/...

    2019

  78. [87]

    Robot: Robustness-oriented testing for deep learning systems,

    J. Wang, J. Chen, Y . Sun, X. Ma, D. Wang, J. Sun, and P. Cheng, “Robot: Robustness-oriented testing for deep learning systems,” in 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), 2021, pp. 300–311

    2021

  79. [88]

    Biasfinder: Metamorphic test generation to uncover bias for sentiment JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 15 analysis systems,

    M. H. Asyrofi, Z. Yang, I. N. B. Yusuf, H. J. Kang, F. Thung, and D. Lo, “Biasfinder: Metamorphic test generation to uncover bias for sentiment JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 15 analysis systems,” IEEE Transactions on Software Engineering, vol. 48, n...

    2021

  80. [89]

    A systematic literature review on the use of deep learning in software engineering research,

    C. Watson, N. Cooper, D. N. Palacio, K. Moran, and D. Poshyvanyk, “A systematic literature review on the use of deep learning in software engineering research,” ACM Trans. Softw. Eng. Methodol. , vol. 31, no. 2, mar 2022. [Online]. Available: https://doi.org/10.1145/3485275

    2022 doi

  81. [90]

    Antidotert: Run-time detection and correction of poison attacks on neural networks,

    M. Usman, Y . Sun, D. Gopinath, and C. S. Pasareanu, “Antidotert: Run-time detection and correction of poison attacks on neural networks,” CoRR, vol. abs/2202.01179, 2022. [Online]. Available: https://arxiv.org/abs/2202.01179

    2022 arXiv

  82. [91]

    A probabilistic framework for mutation testing in deep neural networks,

    F. Tambon, F. Khomh, and G. Antoniol, “A probabilistic framework for mutation testing in deep neural networks,” Information and Software Technology, vol. 155, p. 107129, 2023. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0950584922002385

    2023

  83. [92]

    Realizing self-adaptive systems via online reinforcement learning and feature-model-guided exploration,

    A. Metzger, C. Quinton, Z. ´A. Mann, L. Baresi, and K. Pohl, “Realizing self-adaptive systems via online reinforcement learning and feature-model-guided exploration,” Computing, Mar 2022. [Online]. Available: https://doi.org/10.1007/s00607-022-01052-x Ruihan Zhang is currently...

    2022 doi

  84. [162]

    18 667–18 686

    PMLR, 17–23 Jul 2022, pp. 18 667–18 686. [Online]. Available: https://proceedings.mlr.press/v162/robey22a.html

    2022

  85. [2005]

    Available: https://proceedings.neurips.cc/paper/2005/ file/a7f592cef8b130a6967a90617db5681b-Paper.pdf

    [Online]. Available: https://proceedings.neurips.cc/paper/2005/ file/a7f592cef8b130a6967a90617db5681b-Paper.pdf

    2005

  86. [2012]

    He is the co-founder of the PAT model checker

    Jun’s research interests include software engineering, formal methods, program analysis and cyber-security. He is the co-founder of the PAT model checker

  87. [2017]

    Available: https://proceedings.neurips.cc/paper/2017/ file/e077e1a544eec4f0307cf5c3c721d944-Paper.pdf

    [Online]. Available: https://proceedings.neurips.cc/paper/2017/ file/e077e1a544eec4f0307cf5c3c721d944-Paper.pdf

    2017

  88. [2019]

    Available: http://arxiv.org/abs/1911.05268

    [Online]. Available: http://arxiv.org/abs/1911.05268

    1911 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.