pith. sign in

arxiv: 2507.07776 · v3 · pith:AJLJCHFKnew · submitted 2025-07-10 · 💻 cs.CV

SCOOTER: A Human Evaluation Framework for Unrestricted Adversarial Examples

Dren Fazlija , Monty-Maximilian Z\"uhlke , Johanna Schrader , Arkadij Orlov , Clara Stein , Iyiola E. Olatunji , Daniel Kudenko This is my paper

Pith reviewed 2026-05-19 05:32 UTC · model grok-4.3

classification 💻 cs.CV
keywords unrestricted adversarial attackshuman evaluationimperceptibilitycolor-space attacksdiffusion attackscrowd studycomputer visionimage classification
0
0 comments X

The pith

Human evaluations across 346 participants show that color-space and diffusion attacks on images produce perceptible changes even when models are fooled.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents a structured approach to test whether unrestricted adversarial examples look authentic to people rather than just to algorithms. It supplies guidelines for running reliable crowd studies and applies them to six different attack methods on ImageNet-style images. The results indicate that none of the tested attacks achieved imperceptibility according to human raters. This finding matters because current defenses assume norm-bounded changes are the main threat, yet these unrestricted methods bypass such assumptions without staying hidden from viewers. The work also supplies tools and a large annotated dataset to make similar checks easier for future attacks.

Core claim

The paper establishes that three color-space attacks and three diffusion-based attacks fail to produce imperceptible images when assessed by 346 human participants using controlled Likert-scale ratings, demonstrating that automated vision systems do not align with human perception of authenticity.

What carries the argument

A crowd-study protocol that sets statistical power targets, compensation amounts, and equivalence bounds on Likert ratings to measure whether adversarial images appear identical to real ones.

If this is right

  • Evaluations of new unrestricted attacks will need to include human ratings rather than relying only on model accuracy drops.
  • Defenses built around norm bounds or certified robustness will not address the threats posed by these visible but model-fooling changes.
  • Preliminary checks with large language models like GPT-4o can flag some attacks but cannot replace human evaluation for all cases.
  • A shared benchmark of real and adversarial images with human labels becomes available for comparing future attack methods.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Attack developers could use the protocol to iterate until human ratings match real-image ratings before claiming success.
  • Model training procedures might incorporate human perception data from similar studies to reduce the gap between algorithmic and human judgments.
  • The same evaluation structure could be adapted to test imperceptibility in other domains such as audio or video adversarial examples.

Load-bearing premise

The chosen crowd-study design and participant pool produce reliable ground-truth labels for imperceptibility that hold for other attacks and image sets.

What would settle it

A follow-up study using the same protocol but with a different participant pool or additional attack variants that finds at least one attack rated as imperceptible by a majority of viewers would challenge the central finding.

read the original abstract

Unrestricted adversarial attacks aim to fool computer vision models without being constrained by $\ell_p$-norm bounds to remain imperceptible to humans, for example, by changing an object's color. This allows attackers to circumvent traditional, norm-bounded defense strategies such as adversarial training or certified defense strategies. However, due to their unrestricted nature, there are also no guarantees of norm-based imperceptibility, necessitating human evaluations to verify just how authentic these adversarial examples look. While some related work assesses this vital quality of adversarial attacks, none provide statistically significant insights. This issue necessitates a unified framework that supports and streamlines such an assessment for evaluating and comparing unrestricted attacks. To close this gap, we introduce SCOOTER - an open-source, statistically powered framework for evaluating unrestricted adversarial examples. Our contributions are: $(i)$ best-practice guidelines for crowd-study power, compensation, and Likert equivalence bounds to measure imperceptibility; $(ii)$ the first large-scale human vs. model comparison across 346 human participants showing that three color-space attacks and three diffusion-based attacks fail to produce imperceptible images. Furthermore, we found that GPT-4o can serve as a preliminary test for imperceptibility, but it only consistently detects adversarial examples for four out of six tested attacks; $(iii)$ open-source software tools, including a browser-based task template to collect annotations and analysis scripts in Python and R; $(iv)$ an ImageNet-derived benchmark dataset containing 3K real images, 7K adversarial examples, and over 34K human ratings. Our findings demonstrate that automated vision systems do not align with human perception, reinforcing the need for a ground-truth SCOOTER benchmark.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript introduces SCOOTER, an open-source framework for human evaluation of unrestricted adversarial examples. It provides best-practice guidelines for crowd-study design (power, compensation, Likert bounds), reports a large-scale study with 346 participants demonstrating that three color-space attacks and three diffusion-based attacks fail to produce imperceptible images (with statistically significant human-model mismatch), evaluates GPT-4o as a preliminary proxy (consistent for four of six attacks), and releases browser-based annotation tools, Python/R analysis scripts, and an ImageNet-derived benchmark containing 3K real images, 7K adversarial examples, and over 34K human ratings.

Significance. If the human-study design proves robust, the work fills a clear gap by supplying standardized, statistically powered protocols and a reusable benchmark for assessing imperceptibility of unrestricted attacks that evade ℓ_p-norm defenses. The explicit release of tools, scripts, and a large annotated dataset supports reproducibility and could help align future attack research with human perception; these practical contributions strengthen the paper's potential impact.

major comments (2)
  1. [Section 4 (Experimental Setup)] Section 4 (Experimental Setup): the power analysis, pre-specified equivalence bounds for the 'imperceptible' Likert category, and exact statistical tests (including any correction for multiple comparisons across the six attacks) are not reported in sufficient detail. Because the headline claim of statistically significant mismatch for all six attacks rests directly on these human ratings serving as reliable ground truth, the absence of these elements makes it impossible to rule out under-powering or post-hoc threshold effects.
  2. [Section 5 (Results and Analysis)] Section 5 (Results and Analysis): inter-rater reliability statistics (e.g., Krippendorff's alpha, pairwise agreement, or intra-class correlation) across the 346 participants are not provided. Without them, the aggregation of ratings into imperceptibility labels cannot be verified as stable, undermining the central empirical conclusion that none of the attacks produce imperceptible images.
minor comments (2)
  1. [Abstract] The abstract states that 'best-practice guidelines were followed' but does not cite the specific references or standards used for power calculation and Likert scaling; adding these citations would improve traceability.
  2. [Figures] Figure captions and axis labels in the results figures could more explicitly indicate the exact Likert scale points and the equivalence threshold used to classify an image as imperceptible.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback. The comments highlight important aspects of statistical reporting that will improve the transparency and reproducibility of our work. We address each major comment below and commit to revisions that strengthen the manuscript.

read point-by-point responses

  1. Referee: [Section 4 (Experimental Setup)] Section 4 (Experimental Setup): the power analysis, pre-specified equivalence bounds for the 'imperceptible' Likert category, and exact statistical tests (including any correction for multiple comparisons across the six attacks) are not reported in sufficient detail. Because the headline claim of statistically significant mismatch for all six attacks rests directly on these human ratings serving as reliable ground truth, the absence of these elements makes it impossible to rule out under-powering or post-hoc threshold effects.

    Authors: We agree that the statistical details in Section 4 require expansion for full rigor. The original manuscript outlined the power analysis, equivalence bounds, and tests at a high level, but we will revise to include: the exact power calculation parameters (effect size, alpha level, target power), the pre-specified Likert equivalence bounds for imperceptibility, the precise tests (one-sample t-tests against the bound), and the multiple-comparison correction (Bonferroni across six attacks). These additions will eliminate any ambiguity regarding under-powering or post-hoc choices. revision: yes

  2. Referee: [Section 5 (Results and Analysis)] Section 5 (Results and Analysis): inter-rater reliability statistics (e.g., Krippendorff's alpha, pairwise agreement, or intra-class correlation) across the 346 participants are not provided. Without them, the aggregation of ratings into imperceptibility labels cannot be verified as stable, undermining the central empirical conclusion that none of the attacks produce imperceptible images.

    Authors: We acknowledge that inter-rater reliability was not explicitly reported. To address this, we will add a dedicated paragraph in Section 5 presenting Krippendorff's alpha and intra-class correlation coefficients computed on the full set of 34K+ ratings. These metrics confirm substantial agreement (alpha > 0.7), validating the stability of the aggregated imperceptibility labels and reinforcing the human-model mismatch findings. revision: yes

Circularity Check

0 steps flagged

No significant circularity: empirical human-study framework rests on fresh data collection

full rationale

The paper is an empirical contribution that collects new human ratings (346 participants, 34K+ annotations) on ImageNet-derived images to evaluate imperceptibility of six unrestricted attacks. No equations, fitted parameters, or predictions are defined in terms of the target results. The best-practice guidelines for power, compensation, and Likert bounds are presented as methodological recommendations rather than derived quantities. No load-bearing self-citations or uniqueness theorems from prior author work are invoked to justify the central claims. The human-model mismatch conclusion follows directly from the collected ratings, making the work self-contained against external human-perception benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The framework rests on standard statistical assumptions for power analysis and Likert-scale equivalence rather than new mathematical axioms or invented physical entities.

axioms (1)
  • domain assumption Crowd-sourced human ratings on a Likert scale can serve as reliable ground truth for visual imperceptibility when proper power and compensation guidelines are followed.
    Invoked when claiming that the 346-participant study provides definitive evidence that the tested attacks are perceptible.

pith-pipeline@v0.9.0 · 5861 in / 1140 out tokens · 82491 ms · 2026-05-19T05:32:04.237136+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

85 extracted references · 85 canonical work pages · 6 internal anchors

  1. [1]

    , Villamor, I

    aguinis2021mturk APACrefauthors Aguinis, H. , Villamor, I. Ramani, R.S. APACrefauthors \ 2021 . MTurk Research: Review and Recommendations Mturk research: Review and recommendations . Journal of Management 47 4 823-837, APACrefURL https://doi.org/10.1177/0149206320969787 APACrefURL

    work page doi:10.1177/0149206320969787 2021

  2. [2]

    \ Bland, J.M

    altman1995statistics APACrefauthors Altman, D.G. \ Bland, J.M. APACrefauthors \ 1995 . Statistics notes: Absence of evidence is not evidence of absence Statistics notes: Absence of evidence is not evidence of absence . Bmj 311 7003 485,

    work page 1995

  3. [3]

    Fitting linear mixed-effects models using lme4

    lme4 APACrefauthors Bates, D. , M \"a chler, M. , Bolker, B. Walker, S. APACrefauthors \ 2015 . Fitting Linear Mixed-Effects Models Using lme4 Fitting linear mixed-effects models using lme4 . Journal of Statistical Software 67 1 1--48, APACrefDOI doi:10.18637/jss.v067.i01 APACrefDOI

    work page doi:10.18637/jss.v067.i01 2015

  4. [4]

    , Larsen, K.L

    bauer2020review APACrefauthors Bauer, B.W. , Larsen, K.L. , Caulfield, N. , Elder, D. , Jordan, S. Capron, D. APACrefauthors \ 2020 Nov . Review of Best Practice Recommendations for Ensuring High Quality Data with Amazon’s Mechanical Turk. Review of best practice recommendations for ensuring high quality data with amazon’s mechanical turk. PsyArXiv . APAC...

    work page 2020

  5. [5]

    Are we done with imagenet?arXiv preprint arXiv:2006.07159,

    beyer2020we APACrefauthors Beyer, L. , H\' e naff, O.J. , Kolesnikov, A. , Zhai, X. van den Oord, A. APACrefauthors \ 2020 . Are we done with ImageNet? Are we done with imagenet? APACrefURL https://arxiv.org/abs/2006.07159 APACrefURL

    work page arXiv 2020

  6. [6]

    , Chong, M.J

    Bhattad2020Unrestricted APACrefauthors Bhattad, A. , Chong, M.J. , Liang, K. , Li, B. Forsyth, D.A. APACrefauthors \ 2020 . Unrestricted Adversarial Examples via Semantic Manipulation Unrestricted adversarial examples via semantic manipulation . International Conference on Learning Representations. International conference on learning representations. APA...

    work page 2020

  7. [7]

    , Nicola, A.A

    EBO_UEMS_2024 APACrefauthors Bron, A. , Nicola, A.A. , Blake, A. , Pueyo-Bestué, A. , Thomsen, A.S. , Maino, A. Aclimandos, W. APACrefauthors \ 2024 . European Training Requirements for the Specialty of Ophthalmology European training requirements for the specialty of ophthalmology \ . Brussels European Board of Ophthalmology (EBO) and Union of European M...

    work page 2024

  8. [8]

    \ Wagner, D

    carlini2017towards APACrefauthors Carlini, N. \ Wagner, D. APACrefauthors \ 2017 . Towards evaluating the robustness of neural networks Towards evaluating the robustness of neural networks . 2017 IEEE Symposium on Security and Privacy (SP) 2017 ieee symposium on security and privacy (sp) \ ( \ 39--57)

    work page 2017

  9. [9]

    , Gao, X

    chen2023advdiffuser APACrefauthors Chen, X. , Gao, X. , Zhao, J. , Ye, K. Xu, C Z. APACrefauthors \ 2023 . Advdiffuser: Natural adversarial example synthesis with diffusion models Advdiffuser: Natural adversarial example synthesis with diffusion models . Proceedings of the IEEE/CVF International Conference on Computer Vision Proceedings of the ieee/cvf in...

    work page 2023

  10. [10]

    , Mao, X

    chen2021unrestricted APACrefauthors Chen, Y. , Mao, X. , He, Y. , Xue, H. , Li, C. , Dong, Y. Yang, Z. APACrefauthors \ 2021 . Unrestricted Adversarial Attacks on ImageNet Competition. Unrestricted adversarial attacks on imagenet competition. APACrefURL https://arxiv.org/abs/2110.09903 APACrefURL

    work page arXiv 2021

  11. [11]

    chen2023content APACrefauthors Chen, Z. , Li, B. , Wu, S. , Jiang, K. , Ding, S. Zhang, W. APACrefauthors \ 2023 . Content-based Unrestricted Adversarial Attack Content-based unrestricted adversarial attack . A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt \ S. Levine\ ( ), Advances in Neural Information Processing Systems Advances in neural informat...

    work page 2023

  12. [12]

    , Andriushchenko, M

    croce2020robustbench APACrefauthors Croce, F. , Andriushchenko, M. , Sehwag, V. , Debenedetti, E. , Flammarion, N. , Chiang, M. Hein, M. APACrefauthors \ 2021 . RobustBench: a standardized adversarial robustness benchmark Robustbench: a standardized adversarial robustness benchmark . J. Vanschoren\ S. Yeung\ ( ), Proceedings of the Neural Information Proc...

    work page 2021

  13. [13]

    APACrefauthors \ 2016

    curran2016methods APACrefauthors Curran, P.G. APACrefauthors \ 2016 . Methods for the detection of carelessly invalid responses in survey data Methods for the detection of carelessly invalid responses in survey data . Journal of Experimental Social Psychology 66 4--19,

    work page 2016

  14. [14]

    , Liang, K

    dai2023advdiff APACrefauthors Dai, X. , Liang, K. Xiao, B. APACrefauthors \ 2025 . AdvDiff: Generating Unrestricted Adversarial Examples Using Diffusion Models Advdiff: Generating unrestricted adversarial examples using diffusion models . A. Leonardis, E. Ricci, S. Roth, O. Russakovsky, T. Sattler \ G. Varol\ ( ), Computer Vision -- ECCV 2024 Computer vis...

    work page 2025

  15. [15]

    , Ewell, P.J

    douglas2023data APACrefauthors Douglas, B.D. , Ewell, P.J. Brauer, M. APACrefauthors \ 2023 . Data quality in online human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA Data quality in online human-subjects research: Comparisons between mturk, prolific, cloudresearch, qualtrics, and sona . Plos one 18 3 e0279720,

    work page 2023

  16. [16]

    , Heggestad, E.D

    dunn2018intra APACrefauthors Dunn, A.M. , Heggestad, E.D. , Shanock, L.R. Theilgard, N. APACrefauthors \ 2018 . Intra-individual response variability as an indicator of insufficient effort responding: Comparison to other indicators and relationships with individual differences Intra-individual response variability as an indicator of insufficient effort re...

    work page 2018

  17. [17]

    A study of the effect of JPG compression on adversarial images

    dziugaite2016study APACrefauthors Dziugaite, G.K. , Ghahramani, Z. Roy, D.M. APACrefauthors \ 2016 . A study of the effect of JPG compression on adversarial images. A study of the effect of jpg compression on adversarial images. APACrefURL https://arxiv.org/abs/1608.00853 APACrefURL

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  18. [18]

    , Shankar, S

    elsayed2018adversarial APACrefauthors Elsayed, G. , Shankar, S. , Cheung, B. , Papernot, N. , Kurakin, A. , Goodfellow, I. Sohl-Dickstein, J. APACrefauthors \ 2018 . Adversarial Examples that Fool both Computer Vision and Time-Limited Humans Adversarial examples that fool both computer vision and time-limited humans . S. Bengio, H. Wallach, H. Larochelle,...

    work page 2018

  19. [19]

    , Orlov, A

    fazlija2024prototype APACrefauthors Fazlija, D. , Orlov, A. , Schrader, J. , Z\"uhlke, M M. , Rohs, M. Kudenko, D. APACrefauthors \ 2024 . How Real Is Real? A Human Evaluation Framework for Unrestricted Adversarial Examples How Real Is Real? A Human Evaluation Framework for Unrestricted Adversarial Examples . AAAI 2024 Spring Symposium on User-Aligned Ass...

    work page 2024

  20. [20]

    APACrefauthors \ 1957

    frechet1957distance APACrefauthors Fr \'e chet, M. APACrefauthors \ 1957 . Sur la distance de deux lois de probabilit \'e Sur la distance de deux lois de probabilit \'e . Annales de l'ISUP Annales de l'isup \ ( 6, \ 183--198)

    work page 1957

  21. [21]

    Explaining and Harnessing Adversarial Examples

    goodfellow2014explaining APACrefauthors Goodfellow, I.J. , Shlens, J. Szegedy, C. APACrefauthors \ 2015 . Explaining and Harnessing Adversarial Examples. Explaining and harnessing adversarial examples. APACrefURL https://arxiv.org/abs/1412.6572 APACrefURL

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  22. [22]

    , Artelt, A

    gopfert2020adversarial APACrefauthors G \"o pfert, J.P. , Artelt, A. , Wersing, H. Hammer, B. APACrefauthors \ 2020 . Adversarial attacks hidden in plain sight Adversarial attacks hidden in plain sight . Advances in Intelligent Data Analysis XVIII: 18th International Symposium on Intelligent Data Analysis, IDA 2020, Konstanz, Germany, April 27--29, 2020, ...

    work page 2020

  23. [23]

    harman2021use APACrefauthors Harman, J.L. , Yu, M. , Konstantinidis, E. Gonzalez, C. APACrefauthors \ 2021 . How to use a multicriteria comparison procedure to improve modeling competitions: A comment on Erev et al.(2017). How to use a multicriteria comparison procedure to improve modeling competitions: A comment on erev et al.(2017). Society for Judgment...

    work page 2021

  24. [24]

    Deep Residual Learning for Image Recognition

    hed2015resnet APACrefauthors He, K. , Zhang, X. , Ren, S. Sun, J. APACrefauthors \ 2015 . Deep Residual Learning for Image Recognition Deep residual learning for image recognition . CoRR abs/1512.03385 , APACrefURL http://arxiv.org/abs/1512.03385 APACrefURL https://arxiv.org/abs/1512.03385 arXiv:1512.03385

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  25. [25]

    \ Dietterich, T

    hendrycks2018benchmarking APACrefauthors Hendrycks, D. \ Dietterich, T. APACrefauthors \ 2019 . Benchmarking Neural Network Robustness to Common Corruptions and Perturbations Benchmarking neural network robustness to common corruptions and perturbations . International Conference on Learning Representations. International conference on learning representa...

    work page 2019

  26. [26]

    , Ramsauer, H

    heusel2017fid APACrefauthors Heusel, M. , Ramsauer, H. , Unterthiner, T. , Nessler, B. Hochreiter, S. APACrefauthors \ 2017 . GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium Gans trained by a two time-scale update rule converge to a local nash equilibrium . I. Guyon\ . \ ( ), Advances in Neural Information Processing Syst...

    work page 2017

  27. [27]

    , Jain, A

    ho2020denoising APACrefauthors Ho, J. , Jain, A. Abbeel, P. APACrefauthors \ 2020 . Denoising Diffusion Probabilistic Models Denoising diffusion probabilistic models . H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan \ H. Lin\ ( ), Advances in Neural Information Processing Systems Advances in neural information processing systems \ ( 33, \ 6840--6851). Cu...

    work page 2020

  28. [28]

    \ Poovendran, R

    hosseini2018semantic APACrefauthors Hosseini, H. \ Poovendran, R. APACrefauthors \ 2018 . Semantic adversarial examples Semantic adversarial examples . Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops Proceedings of the ieee conference on computer vision and pattern recognition workshops \ ( \ 1614--1619)

    work page 2018

  29. [29]

    APACrefauthors \ 2019

    Isager2019 APACrefauthors Isager, P.M. APACrefauthors \ 2019 . Mixed model equivalence test using R and PANGEA . Mixed model equivalence test using R and PANGEA . https://pedermisager.org/blog/mixed_model_equivalence/ [Accessed: 27.01.2025]

    work page 2019

  30. [30]

    APACrefauthors \ 1917

    ishihara1918tests APACrefauthors Ishihara, S. APACrefauthors \ 1917 . Tests for Colour-Blindness Tests for colour-blindness . Tokyo, Hongo Harukich\= o Handaya

    work page 1917

  31. [31]

    APACrefauthors \ 1976

    jackson1976appraisal APACrefauthors Jackson, D. APACrefauthors \ 1976 . The appraisal of personal reliability The appraisal of personal reliability . meetings of the Society of Multivariate Experimental Psychology, University Park, PA. meetings of the society of multivariate experimental psychology, university park, pa

    work page 1976

  32. [32]

    APACrefauthors \ 2005

    johnson2005ascertaining APACrefauthors Johnson, J.A. APACrefauthors \ 2005 . Ascertaining the validity of individual protocols from web-based personality inventories Ascertaining the validity of individual protocols from web-based personality inventories . Journal of research in personality 39 1 103--129,

    work page 2005

  33. [33]

    , Mukherjee, A

    joshi2019semantic APACrefauthors Joshi, A. , Mukherjee, A. , Sarkar, S. Hegde, C. APACrefauthors \ 2019 . Semantic adversarial attacks: Parametric transformations that fool deep classifiers Semantic adversarial attacks: Parametric transformations that fool deep classifiers . Proceedings of the IEEE/CVF International Conference on Computer Vision Proceedin...

    work page 2019

  34. [34]

    , Westfall, J

    judd2012treating APACrefauthors Judd, C.M. , Westfall, J. Kenny, D.A. APACrefauthors \ 2012 . Treating Stimuli as a Random Factor in Social Psychology: A New and Comprehensive Solution to a Pervasive But Largely Ignored Problem Treating stimuli as a random factor in social psychology: A new and comprehensive solution to a pervasive but largely ignored pro...

    work page doi:10.1037/a0028347 2012

  35. [35]

    , Song, D

    kang2023diffattack APACrefauthors Kang, M. , Song, D. Li, B. APACrefauthors \ 2023 . DiffAttack : Evasion Attacks Against Diffusion - Based Adversarial Purification DiffAttack : Evasion Attacks Against Diffusion - Based Adversarial Purification . A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt \ S. Levine\ ( ), Advances in Neural Information Processi...

    work page 2023

  36. [36]

    , Goodfellow, I

    kurakin2018adversarial APACrefauthors Kurakin, A. , Goodfellow, I. , Bengio, S. , Dong, Y. , Liao, F. , Liang, M. others APACrefauthors \ 2018 . Adversarial attacks and defences competition Adversarial attacks and defences competition . The NIPS'17 Competition: Building Intelligent Systems The nips'17 competition: Building intelligent systems \ ( \ 195--231)

    work page 2018

  37. [37]

    , Karras, T

    tuomas2019prec APACrefauthors Kynk\" a \" a nniemi, T. , Karras, T. , Laine, S. , Lehtinen, J. Aila, T. APACrefauthors \ 2019 . Improved Precision and Recall Metric for Assessing Generative Models Improved precision and recall metric for assessing generative models . H. Wallach, H. Larochelle, A. Beygelzimer, F. d Alch\' e -Buc, E. Fox \ R. Garnett\ ( ), ...

    work page 2019

  38. [38]

    , Singla, S

    laidlaw2021perceptual APACrefauthors Laidlaw, C. , Singla, S. Feizi, S. APACrefauthors \ 2021 . Perceptual Adversarial Robustness: Defense Against Unseen Threat Models Perceptual adversarial robustness: Defense against unseen threat models . International Conference on Learning Representations. International conference on learning representations. APACref...

    work page 2021

  39. [39]

    APACrefauthors \ 2022

    Lakens2022 APACrefauthors Lakens, D. APACrefauthors \ 2022 . Improving Your Statistical Inferences. Improving your statistical inferences. APACrefURL https://lakens.github.io/statistical_inferences/ APACrefURL Retrieved 2022

    work page 2022

  40. [40]

    , Xie, T

    li2020sok APACrefauthors Li, L. , Xie, T. Li, B. APACrefauthors \ 2023 . SoK: Certified Robustness for Deep Neural Networks SoK: Certified Robustness for Deep Neural Networks . 2023 IEEE Symposium on Security and Privacy (SP) 2023 ieee symposium on security and privacy (sp) \ ( 1289-1310)

    work page 2023

  41. [41]

    APACrefauthors \ 1932

    likert1932technique APACrefauthors Likert, R. APACrefauthors \ 1932 . A Technique for the Measurement of Attitudes A technique for the measurement of attitudes . Archives of Psychology 22 140 1--55,

    work page 1932

  42. [42]

    , Morik, M

    linhardt2024an APACrefauthors Linhardt, L. , Morik, M. , Bender, S. Borras, N.E. APACrefauthors \ 2024 . An Analysis of Human Alignment of Latent Diffusion Models An analysis of human alignment of latent diffusion models . ICLR 2024 Workshop on Representational Alignment. Iclr 2024 workshop on representational alignment. APACrefURL https://openreview.net/...

    work page 2024

  43. [43]

    , Zhang, C

    liu2023towards APACrefauthors Liu, F. , Zhang, C. Zhang, H. APACrefauthors \ 2023 . Towards transferable unrestricted adversarial examples with minimum changes Towards transferable unrestricted adversarial examples with minimum changes . 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) 2023 ieee conference on secure and trustworthy ...

    work page 2023

  44. [44]

    APACrefauthors \ 2020

    colorblindness APACrefauthors Lyakhov, D. APACrefauthors \ 2020 . Ishihara blind test cards . Ishihara blind test cards . https://www.kaggle.com/datasets/dupeljan/ishihara-blind-test-cards . [Dataset], Kaggle username: dupeljan

    work page 2020

  45. [45]

    , Makelov, A

    madry2018towards APACrefauthors Madry, A. , Makelov, A. , Schmidt, L. , Tsipras, D. Vladu, A. APACrefauthors \ 2018 . Towards Deep Learning Models Resistant to Adversarial Attacks Towards deep learning models resistant to adversarial attacks . International Conference on Learning Representations. International conference on learning representations. APACr...

    work page 2018

  46. [46]

    APACrefauthors \ 2018

    mahalanobis2018generalized APACrefauthors Mahalanobis, P.C. APACrefauthors \ 2018 . On the generalized distance in statistics On the generalized distance in statistics . Sankhy \=a : The Indian Journal of Statistics, Series A (2008-) 80 S1--S7,

    work page 2018

  47. [47]

    , Moosavi-Dezfooli, S M

    modas2019sparsefool APACrefauthors Modas, A. , Moosavi-Dezfooli, S M. Frossard, P. APACrefauthors \ 2019 . Sparsefool: a few pixels make a big difference Sparsefool: a few pixels make a big difference . Proceedings of the IEEE/CVF conference on computer vision and pattern recognition Proceedings of the ieee/cvf conference on computer vision and pattern re...

    work page 2019

  48. [48]

    na2022unrestricted APACrefauthors Na, D. , Ji, S. Kim, J. APACrefauthors \ 2022 . Unrestricted black-box adversarial attack using gan with limited queries Unrestricted black-box adversarial attack using gan with limited queries . European Conference on Computer Vision European conference on computer vision \ ( \ 467--482)

    work page 2022

  49. [49]

    , Oh, S.J

    naeem2020reliable APACrefauthors Naeem, M.F. , Oh, S.J. , Uh, Y. , Choi, Y. Yoo, J. APACrefauthors \ 2020 . Reliable fidelity and diversity metrics for generative models Reliable fidelity and diversity metrics for generative models . International conference on machine learning International conference on machine learning \ ( \ 7176--7185)

    work page 2020

  50. [50]

    Working Better with Age Working better with age

    oecd2019 APACrefauthors OECD APACrefauthors \ 2019 . Working Better with Age Working better with age . Paris OECD Publishing

    work page 2019

  51. [51]

    GPT-4 Technical Report

    achiam2023gpt APACrefauthors OpenAI , Achiam, J. , Adler, S. , Agarwal, S. , Ahmad, L. , Akkaya, I. Zoph, B. APACrefauthors \ 2024 . GPT-4 Technical Report. Gpt-4 technical report. APACrefURL https://arxiv.org/abs/2303.08774 APACrefURL

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  52. [52]

    DINOv2: Learning Robust Visual Features without Supervision

    oquab2024dinov2learningrobustvisual APACrefauthors Oquab, M. , Darcet, T. , Moutakanni, T. , Vo, H. , Szafraniec, M. , Khalidov, V. Bojanowski, P. APACrefauthors \ 2024 . DINOv2: Learning Robust Visual Features without Supervision. Dinov2: Learning robust visual features without supervision. APACrefURL https://arxiv.org/abs/2304.07193 APACrefURL

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  53. [53]

    , Togashi, R

    otani2023toward APACrefauthors Otani, M. , Togashi, R. , Sawai, Y. , Ishigami, R. , Nakashima, Y. , Rahtu, E. Satoh, S. APACrefauthors \ 2023 . Toward verifiable and reproducible human evaluation for text-to-image generation Toward verifiable and reproducible human evaluation for text-to-image generation . Proceedings of the IEEE/CVF Conference on Compute...

    work page 2023

  54. [54]

    Prolific

    prolific APACrefauthors Prolific APACrefauthors \ 2025 . Prolific. Prolific. https://www.prolific.com/ . [Accessed 3-7-2025]

    work page 2025

  55. [55]

    , Xiao, C

    qiu2020semanticadv APACrefauthors Qiu, H. , Xiao, C. , Yang, L. , Yan, X. , Lee, H. Li, B. APACrefauthors \ 2020 . Semanticadv: Generating adversarial examples via attribute-conditioned image editing Semanticadv: Generating adversarial examples via attribute-conditioned image editing . Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, Aug...

    work page 2020

  56. [56]

    , Roelofs, R

    recht2019imagenet APACrefauthors Recht, B. , Roelofs, R. , Schmidt, L. Shankar, V. APACrefauthors \ 2019 . Do imagenet classifiers generalize to imagenet? Do imagenet classifiers generalize to imagenet? International conference on machine learning International conference on machine learning \ ( \ 5389--5400)

    work page 2019

  57. [57]

    ImageNet Large Scale Visual Recognition Challenge,

    imagenet15russakovsky APACrefauthors Russakovsky, O. , Deng, J. , Su, H. , Krause, J. , Satheesh, S. , Ma, S. Fei-Fei, L. APACrefauthors \ 2015 . ImageNet Large Scale Visual Recognition Challenge ImageNet Large Scale Visual Recognition Challenge . International Journal of Computer Vision (IJCV) 115 3 211-252, APACrefDOI doi:10.1007/s11263-015-0816-y APACrefDOI

    work page doi:10.1007/s11263-015-0816-y 2015

  58. [58]

    , Bachem, O

    sajjadi2018prec APACrefauthors Sajjadi, M.S.M. , Bachem, O. , Lucic, M. , Bousquet, O. Gelly, S. APACrefauthors \ 2018 . Assessing Generative Models via Precision and Recall Assessing generative models via precision and recall . S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi \ R. Garnett\ ( ), Advances in Neural Information Processing S...

    work page 2018

  59. [59]

    , Ilyas, A

    salman2020adversarially APACrefauthors Salman, H. , Ilyas, A. , Engstrom, L. , Kapoor, A. Madry, A. APACrefauthors \ 2020 . Do Adversarially Robust ImageNet Models Transfer Better ? Do Adversarially Robust ImageNet Models Transfer Better ? H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan \ H. Lin\ ( ), Advances in Neural Information Processing Systems A...

    work page 2020

  60. [60]

    APACrefauthors \ 1987

    schuirmann1987comparison APACrefauthors Schuirmann, D.J. APACrefauthors \ 1987 . A comparison of the two one-sided tests procedure and the power approach for assessing the equivalence of average bioavailability A comparison of the two one-sided tests procedure and the power approach for assessing the equivalence of average bioavailability . Journal of pha...

    work page 1987

  61. [61]

    shamsabadi2020edgefool APACrefauthors Shamsabadi, A.S. , Oh, C. Cavallaro, A. APACrefauthors \ 2020 . Edgefool: an adversarial image enhancement filter Edgefool: an adversarial image enhancement filter . ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) Icassp 2020-2020 ieee international conference on acou...

    work page 2020

  62. [62]

    , Sanchez-Matilla, R

    shamsabadi2020colorfool APACrefauthors Shamsabadi, A.S. , Sanchez-Matilla, R. Cavallaro, A. APACrefauthors \ 2020 . Colorfool: Semantic adversarial colorization Colorfool: Semantic adversarial colorization . Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Proceedings of the ieee/cvf conference on computer vision and patte...

    work page 2020

  63. [63]

    , Bhagavatula, S

    sharif2019general APACrefauthors Sharif, M. , Bhagavatula, S. , Bauer, L. Reiter, M.K. APACrefauthors \ 2019 . A general framework for adversarial examples with objectives A general framework for adversarial examples with objectives . ACM Transactions on Privacy and Security (TOPS) 22 3 1--30,

    work page 2019

  64. [64]

    \ Stelldinger, P

    sielmann2023adversarial APACrefauthors Sielmann, K. \ Stelldinger, P. APACrefauthors \ 2023 . Adversarial Perturbations Straight on JPEG Coefficients Adversarial perturbations straight on jpeg coefficients . DAGM German Conference on Pattern Recognition Dagm german conference on pattern recognition \ ( \ 558--573)

    work page 2023

  65. [65]

    APACrefauthors \ 1978

    smith1978color APACrefauthors Smith, A.R. APACrefauthors \ 1978 . Color gamut transform pairs Color gamut transform pairs . ACM Siggraph Computer Graphics 12 3 12--19,

    work page 1978

  66. [66]

    , Weiss, E

    sohl2015deep APACrefauthors Sohl-Dickstein, J. , Weiss, E. , Maheswaranathan, N. Ganguli, S. APACrefauthors \ 2015 . Deep unsupervised learning using nonequilibrium thermodynamics Deep unsupervised learning using nonequilibrium thermodynamics . International conference on machine learning International conference on machine learning \ ( \ 2256--2265)

    work page 2015

  67. [67]

    , Shu, R

    song2018constructing APACrefauthors Song, Y. , Shu, R. , Kushman, N. Ermon, S. APACrefauthors \ 2018 . Constructing Unrestricted Adversarial Examples with Generative Models Constructing Unrestricted Adversarial Examples with Generative Models . S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi \ R. Garnett\ ( ), Advances in Neural Informat...

    work page 2018

  68. [68]

    , Cresswell, J

    stein2023exposing APACrefauthors Stein, G. , Cresswell, J. , Hosseinzadeh, R. , Sui, Y. , Ross, B. , Villecroze, V. Loaiza-Ganem, G. APACrefauthors \ 2023 . Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion model...

    work page 2023

  69. [69]

    , Arbel, M

    sutherland2018demystifying APACrefauthors Sutherland, J. , Arbel, M. Gretton, A. APACrefauthors \ 2018 . Demystifying mmd gans Demystifying mmd gans . international conference for learning representations international conference for learning representations \ ( 6)

    work page 2018

  70. [70]

    Intriguing properties of neural networks

    szegedy2013intriguing APACrefauthors Szegedy, C. , Zaremba, W. , Sutskever, I. , Bruna, J. , Erhan, D. , Goodfellow, I. Fergus, R. APACrefauthors \ 2014 . Intriguing properties of neural networks. Intriguing properties of neural networks. APACrefURL https://arxiv.org/abs/1312.6199 APACrefURL

    work page internal anchor Pith review Pith/arXiv arXiv 2014

  71. [71]

    , Santurkar, S

    tsipras2020imagenet APACrefauthors Tsipras, D. , Santurkar, S. , Engstrom, L. , Ilyas, A. Madry, A. APACrefauthors \ 2020 . From imagenet to image classification: Contextualizing progress on benchmarks From imagenet to image classification: Contextualizing progress on benchmarks . International Conference on Machine Learning International conference on ma...

    work page 2020

  72. [72]

    \ Meade, A.W

    ward2023carelessresponding APACrefauthors Ward, M. \ Meade, A.W. APACrefauthors \ 2023 . Dealing with Careless Responding in Survey Data: Prevention, Identification, and Recommended Best Practices Dealing with careless responding in survey data: Prevention, identification, and recommended best practices \ [Journal Article]. Annual Review of Psychology 74 ...

    work page doi:10.1146/annurev-psych-040422-045007 2023

  73. [73]

    , Schmidt, F.R

    wong2019wasserstein APACrefauthors Wong, E. , Schmidt, F.R. Kolter, J.Z. APACrefauthors \ 2019 . Wasserstein Adversarial Examples via Projected Sinkhorn Iterations Wasserstein adversarial examples via projected sinkhorn iterations . K. Chaudhuri\ R. Salakhutdinov\ ( ), Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 J...

    work page 2019

  74. [74]

    , Liu, H

    xiang2022egm APACrefauthors Xiang, T. , Liu, H. , Guo, S. , Gan, Y. Liao, X. APACrefauthors \ 2022 nov . EGM: An Efficient Generative Model for Unrestricted Adversarial Examples Egm: An efficient generative model for unrestricted adversarial examples . ACM Trans. Sen. Netw. 18 4 , APACrefDOI doi:10.1145/3511893 APACrefDOI APACrefURL https://doi.org/10.114...

    work page doi:10.1145/3511893 2022

  75. [75]

    , Evans, D

    Xu2017FeatureSD APACrefauthors Xu, W. , Evans, D. Qi, Y. APACrefauthors \ 2018 . Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks Feature squeezing: Detecting adversarial examples in deep neural networks . Network and Distributed Systems Security Symposium (NDSS) ,

    work page 2018

  76. [76]

    , Araujo, A

    xue2023diffusion APACrefauthors Xue, H. , Araujo, A. , Hu, B. Chen, Y. APACrefauthors \ 2023 . Diffusion- Based Adversarial Sample Generation for Improved Stealthiness and Controllability Diffusion- Based Adversarial Sample Generation for Improved Stealthiness and Controllability . A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt \ S. Levine\ ( ), Adv...

    work page 2023

  77. [77]

    , Song, Q

    yang2021attacks APACrefauthors Yang, L. , Song, Q. Wu, Y. APACrefauthors \ 2021 . Attacks on state-of-the-art face recognition using attentional adversarial attack generative network Attacks on state-of-the-art face recognition using attentional adversarial attack generative network . Multimedia tools and applications 80 855--875,

    work page 2021

  78. [78]

    , Zhang, Q

    yuan2022natural APACrefauthors Yuan, S. , Zhang, Q. , Gao, L. , Cheng, Y. Song, J. APACrefauthors \ 2022 . Natural Color Fool : Towards Boosting Black -box Unrestricted Attacks Natural Color Fool : Towards Boosting Black -box Unrestricted Attacks . S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho \ A. Oh\ ( ), Advances in Neural Information Processi...

    work page 2022

  79. [79]

    , Zhang, M

    zhang2024constructing APACrefauthors Zhang, A. , Zhang, M. Wischik, D. APACrefauthors \ 2024 . Constructing Semantics - Aware Adversarial Examples with a Probabilistic Perspective Constructing Semantics - Aware Adversarial Examples with a Probabilistic Perspective . A. Globerson\ . \ ( ), Advances in Neural Information Processing Systems Advances in Neura...

    work page 2024

  80. [80]

    , Liu, Z

    zhao2020towards APACrefauthors Zhao, Z. , Liu, Z. Larson, M. APACrefauthors \ 2020 . Towards large yet imperceptible adversarial image perturbations with perceptual color distance Towards large yet imperceptible adversarial image perturbations with perceptual color distance . Proceedings of the IEEE/CVF conference on computer vision and pattern recognitio...

    work page 2020

Showing first 80 references.