Pith. sign in

REVIEW 2 major objections 8 minor 1 cited by

Unraveling Machine Behavior by Multi-Level Bias Analysis and Detection: Methodology and Application to Computer Vision

T0 review · 2 major / 8 minor · reviewed 2026-07-09 · glm-5.2

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Bias leaves a detectable fingerprint inside neural networks at three depths

desk verdict Solid internal-bias detection framework; the gap between internal signals and output-level fairness is real but manageable read the letter →

arxiv 2607.07236 v1 pith:EUW5STGL submitted 2026-07-08 cs.CV

Ignacio Serna , Aythami Morales , Julian Fierrez This is my paper
classification cs.CV
keywords biasnetworkneuralclassificationdetectionmethodmodelsactivations
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that when a convolutional neural network learns from biased training data, the bias is not merely visible in its outputs—it is physically inscribed at three distinct internal levels, each of which can be independently measured. At the level of the learned latent space, biased models produce a geometry where underrepresented groups are more poorly distributed: their samples cluster less cleanly by class, and this structural deficit is quantifiable with a Kolmogorov–Smirnov test on neighbor-probability distributions. At the level of neuron activations, underrepresented groups produce systematically weaker responses in the final convolutional layers, because the network has learned their task-relevant patterns less effectively—a gap measurable with a Mann–Whitney U test. At the deepest level, the convolutional filter parameters themselves encode identifiable bias signatures: a secondary neural network trained only on the weights of task models can classify which demographic group was overrepresented during training, with no access to inputs or outputs. The paper validates all three detectors on more than 127,000 models across face biometrics and a colored-digit benchmark, and shows that as training data moves from heavily skewed toward balanced, the internal disparity—and with it the detectability—decreases smoothly rather than collapsing abruptly.

What carries the argument

Three detection mechanisms operate on a single trained model without retraining it. SpaceBias computes, for each sample in the penultimate latent space, the probability that its k nearest neighbors share its class label, then compares the resulting distributions across demographic groups using the two-sample Kolmogorov–Smirnov statistic. ActivationBias extracts the maximum activation across feature maps in the final convolutional layer for each input, then compares per-group distributions using the normalized Mann–Whitney U statistic (the probability of superiority), yielding a rank-biserial correlation as the bias indicator. WeightBias treats the convolutional filter tensors of a task model

What would settle it

A model trained on perfectly balanced data that nonetheless produces a strongly biased latent space (high KS statistic), low activation for a minority group (high Λ), and a weight signature classifiable as biased by the Level-4 detector—while showing no output disparity—would demonstrate that internal signals are noise rather than bias indicators. Conversely, a model with balanced internals across all three levels but significant output disparity would show the detectors miss real bias.

Watch

Extended reading notes

Core claim

The central claim is that bias in CNNs propagates through a hierarchy of internal representations—latent geometry, layer activations, and filter parameters—and that each level carries an independently detectable, statistically quantifiable trace of that bias. The trace is not a binary artifact: it scales continuously with the severity of the training-data skew, contracting smoothly as the distribution approaches balance and vanishing in unbiased models. This means bias is not an opaque property of the black box but a structured, multi-level phenomenon that can be localized, measured, and potentially targeted for intervention at specific points within the network.

Load-bearing premise

The paper assumes that internal representational disparities—poor latent-space clustering, lower activations, biased filter patterns—are reliable indicators of harmful bias. But the authors themselves note that these signals are correlated with, not proof of, discriminatory output behavior, and that balanced data does not guarantee balanced representations. If internal disparities can exist without output harm, or if output bias can persist without detectable internal traces,

Editorial extensions

If this is right

  • Fairness auditors could inspect a deployed model's internal layers for bias signatures without needing access to the original training data or running large-scale evaluation sets, provided the audit sample is large enough (the paper finds ~1,000 images per group sufficient for reliable statistical tests).
  • The smooth degradation of detection as bias severity decreases suggests a natural threshold for 'acceptable' internal disparity: regulators could set detection-sensitivity targets rather than relying solely on output-parity metrics that may miss compensated bias.
  • The finding that a weight-based detector trained on mild bias transfers upward to strong bias (but not vice versa) implies that future weight-level bias monitors should be calibrated to the subtlest bias one wants to catch, not the most obvious.
  • If internal disparities can exist without output disparities—as the paper acknowledges—then fairness audits that examine only outputs may miss models whose biased representations are currently compensated by downstream layers but could resurface under distribution shift or fine-tuning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The three-level hierarchy may not be independent: if biased filter parameters (Level 4) produce biased activations (Level 3) which produce a biased latent geometry (Level 2), then the levels are causally linked, and intervening at the earliest level (weights) might automatically correct the downstream levels. The paper does not test this causal chain directly, but the consistent ordering across le
  • The cross-severity transfer asymmetry—mild-bias detectors recognizing strong bias but not the reverse—resembles a signal-detection pattern where strong bias is an amplified version of a weak signature. If this generalizes, it would mean bias signatures have a consistent 'shape' that scales with severity rather than changing qualitatively, which would simplify the design of universal bias detectors
  • The observation that balanced training data does not guarantee balanced internal representations (the standard unbiased model was flagged in 52% of draws) implies that data-level debiasing may be insufficient: some representational asymmetries persist and require architectural or loss-function-level interventions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 8 minor

Summary. This paper proposes a multi-level taxonomy for analyzing bias in CNNs, spanning the learned latent space (Level 2: SpaceBias), layer activations (Level 3: ActivationBias), and convolutional filter parameters (Level 4: WeightBias). SpaceBias is a new method using neighbor-probability distributions and a Kolmogorov-Smirnov test; ActivationBias extends prior work (InsideBias) with a Mann-Whitney U test; WeightBias extends prior work (IFBiD) by training a secondary network to classify bias from filter weights alone. Experiments span two domains—gender classification on DiveFace and digit classification on colored-MNIST—with over 127,000 trained models and a controlled severity sweep. The central finding is that internal representational disparities are systematically detectable at all three levels and decrease smoothly as training data approaches balance.

Significance. The paper makes a genuine contribution by providing a unified, multi-level framework that moves bias analysis inside the network rather than treating it as a black box. The experimental scale is impressive: 127,000+ models, two independent domains, and a severity sweep that directly measures how detection degrades with bias strength. The SpaceBias method (Level 2) is entirely new and well-motivated: the use of neighbor-probability distributions with a non-parametric KS test is a principled, training-free approach to quantifying latent-space geometry. The cross-severity transfer experiment (§5.3) is a particularly insightful finding—showing that a detector trained on mild bias transfers upward to strong bias but not vice versa. The statistical methodology is generally sound: non-parametric tests are appropriately chosen for non-normal distributions, and the Bonferroni correction at Level 2 is a responsible choice. The reproducibility of the experimental design (public datasets, controlled bias severity, clear architecture specifications) is a strength.

major comments (2)
  1. The paper validates internal disparity signals against the training-distribution skew, not against output-level performance gaps. This is the load-bearing gap. Table 1 shows the Unbiased ResNet model has accuracy gaps across ethnic groups (95.50 vs 95.35 vs 96.11%), and Appendix B (Table B.10) reveals the standard Unbiased model is flagged biased in 52% of draws at 1,000 images, while Unbiased(+) is flagged only 2%. The authors acknowledge this (§5.2.3: 'balanced training data does not guarantee a balanced representation'), but this acknowledgment actually exposes the core issue: if internal signals fire on models with balanced training data and near-equal output performance, their practical utility for fairness auditing is unclear. The paper never directly compares internal-signal detections against output-based fairness metrics (e.g., accuracy gaps, demographic parity differences) on a
  2. §5.1.2 and Table 4: The severity sweep shows detection performance decreasing smoothly from Severity I to IV, but the paper does not report the corresponding output-level performance gaps at each severity level. Table 3 reports per-color accuracy but not per-group gaps. Without knowing whether Severity IV models (60/20/20 split) actually produce discriminatory outputs, we cannot assess whether the 86% recall (Level 2) and 30% recall (Level 3) at Severity IV represent missed detections of real bias or false alarms on models with negligible output disparity. A table correlating internal-signal magnitude with output-level accuracy gaps across the severity sweep would substantially strengthen the practical claim. This is particularly important because the paper positions these methods as complementary to traditional output-based evaluation (Table 6), yet never directly benchmarks against it.
minor comments (8)
  1. §3.1, Eq. (1): The neighbor-probability formula uses exp(-||xi - xj||^2) without a temperature or bandwidth parameter. The choice of bandwidth affects the probability distribution and hence the KS statistic. The authors should discuss sensitivity to this choice or justify the fixed bandwidth.
  2. §3.1, Definition 1: The latent-space bias indicator Gamma_v is defined as the KS statistic D_n, which depends on sample size. The authors later threshold on Gamma (§5.1.2, Gamma >= 0.06) and argue it is 'largely insensitive to sample size,' but the KS statistic's distribution under H0 does depend on n. The claim should be qualified: the statistic is bounded in [0,1] but not invariant.
  3. Table 1: The Unbiased(+) model is trained with 3x more data (54,000 images) than the Unbiased model (18,000). The improved activation uniformity and lower false-positive rate could be attributable to more data rather than better balance per se. This confound should be acknowledged.
  4. §5.2.1, Figure 7: The 10^lambda rescaling is stated to be for display only, but the choice of base-10 exponent is unusual. A log-scale axis or a different monotonic transform might be more transparent. Clarifying that the Mann-Whitney statistics are computed on raw lambda would help readers.
  5. §5.3, Table 5: The cross-severity transfer experiment is described in prose but lacks a formal table. Given its importance for the practical guidance ('train on the mildest bias one intends to catch'), a dedicated table showing transfer accuracy in both directions would strengthen this important finding.
  6. Appendix A.1: The learning rate schedule is listed as '[schedule, e.g., constant/reduce-on-plateau, factor [..], patience [..]]'—this appears to be an unfilled template placeholder. The actual schedule should be specified for reproducibility.
  7. The paper uses 'gender' as a proxy for sex (§4.1.1) and acknowledges this simplification. A brief note on the distinction between gender (social) and sex (biological) and why the proxy is used would be appropriate given the paper's focus on bias.
  8. Table 6: The 'Applicability' column for WeightBias says 'Reduced' without elaboration. The text (§6) explains the limitations (requires training thousands of models, not transferable), but this should be reflected in the table.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; derivation chain is empirically grounded and self-contained

full rationale

The paper proposes three bias detection methods, each with an independent derivation chain. SpaceBias (Level 2) computes neighbor-probability distributions in the latent space and applies a Kolmogorov-Smirnov test — the statistic Γ is computed from data, not fitted to bias labels, and the detection threshold is calibrated on unbiased models at a fixed false-positive rate. ActivationBias (Level 3) computes final-layer activations λ and applies a Mann-Whitney U test — the indicator Λ is a rank-biserial correlation on observed activations, not a quantity defined in terms of the bias it claims to detect. WeightBias (Level 4) is standard supervised learning: a secondary network is trained on models with known bias labels and evaluated on held-out models from a separate data partition. None of these methods define their output (bias detection) in terms of their input (internal representations) by construction. The two self-citations (InsideBias, IFBiD) are explicitly acknowledged as prior conference papers being extended, and the extensions add independent content (Mann-Whitney U test, the entirely new SpaceBias method, the unified taxonomy, and the MNIST severity sweep). The central novel contribution — the multi-level taxonomy and SpaceBias — is independently grounded. The skeptic's concern that internal signals are validated against training-distribution bias rather than output-level performance gaps is a validity concern, not a circularity concern: the paper does not claim to derive output bias from internal signals by definition. Score 1 reflects the minor, non-load-bearing self-citations for two of three methods, with the central framework standing independently.

Assumptions & free parameters 5 free parameters · 4 assumptions · 1 invented entities

The paper introduces one invented entity (the weight-based bias detector network) which is empirically validated. Free parameters are mostly detection thresholds calibrated to a 5% false-positive rate, which is standard practice. The domain assumptions are the main fragility point—they are empirically supported but not formally derived.

free parameters (5)
  • N (neighborhood size) = 100
    Fixed for headline results; sensitivity analysis in Fig. 5-6 shows stability across 10-100. Chosen to maximize separation.
  • alpha (significance level) = 0.01
    Chosen as 'slightly more conservative' than 0.05 due to high-powered tests at n=6000 per group. Stated in §3.1.
  • Gamma (KS detection threshold) = 0.06
    Set so false-positive rate in unbiased models is 5%. Stated in §5.1.2.
  • rho (activation detection threshold) = 0.32
    Set so false-positive rate in unbiased models is 5%. Stated in §5.2.3.
  • Detector architecture hyperparameters (d, dropout rate) = d varies; dropout=0.1
    Dropout chosen as best among {0.0, 0.1, 0.2, 0.3}. Architecture variants explored in Appendix C.
assumptions (4)
  • domain assumption A biased latent space is one where a particular group is poorly distributed within it (§3.1).
    Foundational premise for SpaceBias. The connection between neighbor-probability distributions and classification difficulty is reasonable but not formally proven for the linear softmax head.
  • domain assumption Underrepresented groups exhibit lower activation levels in final convolutional layers (§3.2).
    Empirical observation from prior InsideBias work. Used as the basis for ActivationBias. The paper provides evidence but treats it as an established fact.
  • domain assumption Bias is encoded in convolutional filter parameters in identifiable patterns (§3.3).
    Premise for WeightBias. Justified empirically by the detector achieving 90% accuracy, but the nature of the patterns is not characterized.
  • domain assumption CNNs are representative of neural network bias behavior generally.
    All experiments use CNNs. The paper acknowledges (§7) that extension to transformers is future work.
invented entities (1)
  • Bias detection network psi(·|Theta) independent evidence
    purpose: Secondary neural network trained to classify bias type from convolutional filter weights of task models.
    Falsifiable: tested on held-out models with known bias labels, achieving up to 90% accuracy. Cross-severity transfer experiment provides additional evidence. Not a postulated physical entity but a trained classifier.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unraveling Machine Behavior by Multi-Level Bias Analysis and Detection: Methodology and Application to Computer Vision." pith.science (2026). https://pith.science/paper/EUW5STGL

@misc{pith2026260707236,
  author       = {Pith},
  title        = {Pith review of: Unraveling Machine Behavior by Multi-Level Bias Analysis and Detection: Methodology and Application to Computer Vision},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EUW5STGL}},
  note         = {Machine review of arXiv:2607.07236}
}
read the original abstract

This study investigates the presence and propagation of bias within Neural Networks through a comprehensive multi-level analysis spanning the learned latent space, layer activations, and the network's parameters. Based on this taxonomy, we propose three bias detection approaches: 1) SpaceBias (new method), which characterizes the latent space prior to the final classification layer using neighbor-probability distributions and quantifies bias with the two-sample Kolmogorov-Smirnov test on the per-group distributions. 2) ActivationBias (extension of the existing method InsideBias), which analyzes the activations of neural network filters and quantifies bias via a Mann-Whitney U test, based on the observed fact that underrepresented groups exhibit lower activation levels in the final convolutional layers. 3) WeightBias (extension of the existing method IFBiD), which uses a secondary neural network trained to identify biased patterns directly in the parameters of task-specific models. Unlike conventional methods, which assess neural network outcomes and treat the model as a black box, our proposed techniques provide insight into how biases manifest within the network architecture itself at different levels, offering a more nuanced and detailed understanding. Experiments are conducted on two complementary applications: gender classification in the DiveFace dataset (72,000 face images) and digit classification on a colored-MNIST benchmark with controlled bias severity. In total, more than 127,000 models with varying degrees and types of bias were trained and evaluated. The severity sweep shows that the internal disparity, and with it the detection performance, decreases smoothly as the training distribution approaches balance. The results highlight the importance of methods that provide deeper insight into the behavior of AI models.

Figures

Figures reproduced from arXiv: 2607.07236 by the authors.

Figure 1
Figure 1. Taxonomy of the effects of bias in the learning process of neural networks. 1. A new method (entirely new in this paper), SpaceBias, the Level-2 latent-space analysis based on neighbor￾probability distributions and the Kolmogorov–Smirnov test, was not part of either conference papers. 2. Extension of InsideBias. ActivationBias extends InsideBias with deeper analyses of activations and a Mann–Whitney U test for inter… view at source ↗
Figure 2
Figure 2. Methodology diagram for analyzing bias at 4 levels: 1) Model Performance; 2) Latent Learned Space; 3) Layer Activations; and 4) Model Parameters. Level 3 (layer activations). Intermediate activations have drawn comparatively little attention as a bias signal, even as evidence accumulates that networks lean on shortcuts: Geirhos et al. [28] frame shortcut learning in general terms, Sagawa et al. [29] show that overpa… view at source ↗
Figure 3
Figure 3. Measuring the Learned Latent Space. 1 st: Compute Clustering Probability. For each sample, calculate the clustering probability pi considering N neighbors. The probability pi of a sample represents the likelihood that neighboring samples in the latent space belong to the same class (e.g. female or male) as the sample under consideration. 2nd: Analyze Probability Distribution. Determine the distribution of clustering… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Distribution of clustering probabilities pi for Caucasian biased and unbiased models. The probability is computed using Eqn. 2 with neighbors N = 100. Caucasian images have higher probabilities than Asian or African/Indian images (higher spike around 1). 5.1. Level 2: …
Figure 5
Figure 5. Figure 5: Kolmogorov–Smirnov (KS ) test statistics and p-values evaluated across ResNet models. Each column represents a specific model variant, plotting the pairwise statistics and p-values of probability distributions between three demographic groups against the number of neig…
Figure 6
Figure 6. Figure 6: Kolmogorov–Smirnov (KS ) test statistics and p-values evaluated across MNIST models. Each column shows the average and standard deviation of 100 models for each degree of bias severity, plotting the pairwise statistics and p-values of probability distributions between …
Figure 7
Figure 7. Figure 7: Per-layer activation across the ResNet models. Group-average activation at every convolutional layer, shown as 10λ , for the three demographic groups (Asian, African/Indian, Caucasian) and, as an out-of-domain control, 1,000 random ImageNet images processed identically…
Figure 8
Figure 8. Figure 8: Activation λ distribution for the last convolutional layer of the Biased ResNet models for the different demographic groups. Red Bias IRed Bias IIRed Bias IIIRed Bias IVUnbiased MODEL 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Activation λ 1 2 3 4 5 6 0 100 200 300 400 500 Frequency …
Figure 9
Figure 9. Figure 9: (Left) Avg. Activation λ for the last convolutional layer of the Red Biased (severity I-IV) and Unbiased models across digit colors. (Right) Activation λ distribution for the Red Biased I and Unbiased models for the different digit colors. 5.2.3. Detection at scale (co…
Figure 10
Figure 10. Figure 10: Detection ROC: Severity IV Recall vs. Unbiased False Positive Rate. 5.3. Detecting Bias in Parameters of the Convolution Filters (Level 4) At this level, a secondary network ψ is trained to classify the bias of a task model only using its convolutional filters, withou…
Figure 11
Figure 11. Figure 11: General architecture of a bias detector with the 1 [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. AIriskEval-edu Demo: Auditing of Pedagogical Risks in Educational Explanations

    cs.AI 2026-07 conditional novelty 4.0 of 10

    A demo audits K-12 explanations for five pedagogical risks with localized evidence and rationales, reporting that a locally fine-tuned Llama-3.1-8B beats GPT-5.5 on most metrics—on a benchmark the authors built themselves.

Reference graph

Works this paper leans on

46 extracted references · 46 canonical work pages · cited by 1 Pith paper

  1. [1]

    Stone, R

    P. Stone, R. Brooks, E. Brynjolfsson, R. Calo, O. Etzioni, G. Hager, J. Hirschberg, S. Kalyanakrishnan, E. Kamar, S. Kraus, et al., Artificial intelligence and life in 2030, One Hundred Year Study on Artificial Intelligence: Report of the 2015-2016 Study Panel (2016) 52

    work page

  2. [2]

    Szegedy, W

    C. Szegedy, W. Zaremba, I. Sutskever, J. B. Estrach, D. Erhan, I. Goodfellow, R. Fergus, Intriguing properties of neural networks, in: International Conference on Learning Representations (ICLR), Banff, Canada, 2014

    work page 2014

  3. [3]

    Tommasi, N

    T. Tommasi, N. Patricia, B. Caputo, T. Tuytelaars, A deeper look at dataset bias, Springer International Publish- ing, 2017, pp. 37–55

    work page 2017

  4. [4]

    Zhang, W

    Q. Zhang, W. Wang, S.-C. Zhu, Examining cnn representations with respect to dataset bias, in: AAAI Confer- ence on Artificial Intelligence, volume 32 ofAAAI’18, AAAI Press, 2018, pp. 4464–4473. 20

    work page 2018

  5. [5]

    A. Wang, A. Narayanan, O. Russakovsky, Revise: A tool for measuring and mitigating bias in visual datasets, in: European Conference on Computer Vision, Springer International Publishing, 2020, pp. 733–751

    work page 2020

  6. [6]

    Buolamwini, T

    J. Buolamwini, T. Gebru, Gender shades: Intersectional accuracy disparities in commercial gender classification, in: S. A. Friedler, C. Wilson (Eds.), Conference on Fairness, Accountability and Transparency, volume 81 of Proceedings of Machine Learning Research, New York, NY , USA, 2018, pp. 77–91

    work page 2018

  7. [7]

    M. Alvi, A. Zisserman, C. Nellåker, Turning a blind eye: Explicit removal of biases and variation from deep neural network embeddings, in: European Conference on Computer Vision (ECCV), Munich, Germany, 2018, pp. 556–572

    work page 2018

  8. [8]

    Serna, A

    I. Serna, A. Morales, J. Fierrez, N. Cebrian, M. Obradovich, I. Rahwan, Algorithmic discrimination: Formu- lation and exploration in deep learning-based face biometrics, in: AAAI Workshop on Artificial Intelligence Safety (SafeAI), New York, NY , USA, 2020, pp. 146–152

    work page 2020

Show all 46 references
  1. [9]

    DeAlcala, I

    D. DeAlcala, I. Serna, A. Morales, J. Fierrez, J. Ortega-Garcia, Measuring bias in AI models: An statistical approach introducing N-Sigma, in: IEEE Conf. on Computers, Software, and Applications (COMPSAC), 2023, pp. 1167–1172

    2023

  2. [10]

    Solano, J

    I. Solano, J. Fierrez, A. Morales, et al., Balancing tails when comparing distributions: Comprehensive equity index (CEI) with application to bias evaluation in operational face biometrics, Pattern Recognition 179 (2026) 113616

    2026

  3. [11]

    Morales, J

    A. Morales, J. Fierrez, R. Vera-Rodriguez, R. Tolosana, Sensitivenets: learning agnostic representations with application to face recognition, IEEE Trans. on Pattern Analysis and Machine Intelligence 43 (2021) 2158–2164. doi:https://doi.org/10.1109/TPAMI.2020.3015420

    2021 doi

  4. [12]

    LeCun, L

    Y . LeCun, L. Bottou, Y . Bengio, P. Haffner, Gradient-based learning applied to document recognition, Proceed- ings of the IEEE 86(11) (1998) 2278–2324

    1998

  5. [13]

    Serna, A

    I. Serna, A. Peña, A. Morales, J. Fierrez, InsideBias: Measuring bias in deep networks and application to face gender biometrics, in: IAPR Intl. Conf. on Pattern Recognition (ICPR), IEEE, 2021, pp. 3720–3727

    2021

  6. [14]

    Serna, D

    I. Serna, D. DeAlcala, A. Morales, J. Fierrez, J. Ortega-Garcia, IFBiD: Inference-free bias detection, in: AAAI Workshop on Artificial Intelligence Safety (SafeAI), 2021

    2021

  7. [15]

    Stock, M

    P. Stock, M. Cisse, Convnets and imagenet beyond accuracy: Understanding mistakes and uncovering biases, in: European Conference on Computer Vision (ECCV), Springer International Publishing, 2018, pp. 498–512

    2018

  8. [16]

    B. Kim, R. Khanna, O. Koyejo, Examples are not enough, learn to criticize! criticism for interpretability, in: Advances in Neural Information Processing Systems (NIPS), Barcelona, Spain, 2016, p. 2288–2296

    2016

  9. [17]

    Schaaf, O

    N. Schaaf, O. de Mitri, H. B. Kim, A. Windberger, M. F. Huber, Towards measuring bias in image classification, in: International Conference on Artificial Neural Networks (ICANN), Springer, Bratislava, Slovakia, 2021, pp. 433–445

    2021

  10. [18]

    Ntoutsi, P

    E. Ntoutsi, P. Fafalios, U. Gadiraju, V . Iosifidis, W. Nejdl, M.-E. Vidal, S. Ruggieri, F. Turini, S. Papadopoulos, E. Krasanakis, et al., Bias in data-driven artificial intelligence systems — An introductory survey, Wiley Interdisciplinary Reviews: Data Mining and Knowledge ...

    2020

  11. [19]

    Mehrabi, F

    N. Mehrabi, F. Morstatter, N. Saxena, K. Lerman, A. Galstyan, A survey on bias and fairness in machine learning, ACM Computing Surveys 54 (2021) 115:1–115:35. doi:10.1145/3457607

    2021 doi

  12. [20]

    Caton, C

    S. Caton, C. Haas, Fairness in machine learning: A survey, ACM Computing Surveys 56 (2024) 1–38. doi:10. 1145/3616865

    2024

  13. [21]

    M. Hort, Z. Chen, J. M. Zhang, F. Sarro, M. Harman, Bias mitigation for machine learning classifiers: A comprehensive survey, ACM Journal on Responsible Computing 1 (2023). doi:10.1145/3610403. 21

    2023 doi

  14. [22]

    Terhörst, J

    P. Terhörst, J. N. Kolf, M. Huber, F. Kirchbuchner, N. Damer, A. M. Moreno, J. Fierrez, A. Kuijper, A compre- hensive study on face recognition biases beyond demographics, IEEE Transactions on Technology and Society 3 (2022) 16–30

    2022

  15. [23]

    DeAndres-Tame, R

    I. DeAndres-Tame, R. Tolosana, P. Melzi, et al., Second FRCSyn-onGoing: Winning solutions and post- challenge analysis to improve face recognition with synthetic data, Information Fusion 120 (2025) 103099. doi:https://doi.org/10.1016/j.inffus.2025.103099

    2025 doi

  16. [24]

    Liang, P

    H. Liang, P. Perona, G. Balakrishnan, Benchmarking algorithmic bias in face recognition: An experimental ap- proach using synthetic faces and human evaluation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 4977–4987

    2023

  17. [25]

    Glüge, M

    S. Glüge, M. Amirian, D. Flumini, T. Stadelmann, How (not) to measure bias in face recognition networks, in: Artificial Neural Networks in Pattern Recognition, Springer International Publishing, 2020, pp. 125–137

    2020

  18. [26]

    Creager, D

    E. Creager, D. Madras, J.-H. Jacobsen, M. Weis, R. Zemel, Flexibly fair representation learning by disentan- glement, in: Proceedings of the 36th International Conference on Machine Learning (ICML), PMLR, 2019, pp. 1436–1445

    2019

  19. [27]

    T. Wang, J. Zhao, M. Yatskar, K.-W. Chang, V . Ordonez, Balanced datasets are not enough: Estimating and mitigating gender bias in deep image representations, in: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 5310–5319. doi:10.1109/I...

    2019 doi

  20. [28]

    Geirhos, J.-H

    R. Geirhos, J.-H. Jacobsen, C. Michaelis, R. Zemel, W. Brendel, M. Bethge, F. A. Wichmann, Shortcut learning in deep neural networks, Nature Machine Intelligence 2 (2020) 665–673. doi:10.1038/s42256-020-00257-z

    2020 doi

  21. [29]

    Sagawa, P

    S. Sagawa, P. W. Koh, T. B. Hashimoto, P. Liang, An investigation of why overparameterization exacerbates spurious correlations, in: Proceedings of the 37th International Conference on Machine Learning (ICML), volume 119 ofProceedings of Machine Learning Research, PMLR, 2020, ...

    2020

  22. [30]

    Neuhaus, M

    Y . Neuhaus, M. Augustin, V . Boreiko, M. Hein, Spurious features everywhere–large-scale detection of harmful spurious features in imagenet, in: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 20235–20246

    2023

  23. [31]

    Kirichenko, P

    P. Kirichenko, P. Izmailov, A. G. Wilson, Last layer re-training is sufficient for robustness to spurious correla- tions, in: International Conference on Learning Representations (ICLR), 2023

    2023

  24. [32]

    G. E. Hinton, S. Roweis, Stochastic neighbor embedding, in: Advances in Neural Information Processing Systems (NeurIPS), volume 15, MIT Press, Cambridge, MA, USA, 2002, p. 857–864

    2002

  25. [33]

    Goldberger, G

    J. Goldberger, G. E. Hinton, S. Roweis, R. R. Salakhutdinov, Neighbourhood components analysis, in: Advances in Neural Information Processing Systems (NeurIPS), volume 17, MIT Press, 2004, pp. 513–520

    2004

  26. [34]

    M. D. Zeiler, R. Fergus, Visualizing and understanding convolutional networks, in: European Conference on Computer Vision (ECCV), Springer, Zurich, Switzerland, 2014, pp. 818–833

    2014

  27. [35]

    LeCun, Y

    Y . LeCun, Y . Bengio, G. Hinton, Deep learning, Nature 521 (2015) 436–444

    2015

  28. [36]

    Glorot, Y

    X. Glorot, Y . Bengio, Understanding the difficulty of training deep feedforward neural networks, in: Y . W. Teh, M. Titterington (Eds.), Conference on Artificial Intelligence and Statistics, volume 9 ofProceedings of Machine Learning Research, PMLR, Chia Laguna Resort, Sardin...

    2010

  29. [37]

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, L. Fei-Fei, ImageNet: A large-scale hierarchical image database, in: Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2009, pp. 248–255. 22

    2009

  30. [38]

    Q. Cao, L. Shen, W. Xie, O. M. Parkhi, A. Zisserman, Vggface2: A dataset for recognising faces scross pose and age, in: International Conference on Automatic Face & Gesture Recognition (FG), IEEE, Lille, France, 2018, pp. 67–74

    2018

  31. [39]

    K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Las Vegas, NV , USA, 2016, pp. 770–778

    2016

  32. [40]

    Ranjan, S

    R. Ranjan, S. Sankaranarayanan, et al., Deep learning for understanding faces: machines may be just as good, or better, than humans, IEEE Signal Processing Magazine 35 (2018) 66–83

    2018

  33. [41]

    D. Bau, J. Zhu, H. Strobelt, A. Lapedriza, B. Zhou, A. Torralba, Understanding the role of individual units in a deep neural network, Proceedings of the National Academy of Sciences 117 (2020)

    2020

  34. [42]

    Serna, Discovering intersectional bias via directional alignment in face recognition embeddings, arXiv preprint arXiv:2510.15520 (2025)

    I. Serna, Discovering intersectional bias via directional alignment in face recognition embeddings, arXiv preprint arXiv:2510.15520 (2025)

    2025

  35. [43]

    Peña, et al., Addressing bias in LLMs: Strategies and application to fair AI-based recruitment, in: Proc

    A. Peña, et al., Addressing bias in LLMs: Strategies and application to fair AI-based recruitment, in: Proc. of the AAAI/ACM Conference on AI, Ethics, and Society (AIES), volume 8, 2025, p. 1976–1987

    2025

  36. [44]

    A. Peña, I. Serna, A. Morales, J. Fierrez, FairCVtest Demo: Understanding bias in multimodal learning with a testbed in fair automatic recruitment, in: ACM Intl. Conf. on Multimodal Interaction (ICMI), 2020, pp. 760–761. Also presented at European Conf. on Computer Vision, ECCV 2020

    2020

  37. [45]

    J. Deng, J. Guo, E. Ververas, I. Kotsia, S. Zafeiriou, RetinaFace: Single-Shot Multi-Level Face Localisation in the Wild, in: Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Seattle, Washington, USA, 2020, pp. 5202–5211

    2020

  38. [46]

    bottleneck

    W. F. Schmidt, M. A. Kraaijveld, R. P. Duin, Feed forward neural networks with random weights, in: Interna- tional Conference on Pattern Recognition (ICPR), IEEE Computer Society, The Hague, Netherlands, 1992, pp. 1–4. 23 Appendix A. Model Architectures Layer Type Configuratio...

    1992

Pith tools

Reviewed July 9, 2026 · model on record in the stance chip above.