Pith. sign in

REVIEW 5 major objections 4 minor 24 references

Impact of Iris Pigmentation on Performance Bias in Visible Iris Verification Systems: A Comparative Study

T0 review · 5 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Visible-light iris recognition performs better on blue irises than on dark irises.

desk verdict The paper's headline claim is not supported by its own tables; the blue-vs-dark comparison is confounded by unmatched datasets, but the raw measurements and fairness tables are worth a second look. read the letter →

arxiv 2411.08490 v1 pith:K3HB7HZP submitted 2024-11-13 cs.CV

Geetanjali Sharma , Abhishek Tandon , Gaurav Jaswal , Aditya Nigam , Raghavendra Ramachandra This is my paper · ORCID
classification cs.CV
keywords irisrecognitionvisiblespectrumpigmentationdemographicbiasfairnessmetricsequalerrorratedeeplearningsmartphonebiometrics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that iris pigmentation biases visible-light iris verification: recognition models consistently achieve lower Equal Error Rates and higher True Match Rates on blue irises than on dark irises across multiple smartphone capture devices. The authors compare three recognition systems—a classical image-processing pipeline, a fine-tuned Vision Transformer, and a ResNet-50 with an SVM classifier—on five datasets drawn from 58 subjects, two blue-iris datasets and three dark-iris datasets. They also quantify demographic fairness with Demographic Parity Difference and Equalized Odds Difference, finding larger disparities for dark irises, especially across different phones. If the claim holds, visible-light iris verification carries a systematic accuracy disadvantage for people with dark eyes, and fairness auditing of such systems should be standard practice.

What carries the argument

The load-bearing machinery is a comparative benchmark built from five smartphone-captured iris datasets (two blue, three dark) taken from a single source, processed into normalized iris strips, and fed to three recognition systems: the classical Open-Iris feature extractor, a fine-tuned ViT-b, and a ResNet-50 with SVM classifier. Performance is scored with Equal Error Rate and True Match Rate at two False Match Rate thresholds, and fairness is scored with Demographic Parity Difference and Equalized Odds Difference. These metrics and the shared dataset source are what allow blue-versus-dark and cross-device comparisons to be made at all.

What would settle it

Recompute the blue-versus-dark comparison on the same five datasets while controlling for image quality indicators such as focus, contrast, and visible texture, or matching subjects across color groups by age and skin tone; if the blue advantage shrinks to near zero once those controls are applied, the claim that pigmentation itself drives the bias is not supported. A simpler direct check is to rereport per-color subject and image counts and quality statistics from the source datasets.

Watch

Extended reading notes

Core claim

The central discovery is that visible-light iris verification is not color-neutral. On blue-iris datasets the Open-Iris pipeline achieves EERs of 0.24% and 0.82%, while on dark-iris datasets the same pipeline's EER rises to 4.86%, 8.00%, and 9.16%, with correspondingly lower True Match Rates. The deep models follow the same direction: ViT-b and ResNet-50 show higher EERs and lower TMRs on dark irises than on blue irises, and their absolute performance is generally poor, so the magnitude of the blue advantage depends on model and device. Fairness metrics (DPD and EoD) show the same pattern, with cross-phone dark-iris comparisons reaching DPD values around 65% and same-phone blue-versus-dark comparisons reaching 81% on one device. The paper concludes that recognition systems generally perform better on blue irises, and that pigmentation, model choice, and capture device jointly determine the size of the bias.

Load-bearing premise

The datasets compared as 'blue' and 'dark' are assumed to differ essentially only in iris color, with no per-subject or per-image accounting for quality, lighting, or demographics, so any measured performance gap is attributed to pigmentation rather than to the devices or the people photographed.

Editorial extensions

If this is right

  • Visible-light iris verification systems deployed on smartphones are predicted to reject or downgrade dark-eyed users more often than blue-eyed users, given the same model and capture device.
  • Fairness metrics like DPD and EoD should be reported alongside EER and TMR whenever iris verification is evaluated across demographic groups.
  • Training on diverse iris colors helps generalization, but the paper's results indicate that improvement is model- and device-specific, so a single diverse training set does not remove the bias.
  • For dark-iris recognition, the classical Open-Iris pipeline outperforms the two deep models tested, suggesting that architecture choice matters more in the dark-iris regime.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the bias is driven by physical contrast rather than algorithmic prejudice, then hardware-side interventions such as active NIR illumination or multi-spectral capture could reduce the gap more directly than model retraining.
  • The small sample of 58 subjects and the device-iris-color correlation in the dataset leave open the possibility that part of the measured effect is a device-quality artifact; a matched-pair study on a single device with many subjects per color would isolate the pigmentation component.
  • One testable extension is to generate synthetic dark-iris training data by degrading texture contrast of blue-iris images, and check whether the deep models' dark-iris EER drops toward the blue level.
  • The same protocol could be applied to NIR-captured irises; if the blue-dark gap disappears under NIR, that would confirm the visible-light optics explanation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper studies whether visible-light iris verification accuracy and fairness vary between blue and dark irises. It uses datasets from [16] captured by three smartphone devices (P1, P2, P3), applies three recognition systems (Open-Iris, ViT-b, ResNet-50), and reports EER, TMR, and fairness metrics DPD and EoD. The authors conclude that recognition systems generally perform better on blue irises, with lower EER and higher TMR, and that fairness varies by device and model.

Significance. The question of pigmentation bias in visible iris recognition is timely and practically relevant, and the paper's stated goal of comparing traditional and deep-learning models across multiple devices is a useful contribution. The paper asks well-defined research questions, benchmarks three systems, and includes fairness metrics, which are appropriate for demographic-bias analysis. However, the central claim is not uniformly supported by the presented evidence: device-matched comparisons are mixed, dataset comparability is unverified, and no uncertainty quantification is provided. If the claim were restricted to model- and device-dependent differences, the study would still be informative but would not support a general statement that visible iris systems perform better on blue irises.

major comments (5)
  1. [Section 4.3, Tables 1-2] The conclusion that "recognition systems generally perform better on blue irises" is not supported by the device-matched comparisons. For ResNet-50 on P1, EER is 19.18% for blue and 17.74% for dark irises; on P2, EER is 19.53% for blue and 17.23% for dark, so dark irises actually perform better in both matched cases. Only Open-Iris and ViT-b favor blue, and the magnitude varies considerably by device. Since the only comparisons that hold device fixed are BI-P1 vs DI-P1 and BI-P2 vs DI-P2, the data support "model- and device-dependent differences" rather than a general blue-iris advantage. The conclusion should be revised accordingly.
  2. [Section 4.1, Tables 2-4] The blue and dark iris subsets are not demonstrated to be comparable. The paper reports 58 unique subjects and five device-specific datasets but gives no per-color subject counts, no image-quality statistics, and no demographic information. Device effects within a single iris color are large: for example, Open-Iris TMR at 1% FMR is 43.10% for DI-P2 versus 94.94% for DI-P3 in Table 2. Without matching on image quality, illumination, and subject characteristics, the observed blue-versus-dark differences could be caused by dataset difficulty or capture conditions rather than by iris pigmentation. The authors should either match the blue and dark subsets on quality/difficulty or analyze the device-confounded comparisons separately.
  3. [Section 4.5] The discussion misquotes the tables. It states that "the Open-Iris model achieved an EER of 0.24% for blue irises captured by a P1, whereas it had an EER of 0.82% for dark irises." Table 1 reports 0.82% for BI-P2 (blue), not for dark irises; Table 2 reports dark-iris Open-Iris EERs of 4.86%, 8.00%, and 9.16%. The claim that "Table 2 highlights that dark irises generally exhibit higher error rates compared to blue irises" is also contradicted by the ResNet-50 rows in the same table. Please correct the discussion to match the data.
  4. [Section 4.2] The evaluation protocol uses a single gallery/probe split with no cross-validation, bootstrap, or significance testing, and all reported metrics are point estimates. With only 58 subjects and roughly five images each, the differences between blue and dark subsets, and especially the fairness metrics in Tables 3-5, need confidence intervals or significance tests to be interpretable. Without such uncertainty quantification, the reported gaps cannot be distinguished from sampling variation.
  5. [Section 4.4] The DPD and EoD fairness metrics are not defined in the paper, and the threshold at which they are computed is only mentioned as "0.1" without specifying whether this is FMR, a score threshold, or a normalized threshold. It is also unclear how the fairness comparisons are built from the same gallery/probe splits used for accuracy, and whether the very large values in Table 5 (e.g., DPD 88.22% and EoD 95.98%) are stable given the small sample. Please provide the metric formulas, the threshold basis, and sensitivity analysis or uncertainty estimates.
minor comments (4)
  1. [Figure 2] The illustrative performance numbers in Figure 2 (e.g., EER 0.29, TMR 98.80/98.50) do not match any row in Tables 1 or 2. Please either label the figure as an illustrative example or make the values consistent with the reported results.
  2. [Section 4.4] The text describes EoD = 27.90% as "small" while the paper elsewhere treats higher EoD as indicating greater bias. Please reconcile these statements or define a threshold for what counts as small.
  3. [Figures 4-5] The DET curves are not explicitly linked to the model/device rows in Tables 1 and 2. Please ensure each curve is labelled with the model and dataset it represents so the reader can map figures to tables.
  4. [General] There are several typographical and consistency issues, including "DpD" versus "DPD", "baised" in Figure 2, and inconsistent use of decimal points (e.g., "0.1" vs "0.1%"). A careful proofreading pass would improve clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the blue-vs-dark comparison is an empirical measurement, not a derivation.

full rationale

The paper does not derive a prediction from fitted parameters; it reports an empirical comparison of three iris recognition systems on blue and dark iris subsets from the public dataset of [16]. The central claim that blue irises yield lower EER and higher TMR is a direct reading of Tables 1 and 2, not a quantity defined in terms of the conclusion. The DPD/EoD fairness metrics are computed from the same similarity scores used for EER/TMR, but this is a definitional measurement rather than a circular step: fairness metrics are by construction functions of score distributions. The only self-citation issue is that one of the present authors co-authored the dataset source [16]; however, the dataset is an external, published acquisition used as experimental input, and the paper's conclusions are not guaranteed by that citation. The blue/dark comparison is statistically confounded by unmatched device subsets (e.g., no blue P3 counterpart, no per-color subject counts), but confounding is a validity threat, not circularity. Therefore no circular step is present.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new entities and fits no constants. Its analysis depends on the comparability of the five reused datasets, the choice of fairness-metric threshold, and the stability of error rates computed from 58 subjects. The main unverified link is that observed blue-vs-dark EER differences reflect pigmentation rather than device or image-quality confounds, since no per-color quality statistics are given.

free parameters (2)
  • Fairness metric threshold = 0.1% FMR
    DPD and EoD are computed at the 0.1% FMR threshold (Section 4.4); the choice affects every reported fairness value and is not varied or justified.
  • Gallery/probe split = 2 gallery, 3 probe images per subject
    The protocol in Section 4.2 fixes 6 genuine and 342 imposter comparisons per subject; different splits would change EER and TMR estimates.
assumptions (3)
  • domain assumption The five datasets from [16] are comparable in subject count, image quality, and capture conditions across blue and dark subsets.
    Section 4.1 states 58 unique subjects and five datasets but gives no per-color subject counts or quality matching; if dark images are systematically lower quality, the observed EER gap is confounded.
  • domain assumption DPD and EoD, as defined in Kotwal and Marcel [11], are appropriate fairness metrics for this small-sample comparison.
    Section 3 invokes DPD/EoD without discussing their limitations for small samples or unbalanced groups.
  • domain assumption The genuine/imposter score protocol (2 gallery vs 3 probe images per subject) yields stable EER and TMR estimates.
    Section 4.2; with only 58 subjects and 348 genuine scores, EER estimates have high variance; no confidence intervals are reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Impact of Iris Pigmentation on Performance Bias in Visible Iris Verification Systems: A Comparative Study." pith.science (2026). https://pith.science/paper/K3HB7HZP

@misc{pith2026241108490,
  author       = {Pith},
  title        = {Pith review of: Impact of Iris Pigmentation on Performance Bias in Visible Iris Verification Systems: A Comparative Study},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K3HB7HZP}},
  note         = {Machine review of arXiv:2411.08490}
}
read the original abstract

Iris recognition technology plays a critical role in biometric identification systems, but their performance can be affected by variations in iris pigmentation. In this work, we investigate the impact of iris pigmentation on the efficacy of biometric recognition systems, focusing on a comparative analysis of blue and dark irises. Data sets were collected using multiple devices, including P1, P2, and P3 smartphones [4], to assess the robustness of the systems in different capture environments [19]. Both traditional machine learning techniques and deep learning models were used, namely Open-Iris, ViT-b, and ResNet50, to evaluate performance metrics such as Equal Error Rate (EER) and True Match Rate (TMR). Our results indicate that iris recognition systems generally exhibit higher accuracy for blue irises compared to dark irises. Furthermore, we examined the generalization capabilities of these systems across different iris colors and devices, finding that while training on diverse datasets enhances recognition performance, the degree of improvement is contingent on the specific model and device used. Our analysis also identifies inherent biases in recognition performance related to iris color and cross-device variability. These findings underscore the need for more inclusive dataset collection and model refinement to reduce bias and promote equitable biometric recognition across varying iris pigmentation and device configurations.

Figures

Figures reproduced from arXiv: 2411.08490 by the authors.

Figure 1
Figure 1. Illustrates an iris recognition system comparing blue and dark irises, using similarity scores and analyzing fairness with DPD and EOD metrics to address demo￾graphic biases. These differences in iris pigmentation [7] might affect the system’s ability to accurately match or verify identity, leading to higher false rejection rates for some groups. Highlights the importance of developing algorithms [17][11] that accou… view at source ↗
Figure 2
Figure 2. Illustration of the iris recognition system’s performance on a dataset with vary￾ing iris pigmentation (dark and blue irises) using three different models: Open-Iris, ViT-b, and ResNet-50. The study evaluates how iris pigmentation influences recog￾nition accuracy and assesses fairness through Demographic Parity Difference (DPD) and Equalized Odds Difference (EoD) metrics to identify potential system biases and demog… view at source ↗
Figure 3
Figure 3. Preprocessed blue and dark irises showing reduced pigmentation differences and noise removal (white pixels), highlighting uniformity for biometric analysis [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: DET curve for comparative analysis among three different models. The X-axis indicates the false match rate and the y-axis indicates the false non-match rate of the Iris Pigmentation (Blue color) dataset. higher EERs ranging from 12.53% to 18.10%, and its TMRs are notab…
Figure 5
Figure 5. Figure 5: DET curve for comparative analysis among three different models. The X-axis indicates the false match rate and the y-axis indicates the false non-match rate of the Iris Pigmentation (Dark color) dataset. well across both iris pigmentation types, achieving low EERs and …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 23 canonical work pages

  1. [16]

    Smartphone based visible iris recognition using dee p sparse filtering

    Kiran B Raja, Ramachandra Raghavendra, Vinay Krishna Ve muri, and Christoph Busch. Smartphone based visible iris recognition using dee p sparse filtering. Pat- tern Recognition Letters, 57:33–42, 2015

    work page 2015

  2. [1]

    Thirdeye: Triplet base d iris recognition with- out normalization

    Sohaib Ahmad and Benjamin Fuller. Thirdeye: Triplet base d iris recognition with- out normalization. In 2019 IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS) , pages 1–9. IEEE, 2019

    work page 2019

  3. [2]

    Iris: Iris recognition inference system of the worldcoin project, 2023

    Worldcoin AI. Iris: Iris recognition inference system of the worldcoin project, 2023

    work page 2023

  4. [3]

    Demographic-relian t algorithmic fairness: Characterizing the risks of demographic data collection in the pursuit of fairness

    McKane Andrus and Sarah Villeneuve. Demographic-relian t algorithmic fairness: Characterizing the risks of demographic data collection in the pursuit of fairness. In Proceedings of the 2022 ACM Conference on Fairness, Account ability, and Trans- parency, pages 1709–1721, 2022

    work page 2022

  5. [4]

    Recent advances in biometric te chnology for mobile devices

    Abhijit Das, Chiara Galdi, Hu Han, Raghavendra Ramachand ra, Jean-Luc Duge- lay, and Antitza Dantcheva. Recent advances in biometric te chnology for mobile devices. In 2018 IEEE 9th International Conference on Biometrics Theor y, Appli- cations and Systems (BTAS) , pages 1–11, 2018

    work page 2018

  6. [5]

    How iris recognition works

    John Daugman. How iris recognition works. In The essential guide to image processing, pages 715–739. Elsevier, 2009

    work page 2009

  7. [6]

    An image is worth 16x16 words: Tr ansformers for image recognition at scale

    Alexey Dosovitskiy et al. An image is worth 16x16 words: Tr ansformers for image recognition at scale. arXiv preprint arXiv:2010.11929 , 2020

    arXiv 2010

  8. [7]

    Iris pigmentation as a quantitative trait: variation in populations of european, east asian and south asian ancestry and associa tion with candidate gene polymorphisms

    Melissa Edwards, David Cha, S Krithika, Monique Johnson, Gillian Cook, and Esteban J Parra. Iris pigmentation as a quantitative trait: variation in populations of european, east asian and south asian ancestry and associa tion with candidate gene polymorphisms. Pigment cell & melanoma research , 29(2):141–162, 2016

    work page 2016

Show all 24 references
  1. [8]

    An o ptimized wave- length band selection for heavily pigmented iris recogniti on

    Yazhuo Gong, David Zhang, Pengfei Shi, and Jingqi Yan. An o ptimized wave- length band selection for heavily pigmented iris recogniti on. IEEE transactions on information forensics and security , 8(1):64–75, 2012

    2012

  2. [9]

    Dee p residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Dee p residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  3. [10]

    Deep learning based off-angle iris recognition

    Ehsaneddin Jalilian, Georg Wimmer, Andreas Uhl, and Mah mut Karakaya. Deep learning based off-angle iris recognition. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICA SSP), pages 4048–

    2022

  4. [11]

    Fairness index measu res to evaluate bias in biometric recognition

    Ketan Kotwal and Sébastien Marcel. Fairness index measu res to evaluate bias in biometric recognition. In International Conference on Pattern Recognition , pages 479–493. Springer, 2022

    2022

  5. [12]

    Robust iris segmentation algorithm in non-cooperat ive environments using interleaved residual u-net

    Yung-Hui Li, Wenny Ramadha Putri, Muhammad Saqlain Asla m, and Ching-Chun Chang. Robust iris segmentation algorithm in non-cooperat ive environments using interleaved residual u-net. Sensors, 21(4):1434, 2021

    2021

  6. [13]

    Presenta tion attack detection on visible spectrum iris recognition by exploring inherent characteristics of light field camera

    Ramachandra Raghavendra and Christoph Busch. Presenta tion attack detection on visible spectrum iris recognition by exploring inherent characteristics of light field camera. In IEEE International Joint Conference on Biometrics , pages 1–8. IEEE, 2014

    2014

  7. [14]

    Exploring the usefulness of light field cameras for biometrics: An empi rical study on face and iris recognition

    Ramachandra Raghavendra, Kiran Bylappa Raja, and Chris toph Busch. Exploring the usefulness of light field cameras for biometrics: An empi rical study on face and iris recognition. IEEE Transactions on Information Forensics and Security , 11(5):922–936, 2015

    2015

  8. [15]

    K.B. Raja, R. Raghavendra, and C. Busch. Iris imaging in v isible spectrum using white led. In Proceedings of the IEEE 7th International Conference on Bio metrics: Theory, Applications, and Systems (BTAS) . IEEE, 2015. 14 Geetanjali Sharma et al

    2015

  9. [17]

    Demographic fairness in biometric system s: What do the ex- perts say? IEEE Technology and Society Magazine , 41(4):71–82, 2022

    Christian Rathgeb, Pawel Drozdowski, Dinusha C Frings, Naser Damer, and Christoph Busch. Demographic fairness in biometric system s: What do the ex- perts say? IEEE Technology and Society Magazine , 41(4):71–82, 2022

    2022

  10. [18]

    Multiscale dynamic graph representation for biometric recognition wi th occlusions

    Min Ren, Yunlong Wang, Yuhao Zhu, Kunbo Zhang, and Zhenan Sun. Multiscale dynamic graph representation for biometric recognition wi th occlusions. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2023

    2023

  11. [19]

    Sequeira, Lulu Chen, James Ferryman, Peter Wild, F ernando Alonso- Fernandez, Josef Bigun, Kiran B

    Ana F. Sequeira, Lulu Chen, James Ferryman, Peter Wild, F ernando Alonso- Fernandez, Josef Bigun, Kiran B. Raja, R. Raghavendra, Chri stoph Busch, Tiago de Freitas Pereira, Sebastien Marcel, Sushree Sangeeta Beh era, Mahesh Gour, and Vivek Kanhangad. Cross-eyed 2017: Cross-spe...

    2017

  12. [20]

    Evaluation of combined visible/nir camera for iris authentication on smartphones

    Shejin Thavalengal, Petronel Bigioi, and Peter Corcora n. Evaluation of combined visible/nir camera for iris authentication on smartphones . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 42–49, 2015

    2015

  13. [21]

    There is an elephant in the room: Towards a critique on the use of fairnes s in biometrics

    Ana Valdivia, Júlia Corbera Serrajòrdia, and Aneta Swia niewicz. There is an elephant in the room: Towards a critique on the use of fairnes s in biometrics. AI and Ethics , 3(4):1407–1422, 2023

    2023

  14. [22]

    Nir iris challenge evaluation in non-cooperative environments: Se gmentation and localiza- tion

    Caiyong Wang, Yunlong Wang, Kunbo Zhang, Jawad Muhammad , Tianhao Lu, Qi Zhang, Qichuan Tian, Zhaofeng He, Zhenan Sun, Yiwen Zhang , et al. Nir iris challenge evaluation in non-cooperative environments: Se gmentation and localiza- tion. In 2021 IEEE International joint confere...

    2021

  15. [23]

    Dualsanet: Dual spa tial attention net- work for iris recognition

    Kai Yang, Zihao Xu, and Jingjing Fei. Dualsanet: Dual spa tial attention net- work for iris recognition. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages 889–897, 2021

    2021

  16. [24]

    In de fense of the triplet loss again: Learning robust person re-identificati on with fast approximated triplet loss and label distillation

    Ye Yuan, Wuyang Chen, Yang Yang, and Zhangyang Wang. In de fense of the triplet loss again: Learning robust person re-identificati on with fast approximated triplet loss and label distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition...

    2020

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.