Pith. sign in

REVIEW 4 major objections 5 minor 20 references

Longitudinal Study of Facial Biometrics at the BEZ: Temporal Variance Analysis

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Pith's one-line read A controlled 2.5-year study of more than 400 people shows face-recognition scores drift slowly, with daily appearance changes causing larger score swings than the passage of time.

desk verdict A genuinely useful controlled longitudinal face dataset with clean descriptive results, whose headline inferential claims outrun the reported statistics and need revision before they can be trusted. read the letter →

arxiv 2507.06858 v2 pith:OOBLP6PV submitted 2025-07-09 cs.CV

classification cs.CV
keywords longitudinalbiometricsfacerecognitiontemporalvariancecomparisonscorescontrolledevaluationagingeffectsre-enrollmentdeltascore
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper reports a controlled longitudinal study in which more than 400 participants were repeatedly photographed at a fixed biometric testing facility over roughly two and a half years, and face-recognition algorithms compared each later photo with the enrollment photo. The central claim is that genuine comparison scores drift downward only slightly with elapsed time, while day-to-day variation in a person's appearance produces larger fluctuations than multi-year aging does. The same data show a statistically significant downward trend in match scores, with commercial off-the-shelf algorithms degrading less than open-source ones. The authors use these findings to argue that long-term controlled testing is indispensable and that periodic re-enrollment is warranted.

What carries the argument

The load-bearing object is the delta-score regression. For each subject, the paper takes the self-comparison score at enrollment as baseline and defines $\Delta s = s_{i0} - s_{ij}$ as the drop in similarity between the enrollment image and a later probe; stacking all mated comparisons gives a population dataset in which the slope $\beta_1$ in $\Delta s = \beta_0 + \beta_1 \tilde{t} + \varepsilon$ is the average per-day drift in genuine match score. This pooling converts thousands of raw similarity scores into a single comparable measure of temporal drift, and comparing the residual spread with that slope is what supports the claim that day-to-day variation exceeds multi-year drift.

What would settle it

Fit per-subject random intercepts and random slopes to the same delta-score data used for Equation (9); if the subject-level slopes are widely dispersed or the fixed-effect slope becomes statistically indistinguishable from zero, then the pooled common-trend claim collapses.

Watch

Extended reading notes

Core claim

The discovery, stated on the paper's own terms, is that facial biometric performance is temporally stable at the population level over 2.5 years: high-quality frontal images of the same people still match their enrollment images, and the decline in genuine similarity is small relative to the spread caused by ordinary day-to-day appearance changes. The paper quantifies this by defining a delta score for each mated comparison and fitting a single linear trend; the fitted slope is negative and reported as statistically significant. Across all tested algorithms the direction is the same, but the magnitude differs, with the COTS systems showing only marginal degradation and the open-source systems showing steeper drift. The authors further state that the observed decline rate aligns closely with current EU ID-card renewal guidelines, and that inter-individual score distances remain much larger than intra-individual temporal variation, so recognition remains reliable across the period.

Load-bearing premise

The trend estimate assumes every person follows the same average rate of facial drift, so if different people age at visibly different rates, the single reported slope could be an averaging artifact rather than a true population law.

Editorial extensions

If this is right

  • If the drift is as slow as reported, high-quality frontal enrollment images remain usable across typical document-renewal periods, so current EU ID-card validity assumptions receive empirical support.
  • Because day-to-day appearance changes outweigh long-term aging in controlled settings, single-session accuracy tests may overstate how stable a face-recognition system will appear in longitudinal use.
  • The difference between COTS and open-source drift rates implies algorithm choice changes how frequently users must re-enroll, with open-source deployments needing shorter renewal cycles or adaptive thresholds.
  • The controlled repeated-measurement design gives temporal resolution that operational datasets such as mugshot archives lack, making this dataset a benchmark for future longitudinal biometric studies.
  • Periodic re-enrollment or adaptive algorithms, rather than one-time enrollment, is the practical safeguard the data supports.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable consequence the authors leave implicit is that a per-subject random-slope model could reveal whether a single population slope is hiding meaningful heterogeneity, such as faster facial drift in older or younger participants.
  • Because the reported extrapolation rests on a linear trend beyond the observed 2.5 years, the claimed alignment with EU renewal policy should be treated as a projection until data covering a full renewal cycle exist.
  • Since the K13 images are high-quality and controlled, the result likely upper-bounds real-world stability; testing the same cohort with lower-quality capture images would show whether capture conditions, rather than aging, dominate score drift in practice.
  • The finding that daily variation exceeds 2.5-year drift suggests operational systems could improve by re-baselining after large appearance changes rather than on a fixed calendar schedule.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper reports a longitudinal face-recognition study conducted at the Biometric Evaluation Center (BEZ) over roughly 2.5 years, with 492 enrolled participants and more than 2,400 controlled frontal K13 images. The authors evaluate five face-recognition algorithms, report score distributions and EER values, analyze the effect of capture angle, and fit a pooled linear regression of a per-subject delta score against time since enrollment (Eq. 9). The central claims are that genuine match scores fluctuate more between individual days than over the entire measurement period, and that there is a clear, statistically significant downward trend in genuine match scores with time since enrollment, with the observed decline aligning closely with EU ID-card renewal guidelines.

Significance. If its central statistical claims were properly supported, this would be a valuable contribution: the controlled BEZ dataset offers unusually dense, regularly sampled longitudinal face data, and the EER and angle analyses in Sections 4.2 and 4.3 are clean, internally consistent descriptive results. The delta-score analysis addresses a question of direct practical relevance to re-enrollment intervals and to the design of longitudinal biometric evaluations. However, the main quantitative claims are currently not supported by the reported statistics: the pooled regression slope, its uncertainty, and any variance decomposition are absent, despite the Conclusion asserting statistical significance and the Abstract asserting day-to-day versus long-term variance relationships.

major comments (4)
  1. [Section 4.4 and Eq. (9)] The pooled linear regression in Eq. (9) is the sole quantitative basis for the Conclusion's claim of a 'clear, statistically significant downward trend,' but the paper never reports the estimated slope beta_1, its standard error, confidence interval, p-value, or any goodness-of-fit measure for any of the five algorithms. Table 8 reports only per-year means and standard deviations of delta scores, which cannot establish statistical significance of a temporal trend. The authors must report the full regression output (slope, uncertainty, and significance) for each algorithm, or the significance claim should be removed.
  2. [Section 3.5 and Eq. (9)] The regression in Eq. (9) pools all mated comparisons and treats each observation as independent, as the authors explicitly acknowledge: 'The regression is performed over the entire set of mated comparisons, without nested modeling per individual.' Because each subject contributes many repeated measurements, the effective sample size is far smaller than the total number of comparisons, and the standard errors and p-values are likely overstated. A mixed-effects model with random intercepts (and possibly random slopes) per subject, or at least cluster-robust standard errors, is needed before the 'statistically significant' conclusion can be drawn. This is a load-bearing issue for the paper's central temporal-drift claim.
  3. [Abstract and Section 4.4] The Abstract's claim that 'these scores fluctuate more significantly between individual days than over the entire measurement period' is not supported by any reported variance decomposition. Section 4.4 makes a similar assertion ('intra-day similarity scores fluctuate stronger than inter-day similarity scores') but provides no quantitative comparison of intra-day versus inter-day variance. The authors should define the two variance components, compute them, and report a test or effect-size estimate; otherwise this headline claim should be removed or substantially softened.
  4. [Section 4.4 and Conclusion] The Conclusion states that the 'observed rate of temporal score decline aligns closely with the European Union's current ID-card renewal guidelines,' but no renewal interval is specified, no policy document is cited, and no quantitative mapping between the fitted slope and any renewal period is provided. This inference is not derivable from the reported regression (whose output is absent) and should be either justified with explicit numbers or removed.
minor comments (5)
  1. [Section 3.5, Eq. (7)] The notation si0 is ambiguous: it should be defined explicitly as the self-comparison score between a subject's enrollment image and itself, not merely as a 'baseline score,' because the delta-score sign convention depends on this definition.
  2. [Table 2] Table 2 contains duplicated rows (D 1.1 appears twice) and inconsistent totals (D 0 lists 2779 measurements, while the individual subsets sum to 2767 + 12 + 2480 + 287 + 2476 + 4). The table should be cleaned and the filtering steps presented as a coherent flow.
  3. [Section 3.4] There is a typo: 'algorithmns' should be 'algorithms.'
  4. [Section 3.5, Eq. (8)] The notation for relative time is inconsistent: Eq. (8) writes 'tilde t_tj' while the surrounding text uses 'tilde t'; the notation should be unified.
  5. [Section 4.2 and Table 5] The EER values are reported with an implausible number of significant digits (3.94 x 10^-6 for four algorithms), and the identical values across COTS1, COTS2, ArcFace, and AdaFace are not discussed. The authors should state whether these are exact or rounded and whether the thresholds differ.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the negative temporal trend is a transparently labeled least-squares fit of the delta-score data (Eq. 9), the delta-score definition (Eq. 7) embeds no fitted parameter, the extrapolation is flagged as hypothetical, and all self-citations are non-load-bearing infrastructure references; remaining weaknesses are statistical rather than circular.

full rationale

The paper's derivation chain is: raw mated comparison scores → delta score Δs = si0 − sij (Eq. 7) → relative time ˜t = tij − ti0 (Eq. 8) → pooled linear regression Δs = β0 + β1·˜t + ε (Eq. 9) → a negative fitted slope, reported qualitatively and via year-wise statistics in Table 8. At no step does the output reduce to its input by construction. Eq. 7 is a shift-and-flip of the raw comparison score; it contains no fitted parameter, so the downward trend is a property of the data fit, not an artifact of the definition. Eq. 9 is explicitly introduced as a model ('The assumption is that all subjects follow a common temporal trend'), and the pooling limitation is stated openly ('The regression is performed over the entire set of mated comparisons, without nested modeling per individual'), so the trend is presented as a fit, not as a derivation or a statistically forced prediction. The forward extrapolation in §4.4 is labeled 'hypothetical.' The abstract's day-vs-period variation claim and the conclusion's 'statistically significant' wording are asserted without a variance decomposition or reported slope, confidence interval, or p-value, but that is a statistical support problem, not circularity. The self-citations ([5], [7], [11], [12]) describe the LokiMk2 station, a usability study, prior synthetic-face data work, and the analysis-infrastructure concept; none carries the temporal-drift claim, which rests solely on the present dataset, so no load-bearing self-citation exists and no uniqueness theorem is imported. Consequently there is no step where an equation equals its input or a fitted value is renamed as a prediction; the honest finding is no significant circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The core quantitative claim rests on a pooled linear fit (Eq. 9) with a hand-chosen real-world threshold (T = 0.755), the implicit assumption that a single enrollment image is a stable baseline, and the stated assumption of a common linear trend across subjects. No new entities are postulated. The headline day-versus-period variance claim is not connected to any fitted quantity in the text.

free parameters (3)
  • beta_1 (temporal drift slope) = Not reported numerically.
    Eq. (9) fits beta_0 + beta_1 * t_tilde to pooled delta scores; beta_1 is the quantity presented as the measured temporal drift, but its value, confidence interval, and p-value are never stated.
  • beta_0 (intercept) = Not reported numerically.
    Linear regression intercept for the pooled delta-score model in Eq. (9).
  • Operating threshold T = 0.755.
    Chosen by hand in Section 4.2 as a 'realistic' operating threshold for the angle-analysis FRR computations; a modeling choice, not a measured quantity.
assumptions (4)
  • domain assumption All subjects share a common linear temporal trend, and the pooled sample points are treated as independent.
    Explicitly stated in Section 3.5: 'The assumption is that all subjects follow a common temporal trend... without nested modeling per individual.' The repeated-measures structure is acknowledged but its effect on significance claims is not handled.
  • domain assumption The initial enrollment image is a stable, time-invariant baseline for each subject.
    All delta scores measure change relative to the single enrollment capture (Eq. 7-8), so any quirk of that capture (expression, lighting, glasses) propagates into every delta score for that subject.
  • domain assumption Comparison scores across different FR algorithms can be interpreted as linear-scaled similarities in [0,1], with delta scores comparable across subjects.
    Section 3.5 defines scores in [0,1]; Section 4.4 notes algorithms are not directly comparable and separate thresholds are used, undercutting the cross-algorithm slope comparison.
  • ad hoc to paper Linear extrapolation of the fitted trend is a meaningful tool for forecasting future recognition performance.
    Sections 4.4 and 5 extrapolate the trendline to suggest a future point where scores fall below operational thresholds. The paper labels this hypothetical, but the extrapolation feeds the discussion of re-enrollment cycles.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Longitudinal Study of Facial Biometrics at the BEZ: Temporal Variance Analysis." pith.science (2026). https://pith.science/paper/OOBLP6PV

@misc{pith2026250706858,
  author       = {Pith},
  title        = {Pith review of: Longitudinal Study of Facial Biometrics at the BEZ: Temporal Variance Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OOBLP6PV}},
  note         = {Machine review of arXiv:2507.06858}
}
read the original abstract

This study presents findings from long-term biometric evaluations conducted at the Biometric Evaluation Center (bez). Over the course of two and a half years, our ongoing research with over 400 participants representing diverse ethnicities, genders, and age groups were regularly assessed using a variety of biometric tools and techniques at the controlled testing facilities. Our findings are based on the General Data Protection Regulation-compliant local bez database with more than 238.000 biometric data sets categorized into multiple biometric modalities such as face and finger. We used state-of-the-art face recognition algorithms to analyze long-term comparison scores. Our results show that these scores fluctuate more significantly between individual days than over the entire measurement period. These findings highlight the importance of testing biometric characteristics of the same individuals over a longer period of time in a controlled measurement environment and lays the groundwork for future advancements in biometric data analysis.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 18 canonical work pages

  1. [1]

    Israel National Cyber Directorate, Israel national cy- ber directorate – gov.il , https : / / www . gov . il / en / departments / israel _ national _ cyber _ directorate / govil - landing - page, Accessed: 2025-04-29, 2025

  2. [2]

    Michigan State University Biometrics Research Group, Biometrics research group at michigan state univer- sity, https : / / biometrics . cse . msu . edu, Accessed: 2025-04-29, 2025

  3. [3]

    Longitudinal Study of Automatic Face Recognition,

    L. Best-Rowden and A. K. Jain, “Longitudinal Study of Automatic Face Recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 40, no. 1, pp. 148–162, Jan. 2018, ISSN : 1939-3539. DOI: 10 . 1109 / TPAMI . 2017 . 2652466. [On- line]. Available: https://ieeexplore.ieee. org/document/7815403/figures (visited on 04/10/2025)

  4. [4]

    Longitudinal study of fin- gerprint recognition,

    S. Yoon and A. K. Jain, “Longitudinal study of fin- gerprint recognition,” en, Proceedings of the National Academy of Sciences , vol. 112, no. 28, pp. 8555– 8560, Jul. 2015, ISSN : 0027-8424, 1091-6490. DOI: 10 . 1073 / pnas . 1410272112. [Online]. Avail- able: https : / / pnas . org / doi / full / 10 . 1073/pnas.1410272112 (visited on 04/10/2025)

  5. [5]

    Customizable Presentation Attack Detection for Improved Resilience of Biometric Applications Us- ing Near-Infrared Skin Detection,

    T. Scheer, M. Rohde, R. Breithaupt, N. Jung, and R. Lange, “Customizable Presentation Attack Detection for Improved Resilience of Biometric Applications Us- ing Near-Infrared Skin Detection,” en,Sensors, vol. 24, no. 8, p. 2389, Jan. 2024, Number: 8 Publisher: Mul- tidisciplinary Digital Publishing Institute, ISSN : 1424-

  6. [6]

    Robust identification and segmentation of the outer skin lay- ers in volumetric fingerprint data,

    A. Kirfel, T. Scheer, N. Jung, and C. Busch, “Robust identification and segmentation of the outer skin lay- ers in volumetric fingerprint data,” Sensors, vol. 22, no. 21, p. 8229, 2022, Open Access under CC BY 4.0. DOI: 10 . 3390 / s22218229. [Online]. Available: https://www.mdpi.com/1424- 8220/22/ 21/8229

  7. [7]

    Forschungsbericht: Zur usability von egates im testlabor des biometriee- valuationszentrums,

    B. Paul, S. K ¨uhne, and T. V oigt, “Forschungsbericht: Zur usability von egates im testlabor des biometriee- valuationszentrums,” Universit¨at Hamburg und RWTH Aachen, Hamburg/Aachen, Tech. Rep., Dec. 2022, Gef¨ordert durch die Deutsche Forschungsgemeinschaft (DFG), Projektzeitraum 2021–2024

  8. [8]

    [On- line]

    MetaHuman — Realistic Person Creator , en-US. [On- line]. Available: https://www.unrealengine. com/en-US/metahuman (visited on 05/06/2025)

Show all 20 references
  1. [9]

    [Online]

    Character Creator: 3D Character Design Software . [Online]. Available:https://www.reallusion. com / character - creator / default . html (visited on 05/06/2025)

  2. [10]

    [Online]

    Human Generator, en-US. [Online]. Available:https: //www.humgen3d.com (visited on 05/06/2025)

  3. [11]

    Enhancing Resilience in Biometric Research: Generation of 3D Synthetic Face Data Using Advanced 3D Character Creation Techniques from High-Fidelity Video Games and Animation,

    F. E. Bl ¨umel, M. Schulz, R. Breithaupt, N. Jung, and R. Lange, “Enhancing Resilience in Biometric Research: Generation of 3D Synthetic Face Data Using Advanced 3D Character Creation Techniques from High-Fidelity Video Games and Animation,” en, Sensors, vol. 24, no. 9, p. 275...

  4. [12]

    Konzept f¨ur ein Zentrum zur Evaluation und Entwicklung biometrischer Systeme,

    A. von Twickel, J. Breuer, and R. Breithaupt, “Konzept f¨ur ein Zentrum zur Evaluation und Entwicklung biometrischer Systeme,” V ereinbarkeit von Biome- trieforschung und Datenschutz , vol. DuD • Daten- schutz und Datensicherheit, no. 1, 2020

  5. [13]

    ArcFace: Additive Angular Margin Loss for Deep Face Recognition,

    J. Deng, J. Guo, J. Yang, N. Xue, I. Kotsia, and S. Zafeiriou, “ArcFace: Additive Angular Margin Loss for Deep Face Recognition,” 2018, Publisher: arXiv Version Number: 4. DOI: 10 . 48550 / ARXIV . 1801 . 07698 . [Online]. Available: https : / / arxiv . org / abs / 1801 . 0769...

  6. [14]

    M. Kim, A. K. Jain, and X. Liu, AdaFace: Quality Adaptive Margin for Face Recognition , Version Num- ber: 2, 2022. DOI: 10 . 48550 / ARXIV . 2204 . 00964. [Online]. Available: https : / / arxiv . org/abs/2204.00964 (visited on 04/10/2025)

  7. [15]

    Dlib-ml: A Machine Learning Toolkit,

    D. E. King, “Dlib-ml: A Machine Learning Toolkit,” Journal of Machine Learning Research 10, pp. 1755- 1758, 2009

  8. [16]

    Deep Face Recognition,

    O. M. Parkhi, A. Vedaldi, and A. Zisserman, “Deep Face Recognition,” en, inProcedings of the British Ma- chine Vision Conference 2015 , Swansea: British Ma- chine Vision Association, 2015, pp. 41.1–41.12, ISBN : 978-1-901725-53-7. DOI: 10.5244/C.29.41. [On- line]. Available: h...

  9. [17]

    International Organization for Standardization, Infor- mation technology — Biometric presentation attack de- tection, 2023

  10. [18]

    International Organization for Standardization, Infor- mation technology — V ocabulary, 2022

  11. [19]

    BSI, BSI Technical Guideline TR-03166, Technical Guideline for Biometric Authentication Components in Devices for Authentication , Sep. 2024. [Online]. Available: https://www.bsi.bund.de. 11

  12. [8220]

    [Online]

    DOI: 10.3390/s24082389. [Online]. Avail- able: https://www.mdpi.com/1424- 8220/ 24/8/2389 (visited on 05/06/2025)

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.