{"id":"d74d7892-9f62-4112-8220-d3e6446ec184","arxiv_id":"2501.16896","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Face recognition models assign different importance to different image frequencies depending on the ethnicity of the face, and intentionally biased models show larger frequency-importance differences.","lead":"This paper tests whether face recognition models rely on different image frequency bands for different ethnicities, using frequency-masking explanations. It reports that low frequencies matter more for African faces, high frequencies more for Asian faces, and that these gaps grow in models trained without one ethnic group.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The frequency-importance measure (Eq. 3) is an uncalibrated OOD perturbation; ethnic differences in low-frequency image statistics could explain the observed 'ethnicity-dependent' importance without any model bias, and Figs. 1–5 show no significance tests.","rationale":"The reader's weakest assumption identifies exactly the load-bearing point: Eq. (3)'s importance scores are not calibrated for the amount of perturbation each frequency band causes, and frequency-masked images are out-of-distribution. My stress-test sharpens this into a concrete, testable confound: ethnic groups differ in low-frequency spectral energy, so the raw similarity drop can differ by ethnicity even for an unbiased model. The paper's own §3.1 limitation statement supports this reading. The second part of the claim (bias amplifies the effect) is additionally weakened by the lack of significance testing, but the more fundamental issue is that the primary measure may not measure what the authors claim. Because the paper could be made convincing by adding the proposed control analysis, a conditional verdict remains appropriate rather than outright rejection. My recommendation is therefore unchanged from the reader's CONDITIONAL, with the condition made more specific: demonstrate that the frequency-importance differences survive normalization by per-band perturbation energy (or some equivalent control for image statistics).","tokens_in":16079,"tokens_out":2780,"duration_ms":31087,"concrete_test":"For each image pair and each frequency band b, compute the spectral energy removed by the mask: E_{b} = ||M_b * F - F||_2 (or the analogous L2 difference in the spatial domain). Then re-compute the mean frequency importance per ethnicity after normalizing h_b by E_b, e.g., h'_b = h_b / E_b. If the ethnicity-dependent ranking and the baseline-vs-biased differences disappear or substantially shrink under this normalization, the headline claim is an artifact of unequal perturbation magnitude rather than differential model reliance. Additionally, apply the same masking pipeline to a deliberately ethnicity-blind model (e.g., a randomly initialized or single-ethnicity-trained network) and check whether similar ethnicity-dependent importance patterns emerge; if they do, the measure does not isolate learned bias.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that FR models assign different frequency importance to different ethnicities, and that this difference grows for more biased models. This rests entirely on the importance measure h_b = |sc(N(j),N(k)) - sc(N(j_{M,b}),N(k_{M,b}))| from Eq. (3), adopted from Ref. [26]. The paper itself acknowledges in §3.1 that frequency-masked images are out-of-distribution for the model, so the observed similarity drop may reflect perturbation artifacts rather than the model's decision-relevant reliance. The critical unaddressed confound is that the magnitude of the perturbation is not controlled across ethnicities: because the frequency bands are defined by Euclidean distance in the DFT grid (§4.4), masking a low-frequency band removes a different amount of spectral energy for images with different low-frequency content. If, for instance, faces of one ethnicity in RFW have more low-frequency energy (due to illumination, skin-tone statistics, or image-acquisition differences), Eq. (3) will produce larger importance values for that ethnicity even for a model that treats all ethnicities identically. The paper's evidence that 'this effect is increased for more biased models' is also based on visual inspection of Figs. 1–5 without statistical tests, error bars, or a control for overall accuracy differences between models. Without controlling for the per-ethnicity perturbation magnitude, the observed frequency-importance differences do not establish ethnicity-dependent spectral priors in FR models.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper investigates ethnicity bias in face recognition (FR) by analyzing how much individual frequency bands contribute to verification decisions. Using the frequency-masking explanation method of Ref. [26], the authors compute a per-band importance score h_b = |sc(N(j),N(k)) - sc(N(j_{M,b}),N(k_{M,b}))| (Eq. 3) for two pre-trained FR models (AdaFace, ElasticFace-Cos) and five ResNet-34 models trained on BUPT-Balanceface (one baseline and four models each trained with one ethnicity excluded). Evaluations on the RFW dataset are used to compare frequency-importance rankings and mean importance distributions across Asian, African, Caucasian, and Indian pairs. The paper's central claim is that different frequencies are important to FR models depending on the ethnicity of the samples, and that this effect is amplified in more biased models, particularly in low-frequency bands.","tokens_in":16439,"tokens_out":4883,"duration_ms":47142,"significance":"If substantiated, the result would be a valuable non-semantic, measurable correlate of demographic bias in face recognition, complementing semantic explanations such as hairstyle or facial hair. The paper has several strengths: it uses standard bias evaluation datasets and protocols (RFW, BFW in the supplementary), considers multiple models with different architectures and losses, follows a published frequency-explanation method, and explicitly acknowledges the out-of-distribution limitation of perturbation-based explanations in Section 3.1. However, the current evidence is descriptive: the central claims rest on visual inspection of plots without significance tests, and the perturbation-magnitude confound identified below is not controlled. The significance is therefore conditional on additional analysis rather than established by the present manuscript.","major_comments":[{"comment":"The central claim that frequency importance differs by ethnicity and increases with model bias is currently supported only by visual inspection of mean ranking plots (Fig. 1) and mean importance bar charts (Figs. 2–5). The text itself acknowledges the large variation between pairs and defers standard deviation plots to the supplementary material (§5.3). Please add inferential statistics over the 6,000-pair ethnicity protocols (e.g., bootstrap confidence intervals for P_{b,e}, permutation tests for ethnicity differences, and paired tests for baseline-vs-biased model differences) and report effect sizes. Without such tests, the observed between-ethnicity differences and their increase under bias cannot be distinguished from sampling variation.","section":"§5.2–§5.3, Figs. 1–5"},{"comment":"The frequency-importance measure in Eq. (3) is not calibrated across ethnicities. Because the DFT bands are defined by Euclidean distance in the frequency grid (§4.4), masking frequency band b always removes the same coordinate set but not the same spectral energy for every image; if ethnic groups differ in low-frequency content (due to illumination, skin-tone statistics, or acquisition differences), Eq. (3) will produce group-dependent importance values even for a model that is entirely ethnicity-agnostic. The out-of-distribution caveat in §3.1 does not address this magnitude confound. Please quantify the per-band spectral energy removed per ethnicity and add a control condition (e.g., equal-energy masking, random-phase perturbation, or pseudo-images with matched spectra) to show that the results reflect model reliance rather than perturbation magnitude.","section":"§3.1, Eq. (3), §4.4"},{"comment":"The comparison supporting the 'more biased models' conclusion is confounded by overall verification accuracy differences. Table 1 shows that each biased model drops sharply on the excluded ethnicity (e.g., M_Afr: 80.25% vs. 92.92% on African), so the absolute similarity differences in Eq. (3) are not comparable across models because the baseline similarity scores differ substantially. Please report relative importance (e.g., the drop divided by the unperturbed similarity score) or otherwise control for the baseline score distribution, and perform the baseline-vs-biased comparison on the same image pairs.","section":"§5.3, Table 1, Eq. (3)"}],"minor_comments":[{"comment":"Equations (4) and (5) reuse the symbol h_b for the raw, min-scaled, and sum-normalized importance values. Moreover, dividing by min(h_b) before sum-normalization is redundant and can be undefined if the minimum is zero; please introduce distinct symbols such as h_b^{(raw)}, h_b^{(min)}, and h_b^{(norm)}.","section":"§3.1, Eqs. (4)–(5)"},{"comment":"The phrase 'rank the obtained mean importance values P_{b,e} for each frequency band b' is ambiguous: it should state explicitly whether the ranking is over ethnicities per band or over bands per ethnicity, and how ties are handled.","section":"§3.3"},{"comment":"The AdaFace ranking is reported as an exception (high frequencies more important for African faces), but this discrepancy is not discussed. A brief interpretation would help the reader understand whether the central claim applies uniformly across models or only to a subset of them.","section":"§5.2, Fig. 1"},{"comment":"There are several typos and grammar issues, including 'its important' (§2.1), 'tent to utilize' (§5.3), 'ethnicitiy' (Eq. 6), and 'bar indicate' (Fig. 3 caption); these should be corrected.","section":"Throughout"},{"comment":"The BFW results are only mentioned as being in the supplementary material; the main text should include a brief summary of whether the BFW findings reproduce the RFW findings.","section":"Supplementary material, BFW"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and relevant question, and the authors are transparent about the method's limitations. However, the headline claim is not yet supported by inferential statistics, and the perturbation-magnitude confound is a genuine threat to validity that requires a control experiment. I see no indication of misrepresentation; the issues are fixable within the scope of the paper, but the current version does not yet warrant acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a worthwhile question and a reasonable first pass, but the paper doesn't yet support its own conclusion. The authors apply a frequency-masking explanation method to ethnicity bias in face recognition and report that different frequencies matter for different ethnicities, and that this difference grows in deliberately biased models. That is a genuinely new angle, and the experimental setup is not lazy: five ResNet models trained on balanced and ethnicity-excluded data, bias measured with SER/STD, plus two off-the-shelf models. Credit where due.\n\nThe problem is the evidence. The main results are visual: mean importance rankings and bar charts, with no significance tests, no error bars in the main figures, and no effect sizes. The paper itself acknowledges in §3.1 that the masking creates out-of-distribution inputs, and later in §5.3 that there is large variation between pairs. There is a sharper confound the paper does not address: the frequency bands are defined by Euclidean distance in the DFT grid, so masking the same band removes different amounts of spectral energy for images with different frequency content. If, say, African faces in RFW have more low-frequency energy (lighting, skin tone, acquisition), the importance score will be larger for that group even for a completely unbiased model. The authors do compare the same pairs across models, which holds image statistics fixed in the model-to-model comparison, but the biased models also have much lower accuracy on the left-out group, and the paper does not control for that. So the second claim—bias increases low-frequency importance—could be a performance confound rather than a shift in spectral reliance.\n\nNone of this kills the idea. The frequency-based route to fairness diagnostics is worth exploring, and the paper provides a concrete protocol to test it. But the claims as written outrun the support. A revised version should include significance tests or at least confidence intervals, show error bars in the main text, and ideally calibrate the importance measure against the amount of spectral energy removed or include a control for image statistics. Releasing code and processed importance scores would help a lot.\n\nBottom line: I would send this to review because the question is important and the work is serious, but I would expect a major revision before publication. I would not cite it as evidence for a causal claim yet, though I might mention it as an unexplored lead.","headline":"A new and sensible question about frequency and ethnicity bias, but the evidence is visual, confounded, and not yet convincing.","tokens_in":16869,"tokens_out":3355,"would_cite":false,"duration_ms":33269,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that ethnicity bias in face recognition is visible in the frequency bands a model relies on, and that this spectral signature grows with training-data bias.","keywords":["face recognition","ethnicity bias","frequency domain","explainability","frequency importance","verification bias","bias amplification"],"falsifier":"Train the same face recognition models on data augmented with frequency-masked images, then rerun the band-masking analysis: if the ethnicity-dependent frequency importance differences disappear, the original observation was an out-of-distribution artifact rather than a genuine bias signature.","tokens_in":15838,"feed_emoji":"👤","tokens_out":7532,"duration_ms":65107,"temperature":0.7,"pith_summary":"This paper argues that a face recognition model's ethnicity bias has a measurable signature in the frequency domain: the model weighs different frequency bands differently depending on the ethnicity of the faces being compared. Using frequency masking, the authors compute a per-band importance score for African, Asian, Caucasian, and Indian verification pairs and compare these scores across seven models. They find that low frequencies are generally more important for African samples and high frequencies more important for Asian samples, and that these differences grow when a model is trained with one ethnicity removed from its data. This offers a non-semantic, quantitative explanation of ethnicity bias, complementary to earlier explanations based on hairstyle, facial hair, makeup, or illumination.","feed_headline":"Face recognition bias shows up in which frequencies a model uses","feed_subtitle":"Different ethnicities are processed with different spectral priorities; the gap widens as model bias grows.","key_machinery":"The operative mechanism is a black-box frequency-masking explanation. Each face image is transformed by the discrete Fourier transform, the spectrum is divided into disjoint frequency bands of width 4 by Euclidean distance, each band is zeroed out in turn, and the image is transformed back; the absolute change in the model's verification similarity score is that band's importance, normalized so the bands sum to 1. Averaging these importance values per ethnicity and per model, and ranking the bands categorically, lets the authors compare which spectral regions drive recognition for each ethnic group and how the pattern shifts with bias.","core_discovery":"The central claim is that different frequency bands carry different importance for face recognition decisions depending on the ethnicity of the input face pair, and that this ethnicity-dependent frequency importance is amplified in more biased models. Across the baseline model and the intentionally biased models, African faces show higher reliance on low-frequency bands than other ethnicities, while Asian faces show higher reliance on high-frequency bands; the two pre-trained models also show ethnicity-dependent differences, though less pronounced. Removing one ethnicity from training systematically shifts the importance profile for that group, most visibly in low-frequency bands. The paper concludes that frequency-based explanations expose a source of ethnicity bias that is invisible to spatial-domain semantic explanations.","pith_inferences":["If the frequency-importance signature is a genuine correlate of bias, it could be turned into an audit metric for face recognition models, detecting disparities even when overall accuracy looks balanced.","Because ethnicity labels are coarse, a natural check is to split the same data by measured skin tone or image brightness; if the low-frequency effect tracks photometric statistics rather than demographic categories, the mechanism is more about image properties than identity groups.","A debiasing experiment could test whether training with frequency-band dropout or a regularization that equalizes per-ethnicity frequency importance reduces verification disparities, turning the diagnostic into a mitigation.","The pairwise analysis could be sharpened by comparing the same face pair across models instead of comparing aggregated distributions, which would remove pair-level variance and isolate the bias-driven shift."],"forward_implications":["For a given model, the relative importance ranking of frequency bands differs by ethnicity, with a recurring pattern of low-frequency dominance for African faces and high-frequency prominence for Asian faces.","Models trained without one ethnicity show larger between-ethnicity gaps in frequency importance, mainly at low-frequency bands, compared with the baseline model.","Excluding an ethnicity from training sharply lowers verification accuracy on that group, and the frequency-importance measurement tracks that drop: the affected group's low-frequency importance increases in most cases.","Two pre-trained models trained on an unbalanced public dataset also display ethnicity-dependent frequency importance, but with smaller between-ethnicity contrasts than the deliberately biased models.","Because the ranking method is categorical, the finding describes trends in spectral reliance rather than exact magnitudes; nominal importance values are reported in supplementary material."],"supporting_citations":[{"why":"Supplies the frequency-based explanation method and the similarity-drop importance measure used throughout the paper.","marker":"[26]"},{"why":"Provides the ethnicity-split face verification benchmark and the evaluation protocol with genuine and impostor pairs.","marker":"[57]"},{"why":"Provides the balanced multi-ethnicity training subsets used to train the baseline and intentionally biased models.","marker":"[58]"},{"why":"Defines the Skewed Error Ratio bias metric and motivates the data-imbalance strategy for amplifying bias.","marker":"[55]"},{"why":"Defines the ElasticFace loss used to train the five models in the study.","marker":"[11]"},{"why":"Supplies one of the pre-trained face recognition models compared in the analysis.","marker":"[33]"},{"why":"Establishes the strategy of training models with one ethnicity subset removed to amplify and explain bias.","marker":"[28]"},{"why":"Motivates the frequency-domain perspective by showing that convolutional neural networks rely on frequency patterns beyond semantic content.","marker":"[54]"}],"fun_headline_variants":["Ethnicity shifts which frequencies face AI relies on","Face recognition bias traced to ethnicity-specific frequencies","Bias in face AI revealed by frequency importance per ethnicity","Different ethnicities need different frequencies in face recognition","Frequency use betrays ethnicity bias in face recognition"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The conclusion rests on the assumption that removing a frequency band and measuring the drop in similarity reveals how much the model relies on that band, rather than simply how surprised the model is by masked images.","fun_headline_variants_meta":{"raw":{"variants":["Ethnicity shifts which frequencies face AI relies on","Face recognition bias traced to ethnicity-specific frequencies","Bias in face AI revealed by frequency importance per ethnicity","Different ethnicities need different frequencies in face recognition","Frequency use betrays ethnicity bias in face recognition"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000157,"raw_usage":{"total_tokens":1129,"prompt_tokens":759,"completion_tokens":370,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":375,"completion_tokens_details":{"reasoning_tokens":297}},"tokens_in":375,"tokens_out":370,"duration_ms":3959,"temperature":1.0,"reasoning_tokens":297,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T05:48:19.501330+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same face recognition models on data augmented with frequency-masked images, then rerun the band-masking analysis: if the ethnicity-dependent frequency importance differences disappear, the original observation was an out-of-distribution artifact rather than a genuine bias signature.","supporting_citations":[{"cited_title":"Beyond Spatial Explanations: Explainable Face Recognition in the Frequency Domain","cited_arxiv_id":"2407.11941","evidence_quote":"Supplies the frequency-based explanation method and the similarity-drop importance measure used throughout the paper."},{"cited_title":"In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020","cited_arxiv_id":null,"evidence_quote":"Defines the Skewed Error Ratio bias metric and motivates the data-imbalance strategy for amplifying bias."}],"review_version":1}