{"id":"a9451ba7-e903-4ba4-b930-e06e04147385","arxiv_id":"2505.16638","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Omitting protected attributes can cut disparate impact substantially with negligible accuracy loss, and new model-multiplicity bounds explain when fairer unaware models exist.","lead":"This paper argues that excluding protected attributes from machine learning models, an idea often dismissed as ineffective for fairness, can reduce group-level disparities in outcomes without materially lowering accuracy. It supports this with new theoretical bounds on model multiplicity and with experiments on census and job-training datasets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Proposition 6 is internally inconsistent in its sign conventions, and even repaired it would bound coefficient-zeroing rather than the retrained unaware models used in the experiments; the empirical existence claim stands, so the verdict should remain conditional.","rationale":"The reader's conditional verdict is appropriate. The central existence claim — that unaware models can match accuracy while reducing disparate impact — is supported by the direct empirical comparisons across multiple datasets, model classes, and protected attributes, and the code is provided. My stress-test confirms the reader's weakest assumption about the theory-empirics gap and strengthens it: Proposition 6 not only models a different procedure (zeroing a coefficient rather than retraining) but is internally inconsistent in its sign conventions. The proof's flipped set is empty under the stated c_G < 0 convention, and the numerical example uses an interval that does not match Eq. (24). Proposition 5 likewise has an undefined delta and a missing accuracy argument. These are genuine flaws in the theoretical contribution, but they do not overturn the empirical demonstration. The paper's own Ethical Considerations also appropriately limits the scope of the claims. Thus the paper should be accepted only conditionally on correcting or weakening the affected theoretical propositions and clarifying the relationship between coefficient-zeroing and retrained unaware models.","tokens_in":22335,"tokens_out":8720,"duration_ms":73996,"concrete_test":"Re-derive Proposition 6 from the logit conditions with the stated convention c_G < 0, and recompute the ACS Income numerical example: verify whether the flipping set is {x in B : sigma(c_G) <= f(x) < 0.5} and whether the claimed 14.5-point DI drop equals P(Q)/P(B) under that set. If the example only works with [0.5, 0.70) or with the opposite sign for c_G, the lower bound is vacuous as written.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's advertised claim is to show 'both theoretically and empirically' that FtU can reduce discrimination without accuracy loss. The empirical part is direct and reasonably solid. The weak point is the theoretical support claimed for logistic regression. In Proposition 6 (Section 4.2), with c_G < 0 for the disadvantaged group, Q(c_G) is defined in Eq. (24) as {x in B : f(x) in [sigma(c_G), 0.5]}. The proof's flipped set, however, is written as {x in B : -c_G <= sigma^{-1}(f(x)) < 0}; for c_G < 0 this interval is empty, and the correct condition is c_G <= sigma^{-1}(f(x)) < 0. Thus the proof's key set is mis-stated. The numerical example compounds this: for c_G = -0.86 it takes P(Q) as women with predictions in [0.5, 0.70) = [sigma(-c_G), 0.5), not [sigma(c_G), 0.5), so the claimed 14.5-point DI reduction is not an instance of Eq. (26). Proposition 5 also uses an undefined delta and does not establish that the constructed G lies in R_epsilon(F). Even if these are fixed, the bound concerns a model obtained by setting the protected coefficient to zero, whereas Section 5 retrains unaware models from scratch; no argument connects those two procedures. The bare existence claim survives on the empirical evidence, but the theory advertised as part of the central claim is not currently load-bearing.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper revisits Fairness through Unawareness (FtU). It argues that omitting a protected attribute from a predictive model can reduce disparate impact with little or no loss in accuracy, and it connects this observation to the model-multiplicity literature. The authors provide theoretical results: general upper bounds on model multiplicity and on changes in disparate impact between classifiers with similar accuracy, and a lower bound for logistic regression showing that an 'unaware' model obtained by zeroing the protected coefficient can reduce DI within a bounded accuracy budget. They substantiate the claim empirically on ACS Income, ACS Employment, and Swiss ALMP data with logistic regression and gradient-boosted trees, and they illustrate the implications on a job-training allocation scenario inspired by the Austrian AMS system. The advertised central claim is that FtU can reduce algorithmic discrimination without necessarily reducing accuracy, both theoretically and empirically.","tokens_in":22643,"tokens_out":9087,"duration_ms":73425,"significance":"If the central claim holds, the paper is practically valuable: it challenges the default assumption that adding protected attributes improves accuracy, and it connects FtU to the less-discriminatory-alternative literature. The empirical part is credible and reproducible: code is provided, and the results show accuracy reductions below 1% with DI reductions between 15% and 80% across many settings. The case study on job-training allocation is relevant and clearly presented. The theoretical part, however, is currently not load-bearing because of sign errors and missing arguments in Propositions 5 and 6; these need repair before the advertised 'both theoretically and empirically' claim is supported. With those repairs, this would be a solid contribution to algorithmic fairness and model multiplicity.","major_comments":[{"comment":"The proposition and its proof are internally inconsistent for c_G < 0. Q(c_G) in Eq. (24) is {x in B : f(x) in [sigma(c_G), 0.5]}, which for c_G = -0.86 is the interval of predicted probabilities [0.297, 0.5]. The proof's flipped-set Eq. (95), however, is written as {x in B : -c_G <= sigma^{-1}(f(x)) < 0}, which is empty when c_G < 0; the correct condition is c_G <= sigma^{-1}(f(x)) < 0. The numerical example then identifies Q with women whose predictions lie in [0.5, 0.70) = [sigma(-c_G), 0.5), which is the post-zeroing prediction interval, not the pre-zeroing interval of Eq. (24); hence the claimed 14.5-point DI reduction is not an instance of Eq. (26). In addition, the proof's calibration bound is stated as sigma(-c_G) <= P(Y|X in Q) <= sigma(c_G), which is reversed for c_G < 0, and Eq. (25) with the stated numbers (sigma(-0.86) approx 0.70, P(Q) = 6.9%) gives about 9.7 percentage points, not the claimed 0.7 percentage points, for the accuracy change.","section":"Section 4.2, Eq. (24)-(27) and Appendix B.2 (proof of Proposition 6)"},{"comment":"The statement uses delta in the right-hand side max_{H in R_{epsilon+delta}(B)} without defining delta or relating it to F or epsilon. The proof constructs G> and G< by modifying F on U = {x : H(x) != B(x)}, but it never shows that either G# lies in R_epsilon(F); no accuracy calculation for F is supplied. Consequently the existence of a model in the epsilon-Rashomon set of F with the claimed DI change is not established. The lower bound may be repairable, but as written it is not load-bearing.","section":"Section 4.1, Proposition 5"},{"comment":"Proposition 6 concerns an unaware classifier F' obtained by setting the protected coefficient of the fitted aware model to zero, while the empirical unaware models in Section 5 are retrained from scratch without the protected attribute. The paper does not argue that these two procedures produce the same or similar classifiers, so the lower bound in Proposition 6 does not necessarily explain the empirical DI reductions plotted in Figure 3. This is a gap between theory and experiments; the empirical existence claim survives, but the claimed theoretical explanation for the logistic-regression results is not currently established.","section":"Section 4.2 vs. Section 5"}],"minor_comments":[{"comment":"The DBR column is labeled 'Disparate Benefit Ratio' but the values (0.15, 0.06, etc.) are differences in base rates as defined in Eq. (3); the label and definition should be corrected.","section":"Table 4"},{"comment":"The sentence 'even a moderate difference of 0.05 closer to the margins corresponds to a large difference of 0.18 near the threshold' is unclear; the 0.05 is a probability difference at the margins and the 0.18 is a probability difference at the threshold induced by the same logit offset; please rephrase.","section":"Section 3.2"},{"comment":"lambda_epsilon is defined with arg max, but ties are not addressed; state a tie-breaking rule or define it as a supremum/infimum.","section":"Section 4.1, Eq. (15)"},{"comment":"Besides the sign issue noted in the major comments, the proof would benefit from explicitly naming the pre-sigmoid variable z = sigma^{-1}(f(x)) to avoid confusion between f and f' intervals.","section":"Appendix B.2, Eq. (95)"},{"comment":"The claim of providing 'the first tight bounds on model multiplicity' should be qualified relative to Black et al. [7], whose bound is recovered in Corollary 1; the novelty is tightness under only the cumulative distribution of p(x), and this should be stated more precisely.","section":"Section 1 and Related Work"}],"recommendation":"major_revision","confidential_remarks":"The paper is worth pursuing. The empirical contribution is solid and the case study is compelling. The theoretical section needs careful repair before publication: Propositions 5 and 6 must be corrected, and the numerical example must be reconciled with the stated equations. If the authors can repair the theory or explicitly re-scope the theoretical claims so they match the retrained-model experiments, this would be acceptable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a useful paper with a solid empirical core and two theoretical results that are not yet ready as stated. The main claim—that unaware models can match aware accuracy while cutting disparate impact—survives on the experiments alone, but the theory advertised as supporting it has real gaps.\n\nWhat is actually new: the tight multiplicity bound in Proposition 2 is a genuine improvement over Black et al., and recovering their bound as a corollary is a nice sanity check. The explicit framing of FtU as a less discriminatory alternative is a good reframing. The empirical work is the strongest part: three datasets, two model classes, code provided, and the effect is large and consistent.\n\nThe soft spots are in the theory. Proposition 5 constructs a G but never shows that G has accuracy within epsilon of F, and delta is undefined. Proposition 6 has a sign bug: Q(c_G) is defined as predictions in [sigma(c_G), 0.5], but the proof uses the condition -c_G <= sigma^{-1}(f(x)) < 0, which is empty when c_G < 0; the numerical example then uses [0.5, 0.70) = [sigma(-c_G), 0.5), so it does not instantiate the stated bound. Beyond the sign issue, the theoretical model is coefficient-zeroing while the experiments retrain from scratch, and there is no argument connecting those two procedures. So the lower bound currently does not explain the observed empirical reductions.\n\nNone of this sinks the empirical claim, but it does mean the theory is not load-bearing as written. The authors should either repair the proofs or explicitly weaken the theoretical claims to match what is actually shown. The empirical contribution alone justifies a serious referee.\n\nWho this is for: anyone thinking about modeling choices in high-stakes tabular prediction, especially practitioners who want a clear demonstration that FtU can be a less discriminatory alternative. Also for researchers in model multiplicity, if the proofs get fixed. I would cite it for the experiments and the framing, and I would bring it to a reading group to discuss the theory-practice gap.\n\nRecommendation: send it to peer review, but expect a revision that fixes Propositions 5 and 6 or clearly delimit what they prove.","headline":"The empirical finding is real and worth engaging; the supporting theory needs repair before the advertised claims can be taken at face value.","tokens_in":23140,"tokens_out":2184,"would_cite":true,"duration_ms":19740,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Omitting protected attributes can substantially reduce disparate impact without a meaningful loss in accuracy.","keywords":["fairness through unawareness","model multiplicity","disparate impact","logistic regression","algorithmic fairness","less discriminatory alternatives","demographic parity","tabular data"],"falsifier":"On a fixed logistic-regression task with a binary protected attribute, compare three models on the same data: the aware model, the unaware model retrained without the protected attribute, and the aware model with the protected coefficient set to zero. If the retrained unaware model's disparate-impact reduction is consistently outside the interval predicted by Proposition 6 from the aware model's $Q(c_G)$, or if its accuracy change exceeds $2σ(−c_G)P(X ∈ Q(c_G))$, then the theoretical mechanism does not explain the empirical result and the central claim would need a different argument.","tokens_in":1785,"feed_emoji":"⚖️","tokens_out":5650,"duration_ms":117890,"temperature":0.7,"pith_summary":"The paper argues that Fairness through Unawareness—leaving protected attributes such as sex or race out of a predictive model—has been dismissed too quickly. Its central claim is that, contrary to the common fairness–accuracy trade-off story, omitting a protected attribute can substantially reduce disparate impact, the gap in positive classification rates between groups, while keeping accuracy essentially unchanged. The authors prove this possibility for logistic regression in theory and demonstrate it empirically on large US Census datasets and a Swiss job-training dataset, with both logistic regression and gradient-boosted trees. They connect the effect to model multiplicity: many models with nearly equal accuracy exist, and an unaware model can serve as a less discriminatory alternative without explicit fairness optimization. The paper's practical conclusion is that including protected attributes should require justification, and that unawareness deserves consideration in high-risk deployments.","feed_headline":"Omitting protected traits cuts disparity without accuracy loss","feed_subtitle":"Census and job-training tests show disparity drops while accuracy stays within one percent.","key_machinery":"The central object is the $ε$-Rashomon set $R_ε(F)$: all classifiers whose accuracy is no less than a reference classifier $F$ minus $ε$. The paper's key quantitative mechanism is the distribution of estimated probabilities $p(x)$ near the $0.5$ decision threshold, encoded by the function $e(λ)$, the extra error incurred by flipping all predictions whose label probability is within $λ$ of $0.5$. This yields a tight upper bound on how much two near-optimal classifiers can disagree, and an upper bound on how much their disparate impact can differ that is inversely proportional to the smaller group's size. For logistic regression, the load-bearing mechanism is the sigmoid's steepness at the threshold: a protected-attribute coefficient $c_G < 0$ shifts all disadvantaged logits, and setting that coefficient to zero changes predictions only on the set $Q(c_G)$ of disadvantaged points with predictions in $[σ(c_G), 0.5]$, giving the accuracy and disparate-impact bounds of Proposition 6.","core_discovery":"The paper establishes that an aware logistic-regression classifier can produce a larger disparate impact than the base-rate difference in the data, and that the corresponding unaware classifier—one that does not use the protected attribute—can stay within a small accuracy loss while removing a fixed fraction of that impact. The theoretical engine is the shape of the sigmoid: a fixed group coefficient $c_G < 0$ shifts every disadvantaged member's logit by a constant, and near the 0.5 decision threshold that constant can push many disadvantaged individuals below the threshold even though it has small effect on average probabilities. On real data the paper observes accuracy reductions under 1% with disparate-impact reductions between roughly 15% and 80% across datasets, model classes, and protected attributes. In a real-world use case modeled on an Austrian job-training allocation algorithm, dropping gender left AUROC essentially unchanged while narrowing the gender gap in high-risk assignments.","pith_inferences":["The paper leaves implicit that the near-threshold mechanism should extend to other smooth classifiers with sigmoid-type outputs, so the empirical pattern likely generalizes beyond the two model classes tested.","A testable extension the authors do not run: measure whether the realized disparate-impact reduction from retrained unaware models scales with the aware model's group coefficient $c_G$ and the mass of $Q(c_G)$; if it does not, the lower-bound mechanism is not the whole story.","A natural next comparison, not made here, is Fairness through Unawareness against explicit interventions such as threshold shifts; combining a small threshold change with unawareness might yield even larger disparate-impact reductions than either alone."],"forward_implications":["In tabular logistic-regression settings, deploying the unaware counterpart of an aware model can serve as a less discriminatory alternative, because accuracy stays within about one percent while disparate impact drops by 15% to 80% in the tested settings.","The multiplicity bounds imply that models with equal accuracy can disagree most on points whose predicted label probability is near 0.5, so near-threshold observations are where fairness differences between models concentrate.","For high-stakes profiling, such as allocating job-training programs, omitting gender can leave the ranking quality essentially unchanged while reducing the gender gap in who is denied training.","The paper's framing suggests that model selection over the Rashomon set, rather than accuracy optimization alone, can be the locus of fairness, and that protected attributes deserve explicit justification before inclusion."],"supporting_citations":[{"why":"Defines Fairness through Unawareness and the standard criticism that omitting protected attributes is insufficient and may lower performance.","marker":"[3]"},{"why":"Supplies the model-multiplicity and Rashomon-set framework, plus the earlier disagreement bound that Proposition 2 refines.","marker":"[7]"},{"why":"Defines the group-wise calibration condition used to connect base-rate differences to classifier disparate impact.","marker":"[11]"},{"why":"States the theoretical fairness-accuracy tradeoff for Bayes-optimal predictors that the paper argues is not decisive for finite empirical model classes.","marker":"[14]"},{"why":"Provides the ACS Income and ACS Employment census datasets used in the main empirical comparisons.","marker":"[18]"},{"why":"Provides the Swiss active labour market policy dataset used to reconstruct the real-world job-profiling case.","marker":"[29]"},{"why":"Supplies the published Austrian job-profiling logistic regression model whose gender coefficient the real-world analysis approximates.","marker":"[25]"},{"why":"Documents the public controversy and justifications around the Austrian profiling system that motivate the practical discussion.","marker":"[1]"},{"why":"Formalizes less discriminatory alternatives as a model-selection criterion, which is the paper's practical lens for unawareness.","marker":"[5]"}],"fun_headline_variants":["Unaware classifiers reduce disparity with negligible accuracy loss","Fairness through unawareness slashes bias without accuracy penalty","Omitting protected traits shrinks discrimination, not accuracy","Model multiplicity reveals FtU cuts bias, keeps accuracy"],"cache_read_input_tokens":25216,"weakest_assumption_plain":"The load-bearing premise is that the unaware model studied in the theory—the aware model with the protected group coefficient removed—matches the unaware model built by retraining without that attribute; the paper gives no proof that these two constructions coincide.","fun_headline_variants_meta":{"raw":{"variants":["Unaware classifiers reduce disparity with negligible accuracy loss","Fairness through unawareness slashes bias without accuracy penalty","Omitting protected traits shrinks discrimination, not accuracy","Model multiplicity reveals FtU cuts bias, keeps accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000162,"raw_usage":{"total_tokens":1212,"prompt_tokens":893,"completion_tokens":319,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":509,"completion_tokens_details":{"reasoning_tokens":256}},"tokens_in":509,"tokens_out":319,"duration_ms":2909,"temperature":1.0,"reasoning_tokens":256,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:57:45.038151+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a fixed logistic-regression task with a binary protected attribute, compare three models on the same data: the aware model, the unaware model retrained without the protected attribute, and the aware model with the protected coefficient set to zero. If the retrained unaware model's disparate-impact reduction is consistently outside the interval predicted by Proposition 6 from the aware model's $Q(c_G)$, or if its accuracy change exceeds $2σ(−c_G)P(X ∈ Q(c_G))$, then the theoretical mechanism does not explain the empirical result and the central claim would need a different argument.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the model-multiplicity and Rashomon-set framework, plus the earlier disagreement bound that Proposition 2 refines."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the group-wise calibration condition used to connect base-rate differences to classifier disparate impact."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"States the theoretical fairness-accuracy tradeoff for Bayes-optimal predictors that the paper argues is not decisive for finite empirical model classes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the ACS Income and ACS Employment census datasets used in the main empirical comparisons."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Swiss active labour market policy dataset used to reconstruct the real-world job-profiling case."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the published Austrian job-profiling logistic regression model whose gender coefficient the real-world analysis approximates."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Documents the public controversy and justifications around the Austrian profiling system that motivate the practical discussion."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Formalizes less discriminatory alternatives as a model-selection criterion, which is the paper's practical lens for unawareness."}],"review_version":1}