{"id":"763dd25f-1930-4afd-8735-d731c0c789a2","arxiv_id":"2502.09765","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A differentiable adjusted parity loss based on soft balanced accuracy trains fair representations without adversarial components, improving demographic parity, equalized odds, and sensitive-feature accuracy on Adult and COMPAS datasets.","lead":"This paper introduces a new training loss, Differential Adjusted Parity, that aims to make machine learning models both accurate and fair by penalizing accuracy differences across sensitive groups. The authors show on two standard datasets that the method can reduce bias metrics while keeping task accuracy high, without the unstable adversarial training used by many prior methods.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Test-set hyperparameter selection and untuned baselines make the reported fairness/accuracy gains unverified as an unbiased estimate.","rationale":"The reader's weakest assumption is exactly the point I find most load-bearing: the paper's quantitative comparisons are selected on the test set. The strongest claim, that DAP improves demographic parity, equalized odds, and sensitive-feature accuracy by the stated percentages, cannot be assessed as an unbiased claim unless hyperparameters are chosen on data not used for final evaluation and baselines receive a comparable tuning effort. I also considered the Table 1 result that balanced and unbalanced soft accuracies show no significant difference, which undermines the 'key element' narrative, but that concern attacks the novelty framing rather than the central comparative claim. The missing explicit loss equation is a reproducibility issue, not the primary threat to the empirical conclusion. The comparison against only adversarial baselines is a scope limitation, but the abstract claims superiority only over adversarial approaches, so it is not internally inconsistent. Given the conditional verdict already issued, the correct action is to require the validation-based re-evaluation: the method may still be effective, but the current evidence does not support the headline margins as unbiased estimates.","tokens_in":10161,"tokens_out":5377,"duration_ms":61751,"concrete_test":"Split each dataset into train/validation/test (for example, 60/20/20 instead of the current 175:25 split, holding out the test portion). Select beta and Omega on the validation set using only validation adjusted parity and fairness metrics; after selection, evaluate once on the untouched test set. Repeat for each baseline with an equivalent validation-based grid search over its own hyperparameters, and report the full grid distributions rather than only the best test-set point. If DAP's test-set gains over best-tuned baselines remain within the reported ranges, the concern is resolved; if the gains shrink below the claimed margins or reverse on some metrics, the empirical superiority claim must be scaled back.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central comparative claim (up to 22.5%, 44.1%, and 40.1% improvements over adversarial baselines) rests on selecting beta and Omega over a 100-combination grid using metrics computed on the test set. Section 5.4 defines the grid, and Section 5.5 states: 'We also obtain an adjusted parity metric for comparing models and selecting best performing hyperparameters.' The reported headline numbers therefore appear to be the best grid point for the test set, not a configuration chosen on a held-out validation split. Meanwhile, the baseline models (CFAIR, LAFTR, FAIR, NODEBIAS) are adapted from reference [14], but no equivalent hyperparameter search or tuning budget is described. Without a matched validation-based selection protocol for both DAP and the baselines, the measured advantage may reflect selection bias and asymmetric tuning rather than a property of the DAP objective. This concern is load-bearing because the abstract's quantified superiority claims are exactly the values that would be affected by such selection.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a differentiable variant of the adjusted parity metric, called Differential Adjusted Parity (DAP), as a unified training objective for fair representation learning. The adjusted parity metric is generalized from binary to N sensitive domains through a normalization term γ, with a derivation for odd N in Appendix A. The authors define soft balanced accuracy (Eqs. 7–12) to make the metric differentiable and state that combining it with the standard cross-entropy loss yields the DAP loss, controlled by hyperparameters β and Ω. Experiments on Adult and COMPAS compare DAP with three adversarial debiasing methods (CFAIR, LAFTR, FAIR) and an unconstrained baseline (NODEBIAS), claiming improvements of up to 22.5% in demographic parity difference, 44.1% in equalized odds difference, and 40.1% in sensitive-feature classification accuracy, at a task-accuracy cost below 2.5%. A five-category sensitive-feature experiment on COMPAS is also reported. The paper argues that, unlike prior non-adversarial approaches, the DAP objective does not admit the degenerate solution of performing equally poorly across all sensitive domains.","tokens_in":10335,"tokens_out":23906,"duration_ms":213436,"significance":"If the empirical claims hold, DAP is a useful addition to non-adversarial fair representation learning: a single differentiable objective without a min-max game, with the adjusted parity metric extended to arbitrary numbers of sensitive domains. The derivation of the maximum standard deviation γ for odd N (Appendix A) is mathematically correct and is a genuine technical extension, and the paper ships code and full result tables (Appendix B). The core claims are falsifiable and are checked against independent standard fairness metrics (demographic parity, equalized odds), not only the proposed metric, which mitigates circularity. The significance is currently bounded by the protocol problems in the major comments: the exact loss is never written down, hyperparameters appear to be selected on the test set, and the authors' own Table 1 shows that the balance component has no measurable effect. If the claims survive a properly held-out protocol with matched baseline tuning, this would be a solid contribution to the fair-representation literature.","major_comments":[{"comment":"The proposed loss function is never explicitly defined. Section 4 derives differentiable soft TP/FP/TN/FN quantities and the soft balanced accuracy S (Eqs. 7–12), and states that substituting S into Eq. (5) 'we can obtain a differential adjusted parity (DAP) loss', while Section 5 says the network is trained with 'a combination of our DAP metric and the standard task cross-entropy loss (Lce)' with weighting hyperparameters β and Ω. The actual expression for the training loss is never given, so the reader cannot determine how the mean-accuracy term, the standard-deviation term, and Lce are combined, nor verify the degeneracy-avoidance argument that is a central selling point (Sections 1 and 6). This omission is load-bearing for reproducibility; please state the loss explicitly, including the exact roles of β and Ω.","section":"Section 4; Eqs. (7)–(13); Section 5.4"},{"comment":"The empirical protocol appears to select hyperparameters on the test set, which biases the headline numbers. Section 5.3 describes only a train/test split (with the ambiguous '175:25 ratio'); Section 5.4 defines a 100-combination grid over Ω and β; and Section 5.5 states that an adjusted parity metric is computed 'for comparing models and selecting best performing hyperparameters', with no validation split described anywhere. In addition, the two balanced random-forest classifiers used for evaluation are trained on 'the encodings in the testing phase' (Section 5.5), so the test set is used both for probe fitting and for model selection. The abstract's quantified claims (22.5%, 44.1%, 40.1%) are therefore, under the protocol as described, the best grid point evaluated on the same test data used for selection rather than unbiased estimates of DAP's performance, and Section 6 even acknowledges that β and Ω require per-dataset calibration. Please add a held-out validation split for hyperparameter selection and for fitting the probes, and re-report the results with the selection-induced bias quantified.","section":"Sections 5.3–5.5"},{"comment":"The paper's own ablation contradicts the claim that soft balanced accuracy is a key element of the method. The abstract calls it 'A key element in this approach', and Section 4 motivates balanced accuracy explicitly by class imbalance ('this measure can be misleading when classes are imbalanced'). Yet Table 1 compares balanced and unbalanced soft accuracies on the imbalanced Adult dataset (75.4% below $50k; 67.3% male) and states 'No significant difference is observed between Balanced and Unbalanced'. If the balance component is not load-bearing in the reported experiments, the claim of novelty should be narrowed to the differentiable soft accuracy, or the authors should demonstrate at least one setting in which balancing matters.","section":"Table 1; Abstract; Section 4"},{"comment":"The baseline comparison is not matched, and no significance tests are reported. The four baselines (CFAIR, LAFTR, FAIR, NODEBIAS) are 'adapted from [14]' with no disclosure of their hyperparameters or tuning budget, whereas DAP receives a 100-combination grid search; the paper reports medians and standard deviations over 5 runs for DAP but does not display uncertainty for the baselines in the figures and performs no statistical test for claims such as 'the lowest EOD and DPD are achieved with DAP' (Section 5.5). Furthermore, the non-adversarial methods discussed in Section 2.2 (VFAE, FBC, FRC, BFA) are never evaluated, even though the central motivation that DAP avoids the degenerate 'equally poor across all sensitive domains' solution is a comparative claim against that family. Please add a matched tuning protocol, significance tests or confidence intervals, and at least one non-adversarial baseline.","section":"Section 5.5; Figures 1–3"}],"minor_comments":[{"comment":"The split 'in a 175:25 ratio' is unclear and likely a typo (perhaps 75:25); a 175:25 split sums to 200 parts and is not a standard notation.","section":"Section 5.3"},{"comment":"In the multi-class COMPAS experiment, the text says balanced race classification accuracy approaches 'the random chance level of 0.33', but the sensitive feature is one-hot encoded into five categories (Section 5.2), giving a chance-level balanced accuracy of 0.20; moreover, the extension of EOD and DPD (Eqs. 1–3), which are defined for binary sensitive attributes, to the five-category setting is never specified, and EOD staying at 0.5 across all β is not discussed as a failure to improve.","section":"Section 5.5; Figure 5"},{"comment":"The symbol Ω is used both for the task-loss weighting hyperparameter (Section 5.4) and for the upper bound of the soft accuracy range (Eq. 16 in Appendix A); please use a distinct symbol for the bound.","section":"Appendix A; Section 5.4"},{"comment":"The end of Section 5.5 tells the reader that the omitted COMPAS sensitivity results 'can be found in section Appendix A', but the relevant plot (Figure 7) appears in Appendix B, and Appendix B contains an unresolved 'Figure ??' cross-reference.","section":"Section 5.5; Appendix B"},{"comment":"There are numerous typos and misspelled affiliation names ('Unviersity', 'Lusanne', 'componenet', 'addititional', 'cirticised'), and the hyperparameter grid 'Ω ∈ {0...100} and β ∈ {0.1...100}' in Section 5.4 is ambiguous as stated (it should be clear that 10 log-spaced values per parameter are used to obtain 100 combinations).","section":"Throughout"},{"comment":"The claim that minimizing prediction inconsistency across sensitive domains 'would minimise the mutual information between the representations and the sensitive feature' is asserted without proof or citation; a formal statement (or a reference) would strengthen the link between the proposed objective and the reported fairness metrics.","section":"Section 4"},{"comment":"The figures plot DAP metrics as functions of β, but the Ω value at which the sweep is performed is not stated in the captions or the text; given the strong Ω-dependence shown in Figure 2, this makes the curves difficult to interpret.","section":"Figures 1–3"}],"recommendation":"major_revision","confidential_remarks":"I support evaluating this manuscript as a major revision. The most consequential issue is the test-set-based hyperparameter selection described in Sections 5.4–5.5, which directly affects the quantified claims in the abstract; this is correctable by re-running with a validation split, but the authors should be asked to do so before acceptance. I would also encourage the editor to ensure that the relationship of this work to reference [19] (adjusted parity, co-authored by one of the present authors) is appropriately positioned in the revision, since the incremental contribution beyond that prior work needs to be stated clearly. Finally, the paper mentions code and results in a zip file in Appendix B, but there is no archival repository or versioned link; given that the loss is not written in the paper, releasing a permanent code link becomes important for reproducibility. None of these concerns reflect on the integrity of the authors, and I found the γ derivation and the general framing sound."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper shows a real idea: a differentiable version of the adjusted parity metric, built on soft balanced accuracies, used as a non-adversarial training loss for fair representations. The gamma normalization for odd numbers of sensitive domains is derived correctly in the appendix (checked for N=3). That extension is new and clean. The qualitative result—DAP beats adversarial baselines on Adult and COMPAS—is plausible, and the use of external fairness metrics (DPD, EOD) helps keep the evaluation from being purely self-referential.\n\nThe soft spots are concentrated in the empirical protocol, and they are load-bearing. The exact DAP loss is never written down; the paper says β and Ω weight the standard deviation and task loss, but no equation is given. A reader cannot reproduce the method from the text alone. More seriously, hyperparameters are selected using test-set metrics: Section 5.5 says adjusted parity is used to 'select best performing hyperparameters,' and the 100-combination grid is searched on the same test set used for reported numbers. The baselines are adapted from an external repo with no matched tuning budget. So the headline improvements (22.5%, 44.1%, 40.1%) may reflect selection bias and asymmetric tuning rather than the objective itself. The paper also claims soft balanced accuracy is a key element, but Table 1 shows no significant difference between balanced and unbalanced variants across all metrics. That directly undercuts the stated contribution. Variance for baselines and significance tests are missing, and non-adversarial baselines (the ones the method is meant to improve on) are not evaluated. The multi-class COMPAS result has a strange feature: adjusted parity drops substantially at high β while DPD improves, and the caption says 'as desired.' That needs explanation.\n\nThe derivations are sound and the writing is mostly clear, but the empirical superiority claim is unverified as an unbiased estimate. This is a solid conference submission that needs major revision, not desk reject. The fix is straightforward: write down the loss, split off a validation set for hyperparameter selection, give baselines the same tuning budget, and report error bars and significance tests.\n\nRecommendation: send to peer review, but expect the referee to ask for the empirical protocol to be redone. I'd bring it to reading group to discuss the selection-bias issue, but I wouldn't cite it yet.\n\nBest,","headline":"A plausible non-adversarial fairness loss with a correct multi-class normalization derivation, but the empirical claims rest on test-set hyperparameter selection and an unsupported key-element claim; needs revision before the numbers can be trusted.","tokens_in":10865,"tokens_out":4039,"would_cite":false,"duration_ms":35901,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A differentiable version of adjusted parity lets one loss train fair representations without an adversary.","keywords":["fair representations","adjusted parity","soft balanced accuracy","non-adversarial debiasing","demographic parity","equalized odds","representation learning"],"falsifier":"Train the same DAP model on Adult and COMPAS with hyperparameters selected on a held-out validation split, then evaluate all fairness metrics on an untouched test split; if the demographic parity, equalized odds, and sensitive-feature accuracy improvements over adversarial baselines fall well below the reported double-digit margins, the central claim would fail.","tokens_in":1455,"feed_emoji":"⚖️","tokens_out":2007,"duration_ms":77643,"temperature":0.7,"pith_summary":"Differential Adjusted Parity (DAP) is a single loss term that trains fair and useful representations without an adversarial network. It turns the adjusted parity metric into a differentiable objective by replacing hard predictions with soft balanced accuracies computed per sensitive domain, then combining mean accuracy with its inconsistency across domains. The paper claims that this removes the degeneracy of earlier non-adversarial methods, in which fairness could be 'satisfied' by failing equally on all sensitive groups. On Adult and COMPAS, DAP reports improvements of up to 22.5% in demographic parity, 44.1% in equalized odds, and 40.1% in sensitive-feature accuracy over adversarial baselines, with less than a 2.5% drop in task accuracy.","feed_headline":"Fairness loss tops adversarial models by up to 44%","feed_subtitle":"Differentiable adjusted parity improves fairness metrics with under 2.5% task-accuracy drop.","key_machinery":"The central object is the Differential Adjusted Parity (DAP) loss, built from soft balanced accuracy. Soft balanced accuracy is the differentiable per-class recall averaged over classes: $S = \\frac{1}{C}\\left\\|\\frac{TP}{TP+FN}\\right\\|_1$, with soft counts $TP=\\sum_x P(x)\\odot L_x$ and analogous expressions for $FP$, $TN$, and $FN$. Computing this quantity separately for each sensitive domain and plugging it into the adjusted parity formula makes $\\Delta_{\\mathrm{adj}}$ a smooth training target that rewards both high and consistent accuracy across groups. The loss combines $\\Delta_{\\mathrm{adj}}$ with the standard task cross-entropy via weights $\\beta$ (on the deviation term) and $\\Omega$ (on the cross-entropy), so no min-max game is needed.","core_discovery":"The paper's central claim is that a differentiable form of adjusted parity can act as the sole fairness mechanism in representation learning, with no adversary. The loss is defined by substituting per-domain soft balanced accuracy into $\\Delta_{\\mathrm{adj}} = \\frac{\\bar{S}-S_R}{1-S_R}\\left(1-\\frac{\\sigma}{\\gamma}\\right)$, where $\\bar{S}$ is mean balanced accuracy across sensitive domains, $\\sigma$ is its standard deviation, $S_R$ is random-predictor accuracy, and $\\gamma$ normalizes by the maximum possible deviation (for odd numbers of domains, $\\gamma = \\sqrt{\\frac14(1-1/N^2)}$). Minimizing this loss jointly with task cross-entropy reduces mutual information between representations and sensitive features, improving demographic parity and equalized odds while keeping representations informative. The paper reports that DAP outperforms adversarial baselines on both datasets and extends to multi-class sensitive attributes.","pith_inferences":["Because the loss is smooth and non-adversarial, it could be dropped into existing autoencoder or variational frameworks in place of maximum-mean-discrepancy penalties without changing the training dynamics; the paper does not test this extension.","The $\\gamma$ normalization implies a concrete prediction: for an odd number of sensitive domains, the maximum achievable adjusted parity is slightly lower than for even numbers, so fairness comparisons across datasets with different domain counts need this normalization to be comparable.","A direct robustness test would be to select $\\beta$ and $\\Omega$ on a held-out validation split rather than the test set; if the reported double-digit margins persist, DAP's advantage is architectural rather than an artifact of test-set selection."],"forward_implications":["Adversarial training is not required: a single cooperative loss can match or beat adversarial models on demographic parity, equalized odds, and sensitive-feature accuracy.","Previous non-adversarial losses that could be gamed by uniformly poor performance no longer have that escape, because low mean accuracy directly lowers the adjusted parity term.","With $\\gamma$ normalizing for the number of domains, the metric and loss extend to multi-class sensitive attributes without forcing a binary reduction of the protected attribute.","Fairness gains come at a controlled cost: less than 2.5% task-accuracy drop on Adult and less than 2% on COMPAS in the reported configurations.","The hyperparameters $\\beta$ and $\\Omega$ let a user trade accuracy against fairness along a tunable axis."],"supporting_citations":[{"why":"supplies the adjusted parity metric that DAP differentiates and extends to arbitrary numbers of sensitive domains.","marker":"[19]"},{"why":"defines the CFAIR adversarial baseline that DAP is compared against and outperforms on fairness metrics.","marker":"[21]"},{"why":"defines the LAFTR adversarial baseline used in the comparison.","marker":"[16]"},{"why":"provides the implementation from which all baseline models and the no-fairness reference were adapted for a common evaluation protocol.","marker":"[14]"},{"why":"supplies the Adult/Census Income dataset used for the income classification experiments.","marker":"[8]"},{"why":"supplies the COMPAS recidivism dataset used to test both binary and multi-class sensitive attributes.","marker":"[7]"}],"fun_headline_variants":["Differentiable parity loss cuts bias up to 44% vs adversarial models","Fairness boost without adversaries: up to 44% better parity","DAP loss: adversarial-beating fairness with under 2.5% accuracy drop","Adversary-free fairness loss improves equalized odds by 44%"],"cache_read_input_tokens":13056,"weakest_assumption_plain":"The load-bearing premise is that selecting hyperparameters with the test set's fairness metrics gives an unbiased estimate of DAP's performance; if a separate validation split is required, the reported gains may shrink or vanish.","fun_headline_variants_meta":{"raw":{"variants":["Differentiable parity loss cuts bias up to 44% vs adversarial models","Fairness boost without adversaries: up to 44% better parity","DAP loss: adversarial-beating fairness with under 2.5% accuracy drop","Adversary-free fairness loss improves equalized odds by 44%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001189,"raw_usage":{"total_tokens":4893,"prompt_tokens":916,"completion_tokens":3977,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":532,"completion_tokens_details":{"reasoning_tokens":3896}},"tokens_in":532,"tokens_out":3977,"duration_ms":29052,"temperature":1.0,"reasoning_tokens":3896,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T20:33:21.056543+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same DAP model on Adult and COMPAS with hyperparameters selected on a held-out validation split, then evaluate all fairness metrics on an untouched test split; if the demographic parity, equalized odds, and sensitive-feature accuracy improvements over adversarial baselines fall well below the reported double-digit margins, the central claim would fail.","supporting_citations":[{"cited_title":"V owels, N","cited_arxiv_id":null,"evidence_quote":"supplies the adjusted parity metric that DAP differentiates and extends to arbitrary numbers of sensitive domains."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"defines the CFAIR adversarial baseline that DAP is compared against and outperforms on fairness metrics."},{"cited_title":"Learning adversarially fair and transferable representations","cited_arxiv_id":null,"evidence_quote":"defines the LAFTR adversarial baseline used in the comparison."},{"cited_title":"Transferlearning_verifyfairness","cited_arxiv_id":null,"evidence_quote":"provides the implementation from which all baseline models and the no-fairness reference were adapted for a common evaluation protocol."},{"cited_title":"Uci machine learning repository, 2017","cited_arxiv_id":null,"evidence_quote":"supplies the Adult/Census Income dataset used for the income classification experiments."},{"cited_title":"Compas risk scales: Demonstrating accuracy, equity, and predictive parity, 2016","cited_arxiv_id":null,"evidence_quote":"supplies the COMPAS recidivism dataset used to test both binary and multi-class sensitive attributes."}],"review_version":1}