Pith. sign in

REVIEW 4 major objections 5 minor 27 references

Simulating Biases for Interpretable Fairness in Offline and Online Classifiers

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Controllable agent-based simulation of loan decisions shows how injected bias is absorbed by classifiers and how each mitigation strategy trades fairness against performance.

desk verdict A useful synthetic-data testbed for fairness auditing, with real reproducibility assets, but its quantitative mitigation rankings rest on single stochastic runs and need error bars before the claims are fully supported. read the letter →

arxiv 2507.10154 v1 pith:N44GYHNT submitted 2025-07-14 stat.ML cs.LG

Ricardo Inácio , Zafeiris Kokkinogenis , Vitor Cerqueira , Carlos Soares This is my paper · ORCID
classification stat.MLcs.LG
keywords agent-basedmodelingsyntheticdatagenerationcontrollablebiasinjectionfairnessauditingmitigationShapleyinteractionsofflinelearningonline
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper builds a simulated loan economy in which members of a privileged group start with consistently better attributes than a protected group, form high-trust peer networks, and receive a label bonus while the protected group receives a penalty. The same simulation is run at 16 combinations of representation imbalance and label prejudice to generate synthetic datasets for both batch and streaming classifiers. The paper's claim is that this is a reusable framework for controllable bias injection, letting auditors study how bias gets absorbed and reinforced by predictive models before real-world deployment. It also claims a second contribution: using second-order Shapley values to make visible how fairness mitigations shift a classifier's reliance on features and feature pairs.

What carries the argument

The engine is an agent-based model of a loan application process. Agents belong to two groups and are born with group-specific distributions for wealth, job, car and house ownership; second-order features such as trust and financial literacy compound these initial gaps; a peer network updates each agent's trust toward the average trust of its neighbors, so privileged agents cluster into high-trust societies that reinforce the original disparity; and a label rule multiplies the qualification score of the privileged group by $(1+\beta)$ and that of the protected group by $(1-\beta)$. The two control knobs are the representation probability $\alpha$ and the prejudice multiplier $\beta$, which the paper uses to inject specific biases into each generated dataset. The explainability component is second-order Shapley values—attributions of individual and pairwise feature contributions—used to depict each model's prediction as a Shapley network whose hubs and thick edges can be compared before and after mitigation.

What would settle it

Using the paper's own simulation code at a fixed representation $\alpha$ and sweeping $\beta$ over 0.0–0.6, the fairness gaps EOD and SPD should widen monotonically with $\beta$; if increasing $\beta$ ever leaves them flat or narrows the gap, the bias-injection controls do not do what is claimed.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that classifiers trained on the ABM-generated data reproduce the group disparities built into the simulation even though the most class-correlated features are hidden from the classifiers, and that mitigation methods behave very differently: demographic-parity constraints push both SPD and EOD close to zero at a consistent performance cost, equalized odds offers a more balanced package, and reweighing gives modest, uneven gains. The explainability analysis then shows that successful mitigation works by diffusion—it spreads predictive importance away from a few dominant hub features and interactions across a broader feature set—except where a feature or interaction is so strongly predictive that attenuation would destroy usable accuracy. Redundant interactions are left alone or even amplified because they do not move the fairness metrics.

Load-bearing premise

The hand-chosen starting conditions of the simulated loan world stand in for how real socioeconomic bias propagates through data and classifiers, and if that transfer fails the quantitative findings describe only the simulation.

Editorial extensions

If this is right

  • Demographic parity is the tested method most likely to equalize approval rates and fairness metrics across groups, but it consistently sacrifices predictive performance in both offline and online pipelines.
  • Equalized odds achieves a more balanced trade-off, with substantial fairness gains and milder performance losses, although offline it can leave groups on ROC curves that never intersect, forcing a sacrifice of the privileged group's performance.
  • Reweighting methods are cheap and model-agnostic but only narrow the fairness gap; under strong label prejudice they leave large SPD and EOD residuals.
  • Overall accuracy and AUC conceal the simulation's group-level disparities, so any fairness audit should report group-disaggregated and sensitive-feature-aware metrics.
  • The ABM framework is transferable: changing the attribute-generation rules and label logic to another domain yields datasets with the same controllable bias structure, enabling fairness audits without access to real sensitive data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • This suggests a practical audit recipe: run a mitigation, compare second-order Shapley networks before and after, and treat any feature or interaction that remains a hub after mitigation as a candidate source of residual unfairness.
  • Because the framework's fairness-mitigation rankings are generated from synthetic data, they are a hypothesis about real loan markets; a direct test would be to repeat the pipeline on public credit datasets with known group structure and see whether the same rankings hold.
  • A natural extension would be to introduce continuous or intersectional sensitive attributes, since the model currently encodes the protected group as a single binary category with a single prejudice knob.
  • The observation that fair predictions often require diffusing attention away from a small set of dominant features implies that feature-engineering choices, not just training-time constraints, could be a primary lever for cost-effective fairness in streaming settings.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a framework for generating synthetic datasets with controllable bias injection using an agent-based model (ABM) of a loan-application process with two demographic groups. The authors train offline (XGBoost) and online (Hoeffding Tree with streaming XGBoost adaptation) classifiers on data from 16 ABM configurations (varying representation probability rep and label-bias factor lbl), evaluate several bias mitigation strategies (automatic reweighing, manual reweighing, exponentiated-gradient with demographic parity or equalized odds constraints), and rank methods on composite performance and fairness scores. They also propose to explain the impact of mitigation on feature use via second-order Shapley values (Shapley interaction networks). The main claims are that the framework successfully embeds biases that classifiers reproduce, that mitigation methods trade off performance and fairness, and that Shapley interaction networks reveal how mitigations diffuse predictive importance across features.

Significance. If the results are robust, the ABM-based synthetic data generator would be a useful testbed for fairness auditing and for comparing mitigation methods, particularly because it covers both offline and online learning settings and makes code available. The inclusion of second-order Shapley values is a sensible application, though the technique itself is not new beyond its use of Shapley interactions. The paper's value is mainly methodological and empirical rather than theoretical. However, the quantitative conclusions about mitigation rankings and trade-offs are currently supported only by single-run point estimates with no uncertainty quantification, which limits their reliability. The manuscript also under-specifies the fairness composite score and the explainability procedure, and the 'no explicit bias' baseline is not actually unbiased, all of which need attention before the claims can be accepted.

major comments (4)
  1. [4.1, 5, Tables 1-3] The paper does not state how many independent simulation runs were performed. The ABM is explicitly stochastic (random agent generation, probabilistic network formation, random transactions, 5% label flips), and the support counts in Tables 2 and 3 (e.g., 632 vs. 579 in Table 2) are consistent with one realization per scenario. Without repeated seeds, confidence intervals, or statistical tests, the rank counts in Table 1 (e.g., Equalized Odds best performance in 14/16 offline scenarios, Demographic Parity best fairness in 16/16) may be unstable and could be noise. This is load-bearing because the central empirical claim about which mitigation methods are preferable depends on these rankings. Please report results over multiple runs with means and standard deviations, or at least provide a sensitivity analysis over seeds.
  2. [4.2, 5, Table 1] The composite fairness score is said to combine SPD and EOD, but the formula is not given and the raw SPD/EOD values are not reported anywhere in the paper (Tables 2 and 3 contain only accuracy, recall, precision, and approval rate). Since the Demographic Parity constraint directly optimizes SPD, ranking that method as best on a composite that includes SPD is partially circular; the reader cannot assess the magnitude of fairness improvements or verify the claim that DP achieves the best fairness in 16/16 scenarios. Please report the SPD/EOD values for all scenarios or make the composite score definition explicit, and discuss the circularity concern.
  3. [3.2, 5.1, Tables 2-3] The scenario with rep = 0.5, lbl = 0.0 is described in Section 5.1 as a case where 'no explicit biases were introduced' and results were 'fairly balanced across groups.' However, even in that scenario the ABM encodes structural bias: group A has wealth range [50,89] and job/car/house probabilities 0.9/0.8/0.6, while group B has [30,59] and 0.7/0.5/0.3. The tables show large approval-rate gaps (e.g., 0.630 vs. 0.147 offline; 0.724 vs. 0.338 online). This means the baseline is not unbiased, and the framing of 'controllable bias injection' as separating explicit from structural biases is misleading. The paper should acknowledge that the lbl=0, rep=0.5 configuration still contains considerable embedded bias, and interpret the results accordingly.
  4. [6, 6.1, 6.2] The proposed 'novel explainability technique' based on second-order Shapley values is not described precisely enough to be reproducible. There are no equations or algorithmic details defining the Shapley interaction values, how the interaction networks are constructed, or how node and edge weights are computed. The text refers to figures but the figures are not included in the manuscript text. Since this is advertised as a contribution, the authors should specify the interaction index used (e.g., Shapley Interaction Index from shapiq), the approximation procedure, and the exact construction of the network plots.
minor comments (5)
  1. [4.4, 5.4] The paper contains several typos and inconsistent terms, such as 'ExponenetiatedGradient' and 'ExponetiatedGradient' for ExponentiatedGradient, 'perditions' for predictions, and 'T able' in table captions. A careful proofreading pass is needed.
  2. [4.1] The sentence 'The ABM is executed for 10000 time steps in each, using different combinations...' is ambiguous about whether 'each' refers to each scenario or each run; clarifying this would help.
  3. [4.2] The paper states that the hidden features are chosen 'to avoid leakage,' but it does not report which features remain as inputs to the classifiers or how many features there are. Listing the full feature set would make the experiments more reproducible.
  4. [4.4] The description of the online ExponentiatedGradient adaptation (wrapping a streaming XGBoost) is underspecified; please clarify the update interval, the number of trees added per update, and how the reduction is trained over streaming data.
  5. [References] The paper does not cite recent work on the pitfalls of evaluating bias mitigation methods solely with between-group metrics, despite citing Goethals et al. [7] in that context; the discussion in Section 2.4 could be expanded to acknowledge limitations of SPD and EOD.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the synthetic-data framework measures the consequences of injected biases rather than deriving its conclusions from its own definitions.

full rationale

Score 0. The paper is an ABM-based simulation and auditing study. It defines controllable bias inputs (group-specific attribute ranges and probabilities, representation probability alpha, and the label boost/penalty beta), generates synthetic loan-application data, trains offline and online classifiers, and then measures fairness and performance metrics on the resulting predictions. The observation that classifiers trained on biased data exhibit group disparities is a sanity check of the simulation pipeline, not a derivation that reduces to the inputs: the classifiers are external learners (XGBoost, Hoeffding trees), the mitigation methods are standard external algorithms (AIF360 reweighing, Fairlearn ExponentiatedGradient), and the paper explicitly compares mitigated versus unmitigated outcomes. No fitted parameter is renamed as a prediction; beta and rep are controlled experimental variables, and SPD/EOD/approval-rate values are measured outputs. There are no self-citations or imported uniqueness theorems that carry the argument. The paper's self-acknowledged caveat in Section 6, that 'the randomness inherent to the mitigation may introduce some variability in the results,' and the single-run point estimates in Tables 2-3 are robustness/external-validity concerns, not circularity. Consequently, no specific circular step can be quoted, and the derivation chain is self-contained as an experimental demonstration.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

All simulation inputs are hand-selected to encode the biases under study; they are not estimated from data. The paper's claims depend on these choices only through the plausibility of the simulation as a proxy for real bias.

free parameters (6)
  • rep (alpha) = 0.5, 0.6, 0.7, 0.8
    Probability of generating group A entities; varied to simulate representation imbalance.
  • lbl (beta) = 0.0, 0.4, 0.5, 0.6
    Multiplicative boost for group A scores and penalty for group B scores (score*(1±beta)); injected to simulate historical/prejudice bias.
  • wealth ranges = A: [50, 89], B: [30, 59]
    Chosen by hand to create structural economic disparity.
  • attribute probabilities (job, car, house) = A: 0.9/0.8/0.6, B: 0.7/0.5/0.3
    Chosen by hand to encode group differences in access to resources.
  • label noise = 0.05
    Probability of flipping the true loan label to make data less trivial.
  • manual reweigh weights = A:0.5, B:1.5
    Ad hoc mitigation weights favoring the protected group.
assumptions (4)
  • standard math Shapley values satisfy additivity and efficiency axioms
    Used implicitly when reporting feature and interaction contributions.
  • domain assumption The ABM's attribute distributions, trust dynamics, and score adjustment faithfully capture how real socioeconomic bias propagates
    Section 3: no calibration to real data; results are about the simulation.
  • domain assumption Hiding features such as wealth and credit_score prevents leakage so the classifier cannot trivially infer group membership
    Section 4.2: the exclusion is necessary for the experimental setup but is not validated.
  • domain assumption The online XGBoost adaptation preserves the behavior of the offline model
    Section 4.4: the streaming wrapper is assumed to be analogous to the batch version, but no comparison is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Simulating Biases for Interpretable Fairness in Offline and Online Classifiers." pith.science (2026). https://pith.science/paper/N44GYHNT

@misc{pith2026250710154,
  author       = {Pith},
  title        = {Pith review of: Simulating Biases for Interpretable Fairness in Offline and Online Classifiers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N44GYHNT}},
  note         = {Machine review of arXiv:2507.10154}
}
read the original abstract

Predictive models often reinforce biases which were originally embedded in their training data, through skewed decisions. In such cases, mitigation methods are critical to ensure that, regardless of the prevailing disparities, model outcomes are adjusted to be fair. To assess this, datasets could be systematically generated with specific biases, to train machine learning classifiers. Then, predictive outcomes could aid in the understanding of this bias embedding process. Hence, an agent-based model (ABM), depicting a loan application process that represents various systemic biases across two demographic groups, was developed to produce synthetic datasets. Then, by applying classifiers trained on them to predict loan outcomes, we can assess how biased data leads to unfairness. This highlights a main contribution of this work: a framework for synthetic dataset generation with controllable bias injection. We also contribute with a novel explainability technique, which shows how mitigations affect the way classifiers leverage data features, via second-order Shapley values. In experiments, both offline and online learning approaches are employed. Mitigations are applied at different stages of the modelling pipeline, such as during pre-processing and in-processing.

Figures

Figures reproduced from arXiv: 2507.10154 by the authors.

Figure 1
Figure 1. Shapley network plots for the least (a) and b)) and most (c) and d)) explicitly biased scenarios in the offline modality. These show the baselines alongside the models constrained by demographic parity. 6.2 Online Classifier Across bias levels, results were more stable overall, with most of the previously documented patterns frequently resurfacing. This might be attributed to the streaming nature of the model, which… view at source ↗
Figure 2
Figure 2. Shapley network plots for the least (e) and f)) and most (g) and h)) explicitly biased scenarios in the online modality. These show the baselines alongside the models constrained by demographic parity [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 21 canonical work pages

  1. [1]

    In: International conference on machine learning

    Agarwal, A., Beygelzimer, A., Dudík, M., Langford, J., Wallach, H.: A reductions approach to fair classification. In: International conference on machine learning. pp. 60–69. PMLR (2018)

    work page 2018

  2. [2]

    Bellamy, R.K.E., Dey, K., Hind, M., Hoffman, S.C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., Nagar, S., Ramamurthy, K.N., Richards, J., Saha, D., Sattigeri, P., Singh, M., Varshney, K.R., Zhang, Y.: AI Fairness 360: An extensible toolkit for detecting, understanding, and mitigating unwanted algo- rithmic bias (Oct 2018),htt...

    arXiv 2018

  3. [3]

    https://doi.org/10.1073/pnas.082080899, https://www.pnas

    Bonabeau, E.: Agent-based modeling: Methods and techniques for simulating hu- mansystems.ProceedingsoftheNationalAcademyofSciences 99(suppl_3),7280– 7287 (2002). https://doi.org/10.1073/pnas.082080899, https://www.pnas. org/doi/abs/10.1073/pnas.082080899

    doi:10.1073/pnas.082080899 2002

  4. [4]

    In: Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining

    Chen, T., Guestrin, C.: Xgboost: A scalable tree boosting system. In: Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. pp. 785–794 (2016)

    2016

  5. [5]

    Annual Review of political science 17(1), 1–20 (2014) 14 Ricardo Inácio, Zafeiris Kokkinogenis, Vitor Cerqueira, and Carlos Soares

    De Marchi, S., Page, S.E.: Agent-based models. Annual Review of political science 17(1), 1–20 (2014) 14 Ricardo Inácio, Zafeiris Kokkinogenis, Vitor Cerqueira, and Carlos Soares

    work page 2014

  6. [6]

    Brookings Institution Press (1996)

    Epstein, J.M., Axtell, R.: Growing artificial societies: social science from the bot- tom up. Brookings Institution Press (1996)

    work page 1996

  7. [7]

    Goethals, S., Calders, T., Martens, D.: Beyond accuracy-fairness: Stop evaluating bias mitigation methods solely on between-group metrics (2024)

    work page 2024

  8. [8]

    Hagberg, A., Swart, P.J., Schult, D.A.: Exploring network structure, dynamics, and function using networkx. Tech. rep., Los Alamos National Laboratory (LANL), Los Alamos, NM (United States) (2008)

    work page 2008

Show all 27 references
  1. [9]

    Expert systems with applications 73, 220–239 (2017)

    Haixiang, G., Yijing, L., Shang, J., Mingyun, G., Yuanyue, H., Bing, G.: Learning from class-imbalanced data: Review of methods and applications. Expert systems with applications 73, 220–239 (2017)

    2017

  2. [10]

    Advances in neural information processing systems29 (2016)

    Hardt, M., Price, E., Srebro, N.: Equality of opportunity in supervised learning. Advances in neural information processing systems29 (2016)

    2016

  3. [11]

    Im- balanced learning: Foundations, algorithms, and applications pp

    Hoens, T.R., Chawla, N.V.: Imbalanced datasets: from sampling to classifiers. Im- balanced learning: Foundations, algorithms, and applications pp. 43–59 (2013)

    2013

  4. [12]

    Neurocomputing 459, 249–289 (2021)

    Hoi, S.C., Sahoo, D., Lu, J., Zhao, P.: Online learning: A comprehensive survey. Neurocomputing 459, 249–289 (2021)

    2021

  5. [13]

    JMIR Medical Informatics 10(5), e36388 (2022)

    Huang, J., Galal, G., Etemadi, M., Vaidyanathan, M.: Evaluation and mitigation of racial bias in clinical machine learning models: scoping review. JMIR Medical Informatics 10(5), e36388 (2022)

    2022

  6. [14]

    Knowledge and information systems33(1), 1–33 (2012)

    Kamiran, F., Calders, T.: Data preprocessing techniques for classification without discrimination. Knowledge and information systems33(1), 1–33 (2012)

    2012

  7. [15]

    example weight- ing

    Klinkenberg, R.: Learning drifting concepts: Example selection vs. example weight- ing. Intelligent data analysis8(3), 281–300 (2004)

    2004

  8. [16]

    Advances in neural information processing systems30 (2017)

    Lundberg, S.M., Lee, S.I.: A unified approach to interpreting model predictions. Advances in neural information processing systems30 (2017)

    2017

  9. [17]

    ACM computing surveys (CSUR)54(6), 1–35 (2021)

    Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., Galstyan, A.: A survey on bias and fairness in machine learning. ACM computing surveys (CSUR)54(6), 1–35 (2021)

    2021

  10. [18]

    Montiel, J., Halford, M., Mastelini, S.M., Bolmier, G., Sourty, R., Vaysse, R., Zoui- tine, A., Gomes, H.M., Read, J., Abdessalem, T., et al.: River: machine learning for streaming data in python (2021)

    2021

  11. [19]

    In: The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track (2024),https://openreview.net/forum?id=knxGmi6SJi

    Muschalik, M., Baniecki, H., Fumagalli, F., Kolpaczki, P., Hammer, B., Hüller- meier, E.: shapiq: Shapley interactions for machine learning. In: The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track (2024),https://openreview.net/for...

    2024

  12. [20]

    Wiley Interdisciplinary Re- views: Data Mining and Knowledge Discovery10(3), e1356 (2020)

    Ntoutsi, E., Fafalios, P., Gadiraju, U., Iosifidis, V., Nejdl, W., Vidal, M.E., Rug- gieri, S., Turini, F., Papadopoulos, S., Krasanakis, E., et al.: Bias in data-driven artificial intelligence systems—an introductory survey. Wiley Interdisciplinary Re- views: Data Mining and ...

    2020

  13. [21]

    Artificial Intelligence Review49, 281–299 (2018)

    Pérez-Sánchez, B., Fontenla-Romero, O., Guijarro-Berdiñas, B.: A review of adap- tive online learning for artificial neural networks. Artificial Intelligence Review49, 281–299 (2018)

    2018

  14. [22]

    Advances in neural information processing systems33, 2798–2810 (2020)

    Savani, Y., White, C., Govindarajulu, N.S.: Intra-processing methods for debiasing neural networks. Advances in neural information processing systems33, 2798–2810 (2020)

    2020

  15. [23]

    Journal of Mathematical Sociology 1(2), 143 – 186 (1971)

    Schelling, T.C.: Dynamic models of segregation. Journal of Mathematical Sociology 1(2), 143 – 186 (1971)

    1971

  16. [24]

    In: International Con- ference on Machine Learning

    Wang, A., Russakovsky, O.: Directional bias amplification. In: International Con- ference on Machine Learning. pp. 10882–10893. PMLR (2021) Simulating Biases for Interpretable Fairness in Classifiers 15

    2021

  17. [25]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Wang, A., Russakovsky, O.: Overwriting pretrained bias with finetuning data. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 3957–3968 (2023)

    2023

  18. [26]

    Weerts, H., Dudík, M., Edgar, R., Jalali, A., Lutz, R., Madaio, M.: Fairlearn: Assessing and improving fairness of ai systems (2023),http://jmlr.org/papers/ v24/23-0389.html

    2023

  19. [27]

    Zhang, H., Zhang, Y., Zhu, S., Xu, D.: Deterministic convergence of complex mini- batch gradient learning algorithm for fully complex-valued neural networks. Neu- rocomputing 407, 185–193 (2020) 16 Ricardo Inácio, Zafeiris Kokkinogenis, Vitor Cerqueira, and Carlos Soares T abl...

    2020

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.