pith. sign in

arxiv: 1907.06430 · v1 · pith:SV7KOJ62new · submitted 2019-07-15 · 📊 stat.ML · cs.LG

A Causal Bayesian Networks Viewpoint on Fairness

Silvia Chiappa , William S. Isaac This is my paper

Pith reviewed 2026-05-24 21:29 UTC · model grok-4.3

classification 📊 stat.ML cs.LG
keywords fairnesscausal Bayesian networksunfair causal pathsdata-generation mechanismCOMPASmachine learninggraphical modelsbias measurement
0
0 comments X

The pith

Unfairness in a dataset appears as the presence of an unfair causal path in the causal Bayesian network representing the data-generation mechanism.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes interpreting unfairness graphically as the existence of unfair causal paths within a causal Bayesian network that models how the data arose. This lens is applied to examples such as the COMPAS pretrial risk tool to illustrate that assessing fairness in any learned model demands attention to the specific unfairness patterns already present in the training data. The authors argue that causal Bayesian networks supply concrete tools both for quantifying those patterns and for constructing models that avoid them even when multiple overlapping sources of unfairness are at work.

Core claim

Unfairness in a dataset can be interpreted as the presence of an unfair causal path in the causal Bayesian network representing the data-generation mechanism. This viewpoint allows careful consideration of the patterns of unfairness underlying the training data when evaluating fairness in a model, and supplies a tool both to measure unfairness in a dataset and to design fair models in complex unfairness scenarios.

What carries the argument

The causal Bayesian network representing the data-generation mechanism, where designated unfair causal paths serve as the graphical signature of unfairness.

If this is right

  • Evaluating fairness of a model requires explicit attention to the patterns of unfairness already embedded in its training data.
  • Causal Bayesian networks can be used to measure the degree of unfairness present in a given dataset.
  • The same networks support the design of fair models even when multiple, overlapping sources of unfairness are present.
  • Re-examination of tools such as COMPAS reveals how the underlying causal paths determine which fairness interventions are appropriate.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The approach could enable interventions that block only the labeled unfair paths while leaving other causal routes intact.
  • Different fairness criteria might be compared by checking whether they consistently identify the same paths as unfair inside one network.
  • The viewpoint suggests a route for auditing deployed systems by recovering their implicit data-generation network and inspecting its paths.

Load-bearing premise

A causal Bayesian network that accurately represents the data-generation process can be constructed and paths inside it can be labeled unfair in a way that remains stable across different fairness definitions.

What would settle it

A concrete dataset in which two standard fairness definitions assign incompatible sets of paths as the unfair ones inside the same causal Bayesian network.

Figures

Figures reproduced from arXiv: 1907.06430 by Silvia Chiappa, William S. Isaac.

Figure 1
Figure 1. Figure 1: Number of black and white defendants in each of two aggregate risk categories [15]. The overall recidivism rate for black defendants is higher than for white defendants (52% vs. 39%), i.e. Y✚⊥⊥A. Within each risk category, the proportion of defendants who reoffend is approximately the same regardless of race, i.e. Y ⊥⊥ A|Yˆ . Black defendants are more likely to be classified as medium or high risk (58% vs.… view at source ↗
Figure 2
Figure 2. Figure 2: Possible CBN underlying the dataset used for COMPAS. As previous research has shown [29,35,44], modern polic￾ing tactics center around targeting a small number of neighborhoods — often disproportionately populated by non-white and low income residents — with recurring patrols and stops. This uneven distribution of police at￾tention, as well as other factors such as funding for pretrial services [31,46], me… view at source ↗
Figure 3
Figure 3. Figure 3: (a): CBN with a confounder C for the effect of A on Y . (b): Modified CBN re￾sulting from intervening on A. The causal effect of A on Y can be seen as the information traveling from A to Y through causal paths, or as the conditional distribution of Y given A restricted to causal paths. This implies that, to compute the causal effect, we need to disregard the information that travels along non-causal paths,… view at source ↗
Figure 4
Figure 4. Figure 4: (a): CBN in which conditioning on C closes the paths A ← C ← X → Y and A ← C → Y but opens the path A ← E → C ← X → Y . (b): CBN with one direct and one indirect causal path from A to Y . Conditioning on C to block an open back-door path may open a closed path on which C is a collider. For example, in the CBN of [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Top: CBN with the direct path from A to Y and the indirect paths passing through M high￾lighted in red. Bottom: CBN cor￾responding to Eq. (1). To estimate the effect along a specific group of causal paths, we can generalize the formulas for the ADE and AIE by replacing the variable in the first term with the one resulting from performing the intervention A = a along the group of interest and A = a¯ along t… view at source ↗
Figure 6
Figure 6. Figure 6: CBN under￾lying a music de￾gree scenario. As an example, assume the CBN in [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: CBN underlying a college admission scenario. In the more complex case in which the path A → D → Y is considered fair, unfairness can instead be quantified with the path-specific effect along the direct path A → Y , PSEaa¯ , given by hYa(Da¯)ip(Ya(Da¯)) − hYa¯ip(Ya¯) . Notice that computing p(Ya(Da¯)) requires knowledge of the CBN. If the CBN structure is not known or estimating its conditional distribution… view at source ↗
Figure 8
Figure 8. Figure 8: Directed (a) acyclic and (b) cyclic graph. A directed acyclic graph (DAG) is a directed graph with no directed paths starting and ending at the same node. For example, the directed graph in [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
read the original abstract

We offer a graphical interpretation of unfairness in a dataset as the presence of an unfair causal path in the causal Bayesian network representing the data-generation mechanism. We use this viewpoint to revisit the recent debate surrounding the COMPAS pretrial risk assessment tool and, more generally, to point out that fairness evaluation on a model requires careful considerations on the patterns of unfairness underlying the training data. We show that causal Bayesian networks provide us with a powerful tool to measure unfairness in a dataset and to design fair models in complex unfairness scenarios.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 3 minor

Summary. The manuscript offers a graphical interpretation of unfairness in a dataset as the presence of an unfair causal path in the causal Bayesian network representing the data-generation mechanism. It revisits the COMPAS pretrial risk assessment debate and argues that causal Bayesian networks provide a tool to measure unfairness in datasets and design fair models in complex scenarios.

Significance. If the interpretive mapping holds, the paper supplies a causal-graphical lens for diagnosing patterns of unfairness in training data and for reconciling conflicting fairness criteria. This pragmatic viewpoint may aid practitioners in complex settings where purely statistical fairness metrics are insufficient, though the contribution is conceptual rather than deductive or empirical.

minor comments (3)
  1. The abstract and introduction repeatedly use the phrase 'unfair causal path' without an explicit operational definition or procedure for labeling paths as unfair; a dedicated subsection clarifying the labeling criteria (even if context-dependent) would strengthen the central claim.
  2. No concrete algorithm, pseudocode, or worked numerical example is referenced for 'measuring unfairness' with CBNs; adding a small illustrative computation on a toy graph would make the measurement claim more tangible.
  3. The discussion of COMPAS would benefit from an explicit causal diagram (even if schematic) showing the designated unfair paths, rather than relying solely on textual description.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for their positive summary of the manuscript and the recommendation for minor revision. The report accurately reflects our contribution as a graphical interpretation of unfairness through causal paths in Bayesian networks, with application to the COMPAS case and complex fairness scenarios. No specific major comments were listed in the report.

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper advances an interpretive viewpoint that unfairness corresponds to designated unfair causal paths in a causal Bayesian network modeling the data-generating process. No equations, quantitative predictions, fitted parameters, or formal derivations appear in the provided text. The central claim is a graphical re-description of existing fairness concepts rather than a result obtained by construction from inputs or self-citations. Standard causal modeling assumptions are invoked without self-referential definitions or load-bearing self-citation chains that would force the conclusion.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The framework rests on the domain assumption that a causal Bayesian network can represent the data-generation mechanism and that unfair paths can be identified within it. No free parameters or new entities are introduced in the abstract.

axioms (1)
  • domain assumption A causal Bayesian network can represent the data-generation mechanism
    Invoked as the basis for interpreting unfairness as an unfair causal path.

pith-pipeline@v0.9.0 · 5604 in / 1063 out tokens · 27242 ms · 2026-05-24T21:29:45.859264+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

48 extracted references · 48 canonical work pages

  1. [1]

    Litigating algorithms: Challenging government use of algorithmic decision systems, 2018

    AI Now Institute. Litigating algorithms: Challenging government use of algorithmic decision systems, 2018

    work page 2018

  2. [2]

    Alexander.The New Jim Crow: Mass Incarceration in the Age of Colorblindness

    M. Alexander.The New Jim Crow: Mass Incarceration in the Age of Colorblindness. The New Press, 2012. A Causal Bayesian Networks Viewpoint on Fairness 17

    work page 2012

  3. [3]

    D. A. Andrews and J. Bonta.Level of Service Inventory – Revised. Multi-Health Systems Toronto, 2000

    work page 2000

  4. [4]

    D. A. Andrews, J. Bonta, and J. S. Wormith. The recent past and near future of risk and/or need assessment.Crime & Delinquency, 52(1):7–27, 2006

    work page 2006

  5. [5]

    Angwin, J

    J. Angwin, J. Larson, S. Mattu, and L. Kirchner. Machine Bias: There’s soft- ware used across the country to predict future criminals. And it’s biased against blacks., May 2016. https://www.propublica.org/article/machine-bias-risk- assessments-in-criminal-sentencing

    work page 2016

  6. [6]

    Arnold, W

    D. Arnold, W. Dobbie, and C. S. Yang. Racial bias in bail decisions.The Quarterly Journal of Economics, 133:1885–1932, 2018

    work page 1932

  7. [7]

    R. Berk, H. Heidari, S. Jabbari, M. Kearns, and A. Roth. Fairness in criminal justice risk assessments: The state of the art.Sociological Methods & Research, 2018

    work page 2018

  8. [8]

    Bogen and A

    M. Bogen and A. Rieke. Help wanted: An examination of hiring algorithms, equity, and bias. Technical report, Upturn, 2018

    work page 2018

  9. [9]

    Brennan, W

    T. Brennan, W. Dieterich, and B. Ehret. Evaluating the predictive validity of the COMPAS risk and needs assessment system.Criminal Justice and Behavior, 36(1):21–40, 2009

    work page 2009

  10. [10]

    Byanjankar, M

    A. Byanjankar, M. Heikkilä, and J. Mezei. Predicting credit risk in peer-to-peer lending: A neural network approach. InIEEE Symposium Series on Computational Intelligence, pages 719–725, 2015

    work page 2015

  11. [11]

    S. Chiappa. Path-specific counterfactual fairness. InThirty-Third AAAI Conference on Artificial Intelligence, 2019

    work page 2019

  12. [12]

    Chiappa and W

    S. Chiappa and W. S. Isaac. A causal Bayesian networks viewpoint on fairness. In E. Kosta, J. Pierson, D. Slamanig, S. Fischer-Hübner, S. Krenn (eds) Privacy and Identity Management. Fairness, Accountability, and Transparency in the Age of Big Data. Privacy and Identity 2018. IFIP Advances in Information and Communication Technology, volume 547. Springer,...

    work page 2018

  13. [13]

    Fairpredictionwithdisparateimpact:Astudyofbiasinrecidivism prediction instruments.Big data, 5(2):153–163, 2017

    A.Chouldechova. Fairpredictionwithdisparateimpact:Astudyofbiasinrecidivism prediction instruments.Big data, 5(2):153–163, 2017

    work page 2017

  14. [14]

    Chouldechova, E

    A. Chouldechova, E. Putnam-Hornstein, D. Benavides-Prado, O. Fialko, and R. Vaithianathan. A case study of algorithm-assisted decision making in child mal- treatment hotline screening decisions.Proceedings of Machine Learning Research, 81:134–148, 2018

    work page 2018

  15. [15]

    Corbett-Davies, E

    S. Corbett-Davies, E. Pierson, A. Feller, S. Goel, and A. Huq. A computer program used for bail and sentencing decisions was labeled biased against blacks. It’s actually not that clear., October 2016.https://www.washingtonpost.com/news/ monkey-cage/wp/2016/10/17/can-an-algorithm-be-racist-our-analysis-is- more-cautious-than-propublicas/?utm_term=.8c6e8c1cfbdf

    work page 2016

  16. [16]

    Corbett-Davies, E

    S. Corbett-Davies, E. Pierson, A. Feller, S. Goel, and A. Huq. Algorithmic deci- sion making and the cost of fairness. InProceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, page 797–806, 2017

    work page 2017

  17. [17]

    The measure and mismeasure of fairness: A critical review of fair machine learning

    S. Corbett-Davies and S.Goel. The measure and mismeasure of fairness: A critical review of fair machine learning.CoRR, abs/1808.00023, 2018

    work page arXiv 2018

  18. [18]

    P. Dawid. Fundamentals of statistical causality. Technical report, University College London, 2007

    work page 2007

  19. [19]

    De Fauw, J

    J. De Fauw, J. R. Ledsam, B. Romera-Paredes, S. Nikolov, N. Tomasev, S. Black- well, H. Askham, X. Glorot, B. O’Donoghue, D. Visentin, G. van den Driessche, B. Lakshminarayanan, C. Meyer, F. Mackinder, S. Bouton, K. Ayoub, R. Chopra, 18 S. Chiappa and W. S. Isaac D. King, A. Karthikesalingam, C. O. Hughes, R. Raine, J. Hughes, D. A. Sim, C. Egan, A. Tufai...

    work page 2018

  20. [20]

    Dieterich, C

    W. Dieterich, C. Mendoza, and T. Brennan. COMPAS risk scales: Demonstrating accuracy equity and predictive parity, 2016

    work page 2016

  21. [21]

    Dwork, M

    C. Dwork, M. Hardt, T. Pitassi, O. Reingold, and R. Zemel. Fairness through awareness. InProceedings of the 3rd Innovations in Theoretical Computer Science Conference, pages 214–226, 2012

    work page 2012

  22. [22]

    Eckhouse, K

    L. Eckhouse, K. Lum, C. Conti-Cook, and J. Ciccolini. Layers of bias: A unified approach for understanding problems with risk assessment.Criminal Justice and Behavior, 46:185–209, 2018

    work page 2018

  23. [23]

    V. Eubanks. Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor. St. Martin’s Press, 2018

    work page 2018

  24. [24]

    Feldman, S

    M. Feldman, S. A. Friedler, J. Moeller, C. Scheidegger, and S. Venkatasubramanian. Certifying and removing disparate impact. InProceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 259–268, 2015

    work page 2015

  25. [25]

    Machine Bias: There’s software used across the country to predict future criminals. And it’s biased against blacks

    A. W. Flores, K. Bechtel, and C. T. Lowenkamp. False positives, false negatives, and false analyses: A rejoinder to "Machine Bias: There’s software used across the country to predict future criminals. And it’s biased against blacks.".Federal Probation, 80(2):38–46, 2016

    work page 2016

  26. [26]

    Note: Bail reform and risk assessment: The cautionary tale of federal sentencing.Harvard Law Review, 131(4):1125–1146, 2018

    Harvard Law School. Note: Bail reform and risk assessment: The cautionary tale of federal sentencing.Harvard Law Review, 131(4):1125–1146, 2018

    work page 2018

  27. [27]

    X. He, J. Pan, O. Jin, T. Xu, B. Liu, T. Xu, Y. Shi, A. Atallah, R. Herbrich, S. Bowers, et al. Practical lessons from predicting clicks on ads at facebook. In Proceedings of the Eighth International Workshop on Data Mining for Online Advertising, pages 1–9, 2014

    work page 2014

  28. [28]

    Hoffman, L

    M. Hoffman, L. B. Kahn, and D. Li. Discretion in hiring.The Quarterly Journal of Economics, 133(2):765–800, 2018

    work page 2018

  29. [29]

    W. S. Isaac. Hope, hype, and fear: The promise and potential pitfalls of artificial intelligence in criminal justice.Ohio State Journal of Criminal Law, 15(2):543–558, 2017

    work page 2017

  30. [30]

    Kleinberg, S

    J. Kleinberg, S. Mullainathan, and M. Raghavan. Inherent trade-offs in the fair determination of risk scores. In8th Innovations in Theoretical Computer Science Conference, pages 43:1–43:23, 2016

    work page 2016

  31. [31]

    J. L. Koepke and D. G. Robinson. Danger ahead: Risk assessment and the future of bail reform.Washington Law Review, 93:1725–1807, 2017

    work page 2017

  32. [32]

    Kourou, T

    K. Kourou, T. P. Exarchos, K. P. Exarchos, M. V. Karamouzis, and D. I. Fotiadis. Machine learning applications in cancer prognosis and prediction.Computational and Structural Biotechnology Journal, 13:8–17, 2015

    work page 2015

  33. [33]

    M. J. Kusner, J. R. Loftus, C. Russell, and R. Silva. Counterfactual fairness. In Advances in Neural Information Processing Systems 30, pages 4069–4079, 2017

    work page 2017

  34. [34]

    K. Lum. Limitations of mitigating judicial bias with machine learning.Nature Human Behaviour, 1(7):1, 2017

    work page 2017

  35. [35]

    Lum and W

    K. Lum and W. Isaac. To predict and serve?Significance, 13(5):14–19, 2016

    work page 2016

  36. [36]

    Malekipirbazari and V

    M. Malekipirbazari and V. Aksakalli. Risk assessment in social lending via random forests. Expert Systems with Applications, 42(10):4621–4631, 2015

    work page 2015

  37. [37]

    S. G. Mayson. Bias in, bias out.Yale Law School Journal, 128, 2019. A Causal Bayesian Networks Viewpoint on Fairness 19

    work page 2019

  38. [38]

    Mitchell, E

    S. Mitchell, E. Potash, and S. Barocas. Prediction-based decisions and fairness: A catalogue of choices, assumptions, and definitions, 2018

    work page 2018

  39. [39]

    Pearl.Causality: Models, Reasoning, and Inference

    J. Pearl.Causality: Models, Reasoning, and Inference. Cambridge University Press, 2000

    work page 2000

  40. [40]

    Pearl, M

    J. Pearl, M. Glymour, and N. P. Jewell.Causal Inference in Statistics: A Primer. Wiley, 2016

    work page 2016

  41. [41]

    Perlich, B

    C. Perlich, B. Dalessandro, T. Raeder, O. Stitelman, and F. Provost. Machine learning for targeted display advertising: Transfer learning in action.Machine learning, 95(1):103–127, 2014

    work page 2014

  42. [42]

    Peters, D

    J. Peters, D. Janzing, and B. Schölkopf.Elements of Causal Inference: Foundations and Learning Algorithms. MIT Press, 2017

    work page 2017

  43. [43]

    Rosenberg and R

    M. Rosenberg and R. Levinson. Trump’s catch-and-detain policy snares many who call the U.S. home. https://www.reuters.com/investigates/special-report/ usa-immigration-court, June 2018

    work page 2018

  44. [44]

    A. D. Selbst. Disparate impact in big data policing.Georgia Law Review, 52:109–195, 2017

    work page 2017

  45. [45]

    Spirtes, C

    P. Spirtes, C. N. Glymour, R. Scheines, D. Heckerman, C. Meek, G. Cooper, and T. Richardson.Causation, Prediction, and Search. MIT Press, 2000

    work page 2000

  46. [46]

    M. T. Stevenson. Assessing risk assessment in action.Minnesota Law Review, 103, 2017

    work page 2017

  47. [47]

    Vaithianathan, T

    R. Vaithianathan, T. Maloney, E. Putnam-Hornstein, and N. Jiang. Children in the public benefit system at risk of maltreatment: Identification via predictive modeling. American Journal of Preventive Medicine, 45(3):354–359, 2013

    work page 2013

  48. [48]

    Zhang and E

    J. Zhang and E. Bareinboim. Fairness in decision-making – the causal explanation formula. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, 2018

    work page 2018