pith. sign in

arxiv: 2604.27011 · v1 · submitted 2026-04-29 · 💻 cs.LG · cs.AI

Automatic Causal Fairness Analysis with LLM-Generated Reporting

Alessia Berarducci , Eric Rossetto , Alessandro Antonucci , Marco Zaffalon This is my paper

Pith reviewed 2026-05-07 13:25 UTC · model grok-4.3

classification 💻 cs.LG cs.AI
keywords causal fairnessautomated analysislarge language modelscounterfactual queriesdataset auditingfairness reportingmachine learning preprocessing
0
0 comments X

The pith

FairMind automates fairness checks on training datasets by computing causal effects from counterfactual queries and using LLMs to generate the reports.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents a software prototype that automates fairness analysis at the dataset level before models are trained. It applies a causal framework to calculate the effects of protected features on outcomes through counterfactual reasoning after data preprocessing. The tool performs closed-form computations of these effects and then prompts large language models in a zero-shot setup to convert the results into readable reports. Examples illustrate that this produces more accurate summaries than asking the models to evaluate fairness directly from the data. Extensions are discussed for ordinal protected variables, continuous targets, and additional decomposition of the effects.

Core claim

FairMind implements closed-form computation of causal fairness effects based on counterfactual queries involving protected attributes, confounders, mediators, and targets, then exploits large language models in a zero-shot setup to generate accurate reports on detected fairness levels, with examples showing advantages relative to direct LLM analysis of the data.

What carries the argument

Closed-form computation of causal effects from counterfactual queries on protected features and outcomes, followed by zero-shot LLM generation of natural-language fairness reports.

If this is right

  • Datasets can be audited for fairness automatically during the preprocessing stage of machine learning pipelines.
  • Users obtain natural-language fairness summaries without needing to interpret causal quantities themselves.
  • The method applies to cases with ordinal protected variables and continuous outcome variables.
  • Additional decomposition results permit more detailed breakdown of fairness effects.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Embedding the prototype inside AutoML frameworks could flag biased data early and reduce the number of unfair models that reach deployment.
  • The same pattern of causal computation plus LLM reporting might extend to automated checks for other dataset properties such as robustness or missing-data patterns.
  • Large-scale tests on real-world datasets from different domains would reveal how often the zero-shot reports align with expert judgment.

Load-bearing premise

The standard fairness model supplies a complete basis for quantifying fairness through counterfactual causal effects from observed data, and LLMs can turn those numerical results into precise reports without extra training or verification steps.

What would settle it

Running the prototype on datasets with independently verified causal effects and human-written fairness reports, then checking whether the computed values and LLM outputs match the independent assessments.

Figures

Figures reproduced from arXiv: 2604.27011 by Alessandro Antonucci, Alessia Berarducci, Eric Rossetto, Marco Zaffalon.

Figure 1
Figure 1. Figure 1: Causal graph depicting the SFM assumptions (a) and an explicit version view at source ↗
Figure 2
Figure 2. Figure 2: Decomposition of the effect of gender on income for the Adult dataset. We adopt the SFM assumptions and consider an SCM as in Fig. 1a. The fairness descriptors in Eqs. (1)–(5) are computed using the formulae in Prop. 1, focusing on the probability of low income when X transitions from male to fe￾male. Positive values therefore correspond to discrimination against females. For this model, all descriptors ar… view at source ↗
Figure 3
Figure 3. Figure 3: Total and direct effects in the Adult dataset with respect to gender for view at source ↗
Figure 4
Figure 4. Figure 4: Income TE for increasing educational levels in the view at source ↗
Figure 5
Figure 5. Figure 5: Comparing a report directly generated from the data by LLM against view at source ↗
read the original abstract

AutoML, intended as the process of automating the application of machine learning to real-world problems, is a key step for AI popularisation. Most AutoML frameworks are not accounting for the potential lack of fairness in the training data and in the corresponding predictions. We introduce \textsc{FairMind}, a software prototype aiming to automatise fairness analysis at the dataset level. We achieve that by resorting to the assumptions of the \emph{standard fairness model}, recently proposed by Ple\v{c}ko and Bareinboim. This allows for a sound fairness evaluation in terms of causal effects, based on \emph{counterfactual} queries involving the target, possibly confounders and mediators, and the different values of an input feature we regard as \emph{protected}. After the necessary data preprocessing, the tool implements a closed-form computation of the effects. LLMs are consequently exploited to generate accurate reports on the fairness levels detected in the training dataset. We achieve that in a zero-shot setup and show by examples the expected advantages with respect to a direct analysis performed by the LLM. To favour applications, extensions to ordinal protected variable and continuous targets and novel decomposition results are also discussed.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript introduces FairMind, a software prototype for automating fairness analysis at the dataset level. It applies the standard fairness model of Plečko and Bareinboim to enable closed-form computation of causal effects (total, direct, and indirect) via counterfactual queries on protected attributes, confounders, and mediators after preprocessing. LLMs are then used in a zero-shot setup to generate reports on detected fairness levels, with examples claimed to show advantages over direct LLM analysis. Extensions for ordinal protected variables, continuous targets, and novel decomposition results are discussed.

Significance. If the closed-form computations are correctly implemented and the LLM reports prove reliable, the work could offer a practical bridge between causal fairness methods and automated tooling, aiding practitioners in dataset-level audits. The explicit use of an established identification model and the extensions to non-binary/continuous cases provide concrete technical value, though the absence of rigorous validation for the LLM component limits immediate impact.

major comments (2)
  1. [§4] §4 (LLM Reporting): The central claim that LLMs generate 'accurate reports' on fairness levels (including effect signs and magnitudes) in zero-shot mode rests only on qualitative examples, with no quantitative metrics such as expert agreement rates, hallucination rates on counterfactual queries, or error analysis against the closed-form outputs. This directly undermines the automation value, as misstated effects would render the reports unusable regardless of correct preprocessing and computation.
  2. [§3.2] §3.2 (Closed-form Computation): While the paper asserts closed-form evaluation under the Plečko-Bareinboim model, no explicit identification formulas, sensitivity checks for unmeasured confounding, or verification that the preprocessing enforces the required assumptions (e.g., no unmeasured confounders between protected attribute and outcome) are provided; this is load-bearing for the soundness of the causal effects.
minor comments (2)
  1. [Abstract] The abstract and introduction could more clearly distinguish the contributions of the closed-form module versus the LLM reporting module, including any pseudocode or interface details for the prototype.
  2. [Figures/Tables] Figure captions and table descriptions lack sufficient detail on how example outputs were generated (e.g., specific LLM prompts or model versions used).

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback on our manuscript. We appreciate the acknowledgment of the potential practical bridge between causal fairness methods and automated tooling that FairMind aims to provide. Below we address each major comment point by point, outlining the revisions we will make to strengthen the paper.

read point-by-point responses

  1. Referee: [§4] §4 (LLM Reporting): The central claim that LLMs generate 'accurate reports' on fairness levels (including effect signs and magnitudes) in zero-shot mode rests only on qualitative examples, with no quantitative metrics such as expert agreement rates, hallucination rates on counterfactual queries, or error analysis against the closed-form outputs. This directly undermines the automation value, as misstated effects would render the reports unusable regardless of correct preprocessing and computation.

    Authors: We agree that the current manuscript relies on qualitative examples to illustrate the LLM-generated reports in zero-shot mode. To strengthen the evidence for the reliability of these reports, we will revise Section 4 to include quantitative validation. This will consist of an evaluation across multiple benchmark datasets measuring agreement rates with expert human assessments of fairness levels, as well as an error analysis comparing LLM outputs (effect signs and magnitudes) against the closed-form computations. We will also report hallucination rates on counterfactual queries where applicable. These additions will provide a more rigorous assessment of the automation value. revision: yes

  2. Referee: [§3.2] §3.2 (Closed-form Computation): While the paper asserts closed-form evaluation under the Plečko-Bareinboim model, no explicit identification formulas, sensitivity checks for unmeasured confounding, or verification that the preprocessing enforces the required assumptions (e.g., no unmeasured confounders between protected attribute and outcome) are provided; this is load-bearing for the soundness of the causal effects.

    Authors: The closed-form computations implemented in FairMind are derived from the identification results in the Plečko and Bareinboim standard fairness model, which is cited in the manuscript. We acknowledge that greater transparency is needed. In the revised version, we will add the explicit identification formulas for the total, direct, and indirect effects under the model. We will also include a dedicated discussion of the preprocessing pipeline and how it enforces the key assumptions (such as no unmeasured confounding between the protected attribute and outcome). Additionally, we will incorporate a sensitivity analysis subsection to evaluate robustness to potential unmeasured confounding. revision: yes

Circularity Check

0 steps flagged

No significant circularity; pipeline relies on external model and example-based LLM claims

full rationale

The paper's core chain adopts the externally cited standard fairness model of Plečko and Bareinboim to enable closed-form counterfactual computations after preprocessing; this is independent prior work with no self-citation or reduction to the present paper's inputs. The subsequent zero-shot LLM reporting step is presented as an application shown via examples rather than any derived prediction, fitted parameter, or self-definitional equivalence. No load-bearing step equates an output to its input by construction, and the derivation remains self-contained against the stated external assumptions and benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the validity of the standard fairness model assumptions for counterfactual queries and on the unverified premise that LLMs produce accurate fairness reports from causal summaries without fine-tuning or calibration.

axioms (1)
  • domain assumption Assumptions of the standard fairness model proposed by Plečko and Bareinboim
    Invoked to enable sound fairness evaluation in terms of causal effects based on counterfactual queries involving the target, confounders, mediators, and protected features.

pith-pipeline@v0.9.0 · 5511 in / 1518 out tokens · 82099 ms · 2026-05-07T13:25:22.079045+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

24 extracted references · 24 canonical work pages

  1. [1]

    Journal of Machine Learning Research25(265), 1–8 (2024) 16 Alessia Berarducci, Eric Rossetto, Alessandro Antonucci, Marco Zaffalon

    Ankan, A., Textor, J.: Pgmpy: A Python toolkit for Bayesian networks. Journal of Machine Learning Research25(265), 1–8 (2024) 16 Alessia Berarducci, Eric Rossetto, Alessandro Antonucci, Marco Zaffalon

    work page 2024

  2. [2]

    Knowledge and Information Systems65(12), 5097–5149 (2023)

    Barbudo, R., Ventura, S., Romero, J.: Eight years of AutoML: categorisation, review and trends. Knowledge and Information Systems65(12), 5097–5149 (2023)

    work page 2023

  3. [3]

    In: Proceedings of the 42nd International Conference on Machine Learn- ing

    Correa, J.D., Bareinboim, E.: Counterfactual graphical models: Constraints and inference. In: Proceedings of the 42nd International Conference on Machine Learn- ing. vol. 267, pp. 11245–11254. PMLR (2025)

    work page 2025

  4. [4]

    In: Proceedings of 5th Annual Future Business Technology Conference (2008)

    Cortez, P., Silva, A.: Using data mining to predict secondary school student per- formance. In: Proceedings of 5th Annual Future Business Technology Conference (2008)

    work page 2008

  5. [5]

    In: Proceedings of the 35th International Conference on Neural Information Processing Systems

    Ding, F., Hardt, M., Miller, J., Schmidt, L.: Retiring adult: New datasets for fair machine learning. In: Proceedings of the 35th International Conference on Neural Information Processing Systems. pp. 6478–6490. Curran (2021)

    work page 2021

  6. [6]

    Foundations of Science3(1), 151–182 (1998)

    Galles, D., Pearl, J.: An Axiomatic Characterization of Causal Counterfactuals. Foundations of Science3(1), 151–182 (1998)

    work page 1998

  7. [7]

    ACM Trans

    Gu, Y., You, H., Cao, J., Yu, M., Fan, H., Qian, S.: Large language models for constructing and optimizing machine learning workflows: A survey. ACM Trans. Softw. Eng. Methodol. (2025)

    work page 2025

  8. [8]

    Mahajan, K.: CausalFairnessInAction: An open source Python library for causal fairnessanalysis.In:ProceedingsoftheNeurIPS2025WorkshoponCausalFairness (2025)

    work page 2025

  9. [9]

    In: Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence

    Pearl, J.: Direct and indirect effects. In: Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence. pp. 411–420. Morgan Kaufmann (2001)

    work page 2001

  10. [10]

    Cambridge University Press (2009)

    Pearl, J.: Causality: Models, Reasoning, and Inference. Cambridge University Press (2009)

    work page 2009

  11. [11]

    Wiley (2016)

    Pearl, J., Glymour, M., Jewell, N.P.: Causal Inference in Statistics: A Primer. Wiley (2016)

    work page 2016

  12. [12]

    Foundations and Trends in Machine Learning17(3), 304–589 (2024)

    Plečko, D., Bareinboim, E.: Causal Fairness Analysis: A Causal Toolkit for Fair Machine Learning. Foundations and Trends in Machine Learning17(3), 304–589 (2024)

    work page 2024

  13. [13]

    In: 2nd AI for Math Workshop (ICML 2025) (2025)

    Song, P., Han, P., Goodman, N.: A survey on large language model reasoning failures. In: 2nd AI for Math Workshop (ICML 2025) (2025)

    work page 2025

  14. [14]

    In: Rogers, A., Boyd-Graber, J., Okazaki, N

    Wang, B., Min, S., Deng, X., Shen, J., Wu, Y., Zettlemoyer, L., Sun, H.: Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Mat- ters. In: Rogers, A., Boyd-Graber, J., Okazaki, N. (eds.) Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 2717–2739. Association for ...

    work page 2023

  15. [15]

    Journal of Ar- tificial Intelligence Research79, 639–677 (2024) Automatic Causal Fairness Analysis with LLM-Generated Reporting 17 A Proofs The formulae in Prop

    Weerts, H., Pfisterer, F., Feurer, M., Eggensperger, K., Bergman, E., Awad, N., Vanschoren, J., Pechenizkiy, M., Bischl, B., Hutter, F.: Can Fairness be Auto- mated? Guidelines and Opportunities for Fairness-aware AutoML. Journal of Ar- tificial Intelligence Research79, 639–677 (2024) Automatic Causal Fairness Analysis with LLM-Generated Reporting 17 A Pr...

    work page 2024

  16. [16]

    Fairness Decomposition Report

    ‘LATEX: ‘ −‘TEXT‘ must be plain English with∗∗no LaTeX∗ ∗. −‘LATEX‘ must be a∗∗complete standalone LaTeX document∗∗ starting with ‘\documentclass ‘ and containing∗∗only valid LaTeX∗∗(no Markdown, no backticks , no commentary) . Use only information present in the JSON. Do∗∗not∗∗invent numbers or fields . If something required is missing , explicitly say i...

    work page

  17. [17]

    ‘Overview of the Fairness Analysis ‘

    work page

  18. [18]

    ‘Decomposition of Effects ‘

    work page

  19. [19]

    Females have a lower probability than Males of Y

    ‘Stepwise Effects Across Ordered Levels of X‘ ( conditional ; see below) −End with a conclusion−style recap (short paragraph) . −The recap MUST clearly state which group (x0 or x1 , using actual group names from JSON) has higher or lower outcome probability/mean. −It MUST explicitly describe the direction of disparity in plain words (e.g. , "Females have ...

    work page

  20. [20]

    Parse JSON: x0/x1 groups , Y type , mediators/confounders , threshold/curve data + selected threshold , stepwise flags + effects_by_step

    work page

  21. [21]

    Decide which subsections apply

    work page

  22. [22]

    Extract key numbers; round to <=3 decimals

    work page

  23. [23]

    Write TEXT with the exact structure and required statements

    work page

  24. [24]

    Now produce the final answer with exactly : −‘TEXT: ‘ Text here

    Convert the same content into a valid LaTeX document obeying all LaTeX rules . Now produce the final answer with exactly : −‘TEXT: ‘ Text here ... −‘LATEX: ‘ All the latex here .. − − − ## USER PROMPT (input−only) {RESULTS_JSON}

    work page