Pith. sign in

REVIEW 4 major objections 3 minor 13 references

A Causal Lens for Learning Long-term Fair Policies

T0 review · 4 major / 3 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read Qualification-gain fairness decomposes into direct, delayed, and spurious policy effects, and the direct effect is what benefit fairness controls.

desk verdict Useful decomposition of long-term fairness, but the benefit-fairness/DPE alignment claim is unsupported as stated and the core equations contain typos. read the letter →

arxiv 2506.11242 v1 pith:DTCLFDCH submitted 2025-06-12 cs.LG cs.AI

classification cs.LGcs.AI
keywords long-termfairnessreinforcementlearningcausaldecompositionbenefitqualificationgainparitypath-specificeffectssequentialdecisionmakingpolicyoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Fairness in machine learning is usually measured on a single decision, but this paper argues that deployed policies keep reshaping people's qualifications over time, so fairness should be judged on the gap in expected qualification gains between protected groups. Its central contribution is a causal decomposition of that gap into a direct policy effect, an indirect effect through the environment, and a spurious effect that exists even under a no-treatment baseline. The paper proves that the direct effect is exactly the between-group disparity in benefit-weighted treatment probabilities, which is the object controlled by benefit fairness, a short-term fairness notion. That proof yields a reconciliation: when benefit does not depend on the protected attribute, enforcing benefit fairness reduces the direct component of long-term unfairness, so short-term and long-term fairness are not inherently opposed. The paper also supplies a policy-gradient algorithm that optimizes qualification-gain parity and benefit fairness together, and evaluates it in loan-simulation environments.

What carries the argument

The load-bearing mechanism is a telescoping decomposition of value functions defined by two hypothetical policies: the baseline policy $\pi_0$ always takes the negative decision $d_0$, and the virtual policy $\pi_{PS}$ computes qualification gains as if under $\pi_0$ while letting state transitions follow $\pi$. Subtracting the value functions gives $V_{\mathrm{do}(\pi,s)}=(V_{\mathrm{do}(\pi,s)}-V_{\mathrm{do}(\pi_{PS},s)})+(V_{\mathrm{do}(\pi_{PS},s)}-V_{\mathrm{do}(\pi_0,s)})+V_{\mathrm{do}(\pi_0,s)}$, and averaging this across $s_+$ and $s_-$ produces the DPE/IPE/SPE split. Proposition 1 converts the direct term into $\pi(d_1|x,s)\Delta(x,s)$ weighted by the state-visitation count, which is the link from the long-term metric to benefit fairness. The same machinery gives the gradient of $C_\pi(\theta)$ through the policy gradient theorem and supplies the baseline adjustment $G_{s_+}-G_{s_-}$ for pre-existing qualification gaps.

What would settle it

In the loan simulator, introduce a hidden variable that increases both the probability of membership in the disadvantaged group and the initial credit score, estimate the parity gap with the paper's observed-data procedure, and compare it with the true interventional gap obtained by hard intervention; a mismatch would falsify the no-confounder assumption.

Watch

Extended reading notes

Core claim

The paper's central claim is that long-term fairness, defined as qualification gain parity $C_\pi(\theta)=\mathbb{E}[V_{\mathrm{do}(\pi,s_+)}(x_1)]-\mathbb{E}[V_{\mathrm{do}(\pi,s_-)}(x_1)]$, is not monolithic: it decomposes as $C_\pi(\theta)=\mathrm{DPE}+\mathrm{IPE}+\mathrm{SPE}$, where $\mathrm{DPE}$ is the direct causal effect of the policy on qualification gains, $\mathrm{IPE}$ is the delayed effect transmitted through the environment, and $\mathrm{SPE}$ is the spurious environment-only gap that would remain under a baseline policy that always denies treatment. The paper proves that the direct component has the closed form $\mathrm{DPE}=\mathbb{E}_{x\sim\pi|s_+}[\pi(d_1|x,s_+)\Delta(x,s_+)]-\mathbb{E}_{x\sim\pi|s_-}[\pi(d_1|x,s_-)\Delta(x,s_-)]$, where $\Delta(x,s)$ is the expected qualification gain from treatment. Because benefit fairness requires equal treatment probability for equal benefit, this identity says that the direct policy component of long-term unfairness is exactly the quantity benefit fairness regulates; when $\Delta$ is independent of the sensitive attribute, enforcing benefit fairness reduces that direct component. This is the basis for the paper's claim that short-term and long-term fairness can be reconciled, and it motivates a policy-gradient objective that penalizes both the qualification-gain gap and a Gini-inspired benefit-fairness term.

Load-bearing premise

The construction assumes that nothing unobserved drives both a person's protected group and their qualifications; if such a confounder exists, the estimated fairness gap is not the true causal effect.

Editorial extensions

If this is right

  • Enforcing benefit fairness through the Gini-inspired penalty reduces the DPE in the loan simulation, so the alignment between benefit fairness and long-term fairness is observable, not just formal.
  • A policy trained with the qualification-gain-parity constraint outperforms advantage-regularized baselines at reducing the Wasserstein distance between group qualification distributions, especially when the baseline qualification gap is included.
  • The IPE changes less than the DPE during optimization, meaning delayed environmental effects are more resistant to policy-gradient constraints than the direct effect of the decisions.
  • The decomposition gives practitioners a diagnostic: monitoring DPE, IPE, and SPE separately can reveal whether remaining unfairness comes from current decisions, from the dynamics the policy induces, or from the environment alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A consequence the authors do not spell out is that, because DPE is a benefit-weighted treatment gap, benefit fairness could be audited from logged decisions and estimated benefits without re-solving the full MDP; this diagnostic use is not developed in the paper.
  • The same DPE/IPE/SPE split should apply to any additive long-term outcome measure, not only qualification gains, provided a gain function with the paper's interval-additivity property exists.
  • The paper's design suggestion that $\Delta$ be made independent of $S$ implies a concrete search problem over qualification gain functions; an explicit algorithm for that search is a natural next step that the paper does not test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper proposes a causal decomposition of long-term fairness in reinforcement learning. Long-term fairness is formalized as qualification gain parity, Cπ(θ) = E[Vdo(π,s+)(x1)] − E[Vdo(π,s−)(x1)], and is decomposed into a Direct Policy Effect (DPE), an Indirect Policy Effect (IPE), and a Spurious Policy Effect (SPE) using two hypothetical policies, a baseline policy π0 and a virtual policy πPS. The paper then links DPE to benefit fairness, proposes a Gini-inspired regularizer Λ to enforce benefit fairness alongside qualification gain parity, and evaluates the resulting PPO variants on a simulated loan environment from D'Amour et al. (2020). The experiments report utility, qualification-gain disparity, decomposition components, and benefit-fairness behavior under different regularization weights.

Significance. If the decomposition and the benefit-fairness connection were established, the paper would provide a useful framework for separating direct, delayed, and spurious sources of long-term unfairness in dynamic decision systems, and a practical way to balance short-term benefit fairness with long-term parity. Strengths include the use of a formal Bellman/policy-gradient setting, a concrete causal decomposition that is largely an algebraic identity, and the release of code for the experiments. However, the central qualitative claim that benefit fairness aligns with reducing DPE is not supported by the stated definitions, and several technical typos in the central proposition and decomposition need correction before the theory can be evaluated fairly.

major comments (4)
  1. [Section 3.4] The displayed definitions of DPE and IPE use Vdo(πPS,s+) in the second expectation for both groups. With these definitions, DPE equals E[Vdo(π,s+)] − E[Vdo(π,s−)] and IPE equals −SPE, so the displayed equality Cπ = DPE + IPE + SPE holds only by cancellation and the components do not isolate the direct and indirect policy effects as claimed. The intended definitions should use Vdo(πPS,s−) in the second expectation of both DPE and IPE. Because the subsequent benefit-fairness reformulation in Section 3.5 relies on the corrected DPE, this indexing error is load-bearing and must be fixed.
  2. [Section 3.5 and Appendix B] Proposition 1 states that Vdo(π,s)(x) − Vdo(πPS,s)(x) = Σ_{x'} ηπ,s(x→x') π(d1|x,s)Δ(x,s). As printed, the summand does not depend on the summation index x', so the right-hand side collapses to π(d1|x,s)Δ(x,s) times the total occupancy mass and is generally false. The proof in Appendix B derives Gπ,s(x') under the sum, so the correct statement should have π(d1|x',s)Δ(x',s) inside the sum. With that correction, the reformulation of DPE as the difference of expected π(d1|x,s)Δ(x,s) under the two group-wise visitation distributions is valid.
  3. [Section 3.5, Definition 2 and Remark] Benefit fairness does not imply a reduced or zero DPE even when Δ is independent of S. Definition 2 only equates treatment probabilities within equal-benefit cells; it imposes no constraint on the group-specific state visitation distributions. For example, with binary X, Δ(x,s)=x, and the benefit-fair policy π(d1|x,s)=x, we have DPE = P_{π|s+}(X=1) − P_{π|s−}(X=1), which is 0.8 if the two groups visit X=1 with probabilities 0.9 and 0.1. The paper's own preceding sentence recognizes that differing state visitation distributions make benefit fairness incompatible with a zero DPE, but the Remark nevertheless asserts a general alignment when Δ is independent of S. The stated sufficient condition, "state visitation distributions in terms of benefit are identical", is not proved and is much stronger than Δ-independence. Since this alignment motivates the Λ regularizer and the paper's reconciliation conclusion, the central qualitative claim is not established by the provided theory.
  4. [Section 4] The causal interpretation of DPE, IPE, SPE, and the baseline correction Gs+ − Gs− requires identifiability conditions that are not stated. The back-door criterion is invoked only to justify P(x|do(s)) = P(x|s), but the counterfactual term P(Xs = x′|X = x) used in Gs is a different quantity; its identification requires a fully specified Markovian SCM or equivalent no-unobserved-confounding assumptions. Similarly, Vdo(πPS,s) is a path-specific interventional quantity, and path-specific effects are identifiable only under additional conditions such as the absence of recanting witnesses. The manuscript should state these assumptions explicitly or restrict the causal interpretation of the decomposed quantities accordingly.
minor comments (3)
  1. [Section 3.3 and Appendix A] The displayed expression for ∇θCπ(θ) contains the factor α = 2(E[Vdo(π,s+)(x0)] − E[Vdo(π,s−)(x0)]). By the policy gradient theorem, the gradient of Cπ itself has no such factor; α is the derivative of (Cπ)², i.e., the gradient of the penalty term in Eq. (2). The text should relabel this as the gradient of the squared constraint, not of Cπ.
  2. [Throughout] There are several typographical errors that should be corrected: 'Geni' should be 'Gini', 'emperical' should be 'empirical', 'instantenous' should be 'instantaneous', 'non-treatement' should be 'non-treatment', and 'PPC-C' in Section 5.2 should be 'PPO-C' for consistency.
  3. [Section 5.2] The text states that in Figure 5 'in both settings, the DPE varies more significantly over time compared with the IPE' and uses this to conclude that policy optimization is more effective at decreasing DPE than IPE. Please clarify whether this is a qualitative visual observation or a quantitative comparison, and report the numerical scale or error bars if the latter is intended.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the long-term fairness decomposition and the DPE-benefit-fairness connection are derived identities with independent content; the only self-citations are notational or evaluation choices and are not load-bearing.

full rationale

The paper's central derivation chain is self-contained rather than circular. Long-term fairness Cπ(θ) is defined as an interventional value-function gap, and the decomposition Cπ = DPE + IPE + SPE is a telescoping algebraic identity (A = (A−B)+(B−C)+C) in which the causal labels are given substantive meaning through the constructed policies π0 and πPS. Proposition 1 is then proved from the Bellman equations and the definition of benefit Δ(x,s), yielding the DPE expression E_{x∼π|s+}[π(d1|x,s+)Δ(x,s+)] − E_{x∼π|s−}[π(d1|x,s−)Δ(x,s−)]. This is a derived equality, not an input fitted to data, and the benefit-fairness connection follows as a stated sufficient condition (identical state-visitation distributions in terms of benefit). The paper explicitly acknowledges that differing visitation distributions can make benefit fairness incompatible with zero DPE, so no fitted parameter is relabeled as a prediction. The baseline qualification adjustment in Section 4 is an additive construction using the back-door criterion under an explicit unconfoundedness assumption; it is a modeling choice, not a hidden circular fit. The only self-citations are to Hu & Zhang (2022) for notation ('do(π)') and to Hu et al. (2024) for using Wasserstein distance as an evaluation metric; neither supplies a load-bearing theorem, a uniqueness claim, or an ansatz that the present derivation depends on. The Remark claiming broad alignment between benefit fairness and decreasing DPE when Δ is S-independent is not rigorously entailed by the equations and is a correctness/rigor concern, not a circularity, because the DPE formula itself is not assumed to imply it. The experiments directly optimize the proposed objectives and compare against external baselines, so no prediction reduces to its own fitted input. Accordingly, no circular step is identified; the minor self-citations do not raise the score above 2.

Assumptions & free parameters 4 free parameters · 5 assumptions · 2 invented entities

The framework uses standard MDP and SCM assumptions, an additive qualification gain, and an unproven identifiability condition. The hyperparameters Beta_C, Beta_KL, Beta_Lambda, and epsilon are chosen by hand and not reported with values. No external entities are introduced beyond the virtual policy and fictitious pre-period used for causal decomposition.

free parameters (4)
  • Beta_C (qualification gain parity coefficient)
    Penalty weight in objective Eq. (2); chosen by hand, and no value is reported in the main text.
  • Beta_Lambda (benefit fairness coefficient)
    Penalty weight in Eq. (5); varied as 'base' versus 'large' in the experiments, but no numeric base value is given.
  • Beta_KL (KL penalty coefficient)
    KL divergence penalty in PPO; Appendix D shows Beta_KL=0 versus 10, but the value used in the main experiments is not reported.
  • epsilon (benefit-distance radius)
    Small constant in the Lambda metric that prevents division by zero and controls the benefit distance within which positive decision rates must match; its value is not specified.
assumptions (5)
  • domain assumption The policy is Markovian and decisions depend only on current state and sensitive attribute: pi(d|x,s).
    Adopted at the start of Section 3.2; this is a standard MDP assumption.
  • domain assumption The sensitive attribute S is unconfounded with qualification states X, so P(x|do(s))=P(x|s).
    Explicitly assumed in Section 4 to apply the back-door criterion; the Section 3.4 decomposition also needs this for the causal interpretation.
  • domain assumption The qualification gain function gs is additive over intervals.
    Stated in Section 3.2; without additivity, the total qualification gain along a trajectory is not the sum of per-step gains.
  • domain assumption The virtual-policy value functions Vdo(piPS,s) and Vdo(pi0,s) are identifiable from data.
    The decomposition in Section 3.4 treats these as computable objects, but the paper provides no identifiability analysis for path-specific effects in this RL setting.
  • ad hoc to paper The benefit Delta(x,s) is independent of the sensitive attribute S, or the state visitation distributions across groups are identical, for benefit fairness to imply a reduced DPE.
    The reconciliation claim in the Remark of Section 3.5 depends on this condition; it is stated as a design suggestion, not proven.
invented entities (2)
  • Virtual policy piPS
    purpose: A hypothetical policy that uses pi for state transitions but pi0 for computing qualification gains, enabling the direct and delayed decomposition.
    It is a modeling device with no observable counterpart; the path-specific effects it defines inherit all model assumptions.
  • Fictitious pre-period t=0
    purpose: An inserted initial time step designed to separate baseline qualification differences from gains accumulated under the policy.
    Introduced in Section 4 to express baseline gaps as counterfactual gains; it is not observable and relies on the no-confounding assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Causal Lens for Learning Long-term Fair Policies." pith.science (2026). https://pith.science/paper/DTCLFDCH

@misc{pith2026250611242,
  author       = {Pith},
  title        = {Pith review of: A Causal Lens for Learning Long-term Fair Policies},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DTCLFDCH}},
  note         = {Machine review of arXiv:2506.11242}
}
read the original abstract

Fairness-aware learning studies the development of algorithms that avoid discriminatory decision outcomes despite biased training data. While most studies have concentrated on immediate bias in static contexts, this paper highlights the importance of investigating long-term fairness in dynamic decision-making systems while simultaneously considering instantaneous fairness requirements. In the context of reinforcement learning, we propose a general framework where long-term fairness is measured by the difference in the average expected qualification gain that individuals from different groups could obtain.Then, through a causal lens, we decompose this metric into three components that represent the direct impact, the delayed impact, as well as the spurious effect the policy has on the qualification gain. We analyze the intrinsic connection between these components and an emerging fairness notion called benefit fairness that aims to control the equity of outcomes in decision-making. Finally, we develop a simple yet effective approach for balancing various fairness notions.

Figures

Figures reproduced from arXiv: 2506.11242 by the authors.

Figure 1
Figure 1. Causal graph for representing the discrete [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Causal graphs for represent￾ing the two hypothetical policies (the reward is omitted from the figures). (a) The baseline policy π0 that always makes the negative decisions. (b) The virtual policy where the state transitions occur as if under π while the qualifica￾tion is obtained as if under π0. The explanation of this decomposition is as follows. For deriving the direct impact, we employ the policy π P S as the ref… view at source ↗
Figure 3
Figure 3. Utility comparison across different settings. [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Qualification gain disparity comparison across different settings. [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Qualification gain disparity decompositions. [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Benefit fairness and DPE for PPO-C and PPO-Cb model variants. [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Comparison of long-term fairness and bank profit in Home Mortgage (Setting 1) and [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Utility earned by the PPO-C variant comparing when the agent was trained with [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: Lending rates for the PPO-Cb variant in setting 2. [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

13 extracted references · 8 canonical work pages

  1. [3]

    We generated the repayment probabilities by fitting a logistic regression model to credit score datasets and used the predicted repayment probability from each credit score level

    These settings differ from each other in initial distributions over credit scores, repayment probabilities, and credit drift likelihoods. We generated the repayment probabilities by fitting a logistic regression model to credit score datasets and used the predicted repayment probability from each credit score level. Setting 1 and 2 both use the same distr...

  2. [9]

    Nuno Mota, Negar Mohammadi, Palash Dey, Krishna P Gummadi, and Abhijnan Chakraborty

    URL https: //kaggle.com/competitions/home-credit-default-risk . Nuno Mota, Negar Mohammadi, Palash Dey, Krishna P Gummadi, and Abhijnan Chakraborty. Fair partitioning of public resources: Redrawing district boundary to minimize spatial inequality in school funding. In Proceedings of the Web Conference 2021, pp. 646–657,

  3. [10]

    Candice Schumann, Jeffrey Foster, Nicholas Mattei, and John Dickerson

    URL https://arxiv.org/abs/1707.06347. Candice Schumann, Jeffrey Foster, Nicholas Mattei, and John Dickerson. We need fairness and explainability in algorithmic hiring. In International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS),

  4. [11]

    Graphgini: Fostering individual and group fairness in graph neural networks

    Anuj Kumar Sirohi, Anjali Gupta, Sayan Ranu, Sandeep Kumar, and Amitabha Bagchi. Graphgini: Fostering individual and group fairness in graph neural networks. arXiv preprint arXiv:2402.12937,

  5. [12]

    A survey on causal reinforce- ment learning

    Yan Zeng, Ruichu Cai, Fuchun Sun, Libo Huang, and Zhifeng Hao. A survey on causal reinforce- ment learning. arXiv preprint arXiv:2302.05209,

  6. [2007]

    A short-term intervention for long-term fairness in the labor market

    Lily Hu and Yiling Chen. A short-term intervention for long-term fairness in the labor market. In Proceedings of the 2018 World Wide Web Conference, pp. 1389–1398,

  7. [2015]

    Survey on fair reinforcement learning: Theory and practice

    Pratik Gajane, Akrati Saxena, Maryam Tavakol, George Fletcher, and Mykola Pechenizkiy. Survey on fair reinforcement learning: Theory and practice. arXiv preprint arXiv:2205.10032,

  8. [2016]

    Runtime monitor- ing of dynamic fairness properties

    Thomas Henzinger, Mahyar Karimi, Konstantin Kueffner, and Kaushik Mallik. Runtime monitor- ing of dynamic fairness properties. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pp. 604–614,

Show all 13 references
  1. [2017]

    Distributionally robust per- formative optimization

    Zhuangzhuang Jia, Yijie Wang, Roy Dong, and Grani A Hanasusanto. Distributionally robust per- formative optimization. arXiv preprint arXiv:2407.01344,

  2. [2018]

    The disparate equilibria of algorithmic decision making when individuals invest ratio- nally

    Lydia T Liu, Ashia Wilson, Nika Haghtalab, Adam Tauman Kalai, Christian Borgs, and Jennifer Chayes. The disparate equilibria of algorithmic decision making when individuals invest ratio- nally. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency...

  3. [2020]

    Fairness is not static: deeper understanding of long term fairness via simulation studies

    Alexander D’Amour, Hansa Srinivasan, James Atwood, Pallavi Baljekar, David Sculley, and Yoni Halpern. Fairness is not static: deeper understanding of long term fairness via simulation studies. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency,...

  4. [2022]

    Striking a balance in fairness for dynamic systems through reinforcement learning

    Yaowei Hu, Jacob Lear, and Lu Zhang. Striking a balance in fairness for dynamic systems through reinforcement learning. In 2023 IEEE International Conference on Big Data (BigData), pp. 662–

  5. [2024]

    Addressing polarization and unfairness in perfor- mative prediction

    Kun Jin, Tian Xie, Yang Liu, and Xueru Zhang. Addressing polarization and unfairness in perfor- mative prediction. arXiv preprint arXiv:2406.16756,

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.