REVIEW 4 major objections 3 minor 13 references
A Causal Lens for Learning Long-term Fair Policies
T0 review · 4 major / 3 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Qualification-gain fairness decomposes into direct, delayed, and spurious policy effects, and the direct effect is what benefit fairness controls.
desk verdict Useful decomposition of long-term fairness, but the benefit-fairness/DPE alignment claim is unsupported as stated and the core equations contain typos. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is a telescoping decomposition of value functions defined by two hypothetical policies: the baseline policy $\pi_0$ always takes the negative decision $d_0$, and the virtual policy $\pi_{PS}$ computes qualification gains as if under $\pi_0$ while letting state transitions follow $\pi$. Subtracting the value functions gives $V_{\mathrm{do}(\pi,s)}=(V_{\mathrm{do}(\pi,s)}-V_{\mathrm{do}(\pi_{PS},s)})+(V_{\mathrm{do}(\pi_{PS},s)}-V_{\mathrm{do}(\pi_0,s)})+V_{\mathrm{do}(\pi_0,s)}$, and averaging this across $s_+$ and $s_-$ produces the DPE/IPE/SPE split. Proposition 1 converts the direct term into $\pi(d_1|x,s)\Delta(x,s)$ weighted by the state-visitation count, which is the link from the long-term metric to benefit fairness. The same machinery gives the gradient of $C_\pi(\theta)$ through the policy gradient theorem and supplies the baseline adjustment $G_{s_+}-G_{s_-}$ for pre-existing qualification gaps.
What would settle it
In the loan simulator, introduce a hidden variable that increases both the probability of membership in the disadvantaged group and the initial credit score, estimate the parity gap with the paper's observed-data procedure, and compare it with the true interventional gap obtained by hard intervention; a mismatch would falsify the no-confounder assumption.
Extended reading notes
Core claim
The paper's central claim is that long-term fairness, defined as qualification gain parity $C_\pi(\theta)=\mathbb{E}[V_{\mathrm{do}(\pi,s_+)}(x_1)]-\mathbb{E}[V_{\mathrm{do}(\pi,s_-)}(x_1)]$, is not monolithic: it decomposes as $C_\pi(\theta)=\mathrm{DPE}+\mathrm{IPE}+\mathrm{SPE}$, where $\mathrm{DPE}$ is the direct causal effect of the policy on qualification gains, $\mathrm{IPE}$ is the delayed effect transmitted through the environment, and $\mathrm{SPE}$ is the spurious environment-only gap that would remain under a baseline policy that always denies treatment. The paper proves that the direct component has the closed form $\mathrm{DPE}=\mathbb{E}_{x\sim\pi|s_+}[\pi(d_1|x,s_+)\Delta(x,s_+)]-\mathbb{E}_{x\sim\pi|s_-}[\pi(d_1|x,s_-)\Delta(x,s_-)]$, where $\Delta(x,s)$ is the expected qualification gain from treatment. Because benefit fairness requires equal treatment probability for equal benefit, this identity says that the direct policy component of long-term unfairness is exactly the quantity benefit fairness regulates; when $\Delta$ is independent of the sensitive attribute, enforcing benefit fairness reduces that direct component. This is the basis for the paper's claim that short-term and long-term fairness can be reconciled, and it motivates a policy-gradient objective that penalizes both the qualification-gain gap and a Gini-inspired benefit-fairness term.
Load-bearing premise
The construction assumes that nothing unobserved drives both a person's protected group and their qualifications; if such a confounder exists, the estimated fairness gap is not the true causal effect.
Editorial extensions
If this is right
- Enforcing benefit fairness through the Gini-inspired penalty reduces the DPE in the loan simulation, so the alignment between benefit fairness and long-term fairness is observable, not just formal.
- A policy trained with the qualification-gain-parity constraint outperforms advantage-regularized baselines at reducing the Wasserstein distance between group qualification distributions, especially when the baseline qualification gap is included.
- The IPE changes less than the DPE during optimization, meaning delayed environmental effects are more resistant to policy-gradient constraints than the direct effect of the decisions.
- The decomposition gives practitioners a diagnostic: monitoring DPE, IPE, and SPE separately can reveal whether remaining unfairness comes from current decisions, from the dynamics the policy induces, or from the environment alone.
Reading between the lines
- A consequence the authors do not spell out is that, because DPE is a benefit-weighted treatment gap, benefit fairness could be audited from logged decisions and estimated benefits without re-solving the full MDP; this diagnostic use is not developed in the paper.
- The same DPE/IPE/SPE split should apply to any additive long-term outcome measure, not only qualification gains, provided a gain function with the paper's interval-additivity property exists.
- The paper's design suggestion that $\Delta$ be made independent of $S$ implies a concrete search problem over qualification gain functions; an explicit algorithm for that search is a natural next step that the paper does not test.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a causal decomposition of long-term fairness in reinforcement learning. Long-term fairness is formalized as qualification gain parity, Cπ(θ) = E[Vdo(π,s+)(x1)] − E[Vdo(π,s−)(x1)], and is decomposed into a Direct Policy Effect (DPE), an Indirect Policy Effect (IPE), and a Spurious Policy Effect (SPE) using two hypothetical policies, a baseline policy π0 and a virtual policy πPS. The paper then links DPE to benefit fairness, proposes a Gini-inspired regularizer Λ to enforce benefit fairness alongside qualification gain parity, and evaluates the resulting PPO variants on a simulated loan environment from D'Amour et al. (2020). The experiments report utility, qualification-gain disparity, decomposition components, and benefit-fairness behavior under different regularization weights.
Significance. If the decomposition and the benefit-fairness connection were established, the paper would provide a useful framework for separating direct, delayed, and spurious sources of long-term unfairness in dynamic decision systems, and a practical way to balance short-term benefit fairness with long-term parity. Strengths include the use of a formal Bellman/policy-gradient setting, a concrete causal decomposition that is largely an algebraic identity, and the release of code for the experiments. However, the central qualitative claim that benefit fairness aligns with reducing DPE is not supported by the stated definitions, and several technical typos in the central proposition and decomposition need correction before the theory can be evaluated fairly.
major comments (4)
- [Section 3.4] The displayed definitions of DPE and IPE use Vdo(πPS,s+) in the second expectation for both groups. With these definitions, DPE equals E[Vdo(π,s+)] − E[Vdo(π,s−)] and IPE equals −SPE, so the displayed equality Cπ = DPE + IPE + SPE holds only by cancellation and the components do not isolate the direct and indirect policy effects as claimed. The intended definitions should use Vdo(πPS,s−) in the second expectation of both DPE and IPE. Because the subsequent benefit-fairness reformulation in Section 3.5 relies on the corrected DPE, this indexing error is load-bearing and must be fixed.
- [Section 3.5 and Appendix B] Proposition 1 states that Vdo(π,s)(x) − Vdo(πPS,s)(x) = Σ_{x'} ηπ,s(x→x') π(d1|x,s)Δ(x,s). As printed, the summand does not depend on the summation index x', so the right-hand side collapses to π(d1|x,s)Δ(x,s) times the total occupancy mass and is generally false. The proof in Appendix B derives Gπ,s(x') under the sum, so the correct statement should have π(d1|x',s)Δ(x',s) inside the sum. With that correction, the reformulation of DPE as the difference of expected π(d1|x,s)Δ(x,s) under the two group-wise visitation distributions is valid.
- [Section 3.5, Definition 2 and Remark] Benefit fairness does not imply a reduced or zero DPE even when Δ is independent of S. Definition 2 only equates treatment probabilities within equal-benefit cells; it imposes no constraint on the group-specific state visitation distributions. For example, with binary X, Δ(x,s)=x, and the benefit-fair policy π(d1|x,s)=x, we have DPE = P_{π|s+}(X=1) − P_{π|s−}(X=1), which is 0.8 if the two groups visit X=1 with probabilities 0.9 and 0.1. The paper's own preceding sentence recognizes that differing state visitation distributions make benefit fairness incompatible with a zero DPE, but the Remark nevertheless asserts a general alignment when Δ is independent of S. The stated sufficient condition, "state visitation distributions in terms of benefit are identical", is not proved and is much stronger than Δ-independence. Since this alignment motivates the Λ regularizer and the paper's reconciliation conclusion, the central qualitative claim is not established by the provided theory.
- [Section 4] The causal interpretation of DPE, IPE, SPE, and the baseline correction Gs+ − Gs− requires identifiability conditions that are not stated. The back-door criterion is invoked only to justify P(x|do(s)) = P(x|s), but the counterfactual term P(Xs = x′|X = x) used in Gs is a different quantity; its identification requires a fully specified Markovian SCM or equivalent no-unobserved-confounding assumptions. Similarly, Vdo(πPS,s) is a path-specific interventional quantity, and path-specific effects are identifiable only under additional conditions such as the absence of recanting witnesses. The manuscript should state these assumptions explicitly or restrict the causal interpretation of the decomposed quantities accordingly.
minor comments (3)
- [Section 3.3 and Appendix A] The displayed expression for ∇θCπ(θ) contains the factor α = 2(E[Vdo(π,s+)(x0)] − E[Vdo(π,s−)(x0)]). By the policy gradient theorem, the gradient of Cπ itself has no such factor; α is the derivative of (Cπ)², i.e., the gradient of the penalty term in Eq. (2). The text should relabel this as the gradient of the squared constraint, not of Cπ.
- [Throughout] There are several typographical errors that should be corrected: 'Geni' should be 'Gini', 'emperical' should be 'empirical', 'instantenous' should be 'instantaneous', 'non-treatement' should be 'non-treatment', and 'PPC-C' in Section 5.2 should be 'PPO-C' for consistency.
- [Section 5.2] The text states that in Figure 5 'in both settings, the DPE varies more significantly over time compared with the IPE' and uses this to conclude that policy optimization is more effective at decreasing DPE than IPE. Please clarify whether this is a qualitative visual observation or a quantitative comparison, and report the numerical scale or error bars if the latter is intended.
Circularity Check
No significant circularity: the long-term fairness decomposition and the DPE-benefit-fairness connection are derived identities with independent content; the only self-citations are notational or evaluation choices and are not load-bearing.
full rationale
The paper's central derivation chain is self-contained rather than circular. Long-term fairness Cπ(θ) is defined as an interventional value-function gap, and the decomposition Cπ = DPE + IPE + SPE is a telescoping algebraic identity (A = (A−B)+(B−C)+C) in which the causal labels are given substantive meaning through the constructed policies π0 and πPS. Proposition 1 is then proved from the Bellman equations and the definition of benefit Δ(x,s), yielding the DPE expression E_{x∼π|s+}[π(d1|x,s+)Δ(x,s+)] − E_{x∼π|s−}[π(d1|x,s−)Δ(x,s−)]. This is a derived equality, not an input fitted to data, and the benefit-fairness connection follows as a stated sufficient condition (identical state-visitation distributions in terms of benefit). The paper explicitly acknowledges that differing visitation distributions can make benefit fairness incompatible with zero DPE, so no fitted parameter is relabeled as a prediction. The baseline qualification adjustment in Section 4 is an additive construction using the back-door criterion under an explicit unconfoundedness assumption; it is a modeling choice, not a hidden circular fit. The only self-citations are to Hu & Zhang (2022) for notation ('do(π)') and to Hu et al. (2024) for using Wasserstein distance as an evaluation metric; neither supplies a load-bearing theorem, a uniqueness claim, or an ansatz that the present derivation depends on. The Remark claiming broad alignment between benefit fairness and decreasing DPE when Δ is S-independent is not rigorously entailed by the equations and is a correctness/rigor concern, not a circularity, because the DPE formula itself is not assumed to imply it. The experiments directly optimize the proposed objectives and compare against external baselines, so no prediction reduces to its own fitted input. Accordingly, no circular step is identified; the minor self-citations do not raise the score above 2.
Assumptions & free parameters
free parameters (4)
- Beta_C (qualification gain parity coefficient)
- Beta_Lambda (benefit fairness coefficient)
- Beta_KL (KL penalty coefficient)
- epsilon (benefit-distance radius)
assumptions (5)
- domain assumption The policy is Markovian and decisions depend only on current state and sensitive attribute: pi(d|x,s).
- domain assumption The sensitive attribute S is unconfounded with qualification states X, so P(x|do(s))=P(x|s).
- domain assumption The qualification gain function gs is additive over intervals.
- domain assumption The virtual-policy value functions Vdo(piPS,s) and Vdo(pi0,s) are identifiable from data.
- ad hoc to paper The benefit Delta(x,s) is independent of the sensitive attribute S, or the state visitation distributions across groups are identical, for benefit fairness to imply a reduced DPE.
invented entities (2)
-
Virtual policy piPS
-
Fictitious pre-period t=0
Cite this review
Pith. "Pith review of A Causal Lens for Learning Long-term Fair Policies." pith.science (2026). https://pith.science/paper/DTCLFDCH
@misc{pith2026250611242,
author = {Pith},
title = {Pith review of: A Causal Lens for Learning Long-term Fair Policies},
year = {2026},
howpublished = {\url{https://pith.science/paper/DTCLFDCH}},
note = {Machine review of arXiv:2506.11242}
}
read the original abstract
Fairness-aware learning studies the development of algorithms that avoid discriminatory decision outcomes despite biased training data. While most studies have concentrated on immediate bias in static contexts, this paper highlights the importance of investigating long-term fairness in dynamic decision-making systems while simultaneously considering instantaneous fairness requirements. In the context of reinforcement learning, we propose a general framework where long-term fairness is measured by the difference in the average expected qualification gain that individuals from different groups could obtain.Then, through a causal lens, we decompose this metric into three components that represent the direct impact, the delayed impact, as well as the spurious effect the policy has on the qualification gain. We analyze the intrinsic connection between these components and an emerging fairness notion called benefit fairness that aims to control the equity of outcomes in decision-making. Finally, we develop a simple yet effective approach for balancing various fairness notions.
Figures
Figures from the paper (6 more)
Reference graph
Works this paper leans on
-
[3]
These settings differ from each other in initial distributions over credit scores, repayment probabilities, and credit drift likelihoods. We generated the repayment probabilities by fitting a logistic regression model to credit score datasets and used the predicted repayment probability from each credit score level. Setting 1 and 2 both use the same distr...
work page 2018
-
[9]
Nuno Mota, Negar Mohammadi, Palash Dey, Krishna P Gummadi, and Abhijnan Chakraborty
URL https: //kaggle.com/competitions/home-credit-default-risk . Nuno Mota, Negar Mohammadi, Palash Dey, Krishna P Gummadi, and Abhijnan Chakraborty. Fair partitioning of public resources: Redrawing district boundary to minimize spatial inequality in school funding. In Proceedings of the Web Conference 2021, pp. 646–657,
work page 2021
-
[10]
Candice Schumann, Jeffrey Foster, Nicholas Mattei, and John Dickerson
URL https://arxiv.org/abs/1707.06347. Candice Schumann, Jeffrey Foster, Nicholas Mattei, and John Dickerson. We need fairness and explainability in algorithmic hiring. In International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS),
-
[11]
Graphgini: Fostering individual and group fairness in graph neural networks
Anuj Kumar Sirohi, Anjali Gupta, Sayan Ranu, Sandeep Kumar, and Amitabha Bagchi. Graphgini: Fostering individual and group fairness in graph neural networks. arXiv preprint arXiv:2402.12937,
-
[12]
A survey on causal reinforce- ment learning
Yan Zeng, Ruichu Cai, Fuchun Sun, Libo Huang, and Zhifeng Hao. A survey on causal reinforce- ment learning. arXiv preprint arXiv:2302.05209,
-
[2007]
A short-term intervention for long-term fairness in the labor market
Lily Hu and Yiling Chen. A short-term intervention for long-term fairness in the labor market. In Proceedings of the 2018 World Wide Web Conference, pp. 1389–1398,
work page 2018
-
[2015]
Survey on fair reinforcement learning: Theory and practice
Pratik Gajane, Akrati Saxena, Maryam Tavakol, George Fletcher, and Mykola Pechenizkiy. Survey on fair reinforcement learning: Theory and practice. arXiv preprint arXiv:2205.10032,
-
[2016]
Runtime monitor- ing of dynamic fairness properties
Thomas Henzinger, Mahyar Karimi, Konstantin Kueffner, and Kaushik Mallik. Runtime monitor- ing of dynamic fairness properties. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pp. 604–614,
work page 2023
Show all 13 references
-
[2017]
Distributionally robust per- formative optimization
Zhuangzhuang Jia, Yijie Wang, Roy Dong, and Grani A Hanasusanto. Distributionally robust per- formative optimization. arXiv preprint arXiv:2407.01344,
-
[2018]
The disparate equilibria of algorithmic decision making when individuals invest ratio- nally
Lydia T Liu, Ashia Wilson, Nika Haghtalab, Adam Tauman Kalai, Christian Borgs, and Jennifer Chayes. The disparate equilibria of algorithmic decision making when individuals invest ratio- nally. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency...
2020
-
[2020]
Fairness is not static: deeper understanding of long term fairness via simulation studies
Alexander D’Amour, Hansa Srinivasan, James Atwood, Pallavi Baljekar, David Sculley, and Yoni Halpern. Fairness is not static: deeper understanding of long term fairness via simulation studies. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency,...
2020
-
[2022]
Striking a balance in fairness for dynamic systems through reinforcement learning
Yaowei Hu, Jacob Lear, and Lu Zhang. Striking a balance in fairness for dynamic systems through reinforcement learning. In 2023 IEEE International Conference on Big Data (BigData), pp. 662–
2023
-
[2024]
Addressing polarization and unfairness in perfor- mative prediction
Kun Jin, Tian Xie, Yang Liu, and Xueru Zhang. Addressing polarization and unfairness in perfor- mative prediction. arXiv preprint arXiv:2406.16756,
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.