Pith. sign in

REVIEW 5 minor 56 references

Fair Resource Allocation in Weakly Coupled Markov Decision Processes

T0 review · 0 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read For symmetric weakly coupled MDPs, maximizing the generalized Gini fairness objective is exactly equivalent to maximizing average reward over permutation-invariant policies, so fair allocation reduces to a standard utilitarian problem.

desk verdict A correct, clean utilitarian reduction for GGF-fair symmetric WCMDPs — solid theory, honest scope, and worth a serious referee. read the letter →

arxiv 2411.09804 v2 pith:KJR7DOFY submitted 2024-11-14 cs.LG

Xiaohui Tu , Yossiri Adulyasak , Nima Akbarzadeh , Erick Delage This is my paper · ORCID
classification cs.LG MSC 90C40
keywords fairresourceallocationweaklycoupledMarkovdecisionprocessesgeneralizedGinifunctionutilitarianreductionpermutation-invariantpoliciescountaggregationrestlessbanditsdeepreinforcementlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that fair sequential resource allocation — maximizing the generalized Gini function of the agents' expected discounted rewards in a weakly coupled Markov decision process — can, when all sub-MDPs are identical and symmetrically coupled, be solved exactly by optimizing the plain utilitarian (average-reward) objective over permutation-invariant policies. This matters because the exact linear-programming formulation of the fair problem is exponential in the number of agents, whereas the utilitarian problem can be attacked with Whittle index policies in restless bandits or, in full generality, with the count-proportion deep reinforcement learning method the paper introduces. The reduction (Theorem 3.4) makes fairness essentially free under symmetry: any optimal permutation-invariant average-reward policy is simultaneously optimal for the generalized Gini objective under every weight vector. Machine-replacement experiments on a restless-bandit benchmark support the claim.

What carries the argument

The load-bearing objects are the symmetric WCMDP (Definition 3.1) and three constructions: the permutation-invariant policy (Definition 3.2), the uniform-value averaging lemma (Lemma 3.3), and the count aggregation MDP (Definition 3.5). Lemma 3.3 constructs, from an arbitrary policy, a permutation-invariant policy by averaging occupancy measures over the permutation group; in a symmetric WCMDP the averaged policy assigns every sub-MDP the same expected discounted reward. Because the generalized Gini function evaluated on a uniform vector equals that vector's mean, the fair objective over the averaged policy collapses to the utilitarian objective, and Theorem 3.4 extracts this collapse as an equivalence of optimal policy sets. The count aggregation MDP then rewrites the utilitarian problem over state counts rather than the full product state space, and its proportion-normalized form is the state and action representation on which the paper's policy network is built.

What would settle it

Enumerate all stationary policies of a small symmetric instance (for example $N=2$, $S=3$, $A=2$), compute each policy's value vector, and check whether every permutation-invariant policy that maximizes average reward also maximizes $\operatorname{GGF}_w$ for several weight vectors such as $w_n \propto 2^{-n}$; a single mismatch would refute Theorem 3.4. A complementary probe perturbs symmetry by shifting one sub-MDP's transition matrix by $\varepsilon$ and measures the gap between the GGF-optimal value and the best permutation-invariant utilitarian value as $\varepsilon$ grows, which would reveal how much of the reduction survives when the assumption holds only approximately.

Watch

Extended reading notes

Core claim

The paper's central discovery is Theorem 3.4 (Utilitarian Reduction): for a symmetric WCMDP, the set $\Pi^*_{1/N,PI}$ of permutation-invariant policies that are optimal for the utilitarian objective is non-empty, and every such policy $\pi^*$ satisfies $\operatorname{GGF}_w[V^{\pi^*}_0] = \max_\pi \operatorname{GGF}_w[V^{\pi}_0]$ for every weight vector $w \in \Delta(N)$. In words, once the environment treats all sub-MDPs interchangeably, the Gini-fair allocation problem and the average-reward problem share their optimal policies inside the permutation-invariant class, so a planner who needs fairness can simply maximize average reward. The proof averages: from any policy, averaging its occupancy measure over all $N!$ index permutations produces a permutation-invariant policy whose value vector is uniform, every sub-MDP receiving the same mean value; the generalized Gini function of a uniform vector is exactly that mean, which is the utilitarian objective. An extension (Corollary B.4.1) shows the same reduction holds for any fairness measure that is concave, permutation-invariant, and constant-vector invariant, including $\alpha$-fairness.

Load-bearing premise

The load-bearing premise is full interchangeability of the agents — identical sub-MDP dynamics, identical reward and resource-consumption functions, and a permutation-invariant initial distribution; if the sub-MDPs are heterogeneous, a policy that maximizes average reward need not maximize the generalized Gini fairness value, and the reduction can fail.

Editorial extensions

If this is right

  • Any algorithm that optimizes the utilitarian objective — including Whittle index policies on restless bandits — becomes a valid solver for the Gini-fair problem whenever the WCMDP is symmetric, which is exactly how the experiments benchmark against the index policy.
  • The exact GGF-LP and its count-aggregation dual solve small instances provably optimally; the count dual replaces joint state-action spaces that grow exponentially with the number of agents by count-state models whose size grows polynomially for fixed state count.
  • By Corollary B.4.1 the reduction carries over to every concave, permutation-invariant, constant-vector-invariant fairness measure, so the same utilitarian machinery serves objectives such as $\alpha$-fairness without new algorithms.
  • Because the CP-DRL network consumes state and resource proportions of fixed dimension, a single trained policy can be evaluated across different numbers of agents and different resource budgets, which the scalability experiments demonstrate up to $N=100$ machines.
  • On the machine-replacement instances, the count-proportion approach matches the LP optimum on small problems and outperforms the Whittle index policy and heuristic baselines as the number of machines grows.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A practical diagnostic follows that the paper leaves implicit: in deployments whose subproblems are near-identical with exchangeable initial conditions, an ordinary utilitarian policy already satisfies the Gini criterion, so fairness-specific machinery may be unnecessary until heterogeneity appears.
  • The theorem implies a testable gap question: how quickly does the difference between the GGF-optimal value and the best value of a permutation-invariant utilitarian policy grow as sub-MDP transition matrices drift apart from identical; measuring this gap at small asymmetry would show how much of the result survives approximate symmetry.
  • The count-proportion architecture suggests a stronger transfer claim than the paper fully tests: priority scores learned on a small population should map to near-optimal policies on much larger ones, which would make the method a candidate for zero-shot fleet scaling.
  • Since the Gini function is one member of the concave invariant welfare family, the same reduction suggests that count-proportion solvers could serve Nash welfare or other social objectives, though the paper demonstrates only the Gini case.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

0 major / 5 minor

Summary. The paper formulates a fair resource-allocation problem in weakly coupled Markov decision processes (WCMDPs) under the generalized Gini social welfare function (GGF), and proposes both exact and scalable solution methods. For the general WCMDP, it gives an exact linear program (GGF-LP, model (3)). Its main theoretical contribution, Theorem 3.4, states that when the WCMDP is symmetric (identical sub-MDPs, identical resource-consumption functions, and a permutation-invariant initial distribution), the GGF-optimal problem reduces to the utilitarian average-reward problem restricted to permutation-invariant policies: every permutation-invariant optimal utilitarian policy is also optimal for the GGF objective with any weight vector w, and at least one such policy exists. The proof proceeds by averaging occupancy measures over all permutations (Lemma 3.3) and using the dominance of the utilitarian GGF (Lemma B.4). The paper then introduces a count-aggregation MDP to reduce the state and action spaces, develops a count-proportion-based deep RL method (CP-DRL), and validates both the reduction and the heuristic on machine-replacement/RMAB experiments with optimal LP, Whittle-index, and heuristic baselines.

Significance. The utilitarian reduction is a strong and practically useful result: it connects a fairness-oriented objective to standard utilitarian solvers, including Whittle-index policies for restless bandits, and it is proved with clean occupancy-measure and rearrangement arguments. The extension in Corollary B.4.1 to concave, permutation-invariant, constant-vector-invariant fairness measures broadens the scope of the reduction. The exact GGF-LP formulation is correct and provides an independent baseline for the experiments, while the count-proportion representation gives a fixed-size state/action encoding that supports the CP-DRL method. The empirical study is reasonably comprehensive, covering optimality against exact LP solutions, flexibility across problem sizes, scalability up to 100 machines, and runtime comparisons, and the code is provided. The main limitations—heterogeneous sub-MDPs and the heuristic nature of CP-DRL—are explicitly scoped in the paper.

minor comments (5)
  1. [Section 3.2, Eq. (4)] The displayed objective contains a spurious factor 1/S in front of the expectation: since \bar r_\phi already averages rewards over the N sub-MDPs, the expected discounted mean reward should be E[\sum_t \gamma^t \bar r_\phi(x_t,u_t)], not (1/S) times this quantity. The factor is policy-independent and therefore does not change the optimizer, but the stated equality to \bar V^{\pi_\phi}_0 is incorrect and should be fixed.
  2. [Section 3.2, Eq. (15)] The transition probability p^{(N)}_\phi(x'|x,u) is written using an arbitrary representative (s,a) with x=f(s) and u=g_s(a), but f^{-1}(x) is a set, so the notation is not precise. The authors should state explicitly that the sum is independent of the chosen representative and give the grouped formula in terms of the counts u.
  3. [Appendix B.3, Lemma 3.3 proof] The constructed policy \bar\pi is defined implicitly through the averaged occupancy measure \bar q; for states with zero occupancy mass the policy \bar\pi(s,a) is undefined unless a convention is stated. The permutation-invariance check is also written with "\propto" rather than equality; adding a short normalization argument would remove this small gap.
  4. [Appendix B.5, Corollary B.4.1] The corollary asserts that the expected-utility model \rho[v]=u^{-1}((1/N)\sum_n u(v_n)) satisfies concavity, permutation invariance, and constant-vector invariance, but the concavity claim is not proved. A one-sentence proof or a suitable citation for the concavity of this class of welfare functions should be included.
  5. [Throughout] There are several small typographical and wording issues: "consistantly" in Section 5, "bar optimal" in the proof of Theorem 3.4 in Appendix B.4, and "contant" in Appendix B.5. I also suggest softening the abstract's "for the first time" priority claim unless a literature search can be cited to support it.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the utilitarian reduction is derived from explicit symmetry and concavity assumptions, with no fitted parameter renamed as a prediction.

full rationale

The paper's central claim, Theorem 3.4, is derived in a self-contained way. Lemma 3.3 constructs a permutation-invariant policy by averaging the occupancy measures of all permuted versions of an arbitrary policy; the averaging is valid because the symmetry conditions in Definition 3.1 make the feasible action set, transition kernel, and initial distribution permutation-invariant. The resulting uniform value representation is then used in Theorem 3.4. Lemma B.4 is a direct inequality between GGF_{1/N} and GGF_w, and the chain of inequalities in equation (12) forces equality through an argument that does not presuppose the theorem's conclusion. The reduction does not fit any parameter to the GGF-optimal value and then call it a prediction; the proof is algebraic and relies only on stated assumptions. The experimental OPT baselines are computed directly from the GGF-LP model (3), which is independent of the utilitarian reduction, and the CP-DRL results are approximate and reported with gaps to OPT. Citations to Siddique et al. (2020) and Gast et al. (2024) are external sources, and the only self-citation (Akbarzadeh and Mahajan, 2019) supplies the machine replacement test problem, which is not load-bearing for the main theorem. The paper explicitly scopes Theorem 3.4 to symmetric WCMDPs, so heterogeneous sub-MDPs being outside the theorem is a stated limitation rather than a hidden circular step. No load-bearing reduction of the derivation to its own inputs was found.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central theoretical result rests on the standard MDP framework and on the symmetry assumption; no free parameters are fitted in the derivation. Experimental parameters (GGF weights, discount factor, PPO hyperparameters, cost presets) are listed for transparency but are not part of the proof.

free parameters (4)
  • GGF weights (exponential schedule) = w_n = (1/2^n) / sum_i (1/2^i)
    Chosen for experiments to emphasize the worst-off agents; not fitted to data and not used in the theoretical derivation.
  • Discount factor gamma = 0.95
    Standard experimental choice for the machine replacement instances.
  • PPO hyperparameters = actor LR 5e-4, critic LR 3e-4, two hidden layers of 64 units
    Described in Appendix F.4; these are implementation choices, not part of the theory.
  • Cost structures = Exponential-RCCC and Quadratic-RCCC
    Two synthetic presets for the machine replacement problem; not fitted.
assumptions (5)
  • standard math Finite state and action spaces, stationary transitions and rewards, discounted infinite horizon (gamma < 1).
    Standard MDP assumptions invoked throughout Section 2.1.
  • domain assumption The WCMDP couples sub-MDPs only through linear per-step resource constraints, and an idle action consuming no resources is always available.
    This defines the weakly coupled structure and guarantees feasible actions exist.
  • domain assumption Symmetric WCMDP (Definition 3.1): all sub-MDPs are identical, resource consumption functions agree across sub-MDPs, and the initial distribution is permutation-invariant.
    This is the load-bearing assumption for Lemma 3.3 and Theorem 3.4.
  • standard math The generalized Gini function satisfies concavity, permutation invariance, and constant-vector invariance (Weymark 1981).
    Used to prove GGF_{1/N} >= GGF_w and the extension in Corollary B.4.1.
  • standard math The set of occupancy measures is convex and in bijection with stationary policies (Puterman Theorem 6.9.1).
    Used in the proof of Lemma 3.3 to average permuted policies into a new stationary policy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Fair Resource Allocation in Weakly Coupled Markov Decision Processes." pith.science (2026). https://pith.science/paper/KJR7DOFY

@misc{pith2026241109804,
  author       = {Pith},
  title        = {Pith review of: Fair Resource Allocation in Weakly Coupled Markov Decision Processes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KJR7DOFY}},
  note         = {Machine review of arXiv:2411.09804}
}
abstract

We consider fair resource allocation in sequential decision-making environments modeled as weakly coupled Markov decision processes, where resource constraints couple the action spaces of $N$ sub-Markov decision processes (sub-MDPs) that would otherwise operate independently. We adopt a fairness definition using the generalized Gini function instead of the traditional utilitarian (total-sum) objective. After introducing a general but computationally prohibitive solution scheme based on linear programming, we focus on the homogeneous case where all sub-MDPs are identical. For this case, we show for the first time that the problem reduces to optimizing the utilitarian objective over the class of "permutation invariant" policies. This result is particularly useful as we can exploit Whittle index policies in the restless bandits setting while, for the more general setting, we introduce a count-proportion-based deep reinforcement learning approach. Finally, we validate our theoretical findings with comprehensive experiments, confirming the effectiveness of our proposed method in achieving fairness.

Figures

Figures reproduced from arXiv: 2411.09804 by the authors.

Figure 1
Figure 1. CP-based Stochastic Policy Neural Network [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. (Colored) Learning Curves for Different Numbers of Machines (N from 3 to 5). Experimental results for the Exponential-RCCC scenario are shown with y-axes starting at 7 for zoom-in. Red dashed lines represent the OPT values, green dashed lines show the WIP performance, blue lines depict CP-DRL learning curves over 800 episodes, and orange lines show the RDM performance. Shaded areas indicate the standard deviation ac… view at source ↗
Figure 3
Figure 3. (Colored) Scalability and Time Efficiency of CP-DRL. Subfigures (a) and (b) show the scalability of CP-DRL with a fixed resource ratio of 0.1. Subfigure (a) presents GGF values across different machine counts, with intervals representing the standard deviation over 5 runs. Subfigure (b) and (c) depicts time per episode in seconds for a fixed resource ratio of 0.1 and 0.5, respectively. In all time plots, the green l… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 52 canonical work pages

  1. [1]

    and Mersereau, A

    Adelman, D. and Mersereau, A. J. (2008). Relaxations of weakly coupled stochastic dynamic programs. Operations Research , 56(3):712--727

    work page 2008

  2. [2]

    and Mahajan, A

    Akbarzadeh, N. and Mahajan, A. (2019). Restless bandits with controlled restarts: Indexability and computation of whittle index. In 2019 IEEE 58th conference on decision and control (CDC) , pages 7294--7300. IEEE

    work page 2019

  3. [3]

    Remembering to Be Fair: Non-Markovian Fairness in Sequential Decision Making

    Alamdari, P. A., Klassen, T. Q., Creager, E., and McIlraith, S. A. (2023). Remembering to be fair: On non-markovian fairness in sequential decisionmaking (preliminary report). arXiv preprint arXiv:2312.04772

    work page Pith review arXiv 2023

  4. [4]

    Atwood, J., Srinivasan, H., Halpern, Y., and Sculley, D. (2019). Fair treatment allocations in social networks. arXiv preprint arXiv:1911.05489

    work page Pith review arXiv 2019

  5. [5]

    Bistritz, I., Baharav, T., Leshem, A., and Bambos, N. (2020). My fair bandit: Distributed learning of max-min fairness with multi-player bandits. In International Conference on Machine Learning , pages 930--940. PMLR

    work page 2020

  6. [6]

    and Lu, T

    Boutilier, C. and Lu, T. (2016). Budget allocation using weakly coupled, constrained markov decision processes. In UAI

    work page 2016

  7. [7]

    H., and Wynter, L

    Cai, D., Lim, S. H., and Wynter, L. (2021). Efficient reinforcement learning in resource allocation problems through permutation invariant multi-task learning. In 2021 60th IEEE Conference on Decision and Control (CDC) , pages 2270--2275

    work page 2021

  8. [8]

    and Haas, C

    Caton, S. and Haas, C. (2024). Fairness in machine learning: A survey. ACM Computing Surveys

    work page 2024

Show all 56 references
  1. [9]

    J., Wang, J

    Chen, R. J., Wang, J. J., Williamson, D. F., Chen, T. Y., Lipkova, J., Lu, M. Y., Sahai, S., and Mahmood, F. (2023). Algorithmic fairness in artificial intelligence for medicine and healthcare. Nature biomedical engineering , 7(6):719--742

    2023

  2. [10]

    Creager, E., Madras, D., Pitassi, T., and Zemel, R. (2020). Causal Modeling for Fairness In Dynamical Systems . In Proceedings of the 37th International Conference on Machine Learning , pages 2185--2195. PMLR

    2020

  3. [11]

    D'Amour, A., Srinivasan, H., Atwood, J., Baljekar, P., Sculley, D., and Halpern, Y. (2020). Fairness is not static: Deeper understanding of long term fairness via simulation studies. In Proceedings of the 2020 Conference on Fairness , Accountability , and Transparency , pages ...

    2020

  4. [12]

    and Mannor, S

    Delage, E. and Mannor, S. (2010). Percentile Optimization for Markov Decision Processes with Parameter Uncertainty . Operations Research

    2010

  5. [13]

    and Jiang, D

    El Shar, I. and Jiang, D. (2024). Weakly coupled deep q-networks. Advances in Neural Information Processing Systems , 36

    2024

  6. [14]

    Elmalaki, S. (2021). Fair-iot: Fairness-aware human-in-the-loop reinforcement learning for harnessing human variability in personalized iot. In Proceedings of the International Conference on Internet-of-Things Design and Implementation , pages 119--132

    2021

  7. [15]

    Farnadi, G., St-Arnaud, W., Babaki, B., and Carvalho, M. (2021). Individual fairness in kidney exchange programs. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 11496--11505

    2021

  8. [16]

    Gajane, P., Saxena, A., Tavakol, M., Fletcher, G., and Pechenizkiy, M. (2022). Survey on fair reinforcement learning: Theory and practice. arXiv preprint arXiv:2205.10032

    2022 arXiv

  9. [17]

    Gast, N., Gaujal, B., and Yan, C. (2024). Reoptimization nearly solves weakly coupled markov decision processes

    2024

  10. [18]

    Ge, Y., Zhao, X., Yu, L., Paul, S., Hu, D., Hsieh, C.-C., and Zhang, Y. (2022). Toward pareto efficient fairness-utility trade-off in recommendation through reinforcement learning. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining , pag...

    2022

  11. [19]

    Ghalme, G., Nair, V., Patil, V., and Zhou, Y. (2022a). Long- Term Resource Allocation Fairness in Average Markov Decision Process ( AMDP ) Environment . Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems

    2022

  12. [20]

    Ghalme, G., Nair, V., Patil, V., and Zhou, Y. (2022b). Long-term resource allocation fairness in average markov decision process (amdp) environment. In Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems , pages 525--533

    2022

  13. [21]

    Hassanzadeh, P., Kreacic, E., Zeng, S., Xiao, Y., and Ganesh, S. (2023). Sequential fair resource allocation under a markov decision process framework. In Proceedings of the Fourth ACM International Conference on AI in Finance , pages 673--680

    2023

  14. [22]

    Hawkins, J. T. (2003). A Langrangian decomposition approach to weakly coupled dynamic optimization problems and its applications . PhD thesis, Massachusetts Institute of Technology

    2003

  15. [23]

    Herlihy, C., Prins, A., Srinivasan, A., and Dickerson, J. P. (2023). Planning to fairly allocate: Probabilistic fairness in the restless bandit setting. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pages 732--740

    2023

  16. [24]

    Jabbari, S., Joseph, M., Kearns, M., Morgenstern, J., and Roth, A. (2017). Fairness in reinforcement learning. In International conference on machine learning , pages 1617--1626. PMLR

    2017

  17. [25]

    and Lu, Z

    Jiang, J. and Lu, Z. (2019). Learning fairness in multi-agent systems. Advances in Neural Information Processing Systems , 32

    2019

  18. [26]

    Kozodoi, N., Jacob, J., and Lessmann, S. (2022). Fairness in credit scoring: Assessment, implementation and profit implications. European Journal of Operational Research , 297(3):1083--1094

    2022

  19. [27]

    and Varakantham, P

    Li, D. and Varakantham, P. (2022a). Efficient resource allocation with fairness constraints in restless multi-armed bandits. In Uncertainty in Artificial Intelligence , pages 1158--1167. PMLR

    2022

  20. [28]

    and Varakantham, P

    Li, D. and Varakantham, P. (2022b). Towards soft fairness in restless multi-armed bandits. arXiv preprint arXiv:2207.13343

    2022 arXiv

  21. [29]

    and Varakantham, P

    Li, D. and Varakantham, P. (2023). Avoiding starvation of arms in restless multi-armed bandit. International Foundation for Autonomous Agents and Multiagent Systems

    2023

  22. [30]

    Liu, C., Chen, C.-X., and Chen, C. (2021). Meta: A city-wide taxi repositioning framework based on multi-agent reinforcement learning. IEEE Transactions on Intelligent Transportation Systems , 23(8):13890--13895

    2021

  23. [31]

    T., Dean, S., Rolf, E., Simchowitz, M., and Hardt, M

    Liu, L. T., Dean, S., Rolf, E., Simchowitz, M., and Hardt, M. (2018). Delayed Impact of Fair Machine Learning . In Proceedings of the 35th International Conference on Machine Learning , pages 3150--3158. PMLR

    2018

  24. [32]

    Liu, W., Liu, F., Tang, R., Liao, B., Chen, G., and Heng, P. A. (2020). Balancing between accuracy and fairness for interactive recommendation with reinforcement learning. In Pacific-asia conference on knowledge discovery and data mining , pages 155--167. Springer

    2020

  25. [33]

    and Walrand, J

    Mo, J. and Walrand, J. (2000). Fair end-to-end window-based congestion control. IEEE/ACM Transactions on networking , 8(5):556--567

    2000

  26. [34]

    Moulin, H. (1991). Axioms of cooperative decision making . Cambridge university press

    1991

  27. [35]

    Puranik, B., Madhow, U., and Pedarsani, R. (2022). Dynamic positive reinforcement for long-term fairness. In ICML 2022 Workshop on Responsible Decision Making in Dynamic Environments

    2022

  28. [36]

    Puterman, M. L. (2005). Markov decision processes: discrete stochastic dynamic programming . John Wiley & Sons

    2005

  29. [37]

    Raghavan, M., Barocas, S., Kleinberg, J., and Levy, K. (2020). Mitigating bias in algorithmic hiring: Evaluating claims and practices. In Proceedings of the 2020 conference on fairness, accountability, and transparency , pages 469--481

    2020

  30. [38]

    Rawls, J. (1971). A theory of justice. Cambridge (Mass.)

    1971

  31. [39]

    and Ma, D

    Reuel, A. and Ma, D. (2024). Fairness in reinforcement learning: A survey

    2024

  32. [40]

    Saure, A., Patrick, J., Tyldesley, S., and Puterman, M. L. (2012). Dynamic multi-appointment patient scheduling for radiation therapy. European Journal of Operational Research , 223(2):573--584

    2012

  33. [41]

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347

    2017 arXiv

  34. [42]

    Segal, M., George, A.-M., and Dimitrakakis, C. (2023). Policy fairness and unknown bias dynamics in sequential allocations. In Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization , EAAMO '23, New York, NY, USA. Association for...

    2023

  35. [43]

    Siddique, U., Sinha, A., and Cao, Y. (2023). Fairness in preference-based reinforcement learning. arXiv preprint arXiv:2306.09995

    2023 arXiv

  36. [44]

    Siddique, U., Weng, P., and Zimmer, M. (2020). Learning fair policies in multi-objective (deep) reinforcement learning with average and discounted rewards. In International Conference on Machine Learning , pages 8905--8915. PMLR

    2020

  37. [45]

    Sood, A., Jain, S., and Gujar, S. (2024). Fairness of exposure in online restless multi-armed bandits. arXiv preprint arXiv:2402.06348

    2024 arXiv

  38. [46]

    van den Broek, E., Sergeeva, A., and Huysman, M. (2020). Hiring algorithms: An ethnography of fairness in practice. In 40th international conference on information systems, ICIS 2019 , pages 1--9. Association for Information Systems

    2020

  39. [47]

    Verma, S., Zhao, Y., Shah, S., Boehmer, N., Taneja, A., and Tambe, M. (2024). Group fairness in predict-then-optimize settings for restless bandits. In The 40th Conference on Uncertainty in Artificial Intelligence

    2024

  40. [48]

    Wen, M., Bastani, O., and Topcu, U. (2021). Algorithms for fairness in sequential decision making. In International Conference on Artificial Intelligence and Statistics , pages 1144--1152. PMLR

    2021

  41. [49]

    Weymark, J. A. (1981). Generalized gini inequality indices. Mathematical Social Sciences , 1(4):409--430

    1981

  42. [50]

    Whittle, P. (1988). Restless bandits: Activity allocation in a changing world. Journal of applied probability , 25(A):287--298

    1988

  43. [51]

    Yu, G., Siddique, U., and Weng, P. (2023). Fair deep reinforcement learning with preferential treatment. In ECAI , pages 2922--2929

    2023

  44. [52]

    Zhang, X. (2022). Near-Optimality for Multi-action Multi-resource Restless Bandits with Many Arms . PhD thesis, Cornell University

    2022

  45. [53]

    Zhang, X., Khaliligarekani, M., Tekin, C., et al. (2019). Group retention when using machine learning in sequential decision making: the interplay between user dynamics and fairness. Advances in Neural Information Processing Systems , 32

    2019

  46. [54]

    Zhang, X., Tu, R., Liu, Y., Liu, M., Kjellstrom, H., Zhang, K., and Zhang, C. (2020). How do fair decisions fare in long-term qualification? Advances in Neural Information Processing Systems , 33:18457--18469

    2020

  47. [55]

    and Gordon, G

    Zhao, H. and Gordon, G. (2019). Inherent tradeoffs in learning fair representations. Advances in neural information processing systems , 32

    2019

  48. [56]

    Zimmer, M., Glanois, C., Siddique, U., and Weng, P. (2021). Learning fair policies in decentralized cooperative multi-agent reinforcement learning. In International Conference on Machine Learning , pages 12967--12978. PMLR

    2021

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.