Navigating the Social Welfare Frontier: Portfolios for Multi-objective Reinforcement Learning

Cheol Woo Kim; Jai Moondra; Lingkai Kong; Madeleine Pollack; Milind Tambe; Shresth Verma; Swati Gupta

REVIEW 4 major objections 5 minor 12 references

Navigating the Social Welfare Frontier: Portfolios for Multi-objective Reinforcement Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A small set of policies can cover every fairness preference at once.

desk verdict Useful MORL portfolio paper with a plausible core idea, but the ESR guarantees currently rest on an invalid derivative interchange and the experiments don't satisfy the theorem's oracle or positivity assumptions. read the letter →

arxiv 2502.09724 v2 pith:JUTIYDUT submitted 2025-02-13 cs.LG

Cheol Woo Kim , Jai Moondra , Shresth Verma , Madeleine Pollack , Lingkai Kong , Milind Tambe , Swati Gupta This is my paper · ORCID

classification cs.LG MSC 68W2591B1468T05

keywords multi-objectivereinforcementlearningsocialwelfarefunctionsgeneralizedp-meanspolicyportfolioapproximationguaranteesfairnessoraclecomplexityscalarizedreturns

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks how a decision-maker can choose a policy when multiple stakeholders have different rewards and the right fairness criterion is not known in advance. It works with generalized $p$-mean social welfare functions, the family that interpolates between Egalitarian welfare at $p = -\infty$, Nash welfare at $p = 0$, and Utilitarian welfare at $p = 1$, and it asks whether a small set of policies can be approximately optimal for every $p \leq 1$ simultaneously. The main result is an algorithm, p-MEAN PORTFOLIO, that constructs such an $\alpha$-approximate portfolio: its size grows like $\ln \kappa / \ln(1/\alpha)$, where $\kappa$ is the ratio between the largest and smallest rewards, and it solves the welfare-maximization problem only $\tilde{O}((\ln \kappa)^2 \ln \ln N / \ln(1/\alpha))$ times. If correct, this means decision-makers can inspect a handful of concrete policies, each tied to a distinct fairness stance, instead of committing to a single welfare function before seeing the consequences. Experiments on a taxi gridworld, a post-disaster allocation problem, and a healthcare intervention task find portfolios of at most ten policies with approximation factors near 1.

What carries the argument

Two objects carry the argument: the generalized $p$-mean $f(x, p) = (1/N) (\sum_i x_i^p)^{1/p}$, with limiting interpretations $f(x, -\infty) = \min_i x_i$ and $f(x, 0) = (\prod_i x_i)^{1/N}$, and the two aggregation rules $v^{(1)}(\pi, p) = \mathbb{E}_{\tau}[f(G(\tau), p)]$ and $v^{(2)}(\pi, p) = f(\mathbb{E}_{\tau}[G(\tau)], p)$. The engine is a line search that, starting from a policy optimal at $p$, bisects $[p, 1]$ to find the largest $b$ such that the same policy is $\alpha$-approximate on the whole interval $[p, b]$; monotonicity of $p$-means in $p$ and a Lipschitz bound $d \ln v^{(\ell)}_*(p)/dp \leq \kappa \ln \kappa$ on the logarithm of the optimal value function convert interval shrinking into the stated size and oracle bounds.

What would settle it

Run p-MEAN PORTFOLIO on a small MDP with $N = 2$ rewards where exact optimal policies are enumerable, with fixed $L$, $U$ and $\alpha = 0.5$, and test every $p$ in a fine grid: finding any $p \leq 1$ where no portfolio policy reaches $\alpha$ times the true optimum, or any instance where the returned portfolio exceeds the Theorem 4.1 size bound, would refute the theorem. A second test is to allow one reward to be zero, which makes $\kappa$ undefined and shows whether the strictly-positive assumption is essential.

Watch

Extended reading notes

Core claim

The paper's central claim is that the entire continuum of $p$-mean welfare objectives in a multi-objective MDP can be compressed into a compact portfolio of policies with a multiplicative guarantee. Formally, Theorem 4.1 states that for an MDP with $N$ reward functions bounded by $L$ and $U$, condition number $\kappa = U/L$, feasible policy set $\Pi$, aggregation rule $\ell \in \{1, 2\}$, and $\alpha \in (0, 1)$, p-MEAN PORTFOLIO returns a portfolio $\Pi'$ such that for every $p \leq 1$ there exists $\pi \in \Pi'$ with $v^{(\ell)}(\pi, p) \geq \alpha \cdot \max_{\pi' \in \Pi} v^{(\ell)}(\pi', p)$. The portfolio size is $O(\ln \kappa / \ln(1/\alpha))$ and the number of oracle calls to the welfare-maximization subroutine is $\tilde{O}((\ln \kappa)^2 \ln \ln N / \ln(1/\alpha))$, with logarithmic factors suppressed. The construction starts from $p_0 = -\ln N / \ln(1/\alpha)$, whose optimal policy is already $\alpha$-optimal for the entire tail $p \leq p_0$, then advances through increasing $p$ values chosen by a bisection line search, ending at $p = 1$.

Load-bearing premise

The load-bearing premise is that an oracle can solve the welfare-maximization problem $\max_{\pi \in \Pi} v^{(\ell)}(\pi, p)$ exactly for every $p$ it is asked about, and that every reward is strictly positive so the condition number $\kappa = U/L$ is defined; if either fails, Theorem 4.1's size and oracle bounds do not govern what the algorithm actually delivers.

Editorial extensions

If this is right

A decision-maker no longer needs to commit to a single $p$ in advance: a portfolio whose size grows logarithmically in the reward ratio contains, for every $p \leq 1$, a policy within factor $\alpha$ of the $p$-mean optimum.
The guarantee holds for both ESR and SER aggregation, so the method covers policies executed once per trajectory and policies whose rewards accumulate over repeated executions.
The oracle-cost bound means that improving $\alpha$ by a constant factor changes the number of welfare-maximization problems solved by only a constant factor, making higher-fidelity portfolios cheap in the oracle model.
Because nearby $p$ values have nearby optimal policies (Proposition 4.2), solving the next welfare-maximization problem can be warm-started from the previous solution, with a gap bounded by $(q - p) U H \kappa \ln \kappa$.
The budget-constrained heuristic gives a practical fallback when exact oracle calls are too expensive; in the reported experiments it matches the full algorithm's coverage with far fewer calls.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The portfolio's members are not arbitrary policies but optima for distinct $p$ values, so the algorithm implicitly produces a navigable map of the fairness frontier; the paper illustrates this but does not develop it as an interface.
In multi-stakeholder RLHF-style alignment, the same construction could offer human stakeholders a short menu of alignment policies corresponding to different welfare aggregations instead of requiring one aggregation rule to be fixed in advance.
The condition-number dependence suggests a stress test the paper does not run: when stakeholder reward scales are highly unequal, the guaranteed portfolio grows, and equalizing reward scales before portfolio construction would shrink it.
Since the theorem assumes an exact oracle, an immediate extension is to quantify the oracle gap empirically in small MDPs where exact optima are enumerable, turning the heuristic comparisons into measured approximation guarantees.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a useful within-subfield paper with a plausible central theorem, a real proof gap in the ESR case, and experiments that don't quite live in the theorem's regime. Worth serious refereeing, but as submitted the guarantees for the expected-scalarized-returns case are unproven.

What's new: this is the first portfolio construction for p-mean welfare objectives in multi-objective RL, covering both ESR and SER aggregation rules, with size bound O(ln kappa / ln(1/alpha)) and an oracle-complexity bound. The line-search-over-p idea and the alpha-approximate portfolio framing come from the same group's earlier EC and SODA work in combinatorial optimization; the extension to sequential decision-making, the warm-start bound, and the budget-constrained heuristic are genuine additions. The paper is honest about the oracle assumption in Section 4, and the code is released.

Where it is soft: the stress-test note is right. Lemma A.8 Part 2 for l=1 claims d/dp ln E_tau[f(G(tau),p)] = E_tau[d/dp ln f(G(tau),p)], which is not linearity of expectation. The correct formula is a weighted average, E[f' ]/E[f], and the submitted proof does not supply the needed bound. Since Theorem 4.1 covers l in {1,2} and Proposition 4.2 also uses the slope bound, the ESR guarantees are not established as written. The gap looks repairable: the pointwise slope bound is kappa ln kappa, and a weighted-average argument should give the same slope on ln v^(1). But it has to be written.

The exact-oracle and strictly-positive-reward assumptions create a second gap between theory and experiment. The theorem needs an exact solver for Problem (3); the healthcare experiments use a Whittle-index heuristic, and the taxi reward includes zeros, so kappa is undefined there. The paper acknowledges the exact-oracle issue in prose but does not reconcile it with the theorem statement. Minor: Table 1's caption says p-MEAN PORTFOLIO consistently outperforms, but in the taxi rows random p sampling ties at portfolio sizes 1 and 2; Table 3 has a 0.100 entry that looks like a typo. These should be cleaned up.

Who it is for: people working on multi-objective RL, fairness in resource allocation, and decision-makers who want a small set of policies rather than committing to a single p. The citation pattern is self-referential but legitimate: the prior portfolio papers are the actual source of the technique. I would bring this to our reading group.

Recommendation: send it to peer review. The core idea is likely sound and the application area is timely, but the referee should require a fixed l=1 slope proof and an explicit statement of when the oracle and positivity assumptions hold in practice.

Referee Report

4 major / 5 minor

Summary. This paper studies α-approximate portfolios for generalized p-mean social welfare functions in multi-objective RL. For p ≤ 1 and the two aggregation rules v(1) (expected scalarized returns) and v(2) (scalarized expected returns), it defines a portfolio as a set of policies Π′ such that for every p some policy in Π′ is within factor α of the optimal value v*(p). The main algorithm, p-MEAN PORTFOLIO (Algorithm 1), iteratively selects p values via a line-search subroutine (Algorithm 2), and Theorem 4.1 claims a portfolio of size O(ln κ / ln(1/α)) with oracle complexity eO((ln κ)^2 ln ln N / ln(1/α)), assuming an exact oracle for Problem (3) and strictly positive rewards with condition number κ. The paper also proposes a budget-constrained heuristic (Algorithm 3), proves a warm-start bound (Proposition 4.2), and reports experiments on a taxi domain, a synthetic disaster-resource-allocation domain, and an RMAB healthcare intervention domain.

Significance. If the main theorem were fully established, this would be a substantial contribution: it reduces a continuum of p-mean fairness objectives to a compact portfolio with explicit approximation guarantees, and it is framed uniformly for both ESR and SER aggregation. The paper also provides a useful empirical study, a public code repository, and a warm-start result that is of independent interest. The main proof, however, contains two incorrect inequalities in Lemma A.8 that are load-bearing for Theorem 4.1 and Proposition 4.2; both are plausibly repairable, but as submitted the formal guarantees for ℓ=1 are not demonstrated. In addition, the experimental section operates outside the theorem's stated assumptions (approximate oracles and zero rewards), so the current version should be revised before the guarantees are presented as established.

major comments (4)

[Appendix A.1, Lemma A.8 Part 2] The proof claims that d/dp ln v(1)(π,p) = Eτ[d/dp ln f(G(τ),p)] by linearity of expectation. This is incorrect: the derivative of a log-expectation is a weighted expectation with weights proportional to f(G(τ),p), not the unweighted expectation. Since the pointwise derivative is bounded by κ ln κ, the inequality d ln v(1)/dp ≤ κ ln κ is recoverable by applying the weighted-average formula, but the submitted argument does not supply that derivation. This gap affects Theorem 4.1's portfolio-size and oracle-complexity guarantees as stated for ℓ=1, and it also affects Proposition 4.2.
[Appendix A.1, Lemma A.8 Part 3] The displayed inequality in the proof of Part 3 has the wrong direction. Since w(q)=maxπ ln v(π,q) and πp maximizes at p, one has w(q) ≥ ln v(πp,q), so [w(q)−w(p)]/(q−p) is at least, not at most, [ln v(πp,q)−ln v(πp,p)]/(q−p). The slope bound on w can be recovered by a standard envelope argument, but the written proof does not establish the bound needed for the LINE SEARCH iteration count in the oracle-complexity analysis.
[Section 4 and Section 5 vs. Assumption 3.1] Theorem 4.1 is stated under an exact oracle for Problem (3) and under Assumption 3.1 with L>0. The taxi environment assigns zero and negative rewards, the healthcare environment uses a 0/1 reward, and the healthcare and taxi experiments use approximate solvers (Whittle-index heuristic and Welfare Q-Learning). Moreover, Section 4 explicitly replaces v*(p) by the value found by the RL algorithm when measuring approximation. Consequently Tables 1–2 and Figure 1 do not operate in the theorem's regime and cannot be read as direct empirical verification of the α-guarantee. The authors should state this mismatch explicitly and clarify what is actually being measured in the experiments.
[Section 4.2, BUDGET-CONSTRAINED PORTFOLIO] The heuristic justifies the approximation u(l) by assuming that v(π_{pm(l)}, p)/v*(p) is monotonically decreasing on each interval [pm(l), pm(l+1)]. The text says this assumption 'holds exactly when the interval is sufficiently small, as this function is continuous in p (Lemma A.8) and attains its maximum at p=pm(l)'. Continuity plus endpoint maximality does not imply monotonicity, and Lemma A.8 does not supply this monotonicity. Since Algorithm 3 is presented as a heuristic this is not fatal, but the stated justification is not valid and should be corrected or explicitly weakened.

minor comments (5)

[Appendix A.1, proof of Lemma A.8 Part 1] There are typos in the normalization step: 'LH = 1' should describe the normalized total-reward bound, and 'ln R' in Case B should be 'ln κ' or a defined quantity.
[Appendix A.1, proof of Theorem 4.1] The bound displayed in equation (8) is O((ln κ)^2 ln ln N / (ln(1/α) ln ln(1/α))), while Theorem 4.1 states eO((ln κ)^2 ln ln N / ln(1/α)); the eO notation should be defined precisely or the two displays reconciled.
[Appendix A.1, proof of Theorem 4.1] The sentence 'We note that this size bound can be tightened slightly' appears to refer to the oracle-complexity bound rather than the portfolio-size bound.
[Section 4.2] The notation '[∞, 1]' appears where '[−∞, 1]' is intended, and the phrase 'partition the interval [∞, 1]' should be fixed.
[Appendix C, Table 3] The Resource Allocation row for portfolio size 4 lists the p-MEAN-PORTFOLIO approximation as 0.100, which is inconsistent with Table 1's value of 1.000 for the same row and is likely a typo.

Circularity Check

0 steps flagged · score 0.0 of 10

No material circularity: the portfolio guarantee is verified by direct oracle checks, and self-citations are background; the notable flaw is a non-circular proof gap.

full rationale

The central derivation in Theorem 4.1 is self-contained and not circular. Algorithm 1 constructs a portfolio by repeatedly solving the exact oracle Problem (3), and Algorithm 2 (LINE SEARCH) terminates only when the current policy satisfies the defining α-approximation inequality, e.g. v(ℓ)(πp, a) ≥ α v(ℓ)* (b). The guarantee is therefore checked against the same quantities used in Definition 3.3; it is a verification loop, not a fitted parameter renamed as a prediction. The size and oracle-complexity bounds are proved in Appendix A from Assumption 3.1 and Lemma A.8, without importing the authors' prior combinatorial portfolio theorems as proof. The self-citations (Gupta et al. 2023, 2025; Verma et al. 2024a,b) are background context or experimental methodology and are not load-bearing for Theorem 4.1. The genuinely notable defect is non-circular: Lemma A.8 Part 2 for ℓ = 1 claims d/dp ln Eτ[f(G(τ),p)] = Eτ[d/dp ln f(G(τ),p)] by linearity of expectation, which is an invalid interchange of log and expectation; this leaves the ESR slope bound and hence the ℓ = 1 complexity proof unproven as submitted. That is a correctness gap in the proof, not a reduction of the theorem to its own assumptions. The exact-oracle and L ≤ Ri ≤ U assumptions are stated limitations, and the taxi and healthcare experiments use approximate solvers and zero-valued rewards, but these do not make the derivation circular.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests on no invented entities. It does rest on an exact oracle, strictly positive rewards, and a standard monotonicity theorem, plus an unproven monotonicity assumption inside the heuristic. No new forces, particles, dimensions, or ledger entries are introduced.

free parameters (1)

p0 in BUDGET-CONSTRAINED PORTFOLIO = -100
In the experiments, the initial p for the budget-constrained heuristic is set to -100 regardless of α or N; this choice is ad hoc and not derived from Theorem 4.1.

assumptions (4)

standard math Generalized p-means are monotonic in p (Lemma 3.2, cited to Bullen).
Used throughout, e.g., in Lemmas A.1, A.4, A.7, to compare values of the same policy or optimal value across different p.
domain assumption All rewards are strictly positive and bounded (Assumption 3.1).
Defines condition number κ = U/L, used in Lemma A.8 and Theorem 4.1; violated by zero rewards in the taxi experiment.
domain assumption An oracle can exactly solve max_π v(ℓ)(π,p) for any queried p (Problem (3)).
The algorithms and guarantees in Section 4 are conditional on this oracle; in practice RL solvers are approximate, as acknowledged for the healthcare experiment.
ad hoc to paper For BUDGET-CONSTRAINED PORTFOLIO, the ratio v(π_{p_m(l)}, p)/v*(p) is monotonically decreasing on each interval [p_m(l), p_m(l+1)].
The paper states this assumption 'holds exactly when the interval is sufficiently small' but uses it globally in the heuristic; it is not proven and the heuristic's performance relies on it.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Navigating the Social Welfare Frontier: Portfolios for Multi-objective Reinforcement Learning." pith.science (2026). https://pith.science/paper/JUTIYDUT

@misc{pith2026250209724,
  author       = {Pith},
  title        = {Pith review of: Navigating the Social Welfare Frontier: Portfolios for Multi-objective Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JUTIYDUT}},
  note         = {Machine review of arXiv:2502.09724}
}

abstract

In many real-world applications of reinforcement learning (RL), deployed policies have varied impacts on different stakeholders, creating challenges in reaching consensus on how to effectively aggregate their preferences. Generalized $p$-means form a widely used class of social welfare functions for this purpose, with broad applications in fair resource allocation, AI alignment, and decision-making. This class includes well-known welfare functions such as Egalitarian, Nash, and Utilitarian welfare. However, selecting the appropriate social welfare function is challenging for decision-makers, as the structure and outcomes of optimal policies can be highly sensitive to the choice of $p$. To address this challenge, we study the concept of an $\alpha$-approximate portfolio in RL, a set of policies that are approximately optimal across the family of generalized $p$-means for all $p \in [-\infty, 1]$. We propose algorithms to compute such portfolios and provide theoretical guarantees on the trade-offs among approximation factor, portfolio size, and computational efficiency. Experimental results on synthetic and real-world datasets demonstrate the effectiveness of our approach in summarizing the policy space induced by varying $p$ values, empowering decision-makers to navigate this landscape more effectively.

Figures

Figures reproduced from arXiv: 2502.09724 by the authors.

**Figure 1.** Portfolio of three policies for the healthcare intervention setting (see Section 5 for details), obtained using Algorithm 1 with α = 0.60. The bar plots show total rewards induced by each policy across different education (top) and age (bottom) brackets. A “score” of 1.0 corresponds to a baseline policy (not shown in the figure) used for comparison. The three policies impact various education and age brackets differ… view at source ↗

**Figure 2.** Side-by-side comparison of entries and p-means for vectors A, B, C. The first inequality follows from Lemma 3.2, which ensures monotonicity in the p parameter. The equality comes directly from the definition of πp, as it maximizes v (1)(π, p). The second inequality uses the properties of the maximum function. The next line applies the mean-value theorem to the function f(G(τ ), p) with respect to the variable p. The… view at source ↗

**Figure 3.** Normalized total reward for each route under different policies in the portfolio generated by p-MEANPORTFOLIO in the taxi environment. E. Experimental Details We include the details of various experiments here. E.1. Taxi Environment Problem Setting This environment consists of a taxi agent whose task is to deliver passengers from source to destination. The world consists of a 6x6 grid and based on the environment se… view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Fraction of need meet for various clusters by various policies in the portfolio obtained by p-MEANPORTFOLIO after H = 4 intervention steps for the natural disaster experiment. Note that different solutions in the portfolio emphasize different clusters. 20 [PITH_FULL_I…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 9 canonical work pages

[1]

p0 ≤ p ≤ a < b≤ 1 at all times, and

work page
[2]

For any given policy π ∈ Π and aggregation rule ℓ ∈ {1, 2}, we have d(ln v(ℓ)(π,p)) dp ≤ κ ln κ, where κ is the condition number defined in Assumption 3.1

work page
[3]

Then, for all distinct p, q≤ 1, w(q) − w(p) q − p ≤ κ ln κ

For a given aggregation rule ℓ ∈ {1, 2}, define w(p) := ln v(ℓ) ∗ (p). Then, for all distinct p, q≤ 1, w(q) − w(p) q − p ≤ κ ln κ. 14 Navigating the Social Welfare Frontier Proof. Part 1. Note that g(βx, p) = ln β + g(x, p) for all x ∈ RN >0, p ≤ 1, and β >0. Therefore, we can assume without loss of generality that LH = 1 and U H= U H LH = κ. That is, ass...

work page
[4]

ISBN 9781450394321

International Foundation for Autonomous Agents and Multiagent Systems. ISBN 9781450394321. Ashish Goel and Adam Meyerson. Simultaneous optimiza- tion via approximate majorization for concave profits or convex costs. Algorithmica, 44:301–323, 2006. Daniel Golovin, Anupam Gupta, Amit Kumar, and Kanat Tangwongsan. All-norms and all-l_p-norms approxi- mation ...

work page arXiv 2008
[10]

reasonable

except in the last iteration, we have v(π, a) < α v∗(b) (otherwise LINE SEARCH terminates by step 2). Denote by a(j−1), b(j−1) the value of a, bin iteration j − 1. Then, since b − a is halved in each iteration, we must have 0 < b(j−1) − a(j−1) = (1 − p)2−(j−1) ≤ (1 − p0)2−(j−1) ≤ 4 lnN ln(1/α) 2−j. However, since the algorithm does not terminate in step j...

work page 2024
[11]

Individual preferences: one reward function per preference (total of N reward functions) which we treat as each stakeholder’s reward function

work page
[12]

In our experiments, we set N = 59 and construct |Π| = 200 balancing policies as feasible set

Balancing functions: a collection of reward functions that trade off among the N preferences, whose corresponding policies form our feasible set Π. In our experiments, we set N = 59 and construct |Π| = 200 balancing policies as feasible set. Learning Policy It is computationally intractable to optimally solve the RMAB problem (Papadimitriou and Tsitsiklis...

work page 1999
[1988]

URL http://www.jstor

ISSN 00219002. URL http://www.jstor. org/stable/3214163. Qingqing Wu, Yong Zeng, and Rui Zhang. Joint tra- jectory and communication design for multi-uav en- abled wireless networks. IEEE Transactions on Wireless Communications, 17(3):2109–2121, 2018. doi: 10.1109/ TWC.2017.2789293. Runzhe Yang, Xingyuan Sun, and Karthik Narasimhan. A generalized algorith...

work page arXiv 2018

Show all 12 references

[2014]

Shresth Verma, Gargi Singh, Aditya Mate, Paritosh Verma, Sruthi Gorantla, Neha Madhiwalla, Aparna Hegde, Divy Thakkar, Manish Jain, Milind Tambe, et al

ISSN 1532-4435. Shresth Verma, Gargi Singh, Aditya Mate, Paritosh Verma, Sruthi Gorantla, Neha Madhiwalla, Aparna Hegde, Divy Thakkar, Manish Jain, Milind Tambe, et al. Expanding impact of mobile health programs: Saheli for maternal and child care. AI Magazine, 44(4):363–376, ...

2023
[2021]

Cyrus Cousins

doi: 10.1109/INFOCOM42981.2021.9488823. Cyrus Cousins. Revisiting fair-pac learning and the axioms of cardinal welfare. In Francisco Ruiz, Jennifer Dy, and Jan-Willem van de Meent, editors, Proceedings of The 26th International Conference on Artificial Intelligence and Statist...

2021
[2023]

Cyrus Cousins, Kavosh Asadi, Elita Lobo, and Michael Littman

URL https://proceedings.mlr.press/ v206/cousins23a.html. Cyrus Cousins, Kavosh Asadi, Elita Lobo, and Michael Littman. On welfare-centric fair reinforcement learning. Reinforcement Learning Journal, 3:1124–1137, 2024. Thomas G Dietterich. Hierarchical reinforcement learning wi...

2024 arXiv
[2024]

URL https://openreview.net/forum? id=ybiUVIxJth. Lucas N. Alegre, Diederik M. Roijers, Ann Nowé, Ana L. C. Bazzan, and Bruno C. da Silva. Sample-efficient multi-objective learning via generalized policy improve- ment prioritization. In Proc. of the 22nd International Conferenc...

2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

p0 ≤ p ≤ a < b≤ 1 at all times, and

work page

[2] [2]

For any given policy π ∈ Π and aggregation rule ℓ ∈ {1, 2}, we have d(ln v(ℓ)(π,p)) dp ≤ κ ln κ, where κ is the condition number defined in Assumption 3.1

work page

[3] [3]

Then, for all distinct p, q≤ 1, w(q) − w(p) q − p ≤ κ ln κ

For a given aggregation rule ℓ ∈ {1, 2}, define w(p) := ln v(ℓ) ∗ (p). Then, for all distinct p, q≤ 1, w(q) − w(p) q − p ≤ κ ln κ. 14 Navigating the Social Welfare Frontier Proof. Part 1. Note that g(βx, p) = ln β + g(x, p) for all x ∈ RN >0, p ≤ 1, and β >0. Therefore, we can assume without loss of generality that LH = 1 and U H= U H LH = κ. That is, ass...

work page

[4] [4]

ISBN 9781450394321

International Foundation for Autonomous Agents and Multiagent Systems. ISBN 9781450394321. Ashish Goel and Adam Meyerson. Simultaneous optimiza- tion via approximate majorization for concave profits or convex costs. Algorithmica, 44:301–323, 2006. Daniel Golovin, Anupam Gupta, Amit Kumar, and Kanat Tangwongsan. All-norms and all-l_p-norms approxi- mation ...

work page arXiv 2008

[5] [10]

reasonable

except in the last iteration, we have v(π, a) < α v∗(b) (otherwise LINE SEARCH terminates by step 2). Denote by a(j−1), b(j−1) the value of a, bin iteration j − 1. Then, since b − a is halved in each iteration, we must have 0 < b(j−1) − a(j−1) = (1 − p)2−(j−1) ≤ (1 − p0)2−(j−1) ≤ 4 lnN ln(1/α) 2−j. However, since the algorithm does not terminate in step j...

work page 2024

[6] [11]

Individual preferences: one reward function per preference (total of N reward functions) which we treat as each stakeholder’s reward function

work page

[7] [12]

In our experiments, we set N = 59 and construct |Π| = 200 balancing policies as feasible set

Balancing functions: a collection of reward functions that trade off among the N preferences, whose corresponding policies form our feasible set Π. In our experiments, we set N = 59 and construct |Π| = 200 balancing policies as feasible set. Learning Policy It is computationally intractable to optimally solve the RMAB problem (Papadimitriou and Tsitsiklis...

work page 1999

[8] [1988]

URL http://www.jstor

ISSN 00219002. URL http://www.jstor. org/stable/3214163. Qingqing Wu, Yong Zeng, and Rui Zhang. Joint tra- jectory and communication design for multi-uav en- abled wireless networks. IEEE Transactions on Wireless Communications, 17(3):2109–2121, 2018. doi: 10.1109/ TWC.2017.2789293. Runzhe Yang, Xingyuan Sun, and Karthik Narasimhan. A generalized algorith...

work page arXiv 2018