Pith. sign in

REVIEW 4 major objections 5 minor 33 references

Enabling Pareto-Stationarity Exploration in Multi-Objective Reinforcement Learning: A Multi-Objective Weighted-Chebyshev Actor-Critic Approach

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MOCHA claims to explore Pareto-stationary policies in multi-objective RL at single-objective sample-complexity rates.

desk verdict Interesting algorithmic idea, but the central derivation has a load-bearing algebraic error and the main theorem is unproven; reject in this form. read the letter →

arxiv 2507.21397 v1 pith:67DUBKR5 submitted 2025-07-29 cs.LG

Fnu Hairi , Jiao Yang , Tianchen Zhou , Haibo Yang , Chaosheng Dong , Fan Yang , Michinari Momma , Yan Gao
show 1 more author
Jia Liu
This is my paper
classification cs.LG
keywords multi-objectivereinforcementlearningParetostationarityweighted-Chebyshevscalarizationactor-criticsamplecomplexitymultiple-gradientdescentfrontexplorationofflinerecommendation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes MOCHA, an actor-critic algorithm for multi-objective reinforcement learning, and argues that it can systematically explore Pareto-stationary policies — policies that no other policy dominates in every reward objective — with finite-time guarantees. The key claim is that for any positive weight vector $p$, MOCHA converges to a neighborhood of a Pareto-stationary solution with a bound of order $1/T$ plus batch, critic, and approximation error terms, and that the total sample complexity per exploration direction is $\tilde{\mathcal{O}}(\epsilon^{-2})$ once learning rates are chosen as specified. This matters because most practical MORL methods either find one arbitrary Pareto-stationary policy or lack rates; MOCHA ties the weight vector $p$ to a specific trade-off and lets a user sweep $p$ to trace the Pareto-stationarity front. The convergence rate matches the single-objective actor-critic rate, so the multi-objective exploration does not, order-wise, cost extra samples.

What carries the argument

The load-bearing object is the Wolfe dual reformulation of the weighted-Chebyshev scalarization, a scalarization that converts a vector objective into $\min_x \|p \odot F(x)\|_\infty$. To explore the Pareto front, the paper converts reward maximization into a regret-minimization problem $\min_\theta \|p\odot(J^*_{\mathrm{ub}}-J(\theta))\|_\infty$, introduces an auxiliary variable $\rho$, and writes the Lagrangian dual with matrix $K_p=\operatorname{diag}(\sqrt{p})\sqrt{G^\top G}\operatorname{diag}(\sqrt{p})$. The condition $K_p\lambda=0$ together with $\mathbf{1}^\top\lambda=1$, $\lambda\ge 0$ characterizes a WC-guided common-descent weighting; because this condition need not hold at every iterate, the algorithm instead minimizes $\|K_p\lambda\|^2 - u\lambda^\top(p\odot(J^*_{\mathrm{ub}}-J(\theta)))$ subject to the simplex constraint, a convex QP that mixes MGDA's descent objective with WC's exploration objective. The policy update then moves along $G_t(p\odot\lambda_t)$, and the momentum update $\lambda_t=(1-\eta_t)\lambda_{t-1}+\eta_t\hat\lambda_t^*$ suppresses cumulative bias from the $\lambda$-weighting. The role of this machinery is to turn a user-specified $p$ into a concrete gradient-combination direction while retaining the $O(1/T)$ convergence structure of single-objective actor-critic.

What would settle it

Set up a two-objective MOMDP with a known nonconvex Pareto front, e.g. a small tabular task where the set of achievable return pairs can be computed exactly; run MOCHA with refined $p$ grids and large batch sizes so the $O(1/B)$ and critic-error terms vanish. If the reached policies do not accumulate on the true Pareto-stationary set, or if the empirical stationarity gap exceeds the $O(1/T+\zeta_{\mathrm{approx}})$ prediction, the central claim fails. More directly, one can test the load-bearing duality step by searching for a parameter $\theta$ of a nonconvex objective where the optimal value of $\min \rho$ subject to $p\odot(J^*_{\mathrm{ub}}-J(\theta))\le \rho\mathbf{1}$ is strictly less than the Wolfe dual value in Eq. (7); such a gap would sever the derivation of the $\lambda$ update.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that the weighted-Chebyshev scalarization and the multiple-gradient descent algorithm can be combined inside an actor-critic loop so that the $\lambda$-weighting that selects a common policy-improvement direction is explicitly guided by a user-chosen exploration weight $p$. Specifically, Algorithm 1 runs parallel TD critics, computes per-objective policy gradients from TD errors, solves a convex quadratic program over $\lambda$ derived from the Wolfe dual of the WC problem, mixes the solution with momentum, and updates $\theta_{t+1} = \theta_t - \alpha G_t(p \odot \lambda_t)$. Theorem 3 states that $E\|\nabla_\theta J(\theta_{\hat T})\lambda_{\hat T}\|_2^2$ is at most $O(1/(T(1+2\sum_t\eta_t/p_{\min}^2)))+O(1/B)+O(\max_j E\|w^j_t-w^{j,*}_t\|_2^2)+O(\zeta_{\mathrm{approx}})$. With $\eta_t=p_{\min}^2/t^2$, the first term is $O(1/T)$, and Corollary 4 converts the bound into an $\tilde{\mathcal{O}}(\epsilon^{-2}\log(\epsilon^{-1}))$ sample complexity for an $\epsilon$-Pareto-stationary solution. The paper also claims that the all-ones $p$ case recovers the weight-free MOAC convergence result, making MOCHA a strict generalization.

Load-bearing premise

The load-bearing premise is that the Wolfe dual reformulation in Eq. (7) is exact for the generally nonconvex regret-minimization objective; if strong duality fails there, the $\lambda$ weighting and hence the actor gradient update lose the principled connection to weighted-Chebyshev scalarization.

Editorial extensions

If this is right

  • For each fixed weight vector $p$ with $p_{\min}>0$, MOCHA reaches an $\epsilon$-Pareto-stationary policy with $\tilde{\mathcal{O}}(\epsilon^{-2})$ samples, so the cost of exploring one trade-off direction is the same order as single-objective actor-critic.
  • Sweeping $p$ over a uniform cover of the simplex yields a discrete approximation of the Pareto-stationarity front; the per-direction complexity does not grow with the number of objectives $M$.
  • Choosing $\eta_t=p_{\min}^2/t^2$ restores the $O(1/T)$ leading term, so the momentum schedule can be set from the smallest entry of $p$; small $p_{\min}$ entries slow the leading term and demand more iterations.
  • With $p=\mathbf{1}$ (after normalization), the results recover the weight-free MOAC bound in [4], which positions MOCHA as its generalization rather than a separate algorithm.
  • On the KuaiRand offline recommendation data, MOCHA reports higher Click, Like, and WatchTime scores than SDMGrad, TSCAC, and MOAC for the tested weight vector, and a wider radar-chart footprint as $p$ varies, matching the exploration claim.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same WC-dual-plus-MGDA trick should transfer to non-RL multi-objective optimization with stochastic gradients, which would extend the paper's insight beyond actor-critic to any setting where a common descent direction is needed.
  • A practical implication the authors leave implicit: to avoid tiny $p_{\min}$ slowdowns, one could adaptively resample or soften weight vectors near simplex boundaries, or choose the exploration set so that $p_{\min}$ is bounded below.
  • If the Wolfe duality step fails for nonconvex objectives, the $\lambda$ update may still work as a heuristic; a testable extension is to measure on small MOMDPs with known Pareto fronts whether the dual-derived $\lambda$ actually produces common ascent directions.
  • The paper's $\tilde{O}(\epsilon^{-2})$ claim is per exploration vector; the total cost of tracing a front over $n$ weight vectors is $n\cdot\tilde{O}(\epsilon^{-2})$, so the real question for practice is how to choose the coarsest $p$-grid that still covers the front.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes MOCHA, a multi-objective actor-critic algorithm that combines weighted-Chebyshev scalarization with an MGDA-style actor update, aiming to systematically explore Pareto-stationary solutions in MORL. The main theoretical claim is a finite-time convergence bound to an ε-Pareto-stationary solution with sample complexity O~(ε^{-2}) for any positive weight vector p. The paper also reports experiments on the KuaiRand dataset comparing MOCHA with several baselines. The central algorithmic step is a derivation, via KKT conditions and a Wolfe dual, of the actor's λ-update from the weighted-Chebyshev problem; this derivation is the key premise for the stated convergence guarantees.

Significance. If the theoretical claims were correct, MOCHA would be a useful contribution to MORL: it would combine the Pareto-front exploration property of Chebyshev scalarization with finite-time Pareto-stationarity guarantees, extending prior MOAC results to systematic front exploration. The paper also presents experiments on a real-world recommendation dataset and attempts to visualize Pareto-stationarity exploration. However, the central derivation and convergence theorem are not verified in the manuscript: Theorem 3 is stated without proof, and the KKT/Wolfe-dual step that generates the actor update appears algebraically incorrect. These are load-bearing issues, not presentational ones, so the claimed contribution is not currently established.

major comments (4)
  1. [IV-C, Theorem 3 and Corollary 4] Theorem 3 is stated without a proof and no appendix or supplementary material is provided. The finite-time Pareto-stationary bound E[||∇J(θ_hat_T)λ_hat_T||^2] ≤ O(...) and the sample-complexity claim in Corollary 4 are the main contributions of the paper, yet the reader cannot verify any of the steps (the handling of the Markovian TD critic, the momentum-based λ update, and the conversion of the critic error bound into total sample complexity). This alone prevents acceptance; a complete proof, or a precise derivation from prior theorems with all parameter choices specified, is required.
  2. [IV-B, Step 2-a, Eqs. (6)-(8)] The derivation of Eq. (7) from Eq. (6) is algebraically incorrect. The Lagrangian of Eq. (6) is L = ρ + Σ_i λ_i(p_i(J_i^ub - J_i(θ)) - ρ). Stationarity with respect to θ gives ∇J(θ) diag(p) λ = 0 (equivalently G diag(p) λ = 0 with G = -∇J). In contrast, the constraint K_p λ = 0 in Eq. (7), where K_p = diag(√p)√(G^T G)diag(√p), is equivalent to G diag(√p)λ = 0, since null(√(G^T G)) = null(G). These are not equivalent conditions unless p_i = 1 or p_i = 0. For example, with d = 1, M = 2, G = (1, -1), p = (0.25, 0.75), the KKT condition gives λ = (0.75, 0.25) on the simplex, whereas K_p λ = 0 gives λ ≈ (0.634, 0.366). Thus the QP in Eq. (8) and the subsequent update θ_{t+1} = θ_t - α G_t(p ⊙ λ_t) do not enforce the WC-scalarization stationarity condition. This invalidates the claimed link between MOCHA and the weighted-Chebyshev objective.
  3. [IV-B, Eq. (7)] Even if the KKT condition were corrected to G diag(p)λ = 0, the paper's use of the Wolfe dual is unjustified for the nonconvex objectives it explicitly targets. The Wolfe dual is only equivalent to the primal under convexity and constraint qualification; with nonconvex J(θ) (as in neural-network policies), strong duality can fail. The manuscript offers no alternative justification for replacing Eq. (6) by the dual formulation in Eq. (7), so the actor update lacks a principled foundation. This is a separate correctness risk from the algebraic mismatch above.
  4. [V, Table II, Table III, and Fig. 1] The experiments use weight vectors that violate the p_min > 0 assumption required by Theorem 3. Table II uses p = (0.2, 0.2, 0.2, 0, 0.4), which has p_min = 0; the one-hot vectors in Fig. 1 contain four zero entries; Table III contains ablation vectors with zeros (e.g., abl1 has dislike = 0) or values of 0.0001. Since Theorem 3 and Corollary 4 require p_min > 0, these experiments do not provide evidence for the theoretical claims. The paper should either restrict the experimental setting to positive weights or extend the theory to handle zero entries.
minor comments (5)
  1. [Throughout] The spelling of the method name is inconsistent: the abstract and title use 'Weighted-Chebychev' while the body uses 'Weighted-Chebyshev'; please standardize.
  2. [IV-B, Step 2-a, Eq. (8)] The phrase 'solving for λ under the current θ-value yields a λ-weighting...' is not precise: Problem (8) contains both λ and θ as optimization variables, but the algorithm fixes θ when solving for λ. This should be stated as a conditional update, not as a joint solution.
  3. [IV-C, Corollary 4] In Corollary 4, the critic error condition is written as E[||w_i^t - w_i^{*,t}||^2] = O(ε) for all i ∈ [M], t ∈ [M]; the range for t should presumably be t ∈ [T], not [M]. Please correct this typo.
  4. [References] Reference [15] and reference [32] appear to be the same work (Xiao, Ban, and Ji, 'Direction-oriented multi-objective learning'), with different venue strings; please deduplicate or cite consistently.
  5. [IV-B, Algorithm 1] After the critic loop, the notation sets w_i^t = w_i^k but w_i^k is already defined inside the loop; clarify whether w_i^t refers to the final critic iterate and use consistent superscripts.

Circularity Check

1 steps flagged · score 5.0 of 10

Central WC-MGDA actor update rests on a self-cited Wolfe-dual ansatz that is not the KKT dual of (6); sample-complexity claims inherit the gap.

  1. ansatz smuggled in via citation [Section IV-B, Step 2-a, Eqs. (6)-(8); relied upon by Section IV-C, Theorem 3]
    "By the KKT stationarity condition on ρ and θ and associating Lagrangian dual variables λ ∈ RM + , it can be readily verified that the Wolfe dual problem of Eq. (6) can be written as [19]: max λ,θ λ⊤(p ⊙ (J∗ ub − J(θ))), s.t. Kpλ = 0, 1⊤λ = 1, λ ∈ RM + , θ ∈ Rd, (7) where Kp := diag(√p)√G⊤Gdiag(√p)."

    The KKT stationarity of (6) in θ is −Σ_i λ_i p_i ∇J_i(θ)=0, i.e. G diag(p)λ=0; but K_pλ=0 is equivalent to G diag(√p)λ=0. These differ unless all p_i are equal (e.g., d=1, M=2, G=(1,−1), p=(0.25,0.75): KKT λ=(0.75,0.25), K_pλ=0 gives λ≈(0.634,0.366)). So (7) is not derived from (6); the only support is [19], a self-citation by three of this paper's authors. The actor QP (8), the λ momentum (9), the update θ_{t+1}=θ_t−αG_t(p⊙λ_t), and Theorem 3 all rely on (7), making a load-bearing premise a self-cited ansatz.

full rationale

I examined two potential sources of circularity: (i) the WC-MGDA actor derivation and (ii) the sample-complexity prediction. The sample-complexity result is not circular: it is a stated bound with explicit learning-rate choices (η_t=p_min^2/t^2, T=Θ(1/ε), B=Θ(1/ε)); the p_min dependence is an analytic consequence of the algorithm's momentum schedule, not a fitted parameter renamed as a prediction. The real issue is the actor-direction derivation. Equation (7) is presented as the Wolfe dual of (6) via KKT, but the KKT condition in θ is G diag(p)λ=0, not K_pλ=0, which is equivalent to G diag(√p)λ=0; a simple two-objective example shows they give different λ. Thus the 'readily verified' step is not a derivation, and the only authority cited is [19], prior work by three co-authors. The QP (8), λ momentum (9), actor update, and Theorem 3 all depend on (7), so a load-bearing premise is imported from a self-citation. I also note Theorem 3 is stated without proof; that is a support gap, not circularity. Because the final sample-complexity statement is not itself a restatement of [4] or [19] (it would be independent if a correct proof were supplied), I do not rate the paper as fully circular; the circularity is partial, at the level of the actor-update premise.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The central claim rests on standard RL assumptions, an external WC-scalarization lemma, and a nonconvex Wolfe dual step that is not justified. The algorithm also has several hand-chosen parameters (p, u, eta_t, step sizes, batch sizes) that are not fully specified or, in the case of p, not always within the theoretical range.

free parameters (5)
  • exploration weight vector p = hand-chosen (e.g., (0.2,0.2,0.2,0,0.4) in Table II; one-hot and ablation vectors in Figs. 1-2)
    The weight vector p is a user-specified input to the WC scalarization and directly controls the actor gradient combination. The theory requires p_min > 0, but the experiments use p_min = 0, outside the stated theory.
  • trade-off parameter u = not specified
    Introduced in Eq. (8) to balance Pareto-front exploration and Pareto-stationarity, but its value is never given and the theorem does not mention it. The lambda solution and therefore the actor update depend on u.
  • momentum coefficients eta_t = set to p_min^2/t^2 in Corollary 4
    The convergence bound in Theorem 3 depends on sum_t eta_t / p_min^2. Choosing eta_t = p_min^2/t^2 removes the p_min dependence from the final O(1/T) rate.
  • actor step size alpha = 1/(3L) with L the Lipschitz constant from Assumption 3(b)
    The theorem sets alpha by an assumed global Lipschitz constant L, which is generally unknown and may not yield a practical step size.
  • critic step size beta, critic iterations N, batch sizes D and B = not specified
    The sample complexity bound relies on the critic error term E[||w_i^t - w_i*,t||^2] = O(epsilon), but the paper does not state the values of beta, N, D, or B needed to achieve this, nor does it report them in the experiments.
assumptions (6)
  • standard math Lemma 1 (from [20]): A solution is weakly Pareto-optimal if and only if it minimizes the weighted-Chebyshev scalarization for some weight vector p in the simplex.
    This external result motivates the use of WC scalarization for Pareto-front exploration. It is a known MOO result and is cited to [20], so it is treated as a background axiom.
  • ad hoc to paper The KKT stationarity conditions and Wolfe dual of Eq. (6) are valid despite the objectives J(theta) being generally nonconvex, yielding Eqs. (7) and (8).
    Invoked in Section IV-B, Step 2-a: 'By the KKT stationarity condition on rho and theta and associating Lagrangian dual variables lambda, it can be readily verified that the Wolfe dual problem of Eq. (6) can be written as...' For nonconvex policy-gradient objectives, KKT and Wolfe duality may have a duality gap, so this is a load-bearing assumption.
  • domain assumption Assumption 1: For all theta, the state Markov chain induced by policy pi_theta is irreducible and aperiodic.
    Standard ergodicity assumption used to ensure a unique stationary distribution and to analyze Markovian TD sampling.
  • domain assumption Assumption 2: The feature matrix Phi is bounded and full rank, and ||phi(s)|| <= 1.
    Standard linear function approximation assumption for the critic; needed for TD convergence.
  • domain assumption Assumption 3: The score function is uniformly bounded and each objective gradient is L-Lipschitz in theta.
    Standard smoothness and boundedness assumptions in actor-critic analysis. They may not hold for neural network policies but are common in the literature.
  • standard math The shift by J_ub in Eq. (4) preserves the Pareto front and Pareto stationarity of the original problem.
    Subtracting per-objective constants does not change the gradient directions or the dominance structure, so this is a straightforward equivalence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enabling Pareto-Stationarity Exploration in Multi-Objective Reinforcement Learning: A Multi-Objective Weighted-Chebyshev Actor-Critic Approach." pith.science (2026). https://pith.science/paper/67DUBKR5

@misc{pith2026250721397,
  author       = {Pith},
  title        = {Pith review of: Enabling Pareto-Stationarity Exploration in Multi-Objective Reinforcement Learning: A Multi-Objective Weighted-Chebyshev Actor-Critic Approach},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/67DUBKR5}},
  note         = {Machine review of arXiv:2507.21397}
}
abstract

In many multi-objective reinforcement learning (MORL) applications, being able to systematically explore the Pareto-stationary solutions under multiple non-convex reward objectives with theoretical finite-time sample complexity guarantee is an important and yet under-explored problem. This motivates us to take the first step and fill the important gap in MORL. Specifically, in this paper, we propose a \uline{M}ulti-\uline{O}bjective weighted-\uline{CH}ebyshev \uline{A}ctor-critic (MOCHA) algorithm for MORL, which judiciously integrates the weighted-Chebychev (WC) and actor-critic framework to enable Pareto-stationarity exploration systematically with finite-time sample complexity guarantee. Sample complexity result of MOCHA algorithm reveals an interesting dependency on $p_{\min}$ in finding an $\epsilon$-Pareto-stationary solution, where $p_{\min}$ denotes the minimum entry of a given weight vector $\mathbf{p}$ in WC-scarlarization. By carefully choosing learning rates, the sample complexity for each exploration can be $\tilde{\mathcal{O}}(\epsilon^{-2})$. Furthermore, simulation studies on a large KuaiRand offline dataset, show that the performance of MOCHA algorithm significantly outperforms other baseline MORL approaches.

Figures

Figures reproduced from arXiv: 2507.21397 by the authors.

Figure 1
Figure 1. Comparison of MOCHA and SDMGrad with five one-hot weight vectors. where CIS(s, a) = min  C, π(a | s) πβ(a | s)  , D is the dataset, C is a positive constant to cap the important sampling, and πβ is the behavior policy. By definition, a larger NCIS score implies a better performance for a corresponding objective. All methods are initialized with same critic and actor parameters. In addition, initial policies for al… view at source ↗
Figure 2
Figure 2. In Figure 2a, we show the Pareto solutions for [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 2
Figure 2. MOCHA and SDMGrad with ablation weight vectors. strengthens the observation that, with increasing number of weight vectors p, MOCHA possess the potential to explore more Pareto solutions. VI. CONCLUSION In this paper, we proposed a multi-objective weighted Chebyshev actor-critic (MOCHA) algorithm for multi-objective reinforcement learning. Our proposed MOCHA method judiciously integrates weighted Chebyshev and actor… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

33 extracted references · 30 canonical work pages

  1. [1]

    A practical guide to multi-objective reinforcement learning and planning,

    C. F. Hayes, R. R uadulescu, E. Bargiacchi, J. Kllstrm, M. Macfarlane, M. Reymond, T. Verstraeten, L. M. Zintgraf, R. Dazeley, F. Heintz et al., “A practical guide to multi-objective reinforcement learning and planning,” Autonomous Agents and Multi-Agent Systems, vol. 36, no. 1, p. 26, 2022

    work page 2022

  2. [2]

    Two-stage constrained actor-critic for short video recommendation,

    Q. Cai, Z. Xue, C. Zhang, W. Xue, S. Liu, R. Zhan, X. Wang, T. Zuo, W. Xie, D. Zheng et al., “Two-stage constrained actor-critic for short video recommendation,” in Proceedings of the ACM Web Conference 2023, 2023, pp. 865–875

    work page 2023

  3. [3]

    R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction. MIT press, 2018

    2018

  4. [4]

    Finite-Time Convergence and Sample Complexity of Actor-Critic Multi-Objective Reinforcement Learning

    T. Zhou, F. Hairi, H. Yang, J. Liu, T. Tong, F. Yang, M. Momma, and Y . Gao, “Finite-time convergence and sample complexity of actor-critic multi-objective reinforcement learning,” arXiv preprint arXiv:2405.03082, 2024

    work page Pith review arXiv 2024

  5. [5]

    Recent theoretical advances in non-convex optimization,

    M. Danilova, P. Dvurechensky, A. Gasnikov, E. Gorbunov, S. Gu- minov, D. Kamzolov, and I. Shibaev, “Recent theoretical advances in non-convex optimization,” in High-Dimensional Optimization and Probability: With a View Towards Data Science. Springer, 2022, pp. 79–163

    work page 2022

  6. [6]

    Federated multi- objective learning,

    H. Yang, Z. Liu, J. Liu, C. Dong, and M. Momma, “Federated multi- objective learning,” 2024

    work page 2024

  7. [7]

    Adaptive weighted sum method for multiobjective optimization: a new method for pareto front generation,

    I. Y . Kim and O. L. de Weck, “Adaptive weighted sum method for multiobjective optimization: a new method for pareto front generation,” Structural and multidisciplinary optimization, vol. 31, no. 2, pp. 105– 116, 2006

    work page 2006

  8. [8]

    Multi-objective lqr with linear scalarization,

    A. Jadbabaie, D. Shah, and S. R. Sinclair, “Multi-objective lqr with linear scalarization,” arXiv preprint arXiv:2408.04488, 2024

    arXiv 2024

Show all 33 references
  1. [9]

    Random hypervolume scalarizations for provable multi-objective black box optimization,

    R. Zhang and D. Golovin, “Random hypervolume scalarizations for provable multi-objective black box optimization,” in International conference on machine learning. PMLR, 2020, pp. 11 096–11 105

    2020

  2. [10]

    Multiple-gradient descent algorithm (mgda) for multi- objective optimization,

    J.-A. Désidéri, “Multiple-gradient descent algorithm (mgda) for multi- objective optimization,” Comptes Rendus Mathematique, vol. 350, no. 5-6, pp. 313–318, 2012

    2012

  3. [11]

    Miettinen, Nonlinear multiobjective optimization

    K. Miettinen, Nonlinear multiobjective optimization. Springer Science & Business Media, 1999, vol. 12

    1999

  4. [12]

    Complexity of gradient descent for multiobjective optimization,

    J. Fliege, A. I. F. Vaz, and L. N. Vicente, “Complexity of gradient descent for multiobjective optimization,” Optimization Methods and Software, vol. 34, no. 5, pp. 949–959, 2019

    2019

  5. [13]

    The stochastic multi-gradient algorithm for multi-objective optimization and its application to supervised machine learning,

    S. Liu and L. N. Vicente, “The stochastic multi-gradient algorithm for multi-objective optimization and its application to supervised machine learning,” Annals of Operations Research, pp. 1–30, 2021

    2021

  6. [14]

    Mitigating gradient bias in multi-objective learning: A provably convergent approach,

    H. D. Fernando, H. Shen, M. Liu, S. Chaudhury, K. Murugesan, and T. Chen, “Mitigating gradient bias in multi-objective learning: A provably convergent approach,” in The Eleventh International Conference on Learning Representations, 2022

    2022

  7. [15]

    Direction-oriented multi-objective learning: Simple and provable stochastic algorithms,

    P. Xiao, H. Ban, and K. Ji, “Direction-oriented multi-objective learning: Simple and provable stochastic algorithms,” arXiv preprint arXiv:2305.18409, 2023

    2023 arXiv

  8. [16]

    Multi-criteria reinforcement learning

    Z. Gábor, Z. Kalmár, and C. Szepesvári, “Multi-criteria reinforcement learning.” in ICML, vol. 98, 1998, pp. 197–205

    1998

  9. [17]

    Reinforcement recommenda- tion with user multi-aspect preference,

    X. Chen, Y . Du, L. Xia, and J. Wang, “Reinforcement recommenda- tion with user multi-aspect preference,” in Proceedings of the Web Conference 2021, 2021, pp. 425–435

    2021

  10. [18]

    Few for many: Tchebycheff set scalarization for many-objective optimization,

    X. Lin, Y . Liu, X. Zhang, F. Liu, Z. Wang, and Q. Zhang, “Few for many: Tchebycheff set scalarization for many-objective optimization,” arXiv preprint arXiv:2405.19650, 2024

    2024

  11. [19]

    A multi-objective/multi-task learning framework induced by pareto stationarity,

    M. Momma, C. Dong, and J. Liu, “A multi-objective/multi-task learning framework induced by pareto stationarity,” in International Conference on Machine Learning. PMLR, 2022, pp. 15 895–15 907

    2022

  12. [20]

    Traversing pareto optimal policies: Provably efficient multi-objective reinforcement learning,

    S. Qiu, D. Zhang, R. Yang, B. Lyu, and T. Zhang, “Traversing pareto optimal policies: Provably efficient multi-objective reinforcement learning,” 2024. [Online]. Available: https://arxiv.org/abs/2407.17466

    2024 arXiv

  13. [21]

    Pareto multi- task learning,

    X. Lin, H.-L. Zhen, Z. Li, Q.-F. Zhang, and S. Kwong, “Pareto multi- task learning,” Advances in neural information processing systems, vol. 32, 2019

    2019

  14. [22]

    Multi-objective reinforcement learning for the expected utility of the return,

    D. M. Roijers, D. Steckelmacher, and A. Nowé, “Multi-objective reinforcement learning for the expected utility of the return,” in Proceedings of the Adaptive and Learning Agents workshop at FAIM, vol. 2018, 2018

    2018

  15. [23]

    On finite-time convergence of actor-critic algorithm,

    S. Qiu, Z. Yang, J. Ye, and Z. Wang, “On finite-time convergence of actor-critic algorithm,” IEEE Journal on Selected Areas in Information Theory, vol. 2, no. 2, pp. 652–664, 2021

    2021

  16. [24]

    Improving sample complexity bounds for (natural) actor-critic algorithms,

    T. Xu, Z. Wang, and Y . Liang, “Improving sample complexity bounds for (natural) actor-critic algorithms,” arXiv preprint arXiv:2004.12956, 2020

    2004 arXiv

  17. [25]

    Finite-time convergence and sample complexity of multi-agent actor-critic reinforcement learning with aver- age reward,

    F. Hairi, J. Liu, and S. Lu, “Finite-time convergence and sample complexity of multi-agent actor-critic reinforcement learning with aver- age reward,” in International Conference on Learning Representations, 2022

    2022

  18. [26]

    Fully decen- tralized multi-agent reinforcement learning with networked agents,

    K. Zhang, Z. Yang, H. Liu, T. Zhang, and T. Basar, “Fully decen- tralized multi-agent reinforcement learning with networked agents,” in International Conference on Machine Learning. PMLR, 2018, pp. 5872–5881

    2018

  19. [27]

    Random hypervolume scalarizations for provable multi-objective black box optimization,

    D. Golovin and Q. Zhang, “Random hypervolume scalarizations for provable multi-objective black box optimization,” ArXiv, vol. abs/2006.04655, 2020. [Online]. Available: https://api.semanticscholar. org/CorpusID:219531433

    2006 arXiv

  20. [28]

    Average cost temporal-difference learning,

    J. N. Tsitsiklis and B. Van Roy, “Average cost temporal-difference learning,” Automatica, vol. 35, no. 11, pp. 1799–1808, 1999

    1999

  21. [29]

    On the convergence of stochastic multi-objective gradient manipulation and beyond,

    S. Zhou, W. Zhang, J. Jiang, W. Zhong, J. Gu, and W. Zhu, “On the convergence of stochastic multi-objective gradient manipulation and beyond,” Advances in Neural Information Processing Systems, vol. 35, pp. 38 103–38 115, 2022

    2022

  22. [30]

    Multi-task learning as multi-objective optimization,

    O. Sener and V . Koltun, “Multi-task learning as multi-objective optimization,” Advances in neural information processing systems, vol. 31, 2018

    2018

  23. [31]

    Theoretical guarantees of fictitious discount algorithms for episodic reinforcement learning and global con- vergence of policy gradient methods,

    X. Guo, A. Hu, and J. Zhang, “Theoretical guarantees of fictitious discount algorithms for episodic reinforcement learning and global con- vergence of policy gradient methods,” arXiv preprint arXiv:2109.06362, 2021

    2021 arXiv

  24. [32]

    Direction-oriented multi-objective learning: Simple and provable stochastic algorithms,

    P. Xiao, H. Ban, and K. Ji, “Direction-oriented multi-objective learning: Simple and provable stochastic algorithms,” Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  25. [33]

    Reinforcement learning to optimize long-term user engagement in recommender systems,

    L. Zou, L. Xia, Z. Ding, J. Song, W. Liu, and D. Yin, “Reinforcement learning to optimize long-term user engagement in recommender systems,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019, pp. 2810– 2818

    2019

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.