REVIEW 4 major objections 4 minor 128 references
Unlocking TriLevel Learning with Level-Wise Zeroth Order Constraints: Distributed Algorithms and Provable Non-Asymptotic Convergence
T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read This paper claims the first distributed, gradient-free framework for trilevel learning, using randomized-smoothing zeroth-order cuts to build a cascaded polynomial relaxation and proving non-asymptotic convergence to an epsilon-stationary…
desk verdict First distributed trilevel zeroth-order framework, but the advertised convergence rate is not backed by the proof: exact phi evaluation, K-round bias, and a self-referential step-size break the main theorem as stated. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the zeroth-order cut: a cutting plane generated from a Gaussian-smoothed, two-point estimator of the gradient of the distance-to-optimality functions $\phi_{\mathrm{in}}$ and $\phi_{\mathrm{out}}$, rather than from true gradients or sub-gradients. These cuts define nested quadratic polynomial feasible sets $P^t_{\mathrm{in}}$ and $P^t_{\mathrm{out}}$, with the inner layer approximating the third-level optimality constraint and the outer layer approximating the second-level optimality constraint. The algorithm alternates local zeroth-order gradient steps on distributed workers with consensus updates in the master, periodically adds and prunes cuts during the first $T_1$ iterations, and then runs zeroth-order descent on the exterior-penalty objective.
What would settle it
Choose a small trilevel instance whose third-level minimizer has a closed form, so the exact $\phi_{\mathrm{in}}$ can be evaluated at every iterate, and compare the trajectory obtained with the exact $\phi_{\mathrm{in}}$ against DTZO's $K$-round approximation. If the approximation error changes the limit point by an amount that does not shrink to zero as $K$ and the penalty parameters grow, then the convergence guarantee for the original trilevel problem fails.
Extended reading notes
Core claim
The paper claims that trilevel learning with no first-order information at any level can be solved in a distributed parameter-server setting by replacing each lower-level optimality constraint with a sequence of zeroth-order cuts. Propositions 1 and 2 show that the original feasible region is contained in the relaxed region formed by the inner and outer cuts, and that the relaxation tightens as cuts are added. The relaxed problem is then reformulated as an exterior-penalty function $F$, and Theorem 1 gives an iteration complexity of $T(\epsilon) = O\big((\sum_i c_i + d(\max_{t\in[T_1]} F_\mu - F_\mu^*))^2 / \epsilon^2 + T_1\big)$, simplified to $O(\sum_i d_i^6 / \epsilon^2)$, for reaching $\|G_t\|^2 \le \epsilon$ where $G_t$ is the gradient norm of the penalty function; Theorem 2 states a communication complexity of $T(\epsilon)(2d_1+3d_2+3d_3)N + 2N\lfloor T_1/T\rfloor T(d_2+d_3)$. The guarantees are for the penalty reformulation, and Appendix H argues that the gap between the penalty problem and the original constrained problem shrinks as the penalty parameters grow.
Load-bearing premise
The load-bearing premise is that the distance-to-optimality functions $\phi_{\mathrm{in}}$ and $\phi_{\mathrm{out}}$ can be evaluated exactly and have $L$-Lipschitz gradients, even though the black-box algorithm only approximates them with $K$ communication rounds and the convergence proof does not account for that approximation error.
Editorial extensions
If this is right
- For any trilevel problem whose objectives are smooth and bounded and whose lower-level constraints are soft, DTZO reaches a first-order stationary point of the penalty reformulation without gradient access.
- The communication cost is linear in the number of workers and in the variable dimensions, with an extra cut-update term controlled by $T_1$.
- The parameter $T_1$ gives a tunable trade-off: a larger $T_1$ refines the polynomial relaxation but increases both iteration and communication complexity.
- If gradients are available at some level, the corresponding estimator or cut can be replaced by first-order information, so the framework extends to grey-box trilevel problems with partial zeroth-order constraints.
Reading between the lines
- The paper's Appendix E notes that exact evaluation of $\phi_{\mathrm{in}}$ and $\phi_{\mathrm{out}}$ requires solving the third- and second-level problems to optimality, while the algorithm approximates them with $K$ communication rounds; the convergence proof treats the exact functions, so as stated it does not quantify that approximation error.
- Because the zeroth-order cut relies only on function evaluations, the mechanism should transfer to trilevel problems with non-differentiable or combinatorial lower-level structure, provided the distance-to-optimality functions remain $L$-smooth; this transfer is not demonstrated in the paper.
- The $O(\sum_i d_i^6/\epsilon^2)$ rate is dominated by dimension factors entering through cut generation; a coordinate- or subspace-sampling estimator could plausibly reduce the dimension dependence, but such an analysis is not given.
- The same inner-and-outer cutting cascade could be extended to four or more nested levels by adding further cut layers, a direction the paper explicitly leaves to future work.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes DTZO, a distributed trilevel zeroth-order optimization framework that replaces lower-level value-function constraints with cutting planes ("zeroth-order cuts") built from Gaussian-smoothed gradients and solves the resulting surrogate by an exterior-penalty method. It claims two structural propositions (Propositions 1 and 2) establishing that the generated cuts relax the true feasible regions, a non-asymptotic iteration complexity of O(Σ_i d_i^6 / ε²) for reaching an ε-stationary point of the penalty function, and a communication complexity result. The experiments on robust hyperparameter optimization and black-box LLM backdoor prompt learning compare DTZO with FedZOO and FedRZObl and report favorable accuracy/attack-success numbers, together with ablations on T1, cut pruning, and smoothing parameter μ.
Significance. If the convergence and cut-validity claims were correct, the paper would be a useful first step: it combines cutting planes with zeroth-order oracles in a distributed trilevel setting, avoids first-order information, and provides explicit complexity expressions. The paper also makes a concrete effort to validate the framework experimentally, including ablations on the trade-off parameter T1 and on inactive-cut removal. However, the correctness concerns below are load-bearing: the main iteration-complexity theorem is self-referential in its step-size choice, the cut-generation steps require exact evaluations of distance-to-argmin functions that the algorithm itself only approximates, and the proof of the cut-relaxation claims proceeds in expectation while the algorithm uses a single random realization. As a result, the advertised rate is not established for the implemented algorithm, and the connection to the original trilevel problem is quantitative only for a surrogate objective.
major comments (4)
- [Theorem 1 / Appendix A, Eqs. (51) and (56)] The step-size in Eq. (51) is set to min{1/(8L(d1+4)), ..., 3/(2(L+1)), 1/√(T(ε)-T1)} and the smoothing parameter is required to satisfy μ ≤ 1/√(T(ε)-T1), where T(ε) is defined in Definition 2 as the unknown first iteration at which ||G^t||² ≤ ε. This makes the step-size depend on the very quantity the theorem is supposed to bound. Moreover, the conclusion "when T(ε) ≥ ... " is nearly tautological, since T(ε) is defined as the first hitting time; the argument shows only an average stationarity bound over [T1, T(ε)-1], not that ||G^{T(ε)}||² ≤ ε follows from the stated lower bound on T(ε). The theorem should be reformulated with a fixed iteration budget, or with a deterministic T, and with a step-size that can be chosen before running the algorithm.
- [Appendix E / Propositions 1–2, Eqs. (9), (11), (87)–(95)] The zeroth-order cuts in Eqs. (9) and (11) are defined through exact evaluations of φ_in and φ_out, which are distances to argmin sets of the lower-level problems. Appendix E explicitly replaces these exact values by K-round distributed approximations (Eqs. (87)–(90) and (92)–(95)), but no bound is given for ||φ_in - φ_in^K|| or for the induced violation of the cut inequality. Therefore the containment claim of Proposition 1, and its outer-layer analog, cannot be transferred to the quantities actually computed by DTZO: a feasible trilevel point may violate the approximate cut by an uncontrolled amount. Since the convergence analysis in Appendix A is performed for the fixed cut sets and not for the K-round approximations, the advertised complexity does not cover the implemented algorithm.
- [Propositions 1 and 2 / Appendix C, Eqs. (64)–(69)] The proof of Proposition 1 shows that feasible points satisfy the cut inequality in expectation over the Gaussian smoothing, i.e., with E[G_in_μ] in place of the random G_in_μ, while the algorithm adds the random cut (9) formed from a single draw. No high-probability or deterministic concentration argument bridges this gap. In addition, the premise that φ_in and φ_out have L-Lipschitz continuous gradients is not established for distance-to-argmin functions of generally nonconvex f2 and f3; without convexity or uniqueness of the lower-level minimizers, such functions are typically only Lipschitz and need not be differentiable with Lipschitz gradients. Both facts are load-bearing for the claimed relaxation results.
- [Theorem 1 / Eq. (15), Definition 1] The theorem bounds the stationarity gap of F, the exterior-penalty function over polynomial cuts, rather than stationarity of the original trilevel problem in Eqs. (2) or (3). Appendix H relates F to the relaxed constrained problem in Eqs. (104)–(110) only qualitatively, as the penalty parameters grow; it does not quantify the distance from an ε-stationary point of F to a feasible or stationary point of the original trilevel problem. Consequently, the claimed O(Σ_i d_i^6 / ε²) rate is a statement about a surrogate objective, and the connection to the original trilevel problem is not established.
minor comments (4)
- [Appendix E, Eq. (88)] The numerator of the inner-level gradient estimator uses f1,j where it should use f3,j; this typo obscures the definition of G_in,j.
- [Theorem 2 vs. Appendix B, Eq. (63)] The communication complexity stated in the main text is T(ε)(2d1+3d2+3d3)N + 2N⌊T1/T⌋T(d2+d3), while Appendix B, Eq. (63), gives 3T(ε)(d1+d2+d3)N + 2N⌊T1/T⌋T(d2+d3); these two expressions differ by T(ε)d1N and should be reconciled.
- [Theorem 1 / Definition 2] The step-size setting and the bound require T(ε) > T1 to be meaningful, since √(T(ε)-T1) appears in denominators; the manuscript does not state this condition or handle the case T(ε) ≤ T1.
- [Eq. (9) and Appendix C, Eq. (67)] The notation d_in = (d1+d2+(N+1)d3+3)^3 is dimensionally inconsistent with its use as a dimension in the Gaussian-smoothed gradient bound; please clarify whether this is a typo in Eq. (9) or in Eq. (67).
Circularity Check
No significant circularity: the core convergence proof is self-contained, although the horizon-dependent step-size and the unquantified K-round approximation of phi_in/phi_out create presentation and correctness gaps.
full rationale
I checked the derivation chain for reductions of a claimed result to its own inputs. The main proof in Appendix A is a self-contained analysis of the zeroth-order penalty method: starting from Assumptions 1-2, the Gaussian smoothed gradient estimators, and the definition of the stationarity gap, it derives a bound on the average squared gradient norm of F. Propositions 1-2 also prove the cut-containment claim by a standard L-smoothness argument, assuming phi_in and phi_out are exact and L-smooth. The step-sizes in Theorem 1 are written using 1/sqrt(T(epsilon)-T1), so T(epsilon) appears inside the schedule; this is a fixed-point presentation rather than a causal schedule, but the proof does not assume the conclusion and can be read as a bound for any T satisfying the stated inequality. This is a technical presentation issue, not a fitted-input or definitional circularity. The more serious gap is in Appendix E: the implemented algorithm replaces the exact distance-to-argmin functions phi_in and phi_out by K-round distributed approximations, while the convergence analysis assumes exact phi values and never bounds ||phi_in - phi_in^K|| or the induced cut violation. This is a missing-support and correctness gap, not a circular reduction. Self-citations such as [4] and [36] are used for the soft-constraint heuristic and the cutting-plane framework, but no load-bearing mathematical result is justified solely by those citations. I therefore find no exhibited circular step.
Assumptions & free parameters
free parameters (6)
- L (Lipschitz constant) =
not reported
- mu (smoothing parameter) =
<= 1/sqrt(T(epsilon)-T1)
- T1 (iterations before stopping cut addition) =
tuned in experiments (4500-8500)
- T (cut update period) =
not reported
- phi_j, lambda_l (penalty parameters) =
not reported
- epsilon_in, epsilon_out (slack constants) =
not reported
assumptions (5)
- domain assumption Bounded domain: ||xi,j - x*_i,j|| <= alpha_i, ||zi - z*_i|| <= alpha_i.
- domain assumption L-smoothness of the penalty function F.
- ad hoc to paper phi_in and phi_out have L-Lipschitz continuous gradients.
- ad hoc to paper Exact evaluation of phi_in and phi_out in cut generation.
- standard math Standard Gaussian smoothing gradient bounds from [46].
Cite this review
Pith. "Pith review of Unlocking TriLevel Learning with Level-Wise Zeroth Order Constraints: Distributed Algorithms and Provable Non-Asymptotic Convergence." pith.science (2026). https://pith.science/paper/N6CRTYWD
@misc{pith2026241207138,
author = {Pith},
title = {Pith review of: Unlocking TriLevel Learning with Level-Wise Zeroth Order Constraints: Distributed Algorithms and Provable Non-Asymptotic Convergence},
year = {2026},
howpublished = {\url{https://pith.science/paper/N6CRTYWD}},
note = {Machine review of arXiv:2412.07138}
}
abstract
Trilevel learning (TLL) found diverse applications in numerous machine learning applications, ranging from robust hyperparameter optimization to domain adaptation. However, existing researches primarily focus on scenarios where TLL can be addressed with first order information available at each level, which is inadequate in many situations involving zeroth order constraints, such as when black-box models are employed. Moreover, in trilevel learning, data may be distributed across various nodes, necessitating strategies to address TLL problems without centralizing data on servers to uphold data privacy. To this end, an effective distributed trilevel zeroth order learning framework DTZO is proposed in this work to address the TLL problems with level-wise zeroth order constraints in a distributed manner. The proposed DTZO is versatile and can be adapted to a wide range of (grey-box) TLL problems with partial zeroth order constraints. In DTZO, the cascaded polynomial approximation can be constructed without relying on gradients or sub-gradients, leveraging a novel cut, i.e., zeroth order cut. Furthermore, we theoretically carry out the non-asymptotic convergence rate analysis for the proposed DTZO in achieving the $\epsilon$-stationary point. Extensive experiments have been conducted to demonstrate and validate the superior performance of the proposed DTZO, e.g., it approximately achieves up to a 40$\%$ improvement in performance.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
A gradient method for multilevel optimization
Ryo Sato, Mirai Tanaka, and Akiko Takeda. A gradient method for multilevel optimization. Advances in Neural Information Processing Systems, 34:7522–7533, 2021
2021
-
[2]
Betty: An automatic differentiation library for multilevel optimization
Sang Keun Choe, Willie Neiswanger, Pengtao Xie, and Eric Xing. Betty: An automatic differentiation library for multilevel optimization. In The Eleventh International Conference on Learning Representations, 2023
2023
-
[3]
When nas meets robustness: In search of robust architectures against adversarial attacks
Minghao Guo, Yuzhe Yang, Rui Xu, Ziwei Liu, and Dahua Lin. When nas meets robustness: In search of robust architectures against adversarial attacks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 631–640, 2020
2020
-
[4]
Provably convergent federated trilevel learning
Yang Jiao, Kai Yang, Tiancheng Wu, Chengtao Jian, and Jianwei Huang. Provably convergent federated trilevel learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 12928–12937, 2024. 11
2024
-
[5]
The computational complexity of multi-level linear programs
Charles Blair. The computational complexity of multi-level linear programs. Annals of Operations Research, 34, 1992
1992
-
[6]
Mixed-integer multi-level optimization through multi-parametric programming
Styliani Avraamidou. Mixed-integer multi-level optimization through multi-parametric programming. 2018
2018
-
[7]
Computational difficulties of bilevel linear programming
Omar Ben-Ayed and Charles E Blair. Computational difficulties of bilevel linear programming. Operations Research, 38(3):556–560, 1990
1990
-
[8]
A review on bilevel optimization: From classical to evolutionary approaches and applications
Ankur Sinha, Pekka Malo, and Kalyanmoy Deb. A review on bilevel optimization: From classical to evolutionary approaches and applications. IEEE Transactions on Evolutionary Computation, 22(2):276–295, 2017
2017
Show all 128 references
-
[9]
Communication-efficient stochastic zeroth-order optimization for federated learning
Wenzhi Fang, Ziyi Yu, Yuning Jiang, Yuanming Shi, Colin N Jones, and Yong Zhou. Communication-efficient stochastic zeroth-order optimization for federated learning. IEEE Transactions on Signal Processing, 70:5058– 5073, 2022
2022
-
[10]
Zeroth-order methods for nondifferentiable, nonconvex, and hierarchical federated optimization
Yuyang Qiu, Uday Shanbhag, and Farzad Yousefian. Zeroth-order methods for nondifferentiable, nonconvex, and hierarchical federated optimization. Advances in Neural Information Processing Systems, 36, 2023
2023
-
[11]
Zeroth-order stochastic variance reduction for nonconvex optimization
Sijia Liu, Bhavya Kailkhura, Pin-Yu Chen, Paishun Ting, Shiyu Chang, and Lisa Amini. Zeroth-order stochastic variance reduction for nonconvex optimization. Advances in Neural Information Processing Systems, 31, 2018
2018
-
[12]
Zo-adamm: Zeroth-order adaptive momentum method for black-box optimization
Xiangyi Chen, Sijia Liu, Kaidi Xu, Xingguo Li, Xue Lin, Mingyi Hong, and David Cox. Zo-adamm: Zeroth-order adaptive momentum method for black-box optimization. Advances in neural information processing systems, 32, 2019
2019
-
[13]
Stochastic zeroth-order optimization in high dimensions
Yining Wang, Simon Du, Sivaraman Balakrishnan, and Aarti Singh. Stochastic zeroth-order optimization in high dimensions. In International conference on artificial intelligence and statistics, pages 1356–1365. PMLR, 2018
2018
-
[14]
Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models
Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM workshop on artificial intelligence and security, pages 15–26, 2017
2017
-
[15]
Zeroth-order non-convex learning via hierarchical dual averaging
Amélie Héliou, Matthieu Martin, Panayotis Mertikopoulos, and Thibaud Rahier. Zeroth-order non-convex learning via hierarchical dual averaging. In International Conference on Machine Learning, pages 4192–4202. PMLR, 2021
2021
-
[16]
A zeroth-order block coordinate descent algorithm for huge-scale black-box optimization
HanQin Cai, Yuchen Lou, Daniel McKenzie, and Wotao Yin. A zeroth-order block coordinate descent algorithm for huge-scale black-box optimization. In International Conference on Machine Learning, pages 1193–1203. PMLR, 2021
2021
-
[17]
Can stochastic zeroth-order frank-wolfe method converge faster for non- convex problems? In International conference on machine learning, pages 3377–3386
Hongchang Gao and Heng Huang. Can stochastic zeroth-order frank-wolfe method converge faster for non- convex problems? In International conference on machine learning, pages 3377–3386. PMLR, 2020
2020
-
[18]
Zeroth-order optimization with weak dimension dependency
Pengyun Yue, Long Yang, Cong Fang, and Zhouchen Lin. Zeroth-order optimization with weak dimension dependency. In The Thirty Sixth Annual Conference on Learning Theory, pages 4429–4472. PMLR, 2023
2023
-
[19]
Zeroth-order optimization for composite problems with functional constraints
Zichong Li, Pin-Yu Chen, Sijia Liu, Songtao Lu, and Yangyang Xu. Zeroth-order optimization for composite problems with functional constraints. InProceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 7453–7461, 2022
2022
-
[20]
Escaping saddle points in zeroth-order optimization: the power of two-point estimators
Zhaolin Ren, Yujie Tang, and Na Li. Escaping saddle points in zeroth-order optimization: the power of two-point estimators. In International Conference on Machine Learning, pages 28914–28975. PMLR, 2023
2023
-
[21]
Black-box generalization: Stability of zeroth-order learning
Konstantinos Nikolakakis, Farzin Haddadpour, Dionysis Kalogerias, and Amin Karbasi. Black-box generalization: Stability of zeroth-order learning. Advances in Neural Information Processing Systems, 35:31525–31541, 2022
2022
-
[22]
Autozoom: Autoencoder-based zeroth order optimization method for attacking black-box neural networks
Chun-Chen Tu, Paishun Ting, Pin-Yu Chen, Sijia Liu, Huan Zhang, Jinfeng Yi, Cho-Jui Hsieh, and Shin- Ming Cheng. Autozoom: Autoencoder-based zeroth order optimization method for attacking black-box neural networks. In Proceedings of the AAAI conference on artificial intelligen...
2019
-
[23]
An optimal structured zeroth-order algorithm for non-smooth optimization
Marco Rando, Cesare Molinari, Lorenzo Rosasco, and Silvia Villa. An optimal structured zeroth-order algorithm for non-smooth optimization. Advances in Neural Information Processing Systems, 36, 2024
2024
-
[24]
A comprehensive linear speedup analysis for asynchronous stochastic parallel optimization from zeroth-order to first-order
Xiangru Lian, Huan Zhang, Cho-Jui Hsieh, Yijun Huang, and Ji Liu. A comprehensive linear speedup analysis for asynchronous stochastic parallel optimization from zeroth-order to first-order. Advances in Neural Information Processing Systems, 29, 2016
2016
-
[25]
Distributed zero-order algorithms for nonconvex multiagent optimization
Yujie Tang, Junshan Zhang, and Na Li. Distributed zero-order algorithms for nonconvex multiagent optimization. IEEE Transactions on Control of Network Systems, 8(1):269–281, 2020
2020
-
[26]
Fine-grained theoretical analysis of federated zeroth-order optimization
Jun Chen, Hong Chen, Bin Gu, and Hao Deng. Fine-grained theoretical analysis of federated zeroth-order optimization. Advances in Neural Information Processing Systems, 36, 2024. 12
2024
-
[27]
Distributed zero-order optimization under adversarial noise
Arya Akhavan, Massimiliano Pontil, and Alexandre Tsybakov. Distributed zero-order optimization under adversarial noise. Advances in Neural Information Processing Systems, 34:10209–10220, 2021
2021
-
[28]
Distributed zeroth order optimization over random networks: A kiefer-wolfowitz stochastic approximation approach
Anit Kumar Sahu, Dusan Jakovetic, Dragana Bajovic, and Soummya Kar. Distributed zeroth order optimization over random networks: A kiefer-wolfowitz stochastic approximation approach. In 2018 IEEE Conference on Decision and Control (CDC), pages 4951–4958. IEEE, 2018
2018
-
[29]
Federated zeroth-order optimization using trajectory-informed surrogate gradients
Yao Shu, Xiaoqiang Lin, Zhongxiang Dai, and Bryan Kian Hsiang Low. Federated zeroth-order optimization using trajectory-informed surrogate gradients. arXiv preprint arXiv:2308.04077, 2023
2023 arXiv
-
[30]
Meta- learning to improve pre-training
Aniruddh Raghu, Jonathan Lorraine, Simon Kornblith, Matthew McDermott, and David K Duvenaud. Meta- learning to improve pre-training. Advances in Neural Information Processing Systems, 34:23231–23244, 2021
2021
-
[31]
Learning from mistakes–a framework for neural architecture search
Bhanu Garg, Li Zhang, Pradyumna Sridhara, Ramtin Hosseini, Eric Xing, and Pengtao Xie. Learning from mistakes–a framework for neural architecture search. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 10184–10192, 2022
2022
-
[32]
Convex optimization algorithms
Dimitri Bertsekas. Convex optimization algorithms. Athena Scientific, 2015
2015
-
[33]
Cutting plane methods in machine learning.Optimization for Machine Learning, pages 185–218, 2011
V ojtech Franc, Sören Sonnenburg, and Tomáš Werner. Cutting plane methods in machine learning.Optimization for Machine Learning, pages 185–218, 2011
2011
-
[34]
Distributed robust optimization (DRO), part I: Framework and example
Kai Yang, Jianwei Huang, Yihong Wu, Xiaodong Wang, and Mung Chiang. Distributed robust optimization (DRO), part I: Framework and example. Optimization and Engineering, 15(1):35–67, 2014
2014
-
[35]
A polyhedral approximation framework for convex and robust distributed optimization
Mathias Bürger, Giuseppe Notarstefano, and Frank Allgöwer. A polyhedral approximation framework for convex and robust distributed optimization. IEEE Transactions on Automatic Control, 59(2):384–395, 2013
2013
-
[36]
Asynchronous distributed bilevel optimization
Yang Jiao, Kai Yang, Tiancheng Wu, Dongjin Song, and Chengtao Jian. Asynchronous distributed bilevel optimization. In The Eleventh International Conference on Learning Representations, 2023
2023
-
[37]
Robust beamforming for downlink multi-cell systems: A bilevel optimization perspective
Xingdi Chen, Yu Xiong, and Kai Yang. Robust beamforming for downlink multi-cell systems: A bilevel optimization perspective. In Proceedings of the AAAI Conference on Artificial Intelligence, 2024
2024
-
[38]
Centralized and federated learning for predictive vnf autoscaling in multi-domain 5g networks and beyond
Tejas Subramanya and Roberto Riggio. Centralized and federated learning for predictive vnf autoscaling in multi-domain 5g networks and beyond. IEEE Transactions on Network and Service Management, 18(1):63–78, 2021
2021
-
[39]
Advances in asynchronous parallel and distributed optimization
Mahmoud Assran, Arda Aytekin, Hamid Reza Feyzmahdavian, Mikael Johansson, and Michael G Rabbat. Advances in asynchronous parallel and distributed optimization. Proceedings of the IEEE, 108(11):2013–2031, 2020
2013
-
[40]
Scalebio: Scalable bilevel optimization for llm data reweighting
Rui Pan, Jipeng Zhang, Xingyuan Pan, Renjie Pi, Xiaoyu Wang, and Tong Zhang. Scalebio: Scalable bilevel optimization for llm data reweighting. arXiv preprint arXiv:2406.19976, 2024
2024 arXiv
-
[41]
A fully first-order method for stochastic bilevel optimization
Jeongyeol Kwon, Dohyun Kwon, Stephen Wright, and Robert D Nowak. A fully first-order method for stochastic bilevel optimization. In International Conference on Machine Learning, pages 18083–18113. PMLR, 2023
2023
-
[42]
A conditional gradient- based method for simple bilevel optimization with convex lower-level problem
Ruichen Jiang, Nazanin Abolfazli, Aryan Mokhtari, and Erfan Yazdandoost Hamedani. A conditional gradient- based method for simple bilevel optimization with convex lower-level problem. In International Conference on Artificial Intelligence and Statistics, pages 10305–10323. PMLR, 2023
2023
-
[43]
Darts: Differentiable architecture search
Hanxiao Liu, Karen Simonyan, and Yiming Yang. Darts: Differentiable architecture search. In International Conference on Learning Representations, 2018
2018
-
[44]
A general stochastic approach to solving problems with hard and soft constraints
Henry A Kautz, Bart Selman, and Yueyen Jiang. A general stochastic approach to solving problems with hard and soft constraints. Satisfiability Problem: Theory and Applications, 35:573–586, 1996
1996
-
[45]
An algorithm with optimal dimension-dependence for zero-order nonsmooth nonconvex stochastic optimization
Guy Kornowski and Ohad Shamir. An algorithm with optimal dimension-dependence for zero-order nonsmooth nonconvex stochastic optimization. Journal of Machine Learning Research, 25(122):1–14, 2024
2024
-
[46]
Stochastic first-and zeroth-order methods for nonconvex stochastic program- ming
Saeed Ghadimi and Guanghui Lan. Stochastic first-and zeroth-order methods for nonconvex stochastic program- ming. SIAM journal on optimization, 23(4):2341–2368, 2013
2013
-
[47]
On penalty-based bilevel gradient descent method
Han Shen and Tianyi Chen. On penalty-based bilevel gradient descent method. arXiv preprint arXiv:2302.05185, 2023
2023 arXiv
-
[48]
Improved penalty method via doubly stochastic gradients for bilevel hyperparameter optimization
Wanli Shi and Bin Gu. Improved penalty method via doubly stochastic gradients for bilevel hyperparameter optimization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 9621–9629, 2021
2021
-
[49]
Convex optimization
Stephen P Boyd and Lieven Vandenberghe. Convex optimization. Cambridge university press, 2004. 13
2004
-
[50]
A unified single-loop alternating gradient projection algorithm for nonconvex-concave and convex-nonconcave minimax problems
Zi Xu, Huiling Zhang, Yang Xu, and Guanghui Lan. A unified single-loop alternating gradient projection algorithm for nonconvex-concave and convex-nonconcave minimax problems. arXiv preprint arXiv:2006.02032, 2020
2006 arXiv
-
[51]
Complexities in projection-free stochastic non-convex minimization
Zebang Shen, Cong Fang, Peilin Zhao, Junzhou Huang, and Hui Qian. Complexities in projection-free stochastic non-convex minimization. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 2868–2876. PMLR, 2019
2019
-
[52]
Enhancing the resilience of llms against grey-box extractions
Hanbo Huang, Yihan Li, Bowen Jiang, Bo Jiang, Lin Liu, Zhuotao Liu, Ruoyu Sun, and Shiyu Liang. Enhancing the resilience of llms against grey-box extractions. In ICML 2024 Next Generation of AI Safety Workshop, 2024
2024
-
[53]
Random gradient-free minimization of convex functions
Yurii Nesterov and Vladimir Spokoiny. Random gradient-free minimization of convex functions. Foundations of Computational Mathematics, 17(2):527–566, 2017
2017
-
[54]
Accelerated zeroth-order method for non-smooth stochastic convex optimization problem with infinite variance
Nikita Kornilov, Ohad Shamir, Aleksandr Lobanov, Darina Dvinskikh, Alexander Gasnikov, Innokentiy Shibaev, Eduard Gorbunov, and Samuel Horváth. Accelerated zeroth-order method for non-smooth stochastic convex optimization problem with infinite variance. Advances in Neural Info...
2024
-
[55]
Distributionally robust federated averaging
Yuyang Deng, Mohammad Mahdi Kamani, and Mehrdad Mahdavi. Distributionally robust federated averaging. Advances in neural information processing systems, 33:15111–15122, 2020
2020
-
[56]
Robust optimization over multiple domains
Qi Qian, Shenghuo Zhu, Jiasheng Tang, Rong Jin, Baigui Sun, and Hao Li. Robust optimization over multiple domains. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 4739–4746, 2019
2019
-
[57]
Stochastic composite mirror descent: Optimal bounds with high probabilities
Yunwen Lei and Ke Tang. Stochastic composite mirror descent: Optimal bounds with high probabilities. Advances in Neural Information Processing Systems, 31, 2018
2018
-
[58]
Asynchronous stochastic gradient descent with delay compensation
Shuxin Zheng, Qi Meng, Taifeng Wang, Wei Chen, Nenghai Yu, Zhi-Ming Ma, and Tie-Yan Liu. Asynchronous stochastic gradient descent with delay compensation. In International conference on machine learning, pages 4120–4129. PMLR, 2017
2017
-
[59]
Momentum-based variance reduction in non-convex sgd
Ashok Cutkosky and Francesco Orabona. Momentum-based variance reduction in non-convex sgd. Advances in neural information processing systems, 32, 2019
2019
-
[60]
Conflict-averse gradient descent for multi-task learning
Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu. Conflict-averse gradient descent for multi-task learning. Advances in Neural Information Processing Systems, 34:18878–18890, 2021
2021
-
[61]
Truncated back-propagation for bilevel optimization
Amirreza Shaban, Ching-An Cheng, Nathan Hatch, and Byron Boots. Truncated back-propagation for bilevel optimization. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 1723–1732. PMLR, 2019
2019
-
[62]
Decentralized stochastic bilevel optimization with improved per-iteration complexity
Xuxing Chen, Minhui Huang, Shiqian Ma, and Krishna Balasubramanian. Decentralized stochastic bilevel optimization with improved per-iteration complexity. In International Conference on Machine Learning, pages 4641–4671. PMLR, 2023
2023
-
[63]
Non-convex bilevel optimization with time-varying objective functions
Sen Lin, Daouda Sow, Kaiyi Ji, Yingbin Liang, and Ness Shroff. Non-convex bilevel optimization with time-varying objective functions. Advances in Neural Information Processing Systems, 36, 2024
2024
-
[64]
Revisiting zeroth-order optimization for memory-efficient llm fine-tuning: A benchmark
Yihua Zhang, Pingzhi Li, Junyuan Hong, Jiaxiang Li, Yimeng Zhang, Wenqing Zheng, Pin-Yu Chen, Jason D Lee, Wotao Yin, Mingyi Hong, et al. Revisiting zeroth-order optimization for memory-efficient llm fine-tuning: A benchmark. In Forty-first International Conference on Machine ...
2024
-
[65]
Subspace selection based prompt tuning with nonconvex nonsmooth black-box optimization
Haozhen Zhang, Hualin Zhang, Bin Gu, and Yi Chang. Subspace selection based prompt tuning with nonconvex nonsmooth black-box optimization. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 4179–4190, 2024
2024
-
[66]
Optimal rates for zero-order convex optimization: The power of two function evaluations
John C Duchi, Michael I Jordan, Martin J Wainwright, and Andre Wibisono. Optimal rates for zero-order convex optimization: The power of two function evaluations. IEEE Transactions on Information Theory , 61(5):2788–2806, 2015
2015
-
[67]
Black-box tuning for language- model-as-a-service
Tianxiang Sun, Yunfan Shao, Hong Qian, Xuanjing Huang, and Xipeng Qiu. Black-box tuning for language- model-as-a-service. In International Conference on Machine Learning, pages 20841–20855. PMLR, 2022
2022
-
[68]
Fairness-guided few-shot prompting for large language models
Huan Ma, Changqing Zhang, Yatao Bian, Lemao Liu, Zhirui Zhang, Peilin Zhao, Shu Zhang, Huazhu Fu, Qinghua Hu, and Bingzhe Wu. Fairness-guided few-shot prompting for large language models. Advances in Neural Information Processing Systems, 36, 2024
2024
-
[69]
Grammar prompting for domain-specific language generation with large language models
Bailin Wang, Zi Wang, Xuezhi Wang, Yuan Cao, Rif A Saurous, and Yoon Kim. Grammar prompting for domain-specific language generation with large language models. Advances in Neural Information Processing Systems, 36, 2024
2024
-
[70]
Black-box prompt learning for pre-trained language models
Shizhe Diao, Zhichao Huang, Ruijia Xu, Xuechun Li, LIN Yong, Xiao Zhou, and Tong Zhang. Black-box prompt learning for pre-trained language models. Transactions on Machine Learning Research, 2022. 14
2022
-
[71]
Poisonprompt: Backdoor attack on prompt-based large language models
Hongwei Yao, Jian Lou, and Zhan Qin. Poisonprompt: Backdoor attack on prompt-based large language models. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7745–7749. IEEE, 2024
2024
-
[72]
Qwen technical report
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023
2023 arXiv
-
[73]
Glue: A multi-task benchmark and analysis platform for natural language understanding
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NL...
2018
-
[74]
Discriminative feature alignment: Improving transferability of unsupervised domain adaptation by gaussian-guided latent alignment
Jing Wang, Jiahong Chen, Jianzhe Lin, Leonid Sigal, and Clarence W de Silva. Discriminative feature alignment: Improving transferability of unsupervised domain adaptation by gaussian-guided latent alignment. Pattern Recognition, 116:107943, 2021
2021
-
[75]
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998
1998
-
[76]
Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms
Han Xiao, Kashif Rasul, and Roland V ollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017
2017 arXiv
-
[77]
Cold case: The lost mnist digits
Chhavi Yadav and Léon Bottou. Cold case: The lost mnist digits. Advances in neural information processing systems, 32, 2019
2019
-
[78]
A primer on zeroth-order optimization in signal processing and machine learning: Principals, recent advances, and applications
Sijia Liu, Pin-Yu Chen, Bhavya Kailkhura, Gaoyuan Zhang, Alfred O Hero III, and Pramod K Varshney. A primer on zeroth-order optimization in signal processing and machine learning: Principals, recent advances, and applications. IEEE Signal Processing Magazine, 37(5):43–54, 2020
2020
-
[79]
Using hard constraints for representing soft constraints
Jean-Charles Régin. Using hard constraints for representing soft constraints. In Integration of AI and OR Techniques in Constraint Programming for Combinatorial Optimization Problems: 8th International Conference, CPAIOR 2011, Berlin, Germany, May 23-27, 2011. Proceedings 8, p...
2011
-
[80]
Combining hard and soft constraints in quantum constraint- satisfaction systems
Ellis Wilson, Frank Mueller, and Scott Pakin. Combining hard and soft constraints in quantum constraint- satisfaction systems. In SC22: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1–14. IEEE, 2022
2022
-
[81]
Bilevel optimization: Convergence analysis and enhanced design
Kaiyi Ji, Junjie Yang, and Yingbin Liang. Bilevel optimization: Convergence analysis and enhanced design. In International conference on machine learning, pages 4882–4892. PMLR, 2021
2021
-
[82]
Model-agnostic meta-learning for fast adaptation of deep networks
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning, pages 1126–1135. PMLR, 2017
2017
-
[83]
Towards deep learning models resistant to adversarial attacks
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018
2018
-
[84]
Revisiting and advancing fast adversarial training through the lens of bi-level optimization
Yihua Zhang, Guanhua Zhang, Prashant Khanduri, Mingyi Hong, Shiyu Chang, and Sijia Liu. Revisiting and advancing fast adversarial training through the lens of bi-level optimization. In International Conference on Machine Learning, pages 26693–26712. PMLR, 2022
2022
-
[85]
Distributed distributionally robust optimization with non-convex objectives
Yang Jiao, Kai Yang, and Dongjin Song. Distributed distributionally robust optimization with non-convex objectives. Advances in neural information processing systems, 35:7987–7999, 2022
2022
-
[86]
Provably faster algorithms for bilevel optimization
Junjie Yang, Kaiyi Ji, and Yingbin Liang. Provably faster algorithms for bilevel optimization. Advances in Neural Information Processing Systems, 34:13670–13682, 2021
2021
-
[87]
Bilevel programming for hyperparameter optimization and meta-learning
Luca Franceschi, Paolo Frasconi, Saverio Salzo, Riccardo Grazzi, and Massimiliano Pontil. Bilevel programming for hyperparameter optimization and meta-learning. In International conference on machine learning, pages 1568–1577. PMLR, 2018
2018
-
[88]
Investigating bi-level optimization for learning and vision from a unified perspective: A survey and beyond
Risheng Liu, Jiaxin Gao, Jin Zhang, Deyu Meng, and Zhouchen Lin. Investigating bi-level optimization for learning and vision from a unified perspective: A survey and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(12):10045–10067, 2021
2021
-
[89]
Self-tuning networks: Bilevel optimization of hyperparameters using structured best-response functions
Matthew Mackay, Paul Vicol, Jonathan Lorraine, David Duvenaud, and Roger Grosse. Self-tuning networks: Bilevel optimization of hyperparameters using structured best-response functions. In International Conference on Learning Representations, 2018
2018
-
[90]
Timeautoad: Autonomous anomaly detection with self- supervised contrastive loss for multivariate time series
Yang Jiao, Kai Yang, Dongjing Song, and Dacheng Tao. Timeautoad: Autonomous anomaly detection with self- supervised contrastive loss for multivariate time series. IEEE Transactions on Network Science and Engineering, 9(3):1604–1619, 2022
2022
-
[91]
Fedal: Black-box federated knowledge distillation enabled by adversarial learning
Pengchao Han, Xingyan Shi, and Jianwei Huang. Fedal: Black-box federated knowledge distillation enabled by adversarial learning. IEEE Journal on Selected Areas in Communications, 2024. 15
2024
-
[92]
Deep learning for classical japanese literature
Tarin Clanuwat, Mikel Bober-Irizar, Asanobu Kitamoto, Alex Lamb, Kazuaki Yamamoto, and David Ha. Deep learning for classical japanese literature. arXiv preprint arXiv:1812.01718, 2018
2018 arXiv
-
[93]
Adadelay: Delay adaptive distributed stochastic optimization
Suvrit Sra, Adams Wei Yu, Mu Li, and Alex Smola. Adadelay: Delay adaptive distributed stochastic optimization. In Artificial Intelligence and Statistics, pages 957–965. PMLR, 2016
2016
-
[94]
Distributed zeroth-order stochastic optimization in time-varying networks
Wenjie Li and Mohamad Assaad. Distributed zeroth-order stochastic optimization in time-varying networks. arXiv preprint arXiv:2105.12597, 2021
2021 arXiv
-
[95]
Projection- free methods for stochastic simple bilevel optimization with convex lower-level problem.Advances in Neural Information Processing Systems, 36, 2024
Jincheng Cao, Ruichen Jiang, Nazanin Abolfazli, Erfan Yazdandoost Hamedani, and Aryan Mokhtari. Projection- free methods for stochastic simple bilevel optimization with convex lower-level problem.Advances in Neural Information Processing Systems, 36, 2024
2024
-
[96]
signsgd via zeroth-order oracle
Sijia Liu, Pin-Yu Chen, Xiangyi Chen, and Mingyi Hong. signsgd via zeroth-order oracle. In International Conference on Learning Representations, 2018
2018
-
[97]
Bome! bilevel optimization made easy: A simple first-order approach
Bo Liu, Mao Ye, Stephen Wright, Peter Stone, and Qiang Liu. Bome! bilevel optimization made easy: A simple first-order approach. Advances in Neural Information Processing Systems, 35:17248–17262, 2022
2022
-
[98]
A novel chattering-free discrete sliding mode controller with disturbance compensation for zinc roasting temperature distribution control
Huiping Liang, Bei Sun, Biao Huang, Yonggang Li, and Chunhua Yang. A novel chattering-free discrete sliding mode controller with disturbance compensation for zinc roasting temperature distribution control. IEEE Transactions on Automation Science and Engineering, 2024
2024
-
[99]
Decentralized multi-level compositional optimization algorithms with level-independent convergence rate
Hongchang Gao. Decentralized multi-level compositional optimization algorithms with level-independent convergence rate. In International Conference on Artificial Intelligence and Statistics, pages 4402–4410. PMLR, 2024
2024
-
[100]
On the convergence of local stochastic compositional gradient descent with momentum
Hongchang Gao, Junyi Li, and Heng Huang. On the convergence of local stochastic compositional gradient descent with momentum. In International Conference on Machine Learning, pages 7017–7035. PMLR, 2022
2022
-
[101]
Optimal algorithms for stochastic bilevel optimization under relaxed smoothness conditions
Xuxing Chen, Tesi Xiao, and Krishnakumar Balasubramanian. Optimal algorithms for stochastic bilevel optimization under relaxed smoothness conditions. arXiv preprint arXiv:2306.12067, 2023
2023 arXiv
-
[102]
Communication-efficient federated bilevel optimization with global and local lower level problems
Junyi Li, Feihu Huang, and Heng Huang. Communication-efficient federated bilevel optimization with global and local lower level problems. Advances in Neural Information Processing Systems, 36, 2024
2024
-
[103]
Federated conditional stochastic optimization
Xidong Wu, Jianhui Sun, Zhengmian Hu, Junyi Li, Aidong Zhang, and Heng Huang. Federated conditional stochastic optimization. Advances in Neural Information Processing Systems, 36, 2024
2024
-
[104]
Nonconvex zeroth-order stochastic admm methods with lower function query complexity
Feihu Huang, Shangqian Gao, Jian Pei, and Heng Huang. Nonconvex zeroth-order stochastic admm methods with lower function query complexity. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024
2024
-
[105]
Asynchronous distributed reinforcement learning for lqr control via zeroth-order block coordinate descent.IEEE Transactions on Automatic Control, 2024
Gangshan Jing, He Bai, Jemin George, Aranya Chakrabortty, and Piyush K Sharma. Asynchronous distributed reinforcement learning for lqr control via zeroth-order block coordinate descent.IEEE Transactions on Automatic Control, 2024
2024
-
[106]
Optimal algorithms for stochastic bilevel optimization under relaxed smoothness conditions
Xuxing Chen, Tesi Xiao, and Krishnakumar Balasubramanian. Optimal algorithms for stochastic bilevel optimization under relaxed smoothness conditions. Journal of Machine Learning Research, 25(151):1–51, 2024
2024
-
[107]
Alternating projected sgd for equality-constrained bilevel optimization
Quan Xiao, Han Shen, Wotao Yin, and Tianyi Chen. Alternating projected sgd for equality-constrained bilevel optimization. In International Conference on Artificial Intelligence and Statistics , pages 987–1023. PMLR, 2023
2023
-
[108]
A two-timescale stochastic algorithm framework for bilevel optimization: Complexity analysis and application to actor-critic
Mingyi Hong, Hoi-To Wai, Zhaoran Wang, and Zhuoran Yang. A two-timescale stochastic algorithm framework for bilevel optimization: Complexity analysis and application to actor-critic. SIAM Journal on Optimization, 33(1):147–180, 2023
2023
-
[109]
Faster rates for the frank-wolfe method over strongly-convex sets
Dan Garber and Elad Hazan. Faster rates for the frank-wolfe method over strongly-convex sets. In International Conference on Machine Learning, pages 541–549. PMLR, 2015
2015
-
[110]
One sample stochastic frank-wolfe
Mingrui Zhang, Zebang Shen, Aryan Mokhtari, Hamed Hassani, and Amin Karbasi. One sample stochastic frank-wolfe. In International Conference on Artificial Intelligence and Statistics , pages 4012–4023. PMLR, 2020
2020
-
[111]
Communication-efficient frank-wolfe algorithm for nonconvex decentralized distributed learning
Wenhan Xian, Feihu Huang, and Heng Huang. Communication-efficient frank-wolfe algorithm for nonconvex decentralized distributed learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 10405–10413, 2021
2021
-
[112]
Parallel and distributed block-coordinate frank-wolfe algorithms
Yu-Xiang Wang, Veeranjaneyulu Sadhanala, Wei Dai, Willie Neiswanger, Suvrit Sra, and Eric Xing. Parallel and distributed block-coordinate frank-wolfe algorithms. In International Conference on Machine Learning, pages 1548–1557. PMLR, 2016. 16
2016
-
[113]
Gradient projection and conditional gradient methods for constrained nonconvex minimization
MV Balashov, BT Polyak, and AA Tremba. Gradient projection and conditional gradient methods for constrained nonconvex minimization. Numerical Functional Analysis and Optimization, 41(7):822–849, 2020
2020
-
[114]
Principled penalty-based methods for bilevel reinforcement learning and rlhf
Han Shen, Zhuoran Yang, and Tianyi Chen. Principled penalty-based methods for bilevel reinforcement learning and rlhf. arXiv preprint arXiv:2402.06886, 2024
2024 arXiv
-
[115]
Domino: Data-driven optimization of bi-level mixed-integer nonlinear problems
Burcu Beykal, Styliani Avraamidou, Ioannis PE Pistikopoulos, Melis Onel, and Efstratios N Pistikopoulos. Domino: Data-driven optimization of bi-level mixed-integer nonlinear problems. Journal of Global Optimization, 78:1–36, 2020
2020
-
[116]
Thinking inside the box: A tutorial on grey-box bayesian optimization
Raul Astudillo and Peter I Frazier. Thinking inside the box: A tutorial on grey-box bayesian optimization. In 2021 Winter Simulation Conference (WSC), pages 1–15. IEEE, 2021
2021
-
[117]
A trust region-based two phase algorithm for constrained black-box and grey-box optimization with infeasible initial point
Ishan Bajaj, Shachit S Iyer, and MM Faruque Hasan. A trust region-based two phase algorithm for constrained black-box and grey-box optimization with infeasible initial point. Computers & Chemical Engineering, 116:306– 321, 2018
2018
-
[118]
Localization and cutting-plane methods
Stephen Boyd and Lieven Vandenberghe. Localization and cutting-plane methods. From Stanford EE 364b lecture notes, 386, 2007. 17 Appendix To improve the readability of the Appendix, we have organized its contents as follows: In Appendix A and B, we delve into the comprehensive...
2007
-
[119]
The communication complexity at each iteration. At each iteration, e.g., (t + 1)th iteration, the workers transmit the updated variables xt+1 1,j , xt+1 2,j , xt+1 3,j to the master, resulting in a communication complexity of PN j=1 P3 i=1 di. Upon receiving these updated loca...
-
[120]
The communication complexity of updating zeroth order cuts. During every iteration T (t < T1), the cutting planes are updated to refine the cascaded polynomial approximation, involving two main steps: 2a) Updating the inner layer polynomial approximation: In this phase, local ...
-
[121]
+ ∂ϕin({xt 3,j },zt 1,zt 2 ′,zt 3) ∂({x3,j },z1,z2 ′,z3) ⊤ {x3,j} z1 z2′ z3 − {xt 3,j} zt 1 zt 2 ′ zt 3 − L 2 || {x3,j} z1 z2′ z3 − {xt 3,j} zt 1 zt 2 ′ zt 3 ||2 = ϕin({xt 3,j}, zt 1, zt 2 ′, zt
-
[122]
(64) According to E[Gin µ ({xt 3,j},zt 1, zt 2 ′, zt 3)] =ϕµ,in({xt 3,j}, zt 1, zt 2 ′, zt 3), taking expectation on both sides of Eq
+Gin µ ({xt 3,j},zt 1, zt 2 ′, zt 3)⊤ {x3,j} z1 z2′ z3 − {xt 3,j} zt 1 zt 2 ′ zt 3 + ∂ϕin({xt 3,j },zt 1,zt 2 ′,zt 3) ∂({x3,j },z1,z2 ′,z3) − Gin µ ({xt 3,j},zt 1, zt 2 ′, zt 3) ⊤ {x3,j} z1 z2′ z3 − {xt 3,j}...
-
[123]
(67) By combining Eq
− ∂ϕin({xt 3,j}, zt 1, zt 2 ′, zt 3) ∂({x3,j}, z1, z2, z3) ||2 ≤ µ2 4 L2(d1 +d2 +(N +1)d3 +3)3. (67) By combining Eq. (66) with Eq. (67), we have that, E[ϕin({x3,j}, z1, z2′, z3)] ≥ E[ϕin({xt 3,j}, zt 1, zt 2 ′, zt 3)] +ϕµ,in({xt 3,j}, zt 1, zt 2 ′, zt 3)⊤ {x3,j}...
-
[124]
+ ∂ϕout({xt 2,j },{xt 3,j },zt 1,zt 2,zt 3) ∂({x2,j },{x3,j },z1,z2,z3) ⊤ {x2,j} {x3,j} z1 z2 z3 − {xt 2,j} {xt 3,j} zt 1 zt 2 zt 3 − L 2 || {x2,j} {x3,j} z1 z2 z3 − {xt 2,j} {xt 3,j} zt ...
-
[125]
{ ˆx2,j} ˆz2 # − arg min {x2,j ′},z2 ′ NP j=1 f2,j( ˆz1, x2,j ′, ˆx3,j) s.t. x2,j ′ = z2′, ∀j, hin l ({ ˆx3,j}, ˆz1, z2′, ˆz3) ≤ εin, ∀l ||2 = ||
+Gout µ (t)⊤ {x2,j} {x3,j} z1 z2 z3 − {xt 2,j} {xt 3,j} zt 1 zt 2 zt 3 + ∂ϕout({xt 2,j },{xt 3,j },zt 1,zt 2,zt 3) ∂({x2,j },{x3,j },z1,z2,z3) − Gout µ (t) ⊤ {x2,j} {x3,j} z1 z2 z3 − {xt ...
-
[126]
COLA for linguistic acceptability; and 3) MRPC for semantic equivalence of sentences. In the black-box trilevel learning problem, we compare the proposed DTZO with the state-of-the-art distributed bilevel zeroth order learning method FedRZObl [10], which is used to address the...
-
[127]
=∇x2,j o({xt 2,j},{xt 3,j},zt 1, zt 2, zt 3) + 1 µ bP p=1 [f1,j(xt 1,j, xt 2,j + µup k,2, xt 3,j) − f1,j(xt 1,j, xt 2,j, xt 3,j)up k,2], (125) 41 Gx3,j ({xt 1,j}, {xt 2,j}, {xt 3,j}, zt 1, zt 2, zt
-
[128]
K Future Work This study is the first work that considers how to address the trilevel zeroth order optimization problems
=∇x3,j o({xt 2,j},{xt 3,j},zt 1, zt 2, zt 3) + 1 µ bP p=1 [f1,j(xt 1,j, xt 2,j, xt 3,j + µup k,3) − f1,j(xt 1,j, xt 2,j, xt 3,j)up k,3], (126) where µin,p = [{µp x3,j }, µp z1 , µp z2 , µp z3 ], µout,p = [{µp x2,j }, {µp x3,j }, µp z1 , µp z2 , µp z3 ], up k,1, up k,2, up k,3,...
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.