First-Order Softmax Weighted Switching Gradient Method for Distributed Stochastic Minimax Optimization with Stochastic Constraints

Abolfazl Hashemi; Antesh Upadhyay; Sang Bin Moon; Zhankun Luo

REVIEW 3 major objections 5 minor 12 references

First-Order Softmax Weighted Switching Gradient Method for Distributed Stochastic Minimax Optimization with Stochastic Constraints

T0 review · 3 major / 5 minor · reviewed 2026-07-15 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single-loop primal-only switching method with softmax weights solves federated worst-case minimax problems under stochastic constraints at the standard Õ(ε⁻⁴) rate.

desk verdict Solid dual-free constrained federated minimax with clean high-prob rates; theory is convex, main DNN experiment is not. read the letter →

arxiv 2603.05774 v2 pith:EZPKGLRL submitted 2026-03-06 cs.LG cs.DC

Zhankun Luo , Antesh Upadhyay , Sang Bin Moon , Abolfazl Hashemi This is my paper · ORCID

classification cs.LGcs.DC

keywords federatedlearningstochasticminimaxoptimizationconstraintsswitchinggradientmethodsoftmaxsmoothingpartialparticipationhigh-probabilityconvergenceprimal-only

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Federated models that only minimize average loss can leave some clients with poor performance, and adding client-specific safety or fairness constraints usually forces dual variables that drift and oscillate under partial participation. This paper shows that a simple first-order switching rule, smoothed by temperature-controlled softmax weights over clients, can optimize the worst-case client loss while keeping the worst-case constraint feasible, without any dual variables or nested loops. Under standard convexity and sub-Gaussian noise assumptions the method attains a unified ε-accuracy on both optimality gap and feasibility at the classical stochastic complexity Õ(ε⁻⁴), with a sharper high-probability dependence O(log 1/δ). Partial participation is handled by an extra stochastic-superiority assumption that bounds client sampling noise; the same analysis also yields a tighter lower bound on the softmax temperature once boundedness of the losses is dropped. Experiments on Neyman-Pearson classification and fair classification illustrate that the primal-only scheme is less sensitive to hyper-parameters than penalty or primal-dual baselines.

What carries the argument

Softmax-weighted switching: at each round the server forms temperature-α softmax weights over participating clients for both objective and constraint values; if the softmax-mean constraint is below a tightened tolerance it steps on the weighted objective gradients, otherwise on the weighted constraint gradients, then averages only the feasible iterates.

What would settle it

On a convex federated Neyman-Pearson or fair-classification instance with known optimum, run the algorithm with the prescribed step-sizes and batch sizes and check whether the averaged feasible iterate simultaneously meets both ε-optimality and ε-feasibility at the predicted sample complexity; failure under verified convexity and sub-Gaussian noise would refute the claim.

Watch

Extended reading notes

Core claim

A single-loop Softmax-Weighted Switching Gradient algorithm, which alternates between a softmax-smoothed worst-case objective gradient and a softmax-smoothed constraint gradient according to a feasibility trigger, returns an averaged iterate that simultaneously satisfies optimality gap ≤ ε and feasibility violation ≤ ε (up to a vanishing sampling term under partial participation) with probability 1-δ at the standard Õ(ε⁻⁴) stochastic oracle complexity.

Load-bearing premise

Every local objective and constraint function must be convex; without that the optimality and feasibility guarantees do not hold.

Editorial extensions

If this is right

Primal-only switching can replace dual-variable methods in federated settings that require both worst-case performance and client-wise safety or fairness constraints.
The high-probability rate improves from O(log^{2} 1/δ) to O(log 1/δ), giving tighter sample sizes for the same failure probability.
Dropping the usual bounded-loss assumption yields a strictly smaller admissible range for the softmax temperature α.
Under the stochastic-superiority condition the same rates extend to partial participation, recovering the full-participation bound as the participation ratio tends to one.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same switching-plus-softmax idea may extend, with weaker rates, to the weakly convex regime already studied for unconstrained switching methods, matching the non-convex fair-classification experiments.
If the stochastic-superiority gap σ can be estimated on-line from client statistics, one could adaptively choose the participation ratio to keep sampling error below a target.
The unified error decomposition (optimization + estimation + sampling) supplies a practical checklist for allocating communication rounds, local batch sizes and client sampling rates.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The real contribution is a single-loop primal-only method that combines softmax weighting of the client max with a Polyak-style switch on the constraint, plus a full high-probability analysis that gets the standard Õ(ε^{-4}) stochastic rate and a cleaner O(log 1/δ) factor. They also drop the global loss bound that Wang et al. needed for the softmax temperature and get a tighter α ≳ ln n / ε' (or ln m under partial participation). That package is useful for people who actually have to keep dual variables from drifting under client sampling.

The math is carefully written. Appendices C–E spell out the Bregman three-point identities, sub-Gaussian concentration, and supermartingale arguments under the stated convexity/Lipschitz/diameter/sub-Gaussian assumptions. The error split into optimization / estimation / sampling terms is explicit, and the partial-participation extension via the stochastic-superiority (FSD) client-gap assumption is a genuine (if non-standard) technical device. The NP classification experiment sits inside the theory and looks clean against penalty and primal-dual baselines.

The soft spots are real but not fatal. Every main theorem leans on convexity of every local f_i and g_i; the fair-classification experiment is a deep net the authors themselves call highly non-convex and non-smooth, so those plots cannot be read as corroboration of the complexity claims. The abstract also promises federated safe RL that never appears in Section 5. Assumption 4.9 is hard to check on real data, and there is no released code. None of that breaks the convex full-participation theory; it just means the empirical section over-reaches.

This is for the constrained-FL / Group-DRO / switching-subgradient crowd. A serious referee should see it. I would cite the dual-free rate and the α bound if I were writing in that area, and I would bring the partial-participation analysis to reading group if we are talking about client sampling noise. Send it to review; ask them to either add a non-convex disclaimer or a weak-convexity extension and to drop the missing RL claim.

Referee Report

3 major / 5 minor

Summary. The paper proposes a Softmax-Weighted Switching Gradient method for distributed stochastic minimax optimization with stochastic constraints in federated learning. It replaces the hard client-wise max with a temperature-controlled softmax, then switches between objective and constraint gradients according to a feasibility check, without dual variables. Under convexity, Lipschitz continuity, diameter, and sub-Gaussian assumptions, Main Theorems 4.6 and 4.7 give high-probability O(ε) optimality-gap and feasibility guarantees for full participation at the standard Õ(ε^{-4}) oracle complexity, with an improved O(log 1/δ) dependence. Under an additional stochastic-superiority assumption (4.9), Theorem 4.10 extends the analysis to partial participation. The softmax temperature lower bound is tightened by dropping a global loss bound. Experiments cover Neyman-Pearson classification (convex) and fair classification with a DNN (non-convex).

Significance. If the analysis holds, the work supplies a clean primal-only single-loop alternative to primal-dual and penalty methods for constrained worst-case federated learning, with an explicit three-way error decomposition (optimization / estimation / sampling) and a sharper high-probability rate. The relaxation of the bounded-loss assumption for the softmax parameter and the partial-participation extension under a stochastic-dominance condition are genuine technical contributions relative to prior softmax-minimax and switching-subgradient literature. The appendices contain a full high-probability derivation under the stated assumptions, which is a clear strength.

major comments (3)

Assumption 4.1 (convexity of every local f_i and g_i) is load-bearing for Main Theorems 4.6, 4.7 and 4.10: non-negative Bregman divergences, the three-point identity (Lemma B.1), and Jensen passage from averaged feasible iterates to the claimed gap/feasibility bounds all require it. Section 5’s fair-classification experiment, however, uses a deep network whose landscape is described as “highly non-convex and non-smooth.” The optimality-gap and feasibility guarantees therefore do not apply to the setting used to claim competitive performance against primal-dual/penalty baselines. Either a non-convex analysis (or stationarity guarantee) or an explicit disclaimer that the DNN results are purely empirical is needed; without it the experimental section cannot be read as corroboration of the central complexity claims.
For partial participation, Assumption 4.9 (relative gaps F−f_i and G−g_i stochastically dominated by Unif[0,σ]) is the structural premise that makes client-sampling error O(σ / |ln(1−r)| n) controllable (Theorem 4.10 and the error-decomposition box). The assumption is non-standard, hard to verify on real federated data, and not checked or even estimated in the experiments (m/n = 0.5). The paper should either provide diagnostic evidence that the ordered-gap condition holds approximately on the Adult/Breast-Cancer partitions, or clearly mark the partial-participation rate as conditional on this premise and discuss failure modes when it is violated.
Abstract and introduction list “federated safe reinforcement learning” among the experimental tasks, yet Section 5 and Appendix F only report NP classification and fair classification. Either the RL experiment must be restored with the same rigor as the other two, or the claim must be removed from the abstract and contribution list so that the empirical scope matches the manuscript body.

minor comments (5)

Abstract vs. body inconsistency on experimental tasks (safe RL mentioned only in abstract) should be resolved for consistency.
Figure 2 caption and surrounding text discuss α-sensitivity; the practical switch threshold is relaxed from ε/2 to ε/1.1 in Appendix F without a corresponding statement in the main algorithm description—clarify that the theory uses ε/2 while practice uses a milder constant.
Notation for masked softmax (Eq. 10) and the localized maxima F(w; I_k), G(w; I_k) (Eq. 11) is introduced late; a short pointer in Section 2 would help.
Typos: “ganrantees” (p. 5), “fo the worst-case” (p. 5), “condtional” / “defintion” in the appendices; also “V on Neumann” in Related Work.
Related Work (Appendix G) is thorough but could more sharply position the O(log 1/δ) improvement against Lan & Zhou (2020b) and the α bound against Wang et al. (2023) in a short comparison table or paragraph in the main text.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: convergence rates follow from standard Bregman/polarization identities, sub-Gaussian concentration and supermartingale arguments under explicit assumptions; softmax lower bound and error decomposition are derived, not fitted or self-defined.

full rationale

The main theorems (4.6, 4.7, 4.10) state high-probability bounds on the averaged feasible iterate under Assumptions 4.1–4.5 (and 4.9 for partial participation). The appendices derive these via the three-point Bregman identity (Lemma B.1), polarization (Lemma B.2), softmax-mean deviation bounds (Lemmas B.4–B.6), sub-Gaussian maximal inequalities and supermartingales (Lemmas B.7–B.14), and a standard error decomposition into optimization, estimation and sampling terms. The softmax temperature lower bound α ≳ ln m / ε′ (or ln n / ε′) is obtained directly from the entropy argument in Lemma B.4 after dropping the boundedness assumption of Wang et al. (2023); it is a free hyper-parameter, not a quantity fitted to data. No step equates a claimed prediction to a fitted input by construction, imports a uniqueness theorem from overlapping authors as an external fact that forces the result, or renames a known pattern. Self-citations (e.g., prior switching-gradient or EM papers by the same group) appear only as background or related-work pointers and are not load-bearing for the complexity claims. The non-convex DNN experiment lies outside the convexity assumption and is therefore a correctness/scope issue, not circularity of the derivation chain. The paper is self-contained against its stated assumptions.

Assumptions & free parameters 5 free parameters · 6 assumptions · 2 invented entities

The central rates rest on standard convex stochastic-optimization axioms plus one nonstandard client-heterogeneity axiom (stochastic superiority / uniformly bounded relative gap) needed for partial participation. Free parameters are the usual algorithmic knobs (stepsizes, α, ε, E, m) with theoretical prescriptions; α’s lower bound is proved rather than fitted. No new physical entities are postulated; the main modeling invention is the stochastic-superiority assumption and the masked-softmax switching rule.

free parameters (5)

softmax temperature α
Algorithmic hyperparameter; theory only gives a lower bound α ≥ 2 ln m / ε′ (or ln n / ε′). Experiments use α=6400 (NP) and α=1 (fair classification)—orders of magnitude apart—so practical performance depends on hand-chosen α.
global/local stepsizes (η, γ) and local steps E
Set theoretically as η = D/(L√(8K)), γ = η/E, with E ≲ √K; in experiments chosen from grids (e.g. η∈{0.1,…,1.5}).
feasibility tolerance ε and practical switch threshold
Theory uses G_k ≤ ε/2; experiments switch at ε/1.1. ε itself is a user tolerance (0.1 NP, 0.05 fair).
participation ratio r=m/n and constraint-satisfied ratio κ=|S|/K
Enter the sampling-error term; κ appears in the partial-participation feasibility residual and is path-dependent.
client-gap scale σ in Assumption 4.9
Controls the partial-participation sampling error O(σ / |ln(1−r)| n); not estimated from data in the paper.

assumptions (6)

domain assumption Assumption 4.1: each local f_i and g_i is convex on Θ
Used for Bregman nonnegativity and Jensen on the averaged feasible iterates in all main theorems.
domain assumption Assumption 4.2–4.3: L-Lipschitz components and diameter D of Θ
Standard for first-order rates; enter every stepsize and error term.
domain assumption Assumptions 4.4–4.5: sub-Gaussian function-value and gradient noise
Drive high-probability concentration (Lemmas B.7–B.10) and the O(log 1/δ) factors.
ad hoc to paper Assumption 4.9: relative gaps F−f_i and G−g_i are stochastically dominated by Unif[0,σ]
Nonstandard structural assumption enabling partial-participation sampling-error bounds (Lemmas B.12–B.14); not implied by convexity alone.
domain assumption Existence of a feasible optimum w* with G(w*)≤0 (and optionally Slater for κ bounds)
Problem setup (*); Slater used only in the optional κ analysis at the end of Appendix E.
standard math Standard concentration / supermartingale tools (Boucheron et al., Durrett)
Used throughout Appendices B–E for sub-Gaussian and sub-gamma tails.

invented entities (2)

Softmax-Weighted Switching Gradient (masked softmax + primal switch for constrained federated minimax)
purpose: Replace dual variables and hard max with a single-loop primal update that prioritizes worst-case clients and feasibility.
Algorithmic construct; efficacy is what the theorems and experiments test. Independent evidence is the stated rates under the listed assumptions, not an external physical prediction.
Stochastic superiority / uniformly bounded relative gap (Assumption 4.9)
purpose: Quantify client sampling noise so partial-participation max over I_k approximates the global max with exponential tails.
Modeling assumption introduced to close the partial-participation analysis; no independent empirical validation of the Unif[0,σ] dominance on real FL datasets is given.

how reviews work

0 comments

Cite this review

Pith. "Pith review of First-Order Softmax Weighted Switching Gradient Method for Distributed Stochastic Minimax Optimization with Stochastic Constraints." pith.science (2026). https://pith.science/paper/EZPKGLRL

@misc{pith2026260305774,
  author       = {Pith},
  title        = {Pith review of: First-Order Softmax Weighted Switching Gradient Method for Distributed Stochastic Minimax Optimization with Stochastic Constraints},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EZPKGLRL}},
  note         = {Machine review of arXiv:2603.05774}
}

abstract

This paper addresses the distributed stochastic minimax optimization problem subject to stochastic constraints. We propose a novel first-order Softmax-Weighted Switching Gradient method tailored for federated learning. Under full client participation, our algorithm achieves the standard $\tilde{\mathcal{O}}(\epsilon^{-4})$ oracle complexity to satisfy a unified bound $\epsilon$ for both the optimality gap and feasibility tolerance. We extend our theoretical analysis to the practical partial participation regime by quantifying client sampling noise through a stochastic superiority assumption. Furthermore, by relaxing standard boundedness assumptions on the objective functions, we establish a strictly tighter lower bound for the softmax hyperparameter. We provide a unified error decomposition and establish a sharp $\mathcal{O}(\log\frac{1}{\delta})$ high-probability convergence guarantee. Ultimately, our framework demonstrates that a single-loop primal-only switching mechanism provides a stable alternative for optimizing worst-case client performance, effectively bypassing the hyperparameter sensitivity and convergence oscillations often encountered in traditional primal-dual or penalty-based approaches. We verify the efficacy of our algorithm via experiment on the Neyman-Pearson (NP) classification, fair classification, and federated safe reinforcement learning tasks.

Figures

Figures reproduced from arXiv: 2603.05774 by the authors.

**Figure 3.** Fair classification. Comparisons against penalty and primal-dual baselines. Top: full participation (E = 1, m = n). Bottom: partial participation (E = 2, m n = 0.5). network, which renders the optimization landscape highly non-convex and non-smooth. We conduct experiments using the Adult income dataset [Kohavi and Becker, 1996]. We compare our method against the penalty-based and primal-dual baselines. As illustrate… view at source ↗

**Figure 2.** α-sensitivity. Impact of temperature α. High α approximates the hard max operator, while low α smooths the objective toward a simple average. We further validate the theoretical efficacy of our approach by varying the softmax parameter (α), as illustrated in [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

**Figure 4.** Federated learning settings. Impact of the number of local epochs E (top row) and the client participation ratio m/n (bottom row). F.3 Fair Classification [PITH_FULL_IMAGE:figures/full_fig_p046_4.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 1 canonical work pages

[1]

Wenke Huang, Mang Ye, Zekun Shi, Guancheng Wan, He Li, Bo Du, and Qiang Yang

URLhttps://openreview.net/forum? id=a6Cagkpmgz. Wenke Huang, Mang Ye, Zekun Shi, Guancheng Wan, He Li, Bo Du, and Qiang Yang. Federated learning for gener- alization, robustness, fairness: A survey and benchmark. IEEE Transactions on Pattern Analysis and Machine Intel- ligence, 46(12):9387–9406, 2024. Yankun Huang and Qihang Lin. Oracle complexity of sing...

arXiv 2024
[2]

Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas

PMLR, 2020. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication- efficient learning of deep networks from decentralized data. InArtificial intelligence and statistics, pages 1273–

2020
[3]

Mehryar Mohri, Gary Sivek, and Ananda Theertha Suresh

Pmlr, 2017. Mehryar Mohri, Gary Sivek, and Ananda Theertha Suresh. Agnostic federated learning. InInternational confer- ence on machine learning, pages 4615–4625. PMLR,

2017
[4]

Mahdi Morafah, Vyacheslav Kungurtsev, Hojin Chang, Chen Chen, and Bill Lin

URLhttps://proceedings.mlr.press/ v97/mohri19a.html. Mahdi Morafah, Vyacheslav Kungurtsev, Hojin Chang, Chen Chen, and Bill Lin. Towards diverse device heterogeneous federated learning via task arithmetic knowledge integra- tion.Advances in Neural Information Processing Systems, 37:127834–127877, 2024. Arkadi Nemirovski. Prox-method with rate of convergen...

2024
[5]

Seonguk Seo, Jinkyu Kim, Geeho Kim, and Bohyung Han

URLhttps://openreview.net/forum? id=ryxGuJrFvS. Seonguk Seo, Jinkyu Kim, Geeho Kim, and Bohyung Han. Relaxed contrastive learning for federated learning. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12279–12288, 2024. Moshe Shaked and J George Shanthikumar.Stochas- tic orders. Springer, 2007. URLhttps: //lin...

arXiv 2024
[6]

Optimization via First-Order Switching Methods: Skew-Symmetric Dynamics and Optimistic Discretization

Springer, 2018. Antesh Upadhyay, Sang Bin Moon, and Abolfazl Hashemi. Optimization via first-order switching methods: Skew- symmetric dynamics and optimistic discretization.arXiv preprint arXiv:2505.09146, 2025. URLhttps:// arxiv.org/abs/2505.09146. Antesh Upadhyay, Sang Bin Moon, and Abolfazl Hashemi. Fedsgm: A unified framework for constraint aware, bid...

work page Pith review arXiv doi:10.1007/bf01448847 2018
[8]

the softmax weightsr k such thatP i∈I[rk]i = 1and[r k]i ≥0and isF t-measurable; (from the defintion ofr k) rk ≡p k1k +q k[1−1 k] = softmax(αf(w k,ξ k))1k + softmax(αg(wk,ξ k))[1−1 k]
[10]

effective

the condtional 1-subgaussian random variablez (i) k,τ such thatE[z (i) k,τ | F t] = 0andE[exp(∥z (i) k,τ ∥2)| F t]≤2and therefore lnE[exp(λ⟨e,z (i) k,τ ⟩)| F t]≤lnE[exp(λ∥z (i) k,τ ∥)| F t]≤ λ2 2 ,∀e∈S d−1,∀λ∈R, and isF t+1-measurable. (from the definition ofz (i) k,τ , independence of allζ (i) k,τ andξ k ≡(ξ (i) k )i∈I and the assumption of sub-Gaussiani...

2013

Show all 12 references

[11]

the indicator1 k ≡1 Gk(wk)≤ ϵ 2 isF t-measurable; (from the definition ofG k(wk,ξ k) =m(g(w k,ξ k), α))
[12]

the softmax weightsr k such thatP i∈Ik [rk]i = 1and[r k]i ≥0and isF t-measurable; (from the defintion ofr k) rk ≡p k1k +q k[1−1 k] = softmax(αf(w k,ξ k))1k + softmax(αg(wk,ξ k))[1−1 k]
[13]

the direction vectorsd k ande (i) k such that∥d k∥ ≤1(from the definition ofd k and the assumption of finite diameter of the parameter spaceΘassumption 4.3) and∥e (i) k ∥ ≤1(from the definition ofe (i) k and the assumption of Lipschitz continuity as- sumption 4.2) and areF t-m...
[14]

effective

the conditional 1-subgaussian random variablez (i) k,τ such thatE[z (i) k,τ | F t] = 0andE[exp(∥z (i) k,τ ∥2)| F t]≤2and thereforelnE[exp(λ⟨e,z (i) k,τ ⟩)| F t]≤lnE[exp(λ∥z (i) k,τ ∥)| F t]≤ λ2 2 ,∀e∈S d−1,∀λ∈R, and isF t+1-measurable. (from the definition ofz (i) k,τ , indepe...

2013

Pith tools

Reviewed July 15, 2026 · model on record in the stance chip above.

[1] [1]

Wenke Huang, Mang Ye, Zekun Shi, Guancheng Wan, He Li, Bo Du, and Qiang Yang

URLhttps://openreview.net/forum? id=a6Cagkpmgz. Wenke Huang, Mang Ye, Zekun Shi, Guancheng Wan, He Li, Bo Du, and Qiang Yang. Federated learning for gener- alization, robustness, fairness: A survey and benchmark. IEEE Transactions on Pattern Analysis and Machine Intel- ligence, 46(12):9387–9406, 2024. Yankun Huang and Qihang Lin. Oracle complexity of sing...

arXiv 2024

[2] [2]

Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas

PMLR, 2020. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication- efficient learning of deep networks from decentralized data. InArtificial intelligence and statistics, pages 1273–

2020

[3] [3]

Mehryar Mohri, Gary Sivek, and Ananda Theertha Suresh

Pmlr, 2017. Mehryar Mohri, Gary Sivek, and Ananda Theertha Suresh. Agnostic federated learning. InInternational confer- ence on machine learning, pages 4615–4625. PMLR,

2017

[4] [4]

Mahdi Morafah, Vyacheslav Kungurtsev, Hojin Chang, Chen Chen, and Bill Lin

URLhttps://proceedings.mlr.press/ v97/mohri19a.html. Mahdi Morafah, Vyacheslav Kungurtsev, Hojin Chang, Chen Chen, and Bill Lin. Towards diverse device heterogeneous federated learning via task arithmetic knowledge integra- tion.Advances in Neural Information Processing Systems, 37:127834–127877, 2024. Arkadi Nemirovski. Prox-method with rate of convergen...

2024

[5] [5]

Seonguk Seo, Jinkyu Kim, Geeho Kim, and Bohyung Han

URLhttps://openreview.net/forum? id=ryxGuJrFvS. Seonguk Seo, Jinkyu Kim, Geeho Kim, and Bohyung Han. Relaxed contrastive learning for federated learning. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12279–12288, 2024. Moshe Shaked and J George Shanthikumar.Stochas- tic orders. Springer, 2007. URLhttps: //lin...

arXiv 2024

[6] [6]

Optimization via First-Order Switching Methods: Skew-Symmetric Dynamics and Optimistic Discretization

Springer, 2018. Antesh Upadhyay, Sang Bin Moon, and Abolfazl Hashemi. Optimization via first-order switching methods: Skew- symmetric dynamics and optimistic discretization.arXiv preprint arXiv:2505.09146, 2025. URLhttps:// arxiv.org/abs/2505.09146. Antesh Upadhyay, Sang Bin Moon, and Abolfazl Hashemi. Fedsgm: A unified framework for constraint aware, bid...

work page Pith review arXiv doi:10.1007/bf01448847 2018

[7] [8]

the softmax weightsr k such thatP i∈I[rk]i = 1and[r k]i ≥0and isF t-measurable; (from the defintion ofr k) rk ≡p k1k +q k[1−1 k] = softmax(αf(w k,ξ k))1k + softmax(αg(wk,ξ k))[1−1 k]

[8] [10]

effective

the condtional 1-subgaussian random variablez (i) k,τ such thatE[z (i) k,τ | F t] = 0andE[exp(∥z (i) k,τ ∥2)| F t]≤2and therefore lnE[exp(λ⟨e,z (i) k,τ ⟩)| F t]≤lnE[exp(λ∥z (i) k,τ ∥)| F t]≤ λ2 2 ,∀e∈S d−1,∀λ∈R, and isF t+1-measurable. (from the definition ofz (i) k,τ , independence of allζ (i) k,τ andξ k ≡(ξ (i) k )i∈I and the assumption of sub-Gaussiani...

2013