{"id":"3cac05b5-56b9-48cd-a0d4-53b4aad33844","arxiv_id":"2412.10961","paper_version":2,"verdict":"REJECT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"Periodically reusing MGDA-style weights reduces multi-objective training cost, with claimed SOTA convergence rates, but the strongly convex proof contains an invalid step.","lead":"PSMGD is a multi-objective optimization algorithm that cuts training time by reusing the MGDA weight vector for several steps instead of recomputing it every iteration. The paper proves convergence rates matching existing methods and claims a 'backpropagation complexity' as low as single-objective SGD when weights are updated rarely.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Strongly convex proof invalid: the O(η_t) weight-recalculation term is absorbed as O(η_t^2) despite setting (1−α_t)=η_t; Theorem 3.7 and the BP-complexity claims are not established.","rationale":"The reader's weakest_assumption identifies exactly the same load-bearing concern: the strongly convex proof absorbs the weight-recalculation error 4F(1−α_t)SB as η_t^2·4FSB after setting 1−α_t=η_t, which is algebraically invalid when η_t<1. My independent reading of Appendix A.5 confirms this. The invalid step is not cosmetic: with constant learning rate η_t=c/T, the unabsorbed O(η_t) term at every R-th timestep sums to a constant, so the recursion cannot yield O(1/T) from the given argument. This is an internal inconsistency in the proof of the paper's strongest theoretical claim. The general convex and non-convex proofs do not rely on the same absorption—their schedules are compatible with the terms as written—so those portions are not implicated by this critique. The experimental results are plausible but cannot substitute for the broken theoretical guarantee. If the authors re-derive the strongly convex result with (1−α_t)=η_t^2 or an equivalent O(η_t^2) schedule, the conclusion might be salvageable, but that is not what the paper states. Therefore the verdict should remain REJECT as written.","tokens_in":25612,"tokens_out":4578,"duration_ms":38408,"concrete_test":"Symbolically verify the step from inequality (118) to (120) by substituting (1−α_t)=η_t=c/T into the term 4F(1−α_t)SB·1{(t+1)%R=0}. The result is 4F(c/T)SB, which cannot be bounded by η_t^2·4FSB when T>c. Then solve the resulting recursion Δ_{t+1} ≤ (1−2μη_t)Δ_t + η_t^2Φ_const + 4F(c/T)SB·1{(t+1)%R=0} for T→∞. If the accumulated R-spaced error term produces a non-vanishing additive constant rather than O(1/T), Theorem 3.7 as stated is unproven; repair would require restating the theorem with (1−α_t)=O(η_t^2) and re-deriving the recursion under that schedule.","verdict_should_be":"REJECT","load_bearing_attack":"The strongly convex proof of Theorem 3.7 contains an invalid algebraic step in Appendix A.5. After deriving the recursion with an O(1−α_t) term for the weight-recalculation cost—specifically the 4F(1−α_t)SB·1{(t+1)%R=0} term in equation (118)—the proof replaces it by η_t^2·4FSB·1{(t+1)%R=0} inside the η_t^2Φ term (equations (120)–(122)), with the annotation 'where we set (1−α_t)=η_t.' That substitution requires (1−α_t) ≤ η_t^2, not (1−α_t)=η_t. For the stated schedule η_t=c/T with c>1/μ and T>c, η_t<1, so the inequality fails: the term is only O(η_t), not O(η_t^2). Summed over the R-spaced recalculation steps, this leaves an additive constant of order (c/R)·4FSB, so the claimed O(1/T) bound (equation (8)) does not follow from the proof. Since Theorem 3.7 is the basis for the state-of-the-art strongly convex rate and for the BP complexity O(S/εR+(R−1)/εR) in Remark 3.9, the headline theoretical guarantee is not established as written. The flaw is internal to the proof, not a disagreement with consensus, and the other two convexity settings appear structurally sound; however, the central claim across all three settings depends on this invalid absorption.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PSMGD, a periodic stochastic multi-gradient descent method for multi-objective optimization. Instead of recomputing the MGDA-type dynamic weights at every iteration, PSMGD recomputes them every R iterations and reuses them in between, with a momentum-style update λ_t = α_t λ_{t-R} + (1−α_t) λ̂*_t at recalculation steps. The authors claim O(1/T) convergence for strongly convex objectives and O(1/√T) for general convex and non-convex objectives, matching the best existing MOO rates, and introduce a 'backpropagation complexity' measure under which PSMGD achieves objective-independent complexity when R = Ω(S). The paper also reports extensive multi-task learning experiments on QM-9, NYU-v2, Multi-MNIST, CityScapes, and CelebA, with a public code repository.","tokens_in":25973,"tokens_out":16561,"duration_ms":147167,"significance":"If the theoretical claims were fully established, the periodic weight-reuse idea would be a useful and practical contribution: it directly addresses the dominant per-iteration cost of gradient-manipulation MOO methods, and the backpropagation-complexity measure is a sensible metric for comparing such methods. The experimental study is broad, includes several standard MTL benchmarks and many baselines, and the public code is a strength. However, the central strongly-convex result currently rests on an invalid algebraic step in the appendix, and the proof's parameter regime is not stated consistently. The general-convex and non-convex proofs appear structurally sound, and the flaw in the strongly-convex proof is localized, so the contribution is plausibly repairable; in its present form, the headline theoretical guarantee and the associated BP-complexity claims are not established.","major_comments":[{"comment":"The strongly-convex proof contains an invalid absorption step. After deriving the recursion with the periodic weight-recalculation term 4F(1−α_t)SB·1{(t+1)%R=0} in equation (118), the proof replaces it inside the η_t²Φ term with 4FSB·1{(t+1)%R=0}, annotated 'where we set (1−α_t)=η_t.' Since 1−α_t=η_t, the original term is O(η_t), not O(η_t²). With η_t=c/T and η_t<1 for large T, summing this term over the roughly T/R recalculation steps contributes an additive Ω(1) constant, so the contraction argument in equation (125) does not yield the claimed O(1/T) bound in equation (8). Because Theorem 3.7 is the basis for the strongly-convex row in Table 1 and for the strongly-convex BP complexity in Remarks 3.9 and 3.10, those claims are not proven as written. A repair would require, at minimum, changing the schedule so that 1−α_t is O(η_t²) (e.g., 1−α_t=η_t²) and then re-checking all affected bounds.","section":"Appendix A.5, Eqs. (118)–(122)"},{"comment":"The statement of Theorem 3.7 is internally incomplete and inconsistent. The theorem states 'Set η=c/T with c>1/μ', but the proof's final display uses the condition c>1/(2μ), and the proof introduces the condition 'where we set (1−α_t)=η_t' only at equation (120), even though the theorem statement does not specify a schedule for α_t. The contraction coefficient and the conditions on c must be stated consistently and derived from the descent inequality actually used; otherwise the theorem cannot be verified as written. This is a separate load-bearing issue from the absorption error in the previous comment, because it affects the reproducible statement of the claimed O(1/T) rate.","section":"Theorem 3.7 and Appendix A.5"}],"minor_comments":[{"comment":"The theorem states that each objective is bounded by F in the form f_s(x)≤F, but the proof of equation (59) uses |f_s(x)|≤F. Please state the needed two-sided boundedness assumption explicitly.","section":"Theorem 3.5 and Appendix A.3"},{"comment":"The lemma's bounds contain the undefined symbol m inside expressions such as √(mσ² V_ξ[λ_t]); from the context this should be S (or the bound should be written with explicit constants).","section":"Lemma A.1"},{"comment":"The experiments fix a 'momentum λ=0.9' and select R∈{4,8,16}, whereas the convergence proofs analyze time-varying α_t with 1−α_t=η_t or the min-schedule in Theorem 3.5. The paper should explain how the experimental hyperparameters relate to the analyzed parameter regime, or state which theoretical setting the experiments are intended to validate.","section":"Appendix B.3"},{"comment":"Backpropagation complexity is defined informally as the number of backpropagation operations needed to 'achieve a specified performance threshold, denoted ε'. Since the convergence theorems are stated in expectation, the BP-complexity claims should be tied to a precise formal statement (e.g., expectation of the relevant metric being below ε) to be checkable.","section":"Definition 3 and Remark 3.9"},{"comment":"The checklist says that all theoretical claims are demonstrated empirically to hold, but the experiments report task metrics and wall-clock time rather than the convergence metrics used in Theorems 3.5–3.7. Please temper this checklist entry or add measurements of the theoretical convergence metrics.","section":"Reproducibility Checklist"}],"recommendation":"major_revision","confidential_remarks":"The reader's REJECT verdict is stronger than my own assessment. The flaw identified in the stress-test does land: the strongly-convex proof's absorption of an O(η_t) periodic term as O(η_t²) is invalid, and this directly affects the headline O(1/T) and BP-complexity claims. However, the error is localized and a plausible repair exists (changing the α_t schedule so that 1−α_t = O(η_t²) and restating Theorem 3.7 accordingly). The general-convex and non-convex proofs appear structurally sound, and the experimental study is extensive. I would therefore invite a revision rather than reject outright; if the authors cannot repair Theorem 3.7, the strongly-convex claims should be removed or scaled back."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper has a genuinely useful idea and two of three proofs look right, but the strongly convex proof has a real gap, so the headline convergence-rate claim is not established as written.\n\nWhat is new: periodically recomputing the MGDA weights and reusing them for R steps, with a momentum update on the weights, is a simple and sensible way to cut the per-iteration cost of gradient manipulation MOO. The new BP-complexity measure is a good way to talk about total backpropagation cost, and the empirical story is solid: five datasets, fourteen baselines, code and training-time measurements, plus an ablation on R. The observation that MGDA weights are stable is honestly motivating, and the paper does not assume it in the proofs.\n\nThe soft spot is real and load-bearing. In Appendix A.5, equation (118) has a term 4F(1−α_t)SB at the recalculation steps. Equations (120)–(122) absorb that term into η_t^2 Φ, with the annotation \"where we set (1−α_t)=η_t.\" To absorb an O(1−α_t) term into an η_t^2 term you need (1−α_t) ≤ η_t^2. With the stated schedule η_t=c/T and c>1/μ, for large T we have η_t<1, so (1−α_t)=η_t > η_t^2. The inequality fails. Summed over the T/R recalculation steps, that term leaves a constant of order cFSB/R, independent of T, so the O(1/T) bound in Theorem 3.7 does not follow. Consequently Remark 3.9's strongly convex BP complexity and Remark 3.10's O(1/ε) objective-independent claim are not supported. This is not a style quibble; it is the central theoretical result.\n\nWhat holds up: the nonconvex and general convex analyses appear structurally sound, and the experiments are extensive and reproducible. The flaw is internal to one proof, not a disagreement with the field.\n\nWho this is for: researchers working on gradient-manipulation MOO who care about practical training time. It deserves a serious referee, but with a request for major revision: either fix the α_t schedule (e.g., 1−α_t ≤ η_t^2, or handle the recalculation term separately) and re-derive the strongly convex rate, or drop that claim. I would not cite the strongly convex result until it is repaired, but the periodic scheme and BP-complexity framing are worth keeping in mind.","headline":"Useful periodic-weight-reuse idea and solid nonconvex/convex analyses, but the strongly convex proof has a load-bearing absorption error, so the headline O(1/T) rate and BP-complexity claim are not established as written.","tokens_in":26475,"tokens_out":2849,"would_cite":false,"duration_ms":22879,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["90C29","90C25","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"PSMGD claims that periodically reusing the dynamic weights of a multi-gradient descent algorithm—rather than solving for them at every step—preserves the best known convergence rates and, when the period is at least the number of…","keywords":["multi-objective optimization","gradient manipulation methods","periodic weight update","stochastic multi-gradient descent","convergence rate","backpropagation complexity","multi-task learning","Pareto stationarity"],"falsifier":"Substitute the stated momentum schedule $1-\\alpha_t=\\eta_t$ into the strongly convex recurrence and check whether the stale-weight term $4F(1-\\alpha_t)SB$ is bounded by a constant times $\\eta_t^2$ for the full trajectory; if it is not, the displayed $\\Phi$ and the $O(1/T)$ bound in Theorem 3.7 do not follow. Re-running the calculation with $1-\\alpha_t=\\eta_t^2$ would show whether a modified schedule restores the rate, and a synthetic strongly convex two-objective problem with a known Pareto point could test the claimed $O(1/\\epsilon)$ backpropagation complexity empirically.","tokens_in":25439,"feed_emoji":"⚡","tokens_out":12400,"duration_ms":98257,"temperature":0.7,"pith_summary":"This paper tries to establish that the most expensive step in gradient-manipulation methods for multi-objective optimization—recomputing dynamic weights by solving a small constrained problem at every iteration—is largely unnecessary. Its proposed algorithm, PSMGD, solves for the weights once every $R$ iterations and reuses them in between, with a momentum update $\\lambda_t = \\alpha_t \\lambda_{t-R} + (1-\\alpha_t)\\hat{\\lambda}^*_t$ to keep them stable. The authors prove that this periodic schedule preserves the best known convergence rates: $O(1/T)$ for strongly convex objectives and $O(1/\\sqrt{T})$ for general convex and non-convex objectives. They also introduce backpropagation complexity, the total number of backward passes to reach accuracy $\\epsilon$, and show that when $R = \\Omega(S)$ PSMGD needs only $O(1/\\epsilon)$ passes for strongly convex problems and $O(1/\\epsilon^2)$ for convex and non-convex ones—the same order as single-objective SGD. If correct, the result makes multi-objective training almost as cheap per step as single-objective training while retaining the conflict-avoiding behavior of gradient manipulation.","feed_headline":"Reusing weights makes multi-objective training as cheap as single-task","feed_subtitle":"PSMGD recalculates gradient weights only every R steps, provably matching single-objective SGD's backpropagation cost.","key_machinery":"The mechanism is periodic weight recalculation with momentum. Every $R$ iterations the algorithm solves the constrained quadratic program $\\min_\\lambda \\|\\sum_{s\\in[S]} \\lambda_s \\nabla f_s(x_t,\\xi_t)\\|^2$ subject to $\\sum_s \\lambda_s = 1$, obtaining a fresh weight vector $\\hat{\\lambda}^*_t$, and then mixes it with the previous weights via $\\lambda_t = \\alpha_t \\lambda_{t-R} + (1-\\alpha_t)\\hat{\\lambda}^*_t$. For the intervening $R-1$ steps the weights are frozen, so the model update requires only one scalarized backpropagation. The proof engine is a variance bound $V_\\xi[\\lambda_t] \\le S^2 B^2 (1-\\alpha_t)^2$ on the stochastic weight vector, which controls the extra error created by stale weights and lets the recalculated-weight error be absorbed into $\\eta_t^2$ terms in the convergence recurrences. That absorption is what turns the per-iteration quadratic-program solve into an amortized cost and yields the objective-independent backpropagation complexity.","core_discovery":"The central claim is that the dynamic weight vector defining a common descent direction does not need to be recomputed every iteration. Theorem 3.5 proves $O(1/\\sqrt{T})$ convergence in the non-convex case, Theorem 3.6 proves $O(1/\\sqrt{T})$ in the general convex case, and Theorem 3.7 proves $O(1/T)$ in the $\\mu$-strongly convex case, each matching the rates of the best existing stochastic multi-objective methods without assuming Lipschitz continuity of the optimal weight function. The new backpropagation complexity measure counts total backward passes; with the recalculation period set to $R = \\Omega(S)$, the bounds become $O(1/\\epsilon)$ and $O(1/\\epsilon^2)$, independent of the number of objectives $S$ and identical to single-objective SGD. The experiments on multi-task learning datasets are offered as evidence that the same performance as full gradient manipulation is reached in less wall-clock time.","pith_inferences":["The same periodic-reuse wrapper could be applied to other gradient-manipulation methods whose weight vectors drift slowly (for example PCGrad, CAGrad, or FAMO); benchmarking each method with and without an $R$-step reuse schedule would test whether the speedup transfers.","Because backpropagation complexity counts only backward passes, it may understate the practical cost of computing all $S$ objective gradients before each weight solve; in architectures where gradient memory traffic dominates, the real speedup could be smaller than $O(S/R)$.","The weight-stability observation suggests an adaptive rule for choosing $R$: monitor the empirical variation of $\\hat{\\lambda}^*_t$ and set $R$ to the largest window with acceptable drift, with the paper's bounds indicating how convergence degrades as the drift term $|\\hat{\\lambda}^*_t-\\lambda_{t-R}|$ grows.","If the strongly convex proof is repaired by changing the momentum schedule, the same periodic-reuse idea could be combined with adaptive optimizers to reduce multi-objective training overhead further."],"forward_implications":["If the central claim holds, setting the recalculation period $R=\\Omega(S)$ makes PSMGD's total backward passes to accuracy $\\epsilon$ of the same order as single-objective SGD: $O(1/\\epsilon)$ for strongly convex objectives and $O(1/\\epsilon^2)$ for convex and non-convex objectives.","The convergence-rate results extend to settings without Lipschitz continuity of the optimal weight function, a condition required by earlier stochastic multi-gradient algorithms.","In multi-task learning, the wall-clock cost of the weight-solving step is amortized over $R$ iterations, so the speedup over full gradient manipulation should grow roughly linearly with $R$ until the weight-stability assumption degrades.","Backpropagation complexity gives practitioners a common metric—total backward passes to a target loss—for comparing multi-objective optimizers on equal footing."],"supporting_citations":[{"why":"Supplies the two lemmas on variance and bias of the stochastically recalculated weight vector that the PSMGD convergence proofs telescope.","marker":"(Zhou et al. 2022)"},{"why":"Defines the stochastic multi-gradient algorithm and the standard assumptions PSMGD extends; its rates are the baselines PSMGD claims to match.","marker":"(Liu and Vicente 2021)"},{"why":"Introduces the multiple-gradient-descent direction and the quadratic program whose solution gives the dynamic weights.","marker":"(Fliege and Svaiter 2000)"},{"why":"Provides the dual form of that quadratic program, which Algorithm 1 solves to obtain a fresh weight vector.","marker":"(Désidéri 2012)"},{"why":"SDMGrad is a state-of-the-art stochastic multi-objective method in the comparison table, providing a non-convex rate baseline.","marker":"(Xiao, Ban, and Ji 2024)"},{"why":"MoDo is a recent comparison-table method that achieves linear convergence for strongly convex objectives, serving as the baseline for PSMGD's strong-convexity claim.","marker":"(Chen et al. 2024)"}],"fun_headline_variants":["Multi-objective training sped up by periodic gradient weight reuse","PSMGD: Reuse weights to accelerate multi-objective optimization","Periodic weight recomputation matches single-task training speed in MOO","Faster multi-objective learning via infrequent weight recalculation","Reusing descent weights every few steps speeds multi-objective training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the error from reusing stale weights shrinks as the square of the step size, yet the momentum schedule chosen in the strongly convex proof only guarantees it shrinks as the step size itself.","fun_headline_variants_meta":{"raw":{"variants":["Multi-objective training sped up by periodic gradient weight reuse","PSMGD: Reuse weights to accelerate multi-objective optimization","Periodic weight recomputation matches single-task training speed in MOO","Faster multi-objective learning via infrequent weight recalculation","Reusing descent weights every few steps speeds multi-objective training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000256,"raw_usage":{"total_tokens":1621,"prompt_tokens":1035,"completion_tokens":586,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":651,"completion_tokens_details":{"reasoning_tokens":499}},"tokens_in":651,"tokens_out":586,"duration_ms":5886,"temperature":1.0,"reasoning_tokens":499,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T15:27:49.004061+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Substitute the stated momentum schedule $1-\\alpha_t=\\eta_t$ into the strongly convex recurrence and check whether the stale-weight term $4F(1-\\alpha_t)SB$ is bounded by a constant times $\\eta_t^2$ for the full trajectory; if it is not, the displayed $\\Phi$ and the $O(1/T)$ bound in Theorem 3.7 do not follow. Re-running the calculation with $1-\\alpha_t=\\eta_t^2$ would show whether a modified schedule restores the rate, and a synthetic strongly convex two-objective problem with a known Pareto point could test the claimed $O(1/\\epsilon)$ backpropagation complexity empirically.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the two lemmas on variance and bias of the stochastically recalculated weight vector that the PSMGD convergence proofs telescope."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"SDMGrad is a state-of-the-art stochastic multi-objective method in the comparison table, providing a non-convex rate baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"MoDo is a recent comparison-table method that achieves linear convergence for strongly convex objectives, serving as the baseline for PSMGD's strong-convexity claim."}],"review_version":1}