{"id":"e90a36dc-ee57-4d40-97d7-718eba1f7f35","arxiv_id":"2509.16959","paper_version":5,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"SON-GOKU measures gradient conflicts between tasks, builds a conflict graph, greedily colors it, and updates one compatible task group per step, reporting gains over multi-task baselines.","lead":"This paper introduces SON-GOKU, a scheduler that groups multi-task objectives by coloring a graph of measured gradient conflicts and trains one compatible group at a time. A generalist should read it because it tests a simple idea, avoiding simultaneous conflicting updates, against sophisticated gradient-surgery and loss-balancing optimizers.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section 5.1's descent guarantee is invalid: Eq. (12) bounds the norm of the active gradient sum, not its inner product with the full objective gradient, so the update can increase F.","rationale":"The reader correctly flags a gap between the theory and the algorithm as run, but the weakest assumption identified there is the reliability of cosine-based conflict proxies. My concern is narrower and more fundamental: even if the cosine estimates and coloring are perfect, Eq. (12) is a norm inequality and does not imply a descent direction for the full objective. Descent is an inner-product statement involving all tasks, including those not activated at the current step; a singleton color class is always τ-compatible, so the claimed descent guarantee would apply to steps that update only one task, which can obviously increase the total loss. The same flaw propagates into Eq. (13), because the deterministic cyclic schedule produces biased gradient estimators, as Appendix I.2 itself acknowledges. This does not disprove the empirical utility of SON-GOKU, but it removes the main theoretical support for the central claim. The paper could be repaired by proving per-cycle descent with explicit control of inactive-task interference, or by switching to randomized scaled color-class sampling so the update is unbiased, but neither is what Algorithm 1 executes. The verdict therefore remains conditional: the theoretical claims need correction or explicit weakening before the paper is taken at face value.","tokens_in":33014,"tokens_out":7564,"duration_ms":72439,"concrete_test":"Two-task counterexample: set ∇L₁(θ)=g₁=(1,0), ∇L₂(θ)=g₂=(-100,1) at a point θ, τ=0, active set S={1} (a valid color class). Inequality (12) holds, but the update θ' = θ - ηg₁ gives F(θ')-F(θ) = η⟨g₁+g₂,-g₁⟩ = 99η > 0, so the full objective increases. To settle the concern, the authors must supply a missing lower bound on ⟨∇F, Σ_{k∈S} g_k⟩ involving inactive tasks, or revise the descent claim to hold over a full update cycle rather than per step.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Section 5.1 claims a τ-compatible active set is a descent direction for the full MTL objective because Eq. (12) lower-bounds ∥Σ_{k∈S} g_k∥². For F = Σ w_k L_k, descent requires ⟨∇F(θ), Σ_{k∈S} g_k⟩ > 0. Eq. (12) gives no control over ⟨∇F, Σ_{k∈S} g_k⟩: ∇F contains inactive-task gradients, whose inner products with the scheduled sum are unconstrained. A singleton S={1} is always τ-compatible; if g₂ is strongly opposed to g₁, the step -ηg₁ increases F. The nonconvex rate Eq. (13) also assumes unbiased stochastic gradients E[g_t|θ_t]=∇F(θ_t), which the deterministic cyclic schedule does not provide (Appendix I.2 concedes the per-step estimator is biased). Thus the descent-to-SGD proof chain is not derived for the algorithm as run; the central theoretical guarantee rests on an unproved inner-product condition.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SON-GOKU, a scheduler for multi-task learning that estimates pairwise gradient interference via EMA-smoothed cosine similarity, builds a thresholded conflict graph, greedily colors it, and activates one color class per training step, with periodic recoloring. It claims theoretical guarantees that τ-compatible active sets preserve descent, that a nonconvex O(1/√T) rate holds with a mild (1+τ) constant, that scheduled group updates are never worse and often better than a single mixed update, that the population conflict graph and task partition can be recovered exactly under a margin assumption, and that greedy coloring gives bounded staleness. Empirically, it reports consistent improvements over baselines across six datasets and further gains when combined with PCGrad, AdaTask, and GradNorm.","tokens_in":33381,"tokens_out":9935,"duration_ms":88652,"significance":"If the theoretical claims held, the paper would offer a simple, low-overhead plug-in scheduler with a rare theoretical treatment of dynamic task grouping, and the six-dataset empirical study with ablations would be a useful contribution. The graph-recovery analysis in Appendix B is internally coherent under its explicit assumptions, and the ablation results support the importance of dynamic recoloring and EMA smoothing. However, several load-bearing theoretical statements are not established as written: the descent claim conflates a norm bound with a full-objective inner-product condition, a key refinement inequality (Proposition 7) is false, the convergence rate in Eq. (13) is not derived for the deterministic schedule actually implemented, and the scheduled-versus-aggregated comparison in Section 5.3 is contradicted by the paper's own Appendix C.4. These issues are central to the paper's claim to explain why grouping and sequential updates improve multi-task learning.","major_comments":[{"comment":"Eq. (12) lower-bounds the squared norm of the active-gradient sum, but the text concludes that the step −ηΣ_{k∈S}g_k is a descent direction for F = Σ_k w_k L_k. Descent requires ⟨∇F(θ), Σ_{k∈S}g_k⟩ > 0, and Eq. (12) gives no control over the components of ∇F coming from inactive tasks. A singleton S={k} is always τ-compatible, and if the inactive-task gradients oppose g_k, the update can increase F. Additionally, Proposition 7 is false as stated: for g1=(1,0) and g2=(−1,0), τeff(S)=1/2 but ∥g1+g2∥²=0 < (1−1/2)(∥g1∥²+∥g2∥²)=1. The positive-part aggregate conflict ratio needs a factor of 2 in the cross-term, and the current version invalidates the 'assumption-free' refinement and its reuse in Appendix F (Eqs. 93–98).","section":"Appendix E, Proposition 7"},{"comment":"Eq. (13) is not derived for Algorithm 1 as run. Theorem 7 in Appendix F assumes unbiased stochastic gradients E[g_t|θ_t]=∇F(θ_t), and Appendix I.2 explicitly concedes that the deterministic cyclic schedule in Algorithm 1 generally violates this condition (Eq. 109). The proposed fixes in I.2.1 either replace the algorithm by randomized class sampling with scaling or appeal to unspecified 'standard analyses' of cyclic block updates; neither establishes Eq. (13) for the implemented method. Moreover, the (1+τ) factor in Eq. (13) does not appear in Theorem 7, and the τeff-based refinement in Appendix F bounds average active-gradient energy, not min_t E∥∇F(θ_t)∥². The claimed rate for the implemented algorithm is therefore unsupported.","section":"Section 5.2/Appendix I"},{"comment":"The claim in Section 5.3 that the scheduled bound is 'never worse' than the aggregated bound is contradicted by Appendix C.4's derivation, which shows that the envelope UB_sch − UB_agg can be positive without additional structure (Eq. 54). Theorem 6's strict-improvement result is conditional on Assumption 7, which postulates negative Hessian-weighted cross-terms between separated groups—exactly the property the scheduler is meant to create rather than a property proved from τ-compatibility or graph recovery. Furthermore, the comparison is between L-smooth upper bounds, not actual objective values, so even Theorem 6 does not show that the scheduled path decreases F more than the mixed step.","section":"Appendix C"},{"comment":"Proposition 8's bounded-staleness statement applies only while the coloring is fixed. Algorithm 1 recolors every R steps, and because R is not required to be a multiple of the current color count m_r, a task can wait across a refresh boundary for longer than Δ+1 steps. For example, with m=2 and R=3, a task in the second color class can be updated at steps 1 and 4, a gap of three steps even though Δ=1. Thus the claim that 'no task waits more than Δ steps between updates' is not guaranteed for the dynamic algorithm; the proof must either align the cycle with refresh boundaries or restate the bound for the refresh-aware schedule.","section":"Section 5.5/Appendix G"},{"comment":"The entire grouping mechanism rests on the assumption that thresholded cosine similarity of EMA-smoothed shared-parameter gradients is a reliable proxy for the interference that actually impedes multi-task learning. The recovery theorems in Appendix B guarantee consistency with the population cosine graph under Assumptions 1–5, but they do not establish that this graph captures the interactions that slow the full objective; conflicts mediated by task-specific heads or by higher-order parameter interactions are outside the model. The paper provides no experiment validating the proxy, such as comparing graph edges against measured negative transfer or per-step objective changes, so the link between the recovered graph and the claimed performance gains remains an assumption.","section":"Sections 4.1–4.2"}],"minor_comments":[{"comment":"Several rows in Table 1 have formatting errors or missing separators, e.g., the SON-GOKU+AdaTask row reads '67 71 6352 0.53...' and the column header 'F&B HEALTH' conflates two datasets. The table should be reformatted and should report standard deviations or the number of seeds, since the text repeatedly claims 'consistent' improvements.","section":"Table 1"},{"comment":"The sentence appealing to 'standard analyses of nonconvex smooth cyclic block updates' has no citation or theorem statement. Either provide a precise reference with the required conditions and constants, or remove the claim and state that the deterministic cyclic case is not analyzed.","section":"Appendix I"},{"comment":"The paper should state clearly whether Eq. (13) applies to the deterministic cyclic schedule, the randomized class-sampling variant, or both, and should define σ² as the variance of the estimator used in each case. The current presentation conflates the two variants.","section":"General"},{"comment":"No code or data availability statement is included, and no details of hyperparameter ranges or sensitivity (e.g., τ∗, β, R) are reported beyond the ablation. Providing these would improve reproducibility.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper is ambitious and the empirical study is broad, but the gap between the stated theory and the implemented algorithm is substantial. The most serious issues are the invalid Proposition 7, the descent-direction conflation in Section 5.1, the unsupported convergence rate for the deterministic schedule, and the contradiction between Section 5.3 and Appendix C.4. These are fixable in principle by revising the claims, proving the cyclic-block rate, or restating the theory for a modified algorithm, but they must be addressed before the paper can be accepted. I would also ask the authors to add statistical support to Table 1 and to validate the cosine-similarity proxy empirically."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read the paper. The kernel of the idea is genuinely neat: EMA-smooth per-task gradients, threshold the pairwise cosines into a conflict graph, color it greedily with Welsh-Powell, and step through one color class at a time, re-coloring every R steps. That is a clean plug-in scheduler, and the ablation (Static One-Shot vs Single-Step vs full) suggests the dynamic refresh and history smoothing are doing real work. The margin-based recovery analysis in Appendix B is the best part—it gives a proper sample-complexity bound for recovering the population conflict graph under a separation assumption, and that argument is internally coherent.\n\nThe problems are the headline guarantees. Section 5.1 claims that a τ-compatible active set gives a descent direction because Eq. (12) lower-bounds the norm of the summed active gradients. But descent needs inner product with the full objective gradient to be positive, not a lower bound on the squared norm of the active sum. The inactive-task gradients are unconstrained; a singleton is always τ-compatible and can be opposed by an inactive task. So the step can increase F. The stress-test note is right that Eq. (12) does not imply descent.\n\nEq. (13)'s nonconvex rate has the same disease. The proof in Appendix F uses E[g_t|θ_t] = ∇F(θ_t). The deterministic cyclic schedule does not satisfy that; Appendix I.2 concedes the per-step estimator is biased. So the claimed (1+τ) rate is not derived for Algorithm 1. The randomized scaling variant would restore unbiasedness, but that is not what the algorithm runs or what the main text promises.\n\nThe scheduled-vs-mixed comparison in Appendix C only gives 'strictly better' under Assumption 7, which essentially assumes the Hessian-weighted cross-terms between separated groups are negative—it bakes in the scheduler's advantage. The strength of the claim in Section 5.3 outruns the assumptions.\n\nEmpirically, the results are suggestive but under-evidenced: no error bars or seeds in Table 1, no code, the closest prior baseline (Jeong & Yoon 2025) is missing, and some datasets are tiny subsets (250 or 2,500 samples). For a paper that leans heavily on theory, these tables need to be more substantial.\n\nWho gets value from this? People working on task grouping and selective updates will find the coloring-plus-EMA recipe worth trying, and the recovery lemma is citable on its own. But the paper overclaims the theory. It deserves a serious referee—with major revision enforced—so the claims can be corrected or withdrawn. I would send it out, asking the authors to prove or drop the descent guarantee and to add baselines and error bars.","headline":"Neat scheduler idea and a solid recovery lemma, but the descent and convergence proofs do not cover the algorithm as run—the claimed guarantees are the paper's main soft spot.","tokens_in":33753,"tokens_out":5748,"would_cite":false,"duration_ms":47669,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A scheduler that activates only one low-conflict color class of tasks per training step improves multi-task learning and preserves the standard SGD convergence rate.","keywords":["multi-task learning","negative transfer","graph coloring","gradient interference","task scheduling","conflict graph","Welsh-Powell algorithm","SGD convergence"],"falsifier":"Construct two tasks whose mean gradient directions are aligned (cosine above $-\\tau$) but whose Hessian cross-terms are strongly negative, so a joint update increases the loss; run SON-GOKU with both in one color class and check whether the descent bound of Equation (12) is violated. Alternatively, fix a refresh window of length $R$ and measure whether task gradient directions rotate by more than the recovery margin $\\gamma$; if they do, the exact-recovery guarantee predicts that the empirical conflict graph should diverge from the population graph.","tokens_in":32803,"feed_emoji":"🎨","tokens_out":5664,"duration_ms":51413,"temperature":0.7,"pith_summary":"The paper introduces SON-GOKU, a scheduler that measures pairwise gradient interference between tasks, builds a conflict graph whose edges join tasks whose averaged gradients point against each other, greedily colors that graph, and updates exactly one color class per training step. Its central claim is that this sequential, low-conflict scheduling reduces negative transfer and improves the final performance of any underlying multi-task optimizer, without retuning the optimizer. The paper backs this with theory: grouped low-conflict updates preserve descent, the method reaches the usual nonconvex SGD rate $O(1/\\sqrt{T})$ up to a small conflict-dependent constant, and over a refresh window scheduled group updates are never worse than a single mixed update and are strictly better when between-group interference is negative. Empirically, the authors report consistent gains over baselines and state-of-the-art multi-task optimizers across six datasets.","feed_headline":"One compatible task group per step speeds MTL","feed_subtitle":"SON-GOKU colors a conflict graph of tasks, updates one color per step, and keeps SGD's 1/√T rate.","key_machinery":"The load-bearing object is the conflict graph $G_\\tau$: vertices are tasks, and an edge exists between tasks $i,j$ when the interference coefficient $\\rho_{ij} = -\\langle \\tilde{g}_i, \\tilde{g}_j\\rangle / (\\|\\tilde{g}_i\\|\\|\\tilde{g}_j\\|)$ exceeds the threshold $\\tau$, where $\\tilde{g}_i$ are EMA-smoothed shared-parameter gradients. Greedy Welsh-Powell coloring partitions the tasks into color classes with no conflict edges inside a class, and the scheduler activates one color class per step in a cyclic order, refreshing the graph every $R$ steps. This mechanism converts the continuous problem of choosing which tasks should update together into a discrete combinatorial one, with the $\\Delta+1$ color bound providing both a bounded update delay and a compact schedule that scales with the worst conflict degree rather than the number of tasks.","core_discovery":"The discovery is that conflict-aware scheduling can be reduced to graph coloring: compute EMA-smoothed gradient cosines, threshold them at a tolerance $\\tau$ to define a conflict graph, apply the Welsh-Powell largest-first greedy coloring, and cyclically activate one color class per step while periodically rebuilding the graph. The paper proves that a $\\tau$-compatible group's aggregate update is a guaranteed descent direction, with the squared norm of the joint update bounded below by $(1-\\tau(|S|-1))$ times the sum of per-task squared norms. With step size $\\eta = c/\\sqrt{T}$, convergence follows the standard nonconvex SGD rate with a mild $(1+\\tau)$ factor, and under a separation margin between conflicting and non-conflicting pairs the empirical conflict graph exactly recovers the population graph with high probability. The paper further claims that sequential group updates dominate a single aggregated update in expected descent when cross-group interactions are sufficiently negative, and that the cyclic schedule keeps every task updated at least once every $\\Delta+1$ steps, where $\\Delta$ is the maximum conflict degree.","pith_inferences":["The theory implicitly predicts that the benefit of scheduling grows with the strength of negative cross-group interference; a direct test would be to construct synthetic task sets with controlled pairwise cosine angles and verify that the advantage over a single mixed update tracks the predicted descent gap.","Because the schedule length depends on the maximum conflict degree $\\Delta$ rather than $K$, the method should scale gracefully to many tasks when most pairs are non-conflicting; measuring the empirical $\\Delta$ on real workloads would test this scaling premise.","The cosine proxy itself is the main extension risk: tasks with aligned mean gradients can still interfere through higher-order Hessian interactions, so a natural next step would be to replace flat gradient cosines with layer-wise or Hessian-weighted conflict measures and check whether the coloring changes materially.","The recovery bound suggests that the EMA refresh window and the tolerance $\\tau$ should be tuned together with the expected drift of task gradients; one testable implication is that an adaptive refresh period based on measured gradient drift would preserve the guarantees at lower average cost."],"forward_implications":["SON-GOKU behaves as a plug-in scheduler: it can wrap existing multi-task optimizers such as PCGrad, AdaTask, and GradNorm, and the paper reports that the combinations improve over the base methods without additional tuning.","The convergence guarantee means the scheduler does not sacrifice the asymptotic $O(1/\\sqrt{T})$ rate; interference control shows up only in the constant via the $(1+\\tau)$ factor, so practitioners can trade a small constant for reduced negative transfer.","The $\\Delta+1$ cyclic schedule provides a bounded staleness guarantee: every task is updated at least once per period, so the method is compatible with analyses that assume bounded inter-update delay.","The exact-recovery result gives a concrete probe budget: if the EMA effective sample size satisfies $n_{\\text{eff}} \\gtrsim \\sigma^2/(m_0^2\\gamma^2)\\log(K/\\delta)$, the estimated conflict graph matches the true one with high probability, linking a practical hyperparameter choice to measurable noise, margin, and task-count quantities.","The ablation study shows that dynamic recoloring and history-smoothed conflict estimates are load-bearing: freezing the coloring or using only the most recent mini-batch gradients degrades performance, supporting the paper's claim that task relationships drift and need periodic refresh."],"supporting_citations":[{"why":"Supplies the largest-first greedy coloring that produces the conflict-free color classes and the $\\Delta+1$ color bound that drives schedule length and staleness.","marker":"(Welsh & Powell, 1967)"},{"why":"Provides the classical nonconvex SGD $O(1/\\sqrt{T})$ convergence bound that the paper extends by the $(1+\\tau)$ factor in its rate theorem.","marker":"(Ghadimi & Lan, 2013)"},{"why":"Supplies the standard SGD convergence framework and noise assumptions used to compare the scheduler's rate with the classical bound.","marker":"(Bottou et al., 2018)"},{"why":"Establishes the bounded-delay assumption that the cyclic color schedule satisfies, connecting grouping to asynchronous SGD guarantees.","marker":"(Niu et al., 2011)"},{"why":"Represents prior task-grouping work based on offline affinity estimation, which SON-GOKU contrasts with online EMA-based interference coloring.","marker":"(Fifty et al., 2021)"},{"why":"Provides PCGrad, both a baseline and a combination partner, whose gradient projection handles residual conflict within scheduler-selected groups.","marker":"(Yu et al., 2020)"},{"why":"Provides GradNorm, a loss-balancing baseline and warm-start partner used in the experimental comparisons.","marker":"(Chen et al., 2018)"},{"why":"Provides AdaTask, an adaptive-learning-rate MTL method that the scheduler is combined with and compared against.","marker":"(Yang et al., 2023)"}],"fun_headline_variants":["Update one task color per step to avoid conflicts","Graph coloring stops MTL gradient clash","SON-GOKU: schedule tasks by conflict graph colors","Color-coded task groups cut gradient interference"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the angle between recent averaged task gradients, thresholded at a chosen tolerance, faithfully measures harmful interference and stays stable within each refresh window; if real conflicts live in higher-order interactions or the gradients swing inside a window, the coloring is built on the wrong graph.","fun_headline_variants_meta":{"raw":{"variants":["Update one task color per step to avoid conflicts","Graph coloring stops MTL gradient clash","SON-GOKU: schedule tasks by conflict graph colors","Color-coded task groups cut gradient interference"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000185,"raw_usage":{"total_tokens":1328,"prompt_tokens":955,"completion_tokens":373,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":571,"completion_tokens_details":{"reasoning_tokens":315}},"tokens_in":571,"tokens_out":373,"duration_ms":3766,"temperature":1.0,"reasoning_tokens":315,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:49:36.943339+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct two tasks whose mean gradient directions are aligned (cosine above $-\\tau$) but whose Hessian cross-terms are strongly negative, so a joint update increases the loss; run SON-GOKU with both in one color class and check whether the descent bound of Equation (12) is violated. Alternatively, fix a refresh window of length $R$ and measure whether task gradient directions rotate by more than the recovery margin $\\gamma$; if they do, the exact-recovery guarantee predicts that the empirical conflict graph should diverge from the population graph.","supporting_citations":[{"cited_title":"This maintains consistent training dynamics across tasks","cited_arxiv_id":null,"evidence_quote":"Provides GradNorm, a loss-balancing baseline and warm-start partner used in the experimental comparisons."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides AdaTask, an adaptive-learning-rate MTL method that the scheduler is combined with and compared against."}],"review_version":1}