{"id":"8aa060c1-60eb-40b6-8185-889575506be0","arxiv_id":"2412.04413","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Task groupings are inferred from one-step gradient distances between tasks at initialization and clustered with graph attention networks, matching TAG/HOA performance at lower cost.","lead":"This paper proposes a cheap way to decide which machine learning tasks should be jointly trained, by measuring how similarly the tasks push a fresh model's weights after a single gradient step. The resulting groupings match previous expensive methods on several benchmarks, though the advertised five-fold speedup and the supporting theory are not fully backed by the paper's own results.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 4.2's bound assumes the one-step update θt*_i is a zero-loss, zero-gradient sample optimum; a finite-η gradient step does not generally reach this, so the theoretical link from SCA distances to joint-trainability is not established.","rationale":"The reader's weakest_assumption is exactly the load-bearing point. The central claim — SCA affinities identify task groupings because of Theorem 4.2/Eq. 15 — depends on the one-step update being a zero-loss, zero-gradient sample optimum. The paper's own Section 3.2 hedges this as 'arguably assumed', and for a finite learning rate it is generally false; one gradient step is not a minimizer. Without this premise, the residual loss and gradient appear in Eq. 9 and the density bound does not follow. I also noticed the missing 1/T in Eq. 11, which is a genuine algebraic error but secondary. These are correctness concerns, not disagreements with the MTL consensus. The empirical contribution is separate and plausible: SCA groupings are comparable to TAG/HOA on several benchmarks at lower compute. That supports a conditional outcome rather than rejection: the authors should repair or relabel the theory, release the SCA learning rate/code, add fair single-model-per-group baselines, and correct the speedup claim. Since my concern matches the reader's, the verdict is unchanged.","tokens_in":22339,"tokens_out":9167,"duration_ms":85455,"concrete_test":"On the CIFAR-10/SVHN/Colorectal setup from Figure 3 (ResNet-50, same initialization), compute θt*_i = θ0 − η∇θℓ_t for 100 samples using the SCA learning rate; if that rate is undisclosed, sweep η ∈ {0.001, 0.01, 0.1, 1.0}. Record the loss and gradient norm at θt*_i. If the loss is not near zero or the gradient norm is not near zero (say, loss reduction < 90% or gradient norm > 1% of its initial value), the premise of Theorem 4.2 is violated. Then check the claimed bound J_i ≤ T^3ψ² on the actual losses; a violation demonstrates that the theoretical link from SCA to task grouping does not hold in the tested regime.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The proof of Theorem 4.2 collapses at the step from Eq. 8 to Eq. 9. It requires that the one-step task-specific update θt*_i = θ0 − η∇θℓ_t(f_{θt}(x_i), y_t^i) is a true sample-wise optimum: ℓ_t(f_{θt*_i}(x_i), y_t^i) = 0 and ∇θℓ_t(f_{θt*_i}(x_i), y_t^i) = 0. This is the only way the Taylor inequality reduces to J_i ≤ T^3ψ². But a finite-η gradient step does not generally reach a stationary point, and the manuscript only labels the assumption as 'arguably assumed' (Section 3.2); it is not proved and is false for generic losses. When the residual loss and gradient are nonzero, Eq. 9 contains extra first-order terms, and the density ψθi no longer bounds J_i. Consequently Eq. 15, the theoretical bridge from SCA distances to joint-trainability, is not established. There is a secondary algebraic slip: Eq. 11 writes θ*_i as the unnormalized sum Σ_j θ^{j*}_i instead of (1/T)Σ_j θ^{j*}_i, which changes the constants, but the zero-loss/gradient premise is the more fundamental defect. The empirical SCA results may still be a useful heuristic, but they do not validate the stated theorem.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SCA (sample-wise convergence-based affinity), a task grouping method for multi-task learning and gradient-based meta-learning that avoids training shared models to measure task affinities. It computes, for a small set of samples, one-step gradient updates from a shared initialization and defines pairwise task affinity as the average L1 distance between the resulting task-specific parameter vectors. A graph attention network and Gaussian mixture model clustering then produce task groups. The paper claims a theoretical bound (Theorem 4.2 and Eq. 15) connecting the density of task-specific sample-wise optima to the average multi-task training loss, thereby justifying the use of SCA distances for grouping. Experiments on several datasets (CelebA, QM9, MIMIC-III, ECG waveforms, and image classification benchmarks) show that SCA groupings perform comparably to TAG and HOA while requiring substantially less computation.","tokens_in":22577,"tokens_out":4239,"duration_ms":43089,"significance":"If the theoretical connection were established, this would be a valuable contribution: task grouping without training any shared model would reduce the computational bottleneck of existing methods. The empirical portion is reasonably broad and the reported speedups are concrete; the results on image classification and ECG tasks show SCA matching or slightly exceeding TAG/HOA, which is credible evidence that the heuristic has practical merit. However, the central theoretical claim is not currently supported: the proof of Theorem 4.2 relies on an unproven and generally false premise, and Eq. (11) contains an algebraic error. The paper therefore stands as an empirically motivated heuristic with an unsupported theoretical narrative. That is still a publishable contribution if reframed, but the manuscript as written overclaims the theoretical grounding.","major_comments":[{"comment":"The step from Eq. (8) to Eq. (9) requires that each task-specific sample-wise optimum θ^{t*}_i satisfies ℓ_t(f_{θ^{t*}_i}(x_i), y_i^t) = 0 and ∇θℓ_t(f_{θ^{t*}_i}(x_i), y_i^t) = 0. The manuscript itself only labels this as \"arguably assumed\" in Section 3.2, and it is not proven. For a generic smooth loss and finite learning rate η, a single gradient step does not reach a stationary point, let alone a zero-loss stationary point. When the residual loss and gradient are nonzero, Eq. (9) contains extra first-order and zeroth-order terms, so the density ψ_{θ_i} no longer upper-bounds J_i. Because Eq. (15) is the theoretical bridge from SCA distances to joint trainability, this assumption is load-bearing. I recommend stating explicit sufficient conditions under which the bound holds (e.g., the per-sample loss is a convex quadratic with appropriate scaling and η chosen to reach the minimum), or reformulating Theorem 4.2 as an approximation with explicit residual terms, or deleting the theorem and presenting SCA as a heuristic.","section":"Section 4.1, proof of Theorem 4.2, Eqs. (8)–(9)"},{"comment":"The proof writes θ*_i = Σ_{j=1}^{T} θ^{j*}_i, but Theorem 4.1, specifically Eq. (6), establishes θ*_i = (1/T) Σ_{j=1}^{T} θ^{j*}_i. The missing factor of 1/T changes the subsequent constants: the T^3 factor in the stated bound is not justified even if the zero-loss/gradient premise were granted. Please re-derive the constant with the correct convex combination.","section":"Eq. (11) in the proof of Theorem 4.2"},{"comment":"The claim that \"Theorem 4.1 holds for both\" multi-task learning and gradient-based meta-learning is asserted without proof. For Reptile with a single inner iteration on a single sample, the update reduces to a form similar to Eq. (4), but the actual Reptile update in Algorithm 1 uses inner-loop batches and multiple inner steps, after which the shared optimum of a training unit is not the average of one-step sample-wise optima from a common initial point. Since the meta-learning experiments in Section 6.2 are a stated contribution, this extension needs a proper derivation for the specific algorithm used, or the claim must be restricted to the one-step, one-sample case.","section":"Section 4.1, \"Extension to meta-learning\""}],"minor_comments":[{"comment":"The abstract says the empirical assessment covers 8 datasets, while the Introduction says \"5 distinct datasets\" and Table 1 lists nine datasets (CelebA, QM9, MIMIC-III, MIMIC-III Waveform, CIFAR-10, SVHN, STL-10, Colorectal, Malaria). Please make the counts consistent.","section":"Abstract and Introduction"},{"comment":"The caption contains the typo \"SVNH\" where the intended abbreviation is \"SVHN\".","section":"Figure 3 caption"},{"comment":"The index i is used both for the outer sample summation and for the two task indices in the inner expression, which is confusing. Please use distinct indices, e.g., a_{τ_p,τ_q} = (1/n) Σ_{k=1}^{n} ||θ^{τ_p*}_k - θ^{τ_q*}_k||_1.","section":"Equation (16)"},{"comment":"The citation for gradient-based meta-learning in the Introduction appears as \"[ ?], [11]\"; a placeholder reference is unresolved.","section":"Section 1, reference citation"},{"comment":"The captions for Figures 11 and 12 both say \"(a) CIFAR\" and \"(b) SVHN\", but the text says these figures concern patient care tasks; the labels appear to be copy-paste errors.","section":"Section 6.4, Figures 11 and 12 captions"}],"recommendation":"major_revision","confidential_remarks":"The empirical core is plausible and the computational speedup is real, but the theoretical section currently overreaches. I would ask the authors to either repair Theorem 4.2 with explicit conditions or reposition the paper as a heuristic method with empirical validation. The missing factor in Eq. (11) and the meta-learning extension also need attention. I do not see grounds for rejection, because the empirical claims are separable from the flawed proof."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know. First, the sample-wise convergence-based affinity (SCA) metric is genuinely new and useful: it computes task affinities from single gradient steps at initialization plus graph clustering, with no shared-model training. Empirically, it matches TAG and HOA on most of the eight datasets, often at lower compute, and the GAT plus GMM grouping is a sensible replacement for branch-and-bound. That is a real contribution. Second, the theory meant to justify it is not sound. Theorem 4.2 requires that one gradient step from θ0 with a finite learning rate lands on a zero-loss, zero-gradient sample optimum. That is false for generic losses and is exactly the property needed to get the bound J_i ≤ T^3 ψ^2. The paper labels this 'arguably assumed' in Section 3.2, but it is load-bearing. Without it, Equation 15 does not connect SCA distances to joint trainability. There is also a minor algebraic slip: Equation 11 writes θ*_i as an unnormalized sum instead of the average; the constants change, but the zero-loss/zero-gradient premise is the fundamental defect.\n\nWhat the paper does well is the empirical story. A hundred samples suffice, relative affinities stay stable across training and sample size, and the consistency check against TAG on QM9 pairs is informative. The limitations section is honest about memory costs and about affinities not being semantic similarity. The soft spots beyond the theory: the abstract's 'five-fold speed enhancement' is not supported by the tables; on CelebA the savings over TAG are closer to 1.5x. The comparison against single-model MTL baselines is confounded because SCA evaluates with two models. And there is no released code, no reported SCA learning rate, and several GAT/GMM hyperparameters are left to the supplement without full values. All of that is fixable.\n\nWho this is for: anyone doing task grouping in MTL or meta-learning who wants a cheap, empirically validated heuristic. The paper deserves a serious referee, but the referee should insist that the theory be relabeled as heuristic or properly fixed, that the speedup claim be corrected, and that code or detailed parameter values be released. My verdict would be conditional acceptance on the empirical method, not on the theorem.","headline":"Cheap and often effective task-affinity heuristic, but the proof that supposedly backs it does not establish the claimed link between SCA distances and joint trainability.","tokens_in":23199,"tokens_out":3037,"would_cite":true,"duration_ms":30099,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","68T05"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proposes SCA, an inter-task affinity score computed from one-step gradient updates on a randomly initialised shared model, and claims it identifies effective task groupings for multi-task learning and gradient-based…","keywords":["task grouping","multi-task learning","negative transfer","sample-wise optimisation landscape","task affinity","graph attention networks","gradient-based meta-learning","SCA scores"],"falsifier":"Train pairs of tasks fully from multiple random initialisations and measure the actual joint-training loss, then compare that ordering with SCA affinity; if SCA ranks pairs opposite to the measured losses in a setting where the gradient norm after one step is far from zero, the theorem's premise fails.","tokens_in":22054,"feed_emoji":"🧩","tokens_out":8964,"duration_ms":83969,"temperature":0.7,"pith_summary":"Task grouping—deciding which tasks to train together—is normally expensive because it requires training candidate multi-task models or measuring gradient effects inside a shared model. This paper claims the decision can be made before any shared training: from a random initialisation, take one gradient step per task on each of a few hundred samples, and measure the average $\\ell^1$ distance between the points reached. Those distances, called sample-wise convergence-based affinity (SCA) scores, act as task affinities, and a graph-attention plus clustering module turns them into task groups. If the claim is right, task grouping becomes a cheap preprocessing step, and the reported experiments on eight datasets show accuracy comparable to or better than state-of-the-art methods with roughly five times less GPU time.","feed_headline":"One gradient step picks which tasks train well together","feed_subtitle":"SCA scores match or beat expensive baselines on eight datasets while cutting GPU time five-fold.","key_machinery":"The load-bearing construction is the sample-wise optimisation landscape: each training sample is split into $T$ task-specific learning problems, and one gradient step from a random shared initialisation $\\theta_0$ yields task-specific 'sample-wise optima' $\\theta^{t*}_i$. Theorem 4.1 shows the shared optimum is their average, and Theorem 4.2 bounds the joint loss by the density $\\psi_{\\theta_i}$ of their pairwise $\\ell^1$ distances. The pairwise distance, averaged over samples, is the SCA score that weights the edges of a task graph; a two-layer graph attention network learns node embeddings that encode higher-order task interactions, and Gaussian mixture clustering on those embeddings produces the final task groups.","core_discovery":"The paper's central claim is that the average multi-task training loss is controlled by the density of one-step task-specific sample-wise optima. Writing $\\theta^{t*}_i = \\theta_0 - \\eta \\nabla_\\theta \\ell_t$ for the point reached by one gradient step on task $t$ and sample $i$, the shared optimum is $\\theta^*_i = \\frac{1}{T}\\sum_t \\theta^{t*}_i$, and Theorem 4.2 gives $J_i \\le T^3 \\psi_{\\theta_i}^2$ for the per-sample average loss, where $\\psi_{\\theta_i}$ is the pairwise $\\ell^1$ scatter of the task-specific optima. Over the whole dataset the bound becomes $J \\le \\frac{T^3}{2n}\\sum_{i=1}^n \\psi_{\\theta_i}^2 + \\frac{n^3}{2}\\Psi_{\\theta^*}^2$ (Eq. 15). The paper therefore proposes pairwise $\\ell^1$ distances between one-step optima, averaged over samples (Eq. 16), as SCA affinities that identify good task groupings without training a shared model. Experiments report that these affinities are stable during training and across sample sizes, and that groupings found this way match or beat HOA and TAG on eight datasets while cutting GPU time roughly five-fold.","pith_inferences":["If the one-step-is-an-optimum assumption is relaxed, SCA can still serve as a first-order proxy for gradient conflict; a cheap probe would be computing SCA after several gradient steps and checking whether task rankings change.","Because SCA depends on the architecture and initialisation, it could double as a tool for choosing which shared architecture to use for a fixed task set, not just which tasks to group.","The memory bottleneck of comparing full parameter vectors, acknowledged by the authors, could be removed with layer-wise or projected distances, making SCA practical for very large models.","The same affinity signal could be used outside static grouping, for example to order tasks in a curriculum or to select collaborators in federated learning where full joint training is too costly."],"forward_implications":["Task affinities can be computed from a randomly initialised network with one forward and backward pass per sampled example, before any shared model is trained.","The same SCA computation applies to gradient-based meta-learning: for one inner Reptile step the shared optimum is again an average of task-specific optima, so grouping can be done inside meta-learning pipelines.","Grouping becomes a graph-clustering problem rather than a search over $2^{|T|}$ model combinations, sidestepping the NP-hard affinity-maximisation formulation.","The reported five-fold GPU-time reduction is accompanied by accuracy that matches or exceeds HOA and TAG across QM9, CelebA, MIMIC-III, ECG waveform, and image-classification tasks.","Only about 100 samples per task are needed to stabilise the SCA scores, so the preprocessing cost does not grow with dataset size."],"supporting_citations":[{"why":"Supplies the sample-wise optimisation landscape assumptions: one gradient step can reach a sample-wise local optimum, and density of these optima bounds training error.","marker":"[18]"},{"why":"Defines the original task-grouping problem and HOA baseline, which trains multi-task models for task pairs; the proposed method is compared against its cost and solution quality.","marker":"[10]"},{"why":"Introduces TAG, the state-of-the-art baseline that infers task affinities by training a single multi-task model and observing per-task gradient effects; also supplies the CelebA model.","marker":"[14]"},{"why":"Provides the graph attention network used to turn pairwise SCA scores into higher-order node embeddings for clustering.","marker":"[19]"},{"why":"Provides the Reptile algorithm used in meta-learning experiments, and the one-inner-iteration setup under which the SCA extension is derived.","marker":"[11]"},{"why":"Bridges multi-task learning and meta-learning, supporting the claim that the shared-optimum averaging result (Theorem 4.1) holds in both settings.","marker":"[13]"},{"why":"Supplies the LSTM-based patient-care model and the Reptile-based joint training framework used in the clinical experiments.","marker":"[12]"},{"why":"Extends sample-wise optimisation analysis to neural initialisation, supporting the use of density of sample-wise optima as a predictor of optimisation behaviour.","marker":"[29]"}],"fun_headline_variants":["One-step gradient distances pick task groups, no shared training","Task affinities from single gradient steps, 5x faster grouping","Sample-wise landscape analysis finds task groups in one pass","Gradient-step scatter based grouping matches costly baselines","Efficient task grouping via one-step optimization geometry"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The theory assumes that one gradient step from the random initialisation already puts the model at a true sample-wise optimum, where the loss and its gradient are both zero; with a finite learning rate, one update generally does not reach such a point.","fun_headline_variants_meta":{"raw":{"variants":["One-step gradient distances pick task groups, no shared training","Task affinities from single gradient steps, 5x faster grouping","Sample-wise landscape analysis finds task groups in one pass","Gradient-step scatter based grouping matches costly baselines","Efficient task grouping via one-step optimization geometry"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000235,"raw_usage":{"total_tokens":1543,"prompt_tokens":1031,"completion_tokens":512,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":647,"completion_tokens_details":{"reasoning_tokens":433}},"tokens_in":647,"tokens_out":512,"duration_ms":6072,"temperature":1.0,"reasoning_tokens":433,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T21:24:25.966157+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train pairs of tasks fully from multiple random initialisations and measure the actual joint-training loss, then compare that ordering with SCA affinity; if SCA ranks pairs opposite to the measured losses in a setting where the gradient norm after one step is far from zero, the theorem's premise fails.","supporting_citations":[{"cited_title":"Gradsign: Model performance inference with theoretical insights,","cited_arxiv_id":null,"evidence_quote":"Supplies the sample-wise optimisation landscape assumptions: one gradient step can reach a sample-wise local optimum, and density of these optima bounds training error."},{"cited_title":"Which tasks should be learned together in multi-task learning?","cited_arxiv_id":null,"evidence_quote":"Defines the original task-grouping problem and HOA baseline, which trains multi-task models for task pairs; the proposed method is compared against its cost and solution quality."},{"cited_title":"Efficiently identifying task groupings for multi-task learning,","cited_arxiv_id":null,"evidence_quote":"Introduces TAG, the state-of-the-art baseline that infers task affinities by training a single multi-task model and observing per-task gradient effects; also supplies the CelebA model."},{"cited_title":"Bridging multi-task learning and meta-learning: Towards efficient training and effective adapta- tion,","cited_arxiv_id":null,"evidence_quote":"Bridges multi-task learning and meta-learning, supporting the claim that the shared-optimum averaging result (Theorem 4.1) holds in both settings."},{"cited_title":"Dynamic neural graphs based federated reptile for semi-supervised multi-tasking in healthcare applications,","cited_arxiv_id":null,"evidence_quote":"Supplies the LSTM-based patient-care model and the Reptile-based joint training framework used in the clinical experiments."},{"cited_title":"Towards theoretically inspired neural initialization optimization,","cited_arxiv_id":null,"evidence_quote":"Extends sample-wise optimisation analysis to neural initialisation, supporting the use of density of sample-wise optima as a predictor of optimisation behaviour."}],"review_version":1}