{"id":"c1cc0449-a764-4a71-b3cb-29f4a975eda2","arxiv_id":"2509.06575","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"A thresholded spectral method for representation-based multi-task learning that provably avoids negative transfer under unknown and large task contamination.","lead":"This paper introduces RAS, a spectral method for multi-task learning that learns a shared representation even when a large, unknown fraction of tasks are outliers or corrupted. It provides non-asymptotic error bounds that stay at or below single-task error, preventing negative transfer.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The theoretical τ in Theorem 9 depends on unknown ε, h, and signal strength, while the implemented τ is a fixed heuristic; for h>0 the required domination ∥N/√T∥_op<τ/4 is not shown, so the claimed adaptivity over single-task learning is not guaranteed by the actual algorithm.","rationale":"The paper's core idea is attractive: threshold singular values by a noise floor and rely on a biased-regularization fallback to avoid negative transfer. The fallback mechanism in Lemma 15 is plausible and gives some independent support for the no-worse-than-single-task part of the claim, and the h=0 experiments are extensive and consistent with the theory. My main concern is a calibration gap between the oracle threshold used in the theorems and the fixed heuristic threshold used in the implementation. The theoretical τ includes unknown quantities such as ε, h, and signal strengths; the paper does not prove that the implemented τ satisfies the required domination when h>0, and the experiments do not cover h>0. If the threshold is too small, the rank-selection and subspace-error guarantees in Propositions 4 and 7 can fail, which removes the adaptive-rate branch of Theorem 9. The no-negative-transfer fallback may still hold, but the paper's advertised adaptivity to heterogeneous inliers is then unsupported. The proof of Proposition 7 also contains an identity that requires the coefficient matrix Θ∗_S to be the effective coefficient matrix A⊤B∗_S rather than the original task-specific coefficients; this is a repair rather than a refutation, but it reinforces that the h>0 case is not fully nailed down. These issues justify the reader's CONDITIONAL verdict; I see no basis for outright rejection, because the h=0 analysis and the fallback logic appear sound and the experiments support the method in that regime.","tokens_in":30347,"tokens_out":24838,"duration_ms":228047,"concrete_test":"Run the Section 5.1 data-generating process with heterogeneous inliers: draw each A(t)∗ with principal angle h ∈ {0, 0.1, 0.3, 0.5} from a central A (Eq. (2)), keep the outlier construction, and use T=100, p=80, n=150, r=10, r_out=10, ε=0.5. For each h, compute the oracle perturbation norm ∥N/√T∥_op from the true β∗, A, and realized single-task estimates, and compare it with τ/4 for the implemented τ=2.5√((p+T)/(nT)). If ∥N/√T∥_op ≥ τ/4 at h=0.5, Proposition 4's premise fails. Then run the full RAS at that h and check whether max_{t∈S}∥β̂(t)−β(t)∗∥ exceeds both Theorem 9's first-term rate and the single-task rate; exceeding single-task would falsify the practical no-negative-transfer claim, while exceeding only the first-term rate would falsify the adaptivity claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The adaptive part of Theorem 9 is proved only for a threshold τ that is proportional to ∥N∥_op, which depends on the unknown contamination proportion ε, the heterogeneity h, and the inlier signal strengths (Section 2.2 and Proposition 4). The implemented RAS uses the fixed value τ = 2.5√((p+T)/(nT)) (Section 5), with no demonstration that this heuristic dominates ∥N∥_op when h>0 or when inlier signals are weak. For h=0 the heuristic is conservative because it replaces |S| by T, but for h>0 the required τ includes an additive term of order h\\barζ/√(1−ε) [σ_max(D∗_S)/(√r σ_min(D∗_S)) ∧ 1]. When this term is non-negligible, the premise ∥N/√T∥_op < τ/4 of Proposition 4 can fail, so the rank-selection step may discard weak inlier directions or retain adversarial directions, and the subspace error bound of Proposition 7 — the basis of the adaptive first branch of Theorem 9 — is not guaranteed. The no-negative-transfer fallback via Lemma 15 may still bound the error at the single-task rate, but the claimed better-than-single-task adaptivity is not delivered by the actual algorithm in exactly the heterogeneous regime the paper advertises. The experiments only run h=0 (§5), leaving this gap untested. A related proof gap appears in Proposition 7: the identity (bA^⊥)⊤eB_S = (bA^⊥)⊤A Θ∗_S holds only if Θ∗_S is redefined as A⊤B∗_S rather than the θ(t)∗ defined through β(t)∗ = A(t)∗θ(t)∗; without that clarification, the h>0 case is not fully established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies representation-based multi-task learning when an unknown, possibly large fraction of tasks are contaminated, and inlier tasks may have heterogeneous representations. It proposes the Robust and Adaptive Spectral method (RAS): fit single-task regressions, form the coefficient matrix, perform an SVD with a data-driven singular-value threshold to select the shared subspace and its dimension, and then apply a biased regularization step that anchors per-task estimates to the learned subspace. The main theoretical results are non-asymptotic bounds on the subspace estimation error and per-task coefficient errors, a guarantee that the method performs no worse than single-task learning via an explicit minimum with the single-task rate, and an extension to transfer learning. The experiments cover low-rank and general-rank outliers, contamination proportions up to 80%, the effect of winsorization, and scaling with the number of tasks.","tokens_in":30811,"tokens_out":6467,"duration_ms":58314,"significance":"If the theoretical claims hold, this is a valuable contribution: RAS requires neither the contamination proportion nor the true representation dimension, handles arbitrary outlier tasks, and its no-negative-transfer guarantee is a practically important improvement over methods whose error bounds degrade with the contamination level. The paper provides detailed proofs in the appendix, and the experimental section is extensive, including comparisons against oracle methods that know the true rank and the contamination proportion. The conceptual idea of using an adaptive threshold calibrated to the perturbation level, and the explicit single-task fallback via the min in Theorem 9, are strengths. However, the central adaptivity claim is currently established only for an oracle threshold that depends on unknown quantities, and the proof of the subspace bound has a gap in the heterogeneous-representation case, so the advertised adaptivity in the h>0 regime is not yet fully supported.","major_comments":[{"comment":"The theoretical threshold τ is defined as τ ≍ sqrt((p+|S|)/(nT)) + h \\barζ / sqrt(1-ε) · (σ_max(D*_S)/(√r σ_min(D*_S)) ∧ 1), which depends on the unknown contamination proportion ε, the heterogeneity h, and inlier signal strengths. The proof of Proposition 4 establishes the required domination ∥N/√T∥_op < 0.25τ only for this oracle choice. In Section 5 the implementation uses the fixed heuristic τ = 2.5√((p+T)/(nT)), which dominates the first term (since T ≥ |S|) but omits the h-dependent heterogeneity term. Consequently, when h > 0 and that term is non-negligible, the premise of Proposition 4, and therefore the rank-selection guarantee, the subspace bound of Proposition 7, and the adaptive first branch of Theorem 9, are not shown to hold for the actual algorithm. Since all experiments in Section 5 use h = 0, this gap is not probed empirically. The authors should either prove that the heuristic threshold satisfies the domination condition under explicit conditions, or provide a genuinely data-dependent threshold that adapts to the unknown heterogeneity and contamination level, with a matching theory.","section":"Section 2.2, Proposition 4, Theorem 9"},{"comment":"The proof of Proposition 7 claims the identity (bA^⊥)^T eB_S = ((bA^⊥)^T A) Θ*_S and then lower-bounds the left-hand side by ∥(bA^⊥)^T A∥_op · σ_r(Θ*_S) using Assumption 2. However, by the definitions in Section 2.1, eB_{:,t} = A A^T β(t)* = A A^T A^{(t)*} θ^{(t)*} for t ∈ S, so the matrix on the left is (bA^⊥)^T A · [A^T A^{(t)*} θ^{(t)*}]_{t∈S}, not (bA^⊥)^T A · [θ^{(t)*}]_{t∈S}, unless A^{(t)*} = A for every inlier task. The smallest singular value of the matrix with columns A^T A^{(t)*} θ^{(t)*} depends on the principal angles between A and the individual A^{(t)*} and can be substantially smaller than σ_r(Θ*_S). Thus the displayed subspace error bound does not follow for h > 0, and the ℓ₂ rate in Theorem 9 that relies on this bound is not established in the heterogeneous setting. Please either redefine Θ*_S to include the rotations A^T A^{(t)*} and derive the resulting bound with explicit dependence on h, or state Proposition 7 and the adaptive branch of Theorem 9 only for h = 0 with a separate treatment for the general case.","section":"Appendix A.3, Proposition 7"},{"comment":"The adaptive first rate in Theorem 9 uses k⋆ = min{r + r_out, r + Tε, p, T} as an upper bound on the estimated rank k̂, but r_out is not defined in the statement of Theorem 9. Moreover, the bound k̂ ≤ r + r_out − r∩ in Lemma 6 requires the outlier coefficient matrix (bBst)_{Sc} to decompose as a rank-r_out matrix plus a perturbation of operator norm at most (3/4)τ. This low-rank-plus-small-perturbation condition is not among the assumptions of Theorem 9, which only states that outlier tasks follow an arbitrary distribution. Without it, k̂ can be as large as min(p,T), and the claimed adaptivity to outlier structure is not proven in general; only the single-task fallback in the second branch of the minimum is unconditional. The authors should either explicitly incorporate the low-rank-outlier condition into Theorem 9, or present the adaptive rate as conditional on Lemma 6 and clarify what, if anything, is guaranteed for general-rank outliers.","section":"Theorem 9 and Lemma 6"}],"minor_comments":[{"comment":"There are several typos: \"theoretical ganrantees\" in Section 1.2, \"kernal\" in the related literature, and \"varing\" in Section 5.4; these should be corrected.","section":"Section 1.2, Section 5.4"},{"comment":"Several axis labels in the figures appear as garbled Unicode escape sequences (for example \"/uni00000013/uni00000011/...\"), making the labels unreadable; the figures should be regenerated with proper text rendering.","section":"Figures 1-6"},{"comment":"The expression for the oracle threshold τ is given in prose rather than as a numbered equation; placing it in a displayed, numbered equation would make the cross-references in Section 3 and the appendix much easier to follow.","section":"Section 2.2"},{"comment":"The notation B*_S = {β(t)*} and B_S = {A A^T β(t)*} is ambiguous because B_S is later used as a matrix; using distinct symbols for the set of vectors and the matrix would improve clarity.","section":"Section 2.1"},{"comment":"The spectral gap assumption is stated as eλ_{r_eff+1} < 0.75τ while the effective signal rank is defined with the level 1.25τ; a remark explaining how these constants relate would help the reader understand the role of the gap condition.","section":"Proposition 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript relies heavily on lemmas from Tian et al. (2023), which is co-authored by one of the present authors; those lemmas are published elsewhere and correctly cited, so this is not disqualifying, but it means the genuinely new technical content is concentrated in the thresholding and subspace analysis, and that is exactly where the h>0 proof gap lies. It may be prudent to involve a reviewer who is not affiliated with that group. The paper's empirical claims are solid for the h=0 case, but the advertised adaptivity to heterogeneous inlier representations is not yet supported by the current proofs."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look if you work on robust MTL or transfer learning, but read it as an engineering-plus-theory paper, not a finished theorem paper. The core idea is genuinely useful: threshold the singular values of the single-task estimate matrix, and you can recover a shared subspace even when a large fraction of tasks are contaminated, without knowing the contamination level or the true rank. The experiments for h=0 are convincing, including up to 80% contamination and a nice demonstration of when winsorization hurts. The extension over Tian et al. (2023) is real: they drop the small-contamination restriction, remove the oracle rank, and allow heterogeneous inlier representations. The adaptive thresholding analysis is the main new technical work.\n\nThe soft spots are two. First, the theory's threshold depends on unknown epsilon, h, and signal strength, while the implementation uses a fixed heuristic tau = 2.5 sqrt((p+T)/(nT)). For h=0 the heuristic is conservative, but for h>0 the required tau has an additive heterogeneity term, and the paper never shows the heuristic dominates the noise in that regime. All experiments are h=0, so the advertised adaptivity in the heterogeneous case is not actually delivered by the algorithm as implemented. The no-negative-transfer fallback still holds, but the better-than-single-task guarantee is only proven for a threshold you don't use.\n\nSecond, and more serious, Proposition 7 has a questionable step. The identity (A_hat^perp)^T B_tilde_S = (A_hat^perp)^T A Theta*_S holds only if Theta*_S is redefined as A^T B*_S, not as the original theta(t)* defined through beta(t)* = A(t)* theta(t)*. As written, the lower bound via sigma_r(Theta*_S) does not follow when A(t)* differs from A. This looks fixable by restating the diversity assumption in the central basis A, but as it stands the h>0 case is not fully proved.\n\nFor whom: statisticians and ML theorists working on robust representation learning, and practitioners who want a tuning-light baseline for contaminated task collections. The paper deserves a serious referee: the method is sound, the empirical work is honest, and the gaps are addressable in revision. Send it out, but ask the authors to fix Proposition 7 and to either adapt the threshold to h>0 or state clearly that the adaptive guarantee is for h=0 only.","headline":"A useful adaptive spectral method for contaminated MTL with a promising h=0 theory, but the h>0 case has a proof gap and the implemented threshold isn't shown to satisfy the theoretical requirement.","tokens_in":31299,"tokens_out":2966,"would_cite":true,"duration_ms":27285,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62J05","62H25","68T05"],"pacs":[],"model":"deepseek-v4-flash","headline":"A thresholded spectral method learns a shared multi-task representation even when an unknown, large fraction of tasks is contaminated, and its per-task estimates never fall below single-task accuracy.","keywords":["multi-task learning","representation learning","robust spectral method","contaminated tasks","adaptive rank selection","negative transfer","transfer learning","non-asymptotic error bounds"],"falsifier":"Run RAS with its fixed threshold $\\tau = 2.5\\sqrt{(p+T)/(nT)}$ in a regime inside the paper's assumptions where inlier heterogeneity $h$ is nonzero and inlier signals are near the signal-to-noise threshold, and check whether the selected rank $\\hat{k}$ misses inlier directions or admits outlier directions and whether the inlier error exceeds $\\sqrt{(p+\\log T)/n}$ with high probability. Exhibiting such an instance would show the implemented threshold violates the inequality that Proposition 4 and Theorem 9 require.","tokens_in":30141,"feed_emoji":"🛡️","tokens_out":11951,"duration_ms":86856,"temperature":0.7,"pith_summary":"Multi-task learning promises statistical gains by sharing a low-dimensional representation across related tasks, but those gains vanish when a sizable share of tasks is corrupted or adversarial. This paper argues that a simple spectral procedure keeps the gains: fit each task separately, stack the coefficient estimates, keep only the singular directions above a data-driven threshold, then shrink per-task estimates toward the recovered subspace. The central theorem states that, with high probability, every inlier task's estimation error is at most the smaller of a multi-task bound and the single-task rate $\\sqrt{(p+\\log T)/n}$ — so the method provably never does worse than learning tasks in isolation, for any contamination proportion below 1 and without knowing the representation dimension. The same construction extends to transfer learning, giving the target task the same safety guarantee against single-task regression.","feed_headline":"Multi-task learning never worse than single-task at 80% contamination","feed_subtitle":"An adaptive singular-value threshold keeps shared representations useful even when most tasks are outliers.","key_machinery":"The load-bearing object is the perturbation matrix $N = \\widehat{B}_{\\mathrm{st}} - \\widetilde{B}$, the difference between the single-task coefficient estimates and the ideal signal matrix, together with the threshold $\\tau \\asymp \\|N\\|_{\\mathrm{op}}$ that cuts the singular values of $\\widehat{B}_{\\mathrm{st}}/\\sqrt{T}$. The threshold performs two jobs at once: it filters out contamination and noise while selecting, without knowing the rank $r$, only directions whose signal clears the noise-and-heterogeneity floor. The minimum with the single-task rate in Theorem 9 is what converts thresholding into a no-negative-transfer guarantee, and a final biased-regularization step anchors per-task estimates to the learned subspace, falling back to the single-task rate when the multi-task bound is unfavorable.","core_discovery":"The paper claims that unknown, arbitrarily structured task contamination does not have to break representation-based multi-task learning. RAS estimates the shared subspace from the leading singular vectors of the single-task coefficient matrix after an adaptive singular-value threshold chosen to dominate the perturbation matrix $N$ (estimation noise plus inlier heterogeneity plus outlier contamination). With the threshold set that way and a mild spectral gap, the estimated rank equals an effective signal rank and the subspace error is controlled by the inlier diversity $\\sigma_{\\min,\\mathrm{in}}$, the heterogeneity $h$, and the contamination fraction $\\varepsilon$. The central result, Theorem 9, bounds each inlier task's $\\ell^2$ estimation error by the minimum of a multi-task rate depending on the estimated rank and the single-task rate $\\sqrt{(p+\\log T)/n}$, which is what guarantees no negative transfer; the only requirement on contamination is $\\varepsilon < 1$, with no prior knowledge of $\\varepsilon$ or $r$.","pith_inferences":["The min-with-single-task structure makes RAS an automatic switch: a user can deploy it without deciding beforehand whether tasks are worth sharing, because in any regime where sharing fails the bound collapses to the single-task rate.","Because the provably valid threshold depends on unknown quantities ($\\varepsilon$, $h$, signal strengths) while the implemented threshold is a fixed heuristic, a natural testable extension is data-driven threshold calibration — for example sample-splitting or bootstrap on singular-value gaps — that would make the finite-sample guarantee fully parameter-free.","The estimated rank $\\hat{k}$ could double as a diagnostic of contamination structure: values near $r$ indicate low-rank outliers below the noise floor, while values near $r + r_{\\mathrm{out}}$ reveal strong outlier signals that the threshold accepted.","Since the argument only requires $\\tau$ to dominate $\\|N\\|_{\\mathrm{op}}$, and the paper notes this reasoning is model-agnostic, the same recipe plausibly extends to nonlinear losses whenever a concentration bound for the single-task estimates is available."],"forward_implications":["Every inlier task keeps an estimation error of at most the single-task rate $\\sqrt{(p+\\log T)/n}$ with high probability, so representation sharing provably cannot hurt, no matter the contamination fraction (as long as it is below 1) and with no knowledge of the true rank.","RAS needs neither the contamination proportion nor the intrinsic dimension as input, unlike the spectral method it benchmarks against; the threshold selects the rank automatically.","Low-rank outlier structure is exploited: the estimated rank satisfies $\\hat{k} \\le r + r_{\\mathrm{out}} - r_{\\cap}$, improving the bound, while general-rank outliers degrade the bound smoothly instead of breaking the method.","In transfer learning, the target-task error is the minimum of the single-task rate $\\sqrt{p/n_0}$ and a representation-based rate, so reusing the learned representation is safe.","Under standard scaling ($\\zeta^{(t)}\\asymp 1$, $\\bar{\\zeta}\\asymp 1$, balanced outlier spectrum), RAS beats single-task learning when $r \\ll |S| \\wedge p$, the outlier dimension or mass is small relative to $p$, and inlier heterogeneity $h$ is small."],"supporting_citations":[{"why":"Supplies the spectral-method baseline (SM) that requires oracle knowledge of $r$ and $\\varepsilon$, the comparison whose bound degrades with contamination, and the technical lemmas reused in the biased-regularization proofs.","marker":"Tian et al. (2023)"},{"why":"Defines the exact shared-representation setting ($h=0$) that RAS generalizes, and the benchmark rates and task-diversity assumptions used throughout.","marker":"Du et al. (2020)"},{"why":"Provides provable meta-learning of linear representations and the theory of task diversity that justifies the inlier assumptions.","marker":"Tripuraneni et al. (2021)"},{"why":"Adaptive and robust multi-task learning with heterogeneous but outlier-free inliers; the comparison point for biased regularization and adaptation to $h$.","marker":"Duan and Wang (2023)"},{"why":"Collaborative learning with shared linear representations; source of the $\\sigma_{\\min,\\mathrm{in}} \\ge c\\sqrt{r}\\,\\bar{\\zeta}$ scaling used in Corollaries 8, 10 and 12.","marker":"Niu et al. (2024)"},{"why":"Theorem 5.39 supplies the sub-Gaussian concentration used to bound the single-task regression perturbation in Lemma 13.","marker":"Vershynin (2010)"},{"why":"Theorem 6.5 supplies the covariance concentration used in Lemma 14 to control the sample-covariance error.","marker":"Wainwright (2019)"},{"why":"Perturbation theory for singular subspaces underlying the principal-angle measure of inlier similarity in condition (2).","marker":"Wedin (1972)"}],"fun_headline_variants":["Spectral MTL thrives even when 80% of tasks are outliers","Adaptive threshold shields MTL from up to 80% contamination","No negative transfer: robust MTL matches single-task worst case","Handles unknown contamination levels without prior knowledge","Outlier-robust representation learning for multi-task models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The guarantees require the singular-value threshold to be at least as large as the worst-case perturbation that contamination, heterogeneity, and noise create in the coefficient matrix, and the threshold that provably works depends on the contamination fraction, the spread among inlier tasks, and their signal strengths — none of which are known; the fixed threshold used in practice is not shown to be large enough in those harder regimes.","fun_headline_variants_meta":{"raw":{"variants":["Spectral MTL thrives even when 80% of tasks are outliers","Adaptive threshold shields MTL from up to 80% contamination","No negative transfer: robust MTL matches single-task worst case","Handles unknown contamination levels without prior knowledge","Outlier-robust representation learning for multi-task models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000276,"raw_usage":{"total_tokens":1637,"prompt_tokens":927,"completion_tokens":710,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":543,"completion_tokens_details":{"reasoning_tokens":626}},"tokens_in":543,"tokens_out":710,"duration_ms":6165,"temperature":1.0,"reasoning_tokens":626,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T16:17:21.895880+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run RAS with its fixed threshold $\\tau = 2.5\\sqrt{(p+T)/(nT)}$ in a regime inside the paper's assumptions where inlier heterogeneity $h$ is nonzero and inlier signals are near the signal-to-noise threshold, and check whether the selected rank $\\hat{k}$ misses inlier directions or admits outlier directions and whether the inlier error exceeds $\\sqrt{(p+\\log T)/n}$ with high probability. Exhibiting such an instance would show the implemented threshold violates the inequality that Proposition 4 and Theorem 9 require.","supporting_citations":[{"cited_title":", Jin, C","cited_arxiv_id":null,"evidence_quote":"Provides provable meta-learning of linear representations and the theory of task diversity that justifies the inlier assumptions."},{"cited_title":"and Wang, K","cited_arxiv_id":null,"evidence_quote":"Adaptive and robust multi-task learning with heterogeneous but outlier-free inliers; the comparison point for biased regularization and adaptation to $h$."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Theorem 6.5 supplies the covariance concentration used in Lemma 14 to control the sample-covariance error."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Perturbation theory for singular subspaces underlying the principal-angle measure of inlier similarity in condition (2)."}],"review_version":1}