{"id":"da81574d-b3d4-4c62-b53b-2331238d4165","arxiv_id":"2602.14231","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Tasks are grouped by cross-task prediction errors, and each group gets its own boosting ensemble, beating baselines on synthetic and real tabular benchmarks.","lead":"A new multi-task learning method, RMB-CLE, clusters tasks by how well each task's model predicts other tasks' data, then trains one boosting ensemble per cluster. It reports gains over single-task, pooling, and multi-task boosting baselines on tabular benchmarks, though not on every dataset.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Cross-task similarity is contaminated by target-task irreducible noise and by diagonal in-domain errors, so clustering may reflect task difficulty rather than functional relatedness.","rationale":"The paper's strongest claim—that RMB-CLE derives inter-task similarity from cross-task errors in a theoretically grounded way that mitigates negative transfer—rests on the assumption that E_{i,j} captures functional mismatch rather than task difficulty. Equation (23) explicitly shows a target-specific irreducible noise term that is independent of the source model. When this noise is large, the similarity vector for that task becomes dominated by a constant offset; cosine distance then groups tasks by noise level rather than by functional relatedness. The inclusion of diagonal entries E_{i,i} (in-sample training errors) is a direct contradiction of the paper's own Remark and further biases the geometry toward task difficulty. This is not merely a theoretical subtlety: it affects the core clustering mechanism that the entire method depends on. The paper's synthetic evaluation uses no added noise, so this issue would not surface there; real-world datasets likely have heterogeneous noise, so the reported improvements may be attributable to noise-based isolation rather than functional clustering. The reader identified exactly this weakest assumption, and the paper's own conclusions admit the lack of difficulty normalization. Thus the concern is load-bearing, but it is addressable (e.g., by removing the diagonal and normalizing by task difficulty). The reader's CONDITIONAL verdict remains appropriate; no stronger adjustment is needed.","tokens_in":33232,"tokens_out":6704,"duration_ms":72253,"concrete_test":"Generate synthetic MTL data with C=5 known functional clusters (as in §4.1, ω=0.9) but add heteroscedastic Gaussian noise: assign each task a σ_i^2 drawn from, e.g., {0.01, 0.5, 2.0} independent of cluster. Run the full RMB-CLE pipeline exactly as written (including diagonal, no normalization) and compute adjusted Rand index between inferred and ground-truth clusters over 100 seeds. Then rerun with (a) diagonal removed (set s_{i,i}=0) and (b) difficulty-normalized similarities (e.g., s_{i,j}=1/(E_{i,j}-E_{i,i}+ε)). If the original's ARI is near chance while (a) or (b) recovers the true clusters, the concern is confirmed; if the original already recovers clusters, the noise contamination is not load-bearing for the core mechanism.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The framework's central justification is that cross-task errors E_{i,j} (Eq. 5) measure functional compatibility. But Eq. (23) decomposes the population risk as E[(η_i-F_j)^2] + E[σ_i^2], where the second term is the target task's irreducible noise. Since the similarity is s_{i,j}=1/(E_{i,j}+ε), this noise term adds a roughly constant offset to every entry of row i of the similarity matrix. Cosine distance (Eq. 8) is not invariant to row-wise additive constants: a very noisy task has an almost constant profile, so it becomes cosine-similar to other noisy tasks and cosine-distant from cleaner tasks even when their underlying functions are close. The method thus partly clusters by noise level, not functional relatedness. The problem is compounded by including the diagonal E_{i,i} in the similarity vector (Eq. 7). E_{i,i} is the in-domain training error, which the paper's own Remark says reflects only task difficulty and should not be used. For a well-fit model this diagonal dominates the vector. The paper acknowledges the need for difficulty normalization in the Conclusions ('Another promising direction is to refine cross-task similarity estimation by accounting for task difficulty'), but the presented theory and algorithm do not do so. Until this is addressed, the claim that cross-task errors provide a 'principled measure of functional similarity' is not supported, and the reported gains on real-world data could stem from implicitly isolating noisy tasks rather than recovering functional clusters.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RMB-CLE, a multi-task boosting framework that estimates inter-task similarity from cross-task generalization errors (Eq. 5), converts these errors to similarities (Eq. 6), embeds tasks as similarity-profile vectors, clusters them with agglomerative average-linkage clustering and silhouette-based model selection, and finally trains cluster-specific LGBM or MTGB ensembles. The authors claim that cross-task errors admit a risk decomposition into functional mismatch and irreducible noise (Section 3.3), that the resulting clusters recover ground-truth structure, and that RMB-CLE consistently outperforms single-task, pooling, and existing multi-task boosting baselines on synthetic and real-world benchmarks.","tokens_in":33619,"tokens_out":6134,"duration_ms":68762,"significance":"If the central claim were supported, the framework would be a useful, model-agnostic addition to the multi-task boosting literature: it is simple, computationally characterized, and ships with public code and data. The empirical protocol is extensive (100 repetitions, task-wise statistical tests, ablations on linkage and pseudo-residual similarity). However, the theoretical justification of the similarity measure is the load-bearing part of the paper, and that justification is currently incomplete: by the paper's own decomposition, cross-task errors are confounded with target-task irreducible noise, and the similarity vectors include the diagonal in-domain errors that the paper's Remark explicitly excludes. The synthetic experiments do not exercise this confound because their data-generating process has no additive noise. The over-strong performance claims in the abstract are also contradicted by a clear, non-marginal exception in the real-world regression results.","major_comments":[{"comment":"The central claim that cross-task errors provide a 'principled measure of functional similarity' is not supported by the paper's own risk decomposition. Eq. (23) gives R_i(F_j) = E[(η_i-F_j)^2] + E[σ_i^2], where the second term is the target task's irreducible noise and is independent of the source model j. Thus each row i of the similarity matrix S in Eq. (7) contains a task-specific, roughly constant offset. Cosine distance (Eq. 8) is not invariant to row-wise additive constants: a noisy task will have a flatter similarity profile and will appear cosine-similar to other noisy tasks, even when the underlying functions η_i are unrelated, while clean tasks will appear mutually close. The Conclusions concede that 'another promising direction is to refine cross-task similarity estimation by accounting for task difficulty,' but the method as presented does not do so. This confound must be re","section":"Section 3.2, Eq. (5)-(8) and Section 3.3.1, Eq. (23)"},{"comment":"The similarity vector s_i = (s_{i,1}, ..., s_{i,m}) includes the diagonal entry s_{i,i}, which is derived from the in-domain training error E_{i,i}. The Remark immediately before Eq. (5) states that 'in-domain accuracy only reflects task difficulty (e.g., noise level or sample size) and not inter-task compatibility.' Including the diagonal directly contradicts this Remark. For a well-fit task, E_{i,i} is small and s_{i,i} can dominate the profile, so the cosine distance in Eq. (8) reflects the tasks' own training errors rather than their transfer relationships. The diagonal should be excluded from the similarity profiles (e.g., set to zero or treated as missing) and the analysis repeated.","section":"Section 3.2, Eq. (7) and Remark after Eq. (5)"},{"comment":"The synthetic experiments do not test the difficulty-confounding concern because the data-generating process in Eq. (36) is deterministic: no additive noise is introduced, so σ_i^2 = 0 for every task and the risk decomposition in Eq. (23) reduces to the functional mismatch term alone. Perfect cluster recovery under these conditions is therefore consistent with a method that clusters by functional mismatch, but it provides no evidence that the method is robust to heterogeneous noise levels. The authors should add synthetic settings where tasks have identical functional forms but different noise levels, and settings where noise is correlated with cluster membership, to determine whether the method clusters by noise or by functional relatedness.","section":"Section 4.1, Eq. (36) and Section 4.3"},{"comment":"The abstract and conclusions claim that RMB-CLE 'consistently outperforms' all baselines, but the Parkinson dataset is a clear exception. In Table 11 (RMSE), ST-GB achieves 0.268 and R-MTGB 0.289, while RMB-CLE-via-LGBM achieves 0.349; in Table 12 (MAE), ST-GB achieves 0.110 and R-MTGB 0.137, while RMB-CLE-via-LGBM achieves 0.203. The text calls the ST-GB advantage 'marginal,' which is inaccurate for a 30-80% relative difference. The paper should either soften the 'consistently outperforms' claim or provide a more detailed analysis of why RMB-CLE fails on this dataset, as it weakens the general robustness claim.","section":"Section 4.4, Tables 11-12"}],"minor_comments":[{"comment":"Algorithm 1 requires Kmax and ε, but these hyperparameters are not specified in the experimental setup or in the hyperparameter search space of Table 3. The authors should state their default values and the range considered, if any, since the silhouette-based selection only determines k within the interval [2, min(m, Kmax)].","section":"Algorithm 1 and Table 3"},{"comment":"Line 24 assigns k⋆ inside the inner loop over tasks, and line 28 reassigns k⋆ after the loop. This is inconsistent with Eq. (11), which defines k⋆ as the maximizer over k of the mean silhouette. The algorithm pseudocode should be aligned with the equation.","section":"Algorithm 1, lines 24-28"},{"comment":"Table 1 uses '3' and '7' as symbols for present/absent, which is confusing; the '7' appears to be a typographical artifact for a check/cross symbol. Several figures (e.g., Figures 2, 5-7) contain OCR-like typos in axis labels ('Clus er', 'Classifica ion', 'R egression'). These should be corrected.","section":"Table 1 and figures"},{"comment":"Prediction requires the task identifier i at test time. For several real-world datasets, tasks are obtained by grouping on an attribute, so the identifier is available; however, this requirement should be stated explicitly in the problem formulation and contrasted with methods that do not need a task identifier at inference.","section":"Eq. (14)"}],"recommendation":"major_revision","confidential_remarks":"The paper is not ready for acceptance in its current form. The main methodological claim—that cross-task errors give a difficulty-free measure of functional similarity—is directly undercut by the paper's own risk decomposition and by the inclusion of diagonal in-domain errors. The issue is fixable in principle (e.g., exclude the diagonal, normalize by task noise, or add synthetic experiments with controlled noise), but until then the reported gains could be attributed to clustering by difficulty rather than by functional relatedness. I recommend major revision rather than rejection because the empirical machinery and the decomposition itself are sound; the burden is on the authors to close the gap between the theory and the similarity measure actually used."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Good paper to know about if you work on tabular multi-task learning. The pipeline is new and clean: train one LGBM per task, build a cross-task error matrix, embed each task as a row, take cosine distances, agglomerative cluster with silhouette selection, and train a local ensemble per cluster. Code and data are public, synthetic experiments are well-designed, and the method matches an oracle cluster-known baseline on synthetic data. That is real evidence the clustering is doing something right, and the real-world gains are broad, with proper task-wise significance tests.\n\nThe risk decomposition in Section 3.3 is correct but elementary — conditional risk equals squared functional mismatch plus irreducible noise. That's textbook, not a contribution. The claimed contribution is that cross-task errors measure functional relatedness rather than task difficulty. Here the paper has a real problem. The similarity vectors include diagonal entries E_{i,i}, the in-domain training errors, which the paper's own Remark says should not be used because they only reflect difficulty. And the irreducible noise term σ_i^2 appears additively in every entry of row i, so a noisy task has a nearly flat similarity profile. Cosine distance is not invariant to row-wise constants, so the method can group tasks by noise level instead of by functional similarity. The paper concedes in the conclusions that difficulty normalization is a future direction, but that's not a footnote — it's the load-bearing part of the claim.\n\nThe empirical results may still hold up: isolating noisy tasks is arguably a useful behavior, and the synthetic recovery of ground-truth clusters suggests the signal is not pure noise. But the paper should either remove the diagonal, normalize the error rows, or revise the theoretical framing so it doesn't claim a measure of functional relatedness when it is really a mix of relatedness and difficulty.\n\nAlso minor: the abstract says 'consistently outperforms' but the Parkinson regression is an exception, and the local ensemble block size is fixed at 100 while baselines are tuned — a fairness loose end.\n\nWho should read this: practitioners who want a working, model-agnostic way to do robust MTL on tabular data, and any researcher thinking about task similarity from transfer errors. It deserves a serious referee: the core idea is new, the experiments are reproducible, and the main flaw is addressable. I'd send it to review and ask for a revision that deals with the diagonal and difficulty contamination head-on.","headline":"Useful, reproducible MTL pipeline; the similarity construction doesn't yet support the paper's strong theoretical claims.","tokens_in":34077,"tokens_out":2916,"would_cite":true,"duration_ms":29862,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A multi-task learning method that measures task similarity by cross-task errors, then clusters tasks and trains a local ensemble per cluster, outperforms single-task and pooling baselines across synthetic and real-world benchmarks.","keywords":["multi-task learning","negative transfer","task clustering","gradient boosting","cross-task error","local ensembling","hierarchical clustering","task similarity"],"falsifier":"Generate a regression dataset with several clusters of tasks, then add a task whose underlying regression function exactly matches one cluster's function but whose label noise is much larger. If RMB-CLE assigns this task to a different cluster (or performs worse than a variant that places it with its functional matches), the similarity geometry is not capturing purely functional relatedness.","tokens_in":33153,"feed_emoji":"🧩","tokens_out":3733,"duration_ms":34651,"temperature":0.7,"pith_summary":"The paper claims that task relatedness in multi-task learning should be measured by how well a model trained on one task predicts another task's data, not by in-domain accuracy. It introduces RMB-CLE, which computes cross-task errors, converts them into similarities, clusters tasks via hierarchical agglomerative clustering with silhouette-based model selection, and trains a local ensemble on each cluster. This structural separation of tasks is meant to avoid negative transfer by isolating incompatible tasks. On synthetic data the method recovers ground-truth clusters exactly and matches an oracle given the true clusters; on real-world datasets it consistently beats single-task, pooling, MTGB, and R-MTGB baselines.","feed_headline":"Cluster tasks by cross-task error for better multi-task learning","feed_subtitle":"A new method measures task relatedness by how well models generalize across tasks, then trains a local ensemble per cluster.","key_machinery":"The central object is the cross-task error matrix E_{i,j}, the loss of a model trained on task j evaluated on task i's data. Similarity is s_{i,j} = 1/(E_{i,j} + ε), and each task is embedded as a row of this matrix; cosine distance between rows defines the task geometry. UPGMA agglomerative clustering on this distance, with silhouette-based selection of the number of clusters, yields task groups; a local boosting ensemble (LGBM or MTGB) is then trained per cluster. The load-bearing identity is the risk decomposition R_i(F_j) = E[(η_i - F_j)^2] + E[σ_i^2], which shows cross-task error captures functional relatedness plus task-specific noise.","core_discovery":"The central claim is that cross-task generalization error provides a principled measure of task compatibility. For regression, the risk of transferring a model F_j to task i decomposes into a functional mismatch term (the squared L2 distance between the conditional functions η_i and F_j) plus an irreducible noise term σ_i^2 that is independent of the source model. For classification, the cross-task error upper-bounds the excess risk relative to the Bayes classifier. RMB-CLE builds a similarity matrix from these cross-task errors, embeds each task as a similarity profile, clusters tasks with UPGMA agglomerative clustering chosen by silhouette score, and trains a local ensemble on each cluster","pith_inferences":["Because the similarity profiles include the diagonal in-domain errors E_{i,i}, a task with high label noise will look dissimilar to every other task even if its underlying function matches; normalizing transfer errors by task difficulty — which the paper defers to future work — would be a natural fix.","The density of cross-task evaluation makes training cost quadratic in the number of tasks; for large task counts, a subsampled or sketch-based approximation of the error matrix would be needed.","The same error-geometry idea could be used for task selection in transfer learning: choose source tasks that minimize the cross-task error on the target rather than using representation-based affinity."],"forward_implications":["Task clusters can be discovered automatically from transfer behavior without metadata, predefined clusters, or low-rank assumptions.","On synthetic data, inferred clusters equal ground-truth clusters, and model performance matches an oracle that knows the true clusters.","The method generalizes R-MTGB's binary inlier/outlier split to multiple clusters, covering more complex task landscapes.","Prediction is fast: one cluster lookup plus a single ensemble evaluation per instance.","Cross-task error-based similarity beats pseudo-residual-based similarity in ablations, indicating the transfer signal matters."],"fun_headline_variants":["Group tasks on transfer error, then ensemble","Error-driven task clustering improves multi-task boosting","How to avoid negative transfer in multi-task learning","Cross-task error: the key to robust MTL clustering"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The paper treats the empirical cross-task error E_{i,j} computed on task i's training data as a pure measure of functional compatibility, but its own risk decomposition contains a task-specific irreducible noise term σ_i^2 that is independent of the source model; a noisy task will appear dissimilar to all tasks, and the similarity vectors include in-domain errors E_{i,i} that the paper says should not be used.","fun_headline_variants_meta":{"raw":{"variants":["Group tasks on transfer error, then ensemble","Error-driven task clustering improves multi-task boosting","How to avoid negative transfer in multi-task learning","Cross-task error: the key to robust MTL clustering"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000356,"raw_usage":{"total_tokens":1757,"prompt_tokens":721,"completion_tokens":1036,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":465,"completion_tokens_details":{"reasoning_tokens":989}},"tokens_in":465,"tokens_out":1036,"duration_ms":8067,"temperature":1.0,"reasoning_tokens":989,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T23:15:20.901788+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate a regression dataset with several clusters of tasks, then add a task whose underlying regression function exactly matches one cluster's function but whose label noise is much larger. If RMB-CLE assigns this task to a different cluster (or performs worse than a variant that places it with its functional matches), the similarity geometry is not capturing purely functional relatedness.","supporting_citations":[],"review_version":1}