{"id":"29fc7b9b-80c7-4eb6-8860-8ea9ed93bf3b","arxiv_id":"2607.16554","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A shared representation's total per-task information is bounded by capacity plus label redundancy; clustered sharing wins exactly when interference reduction exceeds the redundancy it loses.","lead":"This paper derives a capacity–redundancy bound showing the summed task information a shared representation can carry is limited by its capacity plus task-label redundancy, and gives a condition for when splitting tasks into clusters beats a single shared module. It then validates the approach with LoRA fine-tuning experiments.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 5.3's iff sharing rule holds only under unstated saturation of capacity-optimal encoders; the main-text statement is false as written.","rationale":"The load-bearing concern is exactly the saturation assumption the reader identified. The theorem's proof depends on setting the joint-predictive term to zero via Eq. S4.38, which is not part of the theorem statement. Without it, the claimed iff decomposition is incomplete, so the central decision rule is not generally valid. The concern is not speculative: the authors admit the extra term in the appendix. The paper's main contribution is this theorem, so the correctness risk is real. However, the issue is addressable by restating Theorem 5.3 with the saturation condition made explicit (or by proving the condition holds for the empirical LoRA setting). The nonnegativity of Δsh − ΣΔk is a secondary gap, also flagged by the reader. Since the paper's conditional verdict already reflects this, our stress-test does not move the verdict; CONDITIONAL remains appropriate. We agree with the reader's assessment.","tokens_in":22250,"tokens_out":6580,"duration_ms":71241,"concrete_test":"In the linear-Gaussian model of Section S2, solve exactly for the capacity-optimal encoder A under a rank/trace constraint maximizing Σ_t I(Z;Y_t) with I(Z;X) ≤ C. Check whether I(Z*;Y^{1:T}) = C and I(Z*_S_k;Y^{S_k}) = C_k hold at the optimum for various C and task geometries. Construct a case where the optimum does not saturate (e.g., include an irrelevant input coordinate or set the rank below the task-subspace dimension), and compute the exact gain of clustered vs global sharing. If the sign of the gain differs from Δsh − ΣΔk − TC_between(P), the theorem's iff fails. This directly measures the magnitude of the joint-predictive term and identifies when the main-text statement needs qualification.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim—Theorem 5.3's necessary-and-sufficient condition for clustered sharing—is not established for the stated assumptions. The proof (Appendix S4, Eq. S4.36) yields a general gain identity containing a joint-predictive difference term Σ_k I(Z*_S_k;Y^{S_k}) − I(Z*_s;Y^{1:T}). This term is zero only under the saturation assumption (Eq. S4.38): I(Z*_s;Y^{1:T}) = C and I(Z*_S_k;Y^{S_k}) = C_k. The main theorem states only 'capacity-optimal under budget' with I(Z_s;X) ≤ C and I(Z_S_k;X) ≤ C_k. Capacity optimality does not imply these mutual informations saturate their budgets—e.g., if some components of X are irrelevant to Y or the encoder family is rank-limited, the optimum may leave capacity unused. The paper itself concedes after the proof: 'Without the saturation condition the exact gain identity consists of an additional joint-predictive difference term.' Thus Theorem 5.3, as a claim about arbitrary capacity-optimal encoders, is false. Moreover, the label 'interference reduction ≥ 0' for Δsh − ΣΔk is asserted without proof; conditional total correlation is not monotone under conditioning, so this term can be negative in general, further undermining the unqualified 'iff.' The empirical LoRA setting may approximately saturate, but no analytic evidence is given.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a Capacity–Redundancy (CR) identity for multi-task learning: under a shared bottleneck with budget C_s, the total per-task mutual information is bounded by C_s plus the total correlation of the labels. It derives consequences (average per-task decay, capacity scaling), a shared–private extension, subset bounds, and a Gaussian gradient–TC bridge. It then states Theorem 5.3, a clustering-gap identity meant to give an iff condition for clustered sharing to outperform global sharing. The empirical section applies clustered LoRA on GoEmotions and GLUE8, estimating residual coupling from validation residuals and reporting multi-seed confidence intervals.","tokens_in":22718,"tokens_out":7173,"duration_ms":77457,"significance":"If Theorem 5.3 were correct as stated, it would provide a quantitative, distribution-level decision rule for choosing between global and clustered sharing, which is a useful conceptual contribution. The CR bound itself and the Gaussian gradient–TC bridge are correct and add a principled link between gradient similarity and redundancy. The synthetic verification and multi-seed empirical comparisons are strengths. However, the central theorem is overstated: the proof in Appendix S4 requires a saturation assumption that is absent from the main statement, and the 'interference reduction' term is not proven nonnegative. These issues do not invalidate the CR bound but require a substantive revision of the paper's main claim.","major_comments":[{"comment":"The theorem states a decomposition with no joint-predictive term for 'capacity-optimal under budget' encoders. The general identity (S4.36) contains Σ_k I(Z*_S_k; Y(S_k)) − I(Z*_s; Y(1:T)), which cancels only under the saturation condition (S4.38): I(Z*_s; Y(1:T)) = C and I(Z*_S_k; Y(S_k)) = C_k. Capacity optimality under a budget does not imply saturation; the optimum may leave budget unused, e.g., when the label-relevant subspace has lower dimension than the budget. The proof itself concedes after (S4.37) that without the saturation condition the gain identity carries an additional joint-predictive difference term. Therefore the theorem's iff rule is false as stated. The statement must be revised to include saturation, or to state the general identity and give the iff only when the budgets are active.","section":"Theorem 5.3 / Appendix S4, Eq. (S4.36)–(S4.38)"},{"comment":"The theorem labels Δ_sh − Σ_k Δ_k as 'interference reduction ≥ 0' and uses this label to interpret the decomposition. No proof is supplied, and the inequality is not guaranteed: Δ_sh = TC(Y | Z*_s) and Δ_k = TC(Y(S_k) | Z*_S_k) are conditional total correlations with different conditioners and different label subsets. Conditional TC is monotone with respect to the conditioning set only for the same set of variables, not across this comparison. It is easy to construct settings where per-cluster encoders leave within-cluster dependence unresolved while a global encoder with the same total budget removes cross-cluster dependence, giving a negative value. The algebraic identity under saturation remains valid, but the asserted sign is an unsupported claim and should be removed or proved.","section":"Theorem 5.3 body"}],"minor_comments":[{"comment":"The text refers to 'the subset CR bound (Theorem 6.1)', but the correct reference is Corollary 6.1.","section":"Section 7"},{"comment":"There is a stray punctuation artifact: 'TCbetween(P),\\n.' appears after the display equation.","section":"Appendix S4, after Eq. (S4.37)"},{"comment":"The phrase 'Under the Markov chain' would be clearer if the chain Y(1:T) − X − Z_s were explicitly named in the theorem statement, rather than only in the preceding paragraph.","section":"Theorem 3.2"},{"comment":"The sentence 'Table 2 quantifies this effect ε∈{10^-6,10^-5} achieves small error' is a fragment and should be rewritten.","section":"Table 2 / Figure 2"}],"recommendation":"major_revision","confidential_remarks":"This is a borderline major revision rather than reject: the CR bound, the Gaussian bridge, and the empirical study are sound, and the central theorem can be repaired by explicitly assuming budget saturation and correcting the 'interference reduction' sign claim. The abstract and contribution list currently assert the unqualified iff rule, so the authors should revisit those as part of the revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: the paper has a clean conceptual frame for the capacity-redundancy trade-off in MTL, but the advertised if-and-only-if sharing rule (Theorem 5.3) is not proved as stated. The proof in Appendix S4 gives a general gain identity with a joint-predictive difference term; that term only cancels under a saturation condition (Eq. S4.38) that is not included in the theorem statement in the main text. The paper admits this right after the proof. So the headline claim — \"clustered sharing is strictly beneficial iff interference reduction exceeds redundancy loss\" — is not true for arbitrary capacity-optimal encoders, only for those that saturate their budgets. That is a load-bearing gap, not a nitpick. The label \"interference reduction ≥ 0\" is also asserted rather than proved; conditional total correlation does not have to shrink under conditioning, so that term can be negative in general.\n\nThat said, the paper is not empty. The CR identity itself is elementary (DPI plus the chain rule), but the general clustering-gap decomposition (Eq. S4.36) is a useful way to split shared-representation gains into residual coupling, redundancy, and joint-predictive parts. Even without the saturation condition, it is a good diagnostic framing. The Gaussian gradient-TC bridge (Theorem 4.4) is small but genuine: whitened gradient cosine has the same sign as label correlation and, under matched noise, the same ordering, which licenses the common practice of clustering by gradient similarity. The empirical Δ estimator based on validation residuals is also a nice practical tool, and the synthetic bias analysis is a responsible touch.\n\nThe empirical section is honest but weaker than the abstract suggests. Gains on GLUE8 are modest, the random-partition comparison on GoEmotions is available only at one rank with a non-significant p-value around 0.09, and the PCGrad/GradNorm baselines are not compute-matched. The paper does call the random-partition comparison a sanity check, which is fair, but the significance language in the abstract overstates it.\n\nBottom line: worth reading for the conceptual vocabulary and the decomposition, and worth sending to peer review — the issues are addressable. But I would not cite Theorem 5.3 as a proved result until the saturation condition is explicit in the main text and the nonnegativity of the interference term is handled. Major revision needed.","headline":"Useful capacity-redundancy frame and a clean Gaussian bridge, but the headline sharing rule (Thm 5.3) is unproved as stated due to a missing saturation condition.","tokens_in":23055,"tokens_out":3943,"would_cite":false,"duration_ms":42120,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that negative transfer in multi-task learning follows a quantitative identity: the total predictive information a shared encoder can supply across tasks is bounded by its capacity plus task redundancy, and clustered sharin","keywords":["multi-task learning","capacity-redundancy identity","total correlation","negative transfer","task clustering","low-rank adaptation","information bottleneck","gradient cosine similarity"],"falsifier":"In the paper's own Gaussian specialization, compute the left- and right-hand sides of the clustering-gap identity (Equation S4.36) for a rank-constrained encoder that is capacity-optimal but does not saturate its budget: if the joint-predictive term differs from zero, the inequality in Theorem 5.3 no longer predicts the sign of the gain. A concrete falsifying observation would be a case where clustered sharing improves validation performance yet Delta_shared − sum_k Delta_k is smaller than TC_between(P), showing the saturation-free term is decisive in practice.","tokens_in":22188,"feed_emoji":"🧠","tokens_out":8605,"duration_ms":81427,"temperature":0.7,"pith_summary":"This paper claims that negative transfer in multi-task learning is not merely a training artifact but an information-theoretic consequence of limited shared capacity and weak task redundancy. The central claim is a capacity–redundancy identity: the sum over tasks of predictive information carried by a shared representation equals the joint predictive information plus the label redundancy (measured by total correlation) minus a residual coupling term that remains after conditioning on the representation; hence the sum can never exceed capacity plus redundancy. On top of that, the paper proves an if-and-only-if result for clustered sharing: replacing one global encoder with per-cluster encoders improves the sum of per-task informations precisely when the reduction in residual coupling exceeds the cross-cluster redundancy that is lost. If correct, this turns an intuitive trade-off into a measurable decision rule that can guide when to share or specialize parameters in parameter-efficient fine-tuning, including low-rank adapters (LoRA). The paper also proves a Gaussian bridge that justifies using gradient cosine similarity as a proxy for redundancy-based task clustering, and it reports experiments where similarity-clustered adapters reduce residual coupling and outperform size-matched random partitions.","feed_headline":"Cluster tasks only when interference savings beat redundancy loss","feed_subtitle":"An exact identity turns multi-task negative transfer into a measurable capacity-versus-redundancy trade-off.","key_machinery":"The load-bearing object is the exact capacity–redundancy identity, sum_t I(Z_s;Y^(t)) = I(Z_s;Y^(1:T)) + TC(Y^(1:T)) − Delta, where total correlation TC(Y^(1:T)) = sum_t H(Y^(t)) − H(Y^(1:T)) measures label redundancy and Delta = TC(Y^(1:T) | Z_s) is the residual coupling the shared representation leaves unexplained. The identity carries the entire argument: it converts the intuitive capacity-versus-redundancy trade-off into an equation, and subtracting the global and per-cluster versions of it yields Theorem 5.3's clustering-gap decomposition. A supporting mechanism is Theorem 4.4, the gradient–TC bridge, which licenses replacing total-correlation clustering with gradient cosine similarity","core_discovery":"The core discovery is the capacity–redundancy identity sum_t I(Z_s;Y^(t)) = I(Z_s;Y^(1:T)) + TC(Y^(1:T)) − Delta, with Delta = TC(Y^(1:T)|Z_s), giving sum_t I(Z_s;Y^(t)) ≤ C_s + TC(Y^(1:T)). Theorem 5.3 turns this into a gain decomposition: clustered sharing beats global sharing by (Delta_shared − sum_k Delta_k) − TC_between(partition), so it is strictly better when interference reduction exceeds redundancy loss. Theorem 4.4 proves in a Gaussian model that whitened-gradient cosine similarity has the same sign as label correlation and, under matched noise and equal signal power, the same ordering; clustering by it yields the same partition as clustering by total correlation. Delta is estimate","pith_inferences":["The same CR identity could be used prospectively: estimate the two competing terms on a validation set before training and decide the cluster partition by the inequality of Theorem 5.3, rather than after measuring gains.","Because total correlation measures only redundancy, the identity cannot see synergistic task structure; extending the decomposition to O-information or partial information decomposition would reveal cases where tasks help each other beyond shared redundancy, which the present bound treats as unmodeled slack.","Under non-saturating encoders, the clustering rule needs an extra joint-predictive correction; an immediate testable extension is to estimate I(Z_s;Y^(1:T)) − sum_k I(Z_sk;Y^(S_k)) and add it to the gain formula to make the sharing decision robust in realistic under-trained regimes.","The linear rank-growth prediction for low-rank adapters is directly testable: measure saturation of validation performance as T increases at fixed rank, and check whether the rank needed to hold per-task performance roughly doubles when the number of weakly dependent tasks doubles."],"forward_implications":["If tasks are nearly independent (small total correlation), adding tasks to a fixed-capacity shared encoder forces average per-task predictive information to decay like 1/T; keeping a per-task floor requires capacity to grow roughly linearly in T.","Any residual coupling Delta > 0 adds an irreducible term to the aggregate log-loss of decoupled per-task predictors, so private or clustered capacity is not a luxury but a requirement for heterogeneous task sets.","Theorem 5.3 supplies a concrete test: choose a partition with large Delta_shared − sum_k Delta_k and small TC_between(P); a partition that satisfies the inequality is provably better than global sharing at the same total budget.","For low-rank adapters, effective shared capacity grows at most linearly with rank and saturates when the feature covariance is low-rank, so clustered or private routes are needed once the number of distinct task directions outgrows the rank budget.","Validation-residual correlation estimates of Delta give a practical diagnostic: smaller estimated Delta under clustered routing tracks the interference-reduction side of the identity, matching the observed accuracy gains."],"fun_headline_variants":["Share tasks only if interference saved exceeds redundancy lost","Exact identity reveals when clustered MTL beats shared MTL","Gradient similarity orders tasks by redundancy","Measure capacity vs redundancy to stop negative transfer","To share or split tasks? Check interference vs redundancy"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is capacity saturation: the proof of the if-and-only-if clustering rule assumes the capacity-optimal global and per-cluster encoders achieve their budgets exactly (I(Z*_s;Y^(1:T)) = C and I(Z*_{S_k};Y^(S_k)) = C_k); without that, the gain identity carries an extra joint-predictive term, as the paper concedes after the proof.","fun_headline_variants_meta":{"raw":{"variants":["Share tasks only if interference saved exceeds redundancy lost","Exact identity reveals when clustered MTL beats shared MTL","Gradient similarity orders tasks by redundancy","Measure capacity vs redundancy to stop negative transfer","To share or split tasks? Check interference vs redundancy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00081,"raw_usage":{"total_tokens":3383,"prompt_tokens":728,"completion_tokens":2655,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":472,"completion_tokens_details":{"reasoning_tokens":2583}},"tokens_in":472,"tokens_out":2655,"duration_ms":22069,"temperature":1.0,"reasoning_tokens":2583,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T20:35:51.836310+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"In the paper's own Gaussian specialization, compute the left- and right-hand sides of the clustering-gap identity (Equation S4.36) for a rank-constrained encoder that is capacity-optimal but does not saturate its budget: if the joint-predictive term differs from zero, the inequality in Theorem 5.3 no longer predicts the sign of the gain. A concrete falsifying observation would be a case where clustered sharing improves validation performance yet Delta_shared − sum_k Delta_k is smaller than TC_between(P), showing the saturation-free term is decisive in practice.","supporting_citations":[],"review_version":1}