{"id":"affb4e83-8257-446c-a832-1c3b5c46a485","arxiv_id":"2501.03463","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"In a linear model where main and auxiliary responses share features and the coefficient matrix has low rank, optimally weighting each task's OLS estimate yields a feasible estimator that is asymptotically as efficient as an oracle and dominates OLS.","lead":"This paper studies how to improve a main regression estimate using extra auxiliary regressions on the same features. It derives the best way to weight the auxiliary estimates together and proves the combined estimate can be more accurate, with simulations and a vending-machine image example.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Corollary 1 does not follow from Theorem 1 under (C4): the √N-scaled feasible-oracle gap can diverge, so the claimed oracle-equivalence and variance reduction are unproven.","rationale":"The reader's weakest assumption was that d is unknown and selected by cross-validation, which is a legitimate practical gap but not the central mathematical flaw. Even with d known and all four conditions satisfied, the stated rate in Theorem 1 is too slow to imply Corollary 1. The paper's own derivation compares the unscaled feasible-oracle discrepancy with the oracle estimation error, but Corollary 1 scales by √N; the scaled remainder is O_p(√(K+d)+p/√N), which does not vanish under C4. I would credit the oracle weight derivation in (2.3) and the l₂-rate theorem as plausible and useful, but the headline statistical guarantee is the asymptotic normality and variance reduction in Corollary 1, and that guarantee is not supported as stated. The concrete admissible sequence N=m², p=m, K=m, d=1 makes the gap explicit and shows that the proof cannot work without additional assumptions that contradict the paper's stated conditions. Because the central claim is mathematically unproven, the verdict should move from CONDITIONAL to REJECT.","tokens_in":17540,"tokens_out":12141,"duration_ms":124087,"concrete_test":"Analytical check: set N=m², p=m, K=m, d=1, which satisfies (C1)–(C4) (K/p=1/√m→0, p/N=1/m→0). Substitute into Theorem 1's bound after multiplying by √N: the RHS becomes C₁(√(K+d)+p/√N) = C₁(√(m+1)+1) → ∞. Thus Theorem 1 cannot deliver the o_p(1) remainder that Corollary 1 requires. If the authors believe a sharper directional bound exists, it must be stated and proved; without it, the corollary's claimed oracle-equivalence fails in this admissible regime.","verdict_should_be":"REJECT","load_bearing_attack":"The most load-bearing defect is not the unknown rank d; it is the logical gap between Theorem 1 and Corollary 1 even when d is known. Theorem 1 gives ||bβ_bw* − bβ_w*||_2 = O_p(√((K+d)/N) + p/N). By Cauchy–Schwarz, √N α'(bβ_bw* − bβ_w*) = O_p(√(K+d) + p/√N). Corollary 1 requires this quantity to be o_p(1). But Condition (C4) only requires K→∞ and p/N→0; it permits K to grow. Taking p=√N and K=√N satisfies C4, yet the bound becomes √(K+d) ≈ N^{1/4} → ∞, with p/√N = 1. The paper compares the unscaled remainder √((K+d)/N) with the oracle rate √(p/N), which only shows the absolute discrepancy is o(√(p/N)). After the √N normalization used in Corollary 1, this is o(√p), not o(1). Hence Theorem 1 cannot establish that the feasible estimator has the same √N-asymptotic distribution as the oracle, and the variance-reduction claim P(w*) ≤ σ₀² is not established for the feasible estimator. Closing this gap would require much stronger rates, such as K+d = o(1) and p = o(√N), which contradict C4 and the paper's own simulations (e.g., p=√N, K=20).","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a weighted OLS estimator for a primary linear regression that borrows strength from auxiliary tasks sharing the same covariates. Under a low-rank assumption on the combined coefficient matrix B, the set of weight vectors that keep the weighted estimator unbiased is characterized as an affine subspace, and the oracle weight minimizing the noise variance is derived in closed form. The authors state a non-asymptotic bound on the distance between the feasible estimator (which plugs in estimated eigenvectors and residual covariances) and the oracle estimator (Theorem 1), and then claim an asymptotic normality result for the feasible estimator with the same variance as the oracle (Corollary 1). Simulations under linear and logistic models, plus a deep-learning real-data example on smart vending machines, are presented to support the method.","tokens_in":17868,"tokens_out":10047,"duration_ms":92361,"significance":"If the theoretical claims were rigorously established, the paper would offer a simple, interpretable way to exploit auxiliary tasks: the oracle weight (2.3) is elegantly derived, the low-rank assumption is a natural structural condition, and numerical experiments suggest practical gains. The paper also makes a useful connection between auxiliary learning and classical least-squares theory. However, the central asymptotic equivalence between the feasible and oracle estimators is not proven by the arguments in the text, and the real-data implementation is not precisely linked to the theoretical model. These gaps currently prevent the paper from fully delivering on its main claims.","major_comments":[{"comment":"The derivation of Corollary 1 is not valid as written. Theorem 1 gives ||β̂_bŵ* − β̂_w*||_2 = O_p(√((K+d)/N) + p/N). For a fixed nonzero linear functional α, the √N-scaled difference satisfies √N α'(β̂_bŵ* − β̂_w*) = O_p(√(K+d) + p/√N). Under Condition (C4), K may diverge, so this quantity need not be o_p(1). The text compares the unscaled bound with α'(β̂_w* − β(0)) = O_p(√(p/N)), but a fixed linear combination of the oracle estimator's error is actually O_p(1/√N), not O_p(√(p/N)). Thus the Slutsky-type argument in the paragraph before Corollary 1 does not establish the claimed CLT, and the variance-reduction claim P(w*) ≤ σ₀² is not proved for the feasible estimator. The authors need to supply a direct proof of Corollary 1, or state and prove the stronger rate conditions under which it holds, and reconcile those conditions with their simulation settings.","section":"Section 2.3, Corollary 1"},{"comment":"The real-data analysis does not specify how the weighted estimator is computed from the DenseNet softmax outputs. The theory in Section 2 applies to OLS estimators with a common covariate vector X and a residual covariance Σ_ε. For a multi-class softmax model, it is unclear what bB, bΣ_xx, and bΣ_ε represent, and what the 'rank' of the coefficient matrix means for the final fully connected layer. The authors should spell out the exact estimation steps, or state clearly that the deep-learning application is a heuristic extension that is not covered by Theorem 1 and Corollary 1.","section":"Section 4.3 and 4.4"},{"comment":"Condition (C3) assumes the rank d of B is known and fixed, but Section 4.4 selects d by cross-validation (d = 51). The auxiliary task set S_opt is also selected by a search procedure. The theoretical results do not cover data-adaptive choices of d or S_opt, so the reported error rates and standard deviations in the real-data experiment are not directly supported by Corollary 1. The paper should acknowledge this gap or provide additional theory for the selection step.","section":"Condition (C3) and Section 4.4"},{"comment":"The proof of Theorem 1 is relegated to an appendix that is not included in the extracted text provided for review. Since Theorem 1 is the foundation for the subsequent asymptotic claims, the authors must provide the complete proof in the revised version so that the non-asymptotic bound in Eq. (2.4) can be verified.","section":"Theorem 1"}],"minor_comments":[{"comment":"The expression for w*_k is written in a somewhat confusing way; it would be clearer to write w*_k = (1/σ_k²) / (∑_{k'=0}^K 1/σ_{k'}^2), which is what the current formula reduces to.","section":"Section 2.2, Remark 1"},{"comment":"The condition defining Φ2 is ambiguous: 'φ(2)_ij = 0 if i ≤ j ≤ i + K − d + 1 and φ(2)_ij = 1 otherwise'. Please clarify the intended band structure, since d < K+1 in general and the range of j depends on i.","section":"Section 3.1"},{"comment":"The left panel of Figure 9 plots AvgErr for d in [0,71] but the text focuses on d ∈ [40,71]; the axis label and the range should be made consistent, and the location of d=51 should be marked.","section":"Section 4.4, Figure 9"},{"comment":"The paper uses the term 'high-dimensional' broadly, but the theory requires p/N → 0, so p is smaller than N in the asymptotic regime. The title and abstract should be qualified to avoid overstating the range of applicability, particularly because the introduction mentions settings with p > N.","section":"Introduction and Condition (C4)"}],"recommendation":"major_revision","confidential_remarks":"The central theoretical result (Corollary 1) is not justified by the arguments presented, and the real-data experiment sits outside the paper's formal model. The paper's potential contribution is real, but the revision needs to either fix the proof or substantially recalibrate the claims. The authors should also clarify the mismatch between the 'high-dimensional' framing and the actual p/N → 0 assumption."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper is worth a look, but the headline claim needs a close read. The oracle weight in (2.3) is a standard inverse-variance projection over the null space of B, and the paper is honest that this is the core. What is new is the non-asymptotic analysis of the feasible estimator in Theorem 1, with K and p diverging. That bound looks plausible, and the linear simulations are clean. Credit where due: the algebra in Section 2.2 is correct, and the paper does not fit constants to get its main theorem.\n\nThe soft spot is load-bearing. Corollary 1 does not follow from Theorem 1 under Conditions (C1)–(C4). Theorem 1 gives ||β̂_bw* − β̂_w*|| = O_p(√((K+d)/N) + p/N). After multiplying by √N, the remainder is O_p(√(K+d) + p/√N). Condition (C4) allows K=√N and p=√N, making this diverge. So the feasible estimator's √N-asymptotic distribution is not established by the paper's arguments. This is not a minor technicality: the variance-reduction claim P(w*) ≤ σ₀² is proven for the oracle, not for the feasible estimator. The paper needs either a sharper bound on the weight estimation error or a different proof strategy.\n\nTwo smaller issues. The theory assumes d known and p/N→0, so the motivating p>N regime is not covered. And in the real-data section, the weighted estimator is never concretely defined for the DenseNet softmax heads, and the task/rank selection uses the same test splits that produce the headline error rate, so the reported improvement likely benefits from selection bias.\n\nOverall: the paper is not a desk reject. The proof of Theorem 1 and the oracle analysis are genuine contributions, and the topic matters for transfer learning. But Corollary 1 is currently unproven, and the real-data evidence is softer than the text suggests. Send it to a referee who can check the appendix and push on the rates; with a fixed corollary or an honestly weakened statement, it could be a solid paper.","headline":"Correct oracle analysis and a plausible Theorem 1, but Corollary 1 does not follow from the stated rates and the real-data analysis leaks test information.","tokens_in":18383,"tokens_out":3543,"would_cite":false,"duration_ms":32506,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62J05","62F12"],"pacs":[],"model":"deepseek-v4-flash","headline":"Auxiliary learning improves primary regression whenever the task coefficient matrix has rank below the number of tasks; the paper derives the optimal weight and proves the feasible estimator attains it.","keywords":["auxiliary learning","multi-task learning","high-dimensional linear regression","low-rank coefficient matrix","weighted estimator","ordinary least squares","asymptotic efficiency","smart vending machines"],"falsifier":"Under the conditions of Corollary 1, build Wald intervals for α^Tβ^(0) from the feasible estimator; if their empirical coverage is far below nominal as N grows, the asymptotic normality claim is wrong.","tokens_in":17315,"feed_emoji":"📊","tokens_out":10236,"duration_ms":95279,"temperature":0.7,"pith_summary":"This paper claims that the statistical efficiency of a primary linear regression estimator can be improved by pooling ordinary least squares estimates from K auxiliary tasks that share the same covariates, provided the (p × (K+1)) matrix B of true task coefficients has rank d < K+1. It derives an optimal weight w* that minimizes the noise factor w^T Σ_ε w among all weights w satisfying Bw = β(0), gives its closed form, and constructs a feasible estimator that estimates both the weight and the noise covariance from data. The main theorem shows that the feasible estimator is within O_p(√((K+d)/N) + p/N) of the oracle weighted estimator with probability at least 1 − C2 exp(−K), and the corollary gives its asymptotic normality with variance no larger than the primary-only OLS variance. A sympathetic reader would care because this is a rare case where auxiliary or multi-task learning is shown to have a concrete statistical payoff with a closed-form weight formula, not just a heuristic loss-balancing scheme. Simulations on linear and logistic models and a smart-vending-machine image classification application support the theoretical claims.","feed_headline":"Auxiliary tasks provably beat OLS under low-rank coefficient matrix","feed_subtitle":"A closed-form optimal weight over shared-covariate tasks shrinks asymptotic variance to or below OLS.","key_machinery":"The load-bearing mechanism is the right null space of the true coefficient matrix B. The paper assumes rank(B)=d<K+1, so the eigenvectors of B^T B associated with the K+1−d smallest eigenvalues span a null-space matrix Θ, and every weight that preserves the primary coefficient is w = e1 + Θu. The optimal u then comes from a finite-dimensional quadratic minimization: minimize u ↦ (e1+Θu)^T Σ_ε(e1+Θu), whose solution is the closed-form projection w* = e1 − Θ(Θ^TΣ_εΘ)^{−1}Θ^TΣ_εe1. The feasible estimator replaces Θ by the corresponding eigenvectors of bB^T bB and Σ_ε by the residual covariance matrix; Theorem 1 controls the error from those plug-in estimates using sub-Gaussian concentration, and Corollary 1 transfers the oracle's asymptotic distribution to the feasible version.","core_discovery":"On the paper's own terms, the central discovery is that when B = (β^(0),...,β^(K)) is rank-deficient, the feasible weighted estimator bβ_{bw*} = bB bw* reproduces the oracle estimator's first-order behavior. The oracle weight is w* = e1 − Θ(Θ^T Σ_ε Θ)^{−1} Θ^T Σ_ε e1, where Θ spans the right null space of B; this is the weight in the feasible set W = {e1 + Θu : u ∈ $R^{{K−d+1}}$} that minimizes the scalar noise factor w^TΣ_εw. Because e1 is always feasible, P(w*) ≤ σ_0², and therefore Corollary 1 states that √N α^T(bβ_{bw*} − β(0)) converges to a normal distribution with variance at most σ_0² α^$TΣ_xx^{{−1}}$α. In plain terms, auxiliary tasks never hurt asymptotically, and they help whenever the low-rank structure gives the weight more freedom than the trivial e1 choice.","pith_inferences":["The same weight formula would apply to any set of asymptotically normal per-task estimators whose joint covariance is estimable, so the argument is not intrinsically tied to OLS; the paper itself only verifies the logistic case empirically, leaving a theory gap.","Because the theorem assumes d is known while the real-data analysis selects d by cross-validation, the reported confidence intervals may in practice undercover; a bias correction or a theory for estimated rank would be needed.","If the low-rank condition holds only approximately, one can expect the method to remain approximately beneficial when the smallest nonzero singular value of B is not too small; the current proof does not quantify that regime."],"forward_implications":["In the rank-deficient regime d<K+1, the feasible estimator is first-order equivalent to the oracle: its asymptotic variance is P(w*) α^TΣ_xx^{−1}α, which is no larger than the primary-only OLS variance σ_0² α^TΣ_xx^{−1}α.","The gap between the feasible and oracle estimators vanishes at rate √((K+d)/N)+p/N, so uncertainty about the weight and noise covariance does not change the leading-order distribution.","If B has full rank K+1, the feasible set collapses to {e1}, the estimated weight equals e1, and the method reduces to OLS; auxiliary tasks then provide no efficiency gain.","With fixed rank d, increasing the number of auxiliary tasks K expands the feasible set and can only lower P(w*), so additional tasks improve the oracle efficiency as long as K/p→0; simulations show this benefit, while adding tasks outside the low-rank span produces a U-shaped MSE pattern."],"supporting_citations":[{"why":"supplies the sub-Gaussian concentration inequalities used in proving the feasible-oracle discrepancy bound of Theorem 1","marker":"Vershynin (2018)"},{"why":"gives the high-dimensional OLS accuracy baseline Op(sqrt(p/N)) that the weighted estimator is meant to improve and additional concentration tools","marker":"Wainwright (2019)"},{"why":"provides the asymptotic normality for OLS estimators that underlies the oracle CLT behind Corollary 1","marker":"Van der Vaart (2000)"}],"fun_headline_variants":["Auxiliary tasks never hurt: closed-form weight beats OLS under rank deficiency","Low-rank coefficient matrix makes auxiliary learning provably better","Optimal weight over auxiliary tasks shrinks variance to or below OLS","Auxiliary learning: how to combine tasks for better statistical efficiency","Rank-deficient B: weighting auxiliary tasks beats OLS"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's advantage exists only when the matrix of true regression coefficients across primary and auxiliary tasks has rank strictly below the number of tasks plus one, and the user knows that rank; a full-rank matrix or a wrongly chosen rank leaves the method with no gain or with bias.","fun_headline_variants_meta":{"raw":{"variants":["Auxiliary tasks never hurt: closed-form weight beats OLS under rank deficiency","Low-rank coefficient matrix makes auxiliary learning provably better","Optimal weight over auxiliary tasks shrinks variance to or below OLS","Auxiliary learning: how to combine tasks for better statistical efficiency","Rank-deficient B: weighting auxiliary tasks beats OLS"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000497,"raw_usage":{"total_tokens":2418,"prompt_tokens":913,"completion_tokens":1505,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":529,"completion_tokens_details":{"reasoning_tokens":1416}},"tokens_in":529,"tokens_out":1505,"duration_ms":9796,"temperature":1.0,"reasoning_tokens":1416,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:53:27.728991+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Under the conditions of Corollary 1, build Wald intervals for α^Tβ^(0) from the feasible estimator; if their empirical coverage is far below nominal as N grows, the asymptotic normality claim is wrong.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"gives the high-dimensional OLS accuracy baseline Op(sqrt(p/N)) that the weighted estimator is meant to improve and additional concentration tools"}],"review_version":1}