REVIEW 6 cited by
Learning from Similar Linear Representations: Adaptivity, Minimaxity, and Robustness
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Representation multi-task learning (MTL) has achieved tremendous success in practice. However, the theoretical understanding of these methods is still lacking. Most existing theoretical works focus on cases where all tasks share the same representation, and claim that MTL almost always improves performance. Nevertheless, as the number of tasks grows, assuming all tasks share the same representation is unrealistic. Furthermore, empirical findings often indicate that a shared representation does not necessarily improve single-task learning performance. In this paper, we aim to understand how to learn from tasks with \textit{similar but not exactly the same} linear representations, while dealing with outlier tasks. Assuming a known intrinsic dimension, we propose a penalized empirical risk minimization method and a spectral method that are \textit{adaptive} to the similarity structure and \textit{robust} to outlier tasks. Both algorithms outperform single-task learning when representations across tasks are sufficiently similar and the proportion of outlier tasks is small. Moreover, they always perform at least as well as single-task learning, even when the representations are dissimilar. We provide information-theoretic lower bounds to demonstrate that both methods are nearly \textit{minimax} optimal in a large regime, with the spectral method being optimal in the absence of outlier tasks. Additionally, we introduce a thresholding algorithm to adapt to an unknown intrinsic dimension. We conduct extensive numerical experiments to validate our theoretical findings.
Forward citations
Cited by 6 Pith papers
-
Contaminated Multi-task Learning with Heterogeneity: Fundamental Limits and Optimal Algorithms
Filtering-based robust multi-task gradient descent matches minimax rates under task contamination and heterogeneity, removing the √d contamination barrier of regularization and score-based methods.
-
On Non-Stationary Dynamic Pricing: Adaptivity and Optimality
An adaptive dynamic-pricing algorithm achieves, up to logarithmic factors, the minimax optimal regret for both abrupt and smooth non-stationarity in contextual GLM demand, and comes with a matching lower bound.
-
Representation-Aware Distributionally Robust Optimization: A Knowledge Transfer Framework
A representation-aware Wasserstein DRO framework that shrinks estimators toward an external representation subspace, with asymptotic inference and adaptive robustness tuning.
-
Late Fusion Multi-task Learning for Semiparametric Inference with Nuisance Parameters
A late-fusion multi-task learning framework for double machine learning, with theories showing faster rates when tasks share similar parameters, plus a fused kernel method for nuisance parameters.
-
Joint estimation of smooth graph signals from partial linear measurements
A smoothness-penalized least squares estimator jointly recovers graph-indexed signals from partial noisy measurements, and is weakly consistent as the graph grows even when per-vertex measurements are rank-one and mos...
-
Multi-task Learning for Heterogeneous Multi-source Block-Wise Missing Data
A two-step framework combining block-wise imputation with shared and task-specific multi-task learning improves prediction under distribution and posterior shift.
Discussion (0). Sign in to comment.