{"id":"40d00fe5-9c74-4a3b-b0bc-5216b19d6295","arxiv_id":"2505.02972","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"GeoERM learns orthonormal task representations via Riemannian gradient descent with polar retraction, and reports accuracy gains over Euclidean multi-task baselines.","lead":"A new multi-task learning method, GeoERM, places shared task representations on a curved (Riemannian) surface and updates them with geometry-preserving steps. The authors report lower prediction errors and more stable learning than existing multi-task methods on synthetic data and an activity recognition benchmark.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Outlier tasks are optimized together with normal tasks in Eq. (1) and Algorithm 1, so nothing prevents them from pulling the shared center A; the Section 2.1 assertion of protection is not backed by any mechanism.","rationale":"The reader's weakest assumption identifies exactly the load-bearing gap: the algorithm optimizes over all tasks without any mechanism to prevent outliers from distorting the shared representation. This is a genuine internal inconsistency between the modeling claim in Section 2.1 and the actual objective and update rules. The paper's central claim of robustness to heterogeneous and adversarial tasks rests on this point, so it is not a minor issue. However, the core optimization idea—Riemannian gradient plus polar retraction on the Stiefel manifold—is standard and correctly derived, and the empirical results, while not fully robust, suggest the method can work in the tested settings. The concern is addressable by either modifying the algorithm to include robust aggregation or by tempering the robustness claims. Therefore the reader's CONDITIONAL verdict remains appropriate; no adjustment is needed.","tokens_in":796,"tokens_out":704,"duration_ms":85724,"concrete_test":"Using the Section 3.2 simulator, fix a true shared subspace A* and draw outlier tasks' outlier beta from a fixed subspace B* orthogonal to A*. Train GeoERM with outlier proportion epsilon in {0, 0.05, 0.1, 0.2, 0.3} and compute the principal-angle distance between the estimated shared center A_hat and A*, plus the maximum estimation error over normal tasks. If both increase with epsilon, outliers are distorting the learned manifold structure, confirming the concern. As a control, run a robust variant that excludes outliers (or uses a trimmed mean of A(t)); if its error is substantially lower, the distortion is material to the claimed robustness.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.1 states that distinguishing normal from outlier tasks prevents outlier tasks from distorting the learned manifold-based structure, but the objective in Eq. (1) and Algorithm 1 optimize an unweighted average over all T tasks, with no outlier flag, robust loss, trimming, or down-weighting. Lines 5-12 of Algorithm 1 update every A(t) and the shared center A using gradients of the same objective. An outlier task's loss term f(t)(A(t) theta(t)) will pull its A(t) toward directions that fit its arbitrary outlier beta; because A is updated as the mean of the A(t)A(t)^T alignment penalties, those outlier A(t) shift the shared center away from the normal tasks' subspace. The paper provides no mechanism or theoretical bound showing this shift is negligible, and the experiments report only final estimation error, never the distance of the estimated A to the true shared subspace. Thus the central robustness claim is not supported by the algorithm as written.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GeoERM, a multi-task representation learning framework that constrains task-specific representation matrices A^(t) to the Stiefel manifold and optimizes them with Riemannian gradient projection followed by polar retraction. The objective (Eq. (1)) combines per-task losses with a penalty that aligns each A^(t) to a shared center A; a second step refines task coefficients by ridge-type shrinkage toward the learned low-rank structure. The authors claim that this geometry-aware optimization improves estimation accuracy, reduces negative transfer, and remains stable under adversarial label noise, and they support this with synthetic experiments under task heterogeneity and outlier contamination, plus a real-data study on the UCI HAR dataset.","tokens_in":17086,"tokens_out":4235,"duration_ms":48163,"significance":"If the empirical claims held, the paper would provide a simple, general recipe for injecting manifold geometry into existing matrix-factorized MTL methods at the same per-iteration cost. Positive features include the clear algorithm, the release of code, the use of a public benchmark, and the explicit use of retraction to maintain orthonormality. However, the advertised robustness claim is not supported by the algorithm as written or by the experiments: outlier tasks are included in the joint optimization without any robust mechanism, and no adversarial label-noise experiment is actually conducted. The theoretical content is limited to standard retraction facts and a citation of Riemannian gradient descent, which does not cover the Adam-based implementation. The central idea is reasonable and independently testable, but the current evidence is conditional.","major_comments":[{"comment":"The claim that distinguishing normal from outlier tasks \"prevents outlier tasks from distorting the learned manifold-based structure\" is not realized by the algorithm. Equation (1) minimizes an unweighted average over all T tasks, with no outlier indicator, robust loss, trimming, or down-weighting, and Algorithm 1 lines 5-12 updates every A^(t) and the shared center A using gradients of that same objective. An outlier task with arbitrary beta^(t) can therefore pull its own A^(t), and through the penalty term, shift the shared center A away from the normal tasks' subspace. No mechanism or theoretical bound is provided to show this shift is negligible. The paper needs either a robustification mechanism (e.g., robust loss, trimming, or outlier detection) or a substantially weakened claim.","section":"Section 2.1, Eq. (1), Algorithm 1"},{"comment":"The abstract advertises stability \"under adversarial label noise,\" but no experiment in the manuscript manipulates labels. The simulation's \"outlier tasks\" draw coefficients from Uniform(-3,3) and features from N(0,2), which is coefficient and feature contamination, not label noise. The HAR experiment contains no label corruption either. To support the advertised claim, the authors should add experiments with actual flipped or adversarially corrupted labels, or amend the abstract and contribution statements.","section":"Abstract and Section 3.2"},{"comment":"The convergence discussion in Section 2.2 invokes the Riemannian gradient descent result of Absil et al. (2008), but the implemented GeoERM uses the Adam optimizer (Section 3.1) combined with polar retraction. The descent-type inequality and stationarity conclusion for Riemannian gradient descent do not automatically transfer to Adam with retraction, and no separate convergence analysis is given for Algorithm 1. The paper should either provide a convergence guarantee for the actual algorithm or explicitly limit the theoretical claim to the geometric invariance of the iterates.","section":"Section 2.2 and Algorithm 1"},{"comment":"The figures reporting maximum estimation error contain no error bars, confidence bands, or other measures of variability, even though the text says results were averaged over 100 iterations. Without variability information, the claimed consistent improvements over baselines cannot be assessed statistically. Adding error bars or standard-error tables is necessary to support the comparative claims.","section":"Section 3.2 and Figures 2-7"}],"minor_comments":[{"comment":"The displayed objective is missing a summation over t in the penalty term; as written, the penalty depends on an unbound index t. It should be sum over t of the norm term.","section":"Eq. (1)"},{"comment":"The algorithm refers to \"Euclidean gradients of \\bar f (Eq. (1))\", but Eq. (1) defines the full objective and the notation f versus \\bar f is only introduced in Section 2.3. Please make the notation consistent.","section":"Algorithm 1, line 5"},{"comment":"The citation \"Kingma and Adam (2015)\" should be \"Kingma and Ba (2015)\".","section":"Section 3.1"},{"comment":"The sentence \"Our theoretical analysis established the geometric foundations\" overstates the paper's theoretical content; the paper contains retraction properties and a citation of a standard descent result, but no convergence or error-bound theorems for GeoERM itself.","section":"Section 4"},{"comment":"The text says results are averaged over 100 iterations but does not clarify whether these are independent data-generating repetitions or optimization runs; please specify.","section":"Section 3.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is very close in structure, penalty parameters, and simulation design to Tian et al. (2023), which shares an author with this submission. The incremental contribution is the Riemannian optimization layer, which is reasonable, but the framing as a robustness method goes beyond what the algorithm mechanically provides. I would ask the authors to either add a genuine robustification mechanism or substantially soften the robustness claims, and to add experimental support for any label-noise claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The new thing here is straightforward: run pERM-style multi-task learning with the representation matrices constrained to the Stiefel manifold via Riemannian gradient descent and polar retraction. The math for the tangent-space projection and the retraction is correct, and the paper is honest that the geometry tools come from Absil et al. and the objective from Tian et al. (2023). What is new is applying them to the representation matrices in this MTL setup, and that integration seems sensible. The experiments are fairly extensive: synthetic runs across p, n, T, plus the HAR benchmark, and the code link is provided. On the core question, does geometry-aware optimization help, the answer is plausibly yes; the gains over pERM on HAR are small percentage points but consistent across r.\n\nThe soft spots are real but not fatal. The biggest one is the robustness claim. The abstract and Section 2.1 say the method stays stable under outliers and adversarial conditions, but there is no test for adversarial label noise. The synthetic outliers are tasks with arbitrary coefficients and different feature distributions, not label noise. More importantly, neither the objective in Eq. (1) nor Algorithm 1 has any outlier flag, robust loss, trimming, or downweighting. Every task enters the average equally, so an outlier task with a large loss can pull its A(t) and thereby shift the shared center A. The sentence in Section 2.1 claiming the distinction 'prevents outlier tasks from distorting the learned manifold-based structure' is simply not backed by a mechanism or a bound. The experiments may still come out fine because the outlier fraction is small, but the paper needs either a robustification step or a much more careful claim. Second, most figures have no error bars; Table 1 has standard deviations, but the simulation figures don't, which makes it hard to judge whether the gaps are noise. Third, the theoretical section is mostly standard manifold facts, and the paper's own limitations section admits there are no error bounds; the discussion's phrase 'our theoretical analysis established' oversells it. Finally, there is a small inconsistency in the algorithm description: the text says Adam is used, but the update lines show a fixed step size and retraction, no adaptive moments.\n\nNone of this kills the central idea. The optimization is sound and the empirical trend is consistent enough to be worth engineering. This paper is for readers working on multi-task or transfer learning who want an alternative to Euclidean-only methods. I would send it to referees, but ask for a revision that either adds a robust objective or removes the strong robustness claims, adds error bars, and dials down the theory framing.\n\nFor peer review: yes, engage with it, but expect meaningful revisions.","headline":"A reasonable geometry-aware twist on pERM whose robustness claim overreaches; worth reviewing and revising, but not a shift.","tokens_in":17554,"tokens_out":2640,"would_cite":false,"duration_ms":31746,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T05","65K10"],"pacs":[],"model":"deepseek-v4-flash","headline":"Treating task representations as points on the Stiefel manifold and updating them with Riemannian gradients and polar retraction improves multi-task estimation accuracy and robustness.","keywords":["multi-task learning","representation learning","Stiefel manifold","Riemannian optimization","polar retraction","negative transfer","outlier robustness","activity recognition"],"falsifier":"Run a synthetic experiment at fixed $n=100$, $p=50$, $r=5$, $h=0.5$ with an increasing fraction of outlier tasks whose coefficient vectors are drawn with large magnitude but whose feature distributions match the normal tasks; if GeoERM's maximum error on the normal tasks approaches or exceeds that of pERM once the outlier fraction passes some threshold, the claim that manifold geometry alone shields the shared representation from outlier distortion would be contradicted.","tokens_in":16659,"feed_emoji":"📐","tokens_out":7417,"duration_ms":71975,"temperature":0.7,"pith_summary":"Multi-task learning usually treats each task's shared representation as an unconstrained matrix in Euclidean space, even when the representation is required to be orthonormal. GeoERM instead treats each representation matrix as a point on the Stiefel manifold, the curved space of $p \\times r$ matrices with orthonormal columns, and updates it with Riemannian gradients followed by a polar retraction that returns it to the manifold. The paper claims this geometry-aware update improves estimation accuracy on the hardest normal tasks, reduces negative transfer, and remains stable under adversarial label noise, outperforming leading multi-task and single-task baselines. If true, the result would make manifold geometry a standard part of MTL optimization rather than a post-hoc constraint, at no extra per-iteration cost.","feed_headline":"Curved-space updates shrink multi-task estimation error","feed_subtitle":"GeoERM keeps task representations orthonormal during training and reports the lowest errors on hardest tasks and on HAR.","key_machinery":"The load-bearing object is the Stiefel manifold $\\mathrm{St}(p,r)$, the set of $p \\times r$ matrices with orthonormal columns, which gives a curved search space for task representations. The argument runs on two explicit manifold operations: the Riemannian gradient $\\tilde{\\nabla}_{A^{(t)}} f = \\nabla_{A^{(t)}} \\bar f - A^{(t)} \\mathrm{sym}((A^{(t)})^\\top \\nabla_{A^{(t)}} \\bar f)$, which projects the Euclidean gradient onto the tangent space, and the polar retraction $R_{A^{(t)}}(H) = (A^{(t)}+H)(I_r+H^\\top H)^{-1/2}$, which maps the update back onto the manifold while preserving orthonormality. Together they replace the Euclidean update plus naive projection step that can fail Riemannian convergence conditions, and they keep the per-iteration cost comparable to Euclidean baselines.","core_discovery":"The central claim is that explicitly enforcing the Stiefel-manifold geometry of task representation matrices, rather than imposing orthogonality after the fact, yields better and more robust multi-task estimates. GeoERM factorizes each task parameter as $\\beta^{(t)} = A^{(t)}\\theta^{(t)}$ with $A^{(t)} \\in \\mathrm{St}(p,r)$, adds a penalty pulling each $A^{(t)}$ toward a shared center $A$ on the same manifold, and optimizes the objective by projecting Euclidean gradients onto the tangent space and applying the polar retraction $R_A(H) = (A+H)(I_r+H^\\top H)^{-1/2}$. The paper reports that on synthetic heterogeneous-task experiments and on the Human Activity Recognition benchmark, this procedure achieves the lowest maximum estimation error among non-outlier tasks and the lowest classification error across all tested representation dimensions, while remaining robust when a fraction of tasks are outliers.","pith_inferences":["The authors leave implicit that polar retraction is a drop-in replacement for the Euclidean update in existing pERM-style pipelines, so the robustness gain could be tested by ablating only the projection-and-retraction step while keeping the penalty structure fixed.","A natural extension is to ask whether outlier robustness would improve further if Step 1 first identified or down-weighted outlier tasks before fitting the shared center A; the paper's current algorithm does not do this, so such an extension is a testable modification rather than a claim.","If the empirical gains generalize, geometry-aware updates could be applied to other matrix-valued representation spaces, such as Grassmann manifolds for subspaces or SPD manifolds for covariance representations, where the same projection-and-retraction pattern applies.","The paper's own discussion points to the need for Riemannian concentration inequalities; a concrete next step is to prove a finite-sample error bound for the Stiefel-constrained estimator, which would turn the empirical robustness claim into a testable statistical statement."],"forward_implications":["Every iterate of the representation matrices stays exactly on the Stiefel manifold, so orthonormality is preserved during training rather than repaired afterwards.","The two-step structure inherited from pERM-style MTL is preserved, so GeoERM can be applied to any matrix-factorized MTL model with the same per-iteration cost as Euclidean updates.","Empirically, the method lowers maximum estimation error on the hardest normal tasks, which is the metric the authors use to show reduced negative transfer.","On the Human Activity Recognition benchmark, GeoERM reports the lowest classification error at every tested representation dimension, with its best result about $1.02\\%$ at $r=10$.","Robustness to outliers and adversarial label noise is attributed to the geometry-aware update itself, since the algorithm optimizes all tasks in a single objective without a separate outlier-removal stage."],"supporting_citations":[{"why":"Supplies the two-step MTL framework that GeoERM builds on and the pERM baseline it is compared against.","marker":"Tian et al. (2023)"},{"why":"Provides the Riemannian gradient descent convergence theorem and retraction theory used to justify the manifold updates.","marker":"Absil et al. (2008)"},{"why":"Contributes the shared-subspace penalty that pulls each representation toward a common center, along with the ARMUL baseline.","marker":"Duan and Wang (2023)"},{"why":"Provides the method-of-moments baseline for linear representation learning.","marker":"Tripuraneni et al. (2021)"},{"why":"Supplies the AdaptRep baseline used in the numerical comparisons.","marker":"Chua et al. (2021)"},{"why":"Introduces the Human Activity Recognition dataset used for the real-data evaluation.","marker":"Anguita et al. (2013)"},{"why":"Represents the Euclidean high-dimensional statistics framework that the paper says must be adapted for Riemannian concentration inequalities.","marker":"Wainwright (2019)"}],"fun_headline_variants":["Riemannian geometry for task representations lowers errors","Curved-space MTL beats Euclidean baselines in robustness","Manifold-aware multi-task learning shrinks estimation error","Put task representations on a manifold for sharper MTL","GeoERM: optimize on Stiefel manifold for robust MTL"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's robustness claim rests on the assumption that outlier tasks, optimized jointly with normal tasks in a single objective, do not distort the learned shared manifold structure; no outlier-identification or down-weighting step enforces this, so if outliers pull the shared center the guarantee fails.","fun_headline_variants_meta":{"raw":{"variants":["Riemannian geometry for task representations lowers errors","Curved-space MTL beats Euclidean baselines in robustness","Manifold-aware multi-task learning shrinks estimation error","Put task representations on a manifold for sharper MTL","GeoERM: optimize on Stiefel manifold for robust MTL"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000219,"raw_usage":{"total_tokens":1435,"prompt_tokens":932,"completion_tokens":503,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":548,"completion_tokens_details":{"reasoning_tokens":422}},"tokens_in":548,"tokens_out":503,"duration_ms":5574,"temperature":1.0,"reasoning_tokens":422,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T00:38:20.856445+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a synthetic experiment at fixed $n=100$, $p=50$, $r=5$, $h=0.5$ with an increasing fraction of outlier tasks whose coefficient vectors are drawn with large magnitude but whose feature distributions match the normal tasks; if GeoERM's maximum error on the normal tasks approaches or exceeds that of pERM once the outlier fraction passes some threshold, the claim that manifold geometry alone shields the shared representation from outlier distortion would be contradicted.","supporting_citations":[{"cited_title":"(2008), Optimization algorithms on matrix manifolds\\/ , Princeton University Press","cited_arxiv_id":null,"evidence_quote":"Provides the Riemannian gradient descent convergence theorem and retraction theory used to justify the manifold updates."},{"cited_title":"and Wang, K","cited_arxiv_id":null,"evidence_quote":"Contributes the shared-subspace penalty that pulls each representation toward a common center, along with the ARMUL baseline."},{"cited_title":"(2021), Provable meta-learning of linear representations, in International Conference on Machine Learning\\/ , PMLR","cited_arxiv_id":null,"evidence_quote":"Provides the method-of-moments baseline for linear representation learning."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the AdaptRep baseline used in the numerical comparisons."},{"cited_title":"L., et al","cited_arxiv_id":null,"evidence_quote":"Introduces the Human Activity Recognition dataset used for the real-data evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Represents the Euclidean high-dimensional statistics framework that the paper says must be adapted for Riemannian concentration inequalities."}],"review_version":1}