{"id":"da16765f-5329-4aa9-9844-a1a358b887c0","arxiv_id":"2608.04927","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"In gradually changing linear-regression tasks with a shared zero-loss solution, optimal per-task training time scales as 1/N, so total effective training time Ns remains constant.","lead":"Gradual adaptation over many nearby tasks works best when each task is trained for a time that shrinks in proportion to the number of tasks, so the total training effort stays roughly constant as the path is divided more finely. The paper proves this for linear regression and finds supporting trends in rotated MNIST and Yearbook data.","discovery_kind":"first_principles","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The N^{-1} rule holds only for the whole-path common-kernel distance P_N,s; for target-endpoint loss the same model makes s=∞ optimal (zero target loss), so the abstract's unqualified scaling claim is objective-dependent.","rationale":"I read the paper in good faith and the core mathematics appears coherent: Theorem 1 gives a clean dense-task continuum limit with uniform rates, Theorem 2's tail expansions Θ(τ) and Θ(τ^{-1}) for P_τ are plausible from the Kato-transport analysis, and Proposition 3's fixed-time comparison is consistent with the main mechanism. The internal derivations do not exhibit an obvious gap; the high-dimensional random-path experiment also stays within the theorem's assumptions and shows the predicted N^{-1} exact-fitting decay. The load-bearing weakness lies elsewhere: the headline recommendation s_N^* = Θ(N^{-1}) is tied to a specific objective, reduction of squared distance to the common solution set K. In the same model, final-task loss is exactly zero at s = ∞ because the last projection maps the error into V(1), so the target-endpoint objective has a different, indeed opposite, optimum. The paper's own Rotated MNIST and Yearbook panels show this empirically: target loss keeps decreasing while whole-path loss increases. The reader's verdict already identifies this objective-dependence and attaches a condition requiring the caveat to appear in the abstract and practical discussion. My stress-test sharpens the concern by showing that the target objective makes the N^{-1} rule false, not just unproven, but this does not change the appropriate verdict: with the caveat made explicit, the theoretical contribution stands; without it, the abstract overstates the scope. Therefore the reader's CONDITIONAL verdict remains correct, and no verdict adjustment is needed.","tokens_in":22061,"tokens_out":10124,"duration_ms":97460,"concrete_test":"In the rotating rank-one example of Section B.1, evaluate the target-task loss L_1(w_N) = ½ v_N^T H(1) v_N under the discrete updates (5) for N ∈ {4,8,16,32,64} as s ranges over [0,∞]. Since Φ_N,∞ v0 ∈ V(1), this loss is 0 at s = ∞ for every N; record whether the minimizer sits at the boundary (or moves to larger s as N grows) while the P_N,s maximizer scales as N^{-1}. If so, the N^{-1} scaling law is specific to the whole-path common-kernel objective and does not transfer to target-task performance.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Theorem 2 and the abstract's headline s_N^* = Θ(N^{-1}) are stated for the progress measure P_N,s(v0) = ||v0||^2 - ||Φ_N,s v0||^2, i.e. reduction of squared distance to the common solution set K = ∩_t V(t). That measure is not the usual gradual-adaptation objective of good final-task performance. Under the paper's own Section 2.1 assumptions, the final-task population loss L_1(w_N) = ½ v_N^T H(1) v_N is exactly zero when s = ∞, because Φ_N,∞ v0 = P(t_N)···P(t_1)v0 ∈ V(1) = ker H(1). Hence, for target-endpoint loss on the final task, s_N^* = ∞ is an optimizer for every N, not Θ(N^{-1}). The N^{-1} rule is therefore not merely unproven for the target objective; it is contradicted by the same model, which drives the final error into the final task's null space under full fitting. The paper's own experiments display this: Figure 3 (right) and Figure 4 (right) show final-angle and target-year loss continuing to improve after the whole-path mean loss has turned upward. Thus the central claim's practical validity depends entirely on the choice of the whole-path common-kernel distance objective, and the abstract's unqualified statement is misleading without that caveat.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies how per-task training time should scale when a fixed gradual-adaptation path is sampled into N tasks. It considers a smooth path of overparameterized linear regression tasks t∈[0,1] that share a zero-loss parameter w*, with Hessians H(t) of constant rank, and trains each of the N tasks for time s under gradient flow, so each update is the matrix exponential e^{-sH(t_j)}. Defining learning progress P_{N,s}(v0) as the reduction in squared distance to the common solution set K=∩_t ker H(t) (Eq. (6)), the paper proves (Theorem 1) that when Ns_N→τ the discrete updates converge uniformly to the continuum ODE ẋ=-τH(t)x, with O(N^{-1}) rates on compact τ ranges. Theorem 2 characterizes the limiting progress as Θ(τ) for τ→0 and Θ(τ^{-1}) for τ→∞, and concludes that positive limiting progress occurs exactly when Ns_N=Θ(1), giving s_N^*(v0)=Θ(N^{-1}). Proposition 3 compares fixed finite per-task training with exact fitting at first order, and the paper reports experiments on rotating rank-one regression, rotated MNIST, and Yearbook as qualitative consistency checks.","tokens_in":22274,"tokens_out":14122,"duration_ms":144418,"significance":"The mathematical core is nontrivial and, for the measure it analyzes, appears sound and self-contained: the paper gives an exact telescoping decomposition (Proposition 2), a dense-task limit with explicit uniform rates (Lemma 1), Kato-transport expansions for the large-τ tail (Lemma 2), finite-time first-order expansions with a strict coefficient comparison (Lemma 3), and an explicit anisotropic counterexample showing that the finite-vs-exact comparison can reverse under a different distance (Section A.4). The experiments are honestly labeled as qualitative checks, with fixed seeds, bootstrap intervals, and explicit caveats about departures from the theorem's assumptions. The main qualification is scope: the optimal N^{-1} scaling law concerns the whole-path common-kernel distance measure, not final-task loss, and the abstract and introduction currently state the result without this qualification.","major_comments":[{"comment":"The scaling law s_N^* = Θ(N^{-1}) is established only for the progress measure P_{N,s}(v0) = ||v0||^2 - ||Φ_{N,s}v0||^2, i.e., the reduction of squared distance to the common solution set K = ∩_t V(t). This is not the usual gradual-adaptation objective of good final-task performance. Under the same assumptions, the final-task population loss L_1(w_N) = ½ v_N^T H(1) v_N from Eq. (1) is exactly zero at s = ∞, because Φ_{N,∞}v0 = P(t_N)⋯P(t_1)v0 lies in V(1) = ker H(1). Hence for target-endpoint loss, s = ∞ is an optimizer for every N, not Θ(N^{-1}); the N^{-1} rule is contradicted by the same model when the objective is final-task loss rather than whole-path common-kernel distance. The paper's own Figure 3 (right) and Figure 4 (right) demonstrate the operative distinction: final-angle and target-year losses keep improving after whole-path mean loss has turned upward. The abstract's unqualified statement that optimal per-task training times scale as s_N^* = Θ(N^{-1}) is therefore misleading. The claims must either be explicitly restricted to the whole-path common-kernel distance objective, or the target-endpoint objective must be analyzed separately.","section":"Section 2.2 (Eq. (6)), Section 3.2 (Theorem 2), Abstract"}],"minor_comments":[{"comment":"In the sentence 'coupling its resolution N to the local training times N', the second 'N' should be 's_N'; as printed, the sentence is self-referential and confusing.","section":"Section 1"},{"comment":"The Yearbook interpolated optima are non-monotone between N=16 (4.23) and N=32 (5.01). The text says the optimum 'falls overall' but should explicitly acknowledge this non-monotonicity, since it weakens the visual support for a monotone N^{-1} trend.","section":"Table 2"},{"comment":"The experiments use mean cross-entropy over the path as the empirical objective, whereas Proposition 1 relates the theoretical progress measure to the worst-case path loss sup_t L_t(w), not the average. Since the experiments are qualitative consistency checks this is acceptable, but the mismatch should be stated where the proxy is introduced.","section":"Section B.3 and B.4"},{"comment":"The upper bound uses L from Eq. (3) but L is not redefined in Proposition 1; readers must infer it from the spectral bound. A one-line reminder would improve readability.","section":"Proposition 1, Eq. (7)"},{"comment":"In the second bullet, 'prove its Θ(τ) growth' should read 'prove its Θ(τ) growth' or 'prove the Θ(τ) growth'; the sentence as printed has an awkward missing subject.","section":"Contributions, Section 1"}],"recommendation":"major_revision","confidential_remarks":"The mathematical core appears correct for the stated whole-path distance measure, and the proof techniques are solid and well-documented. The main risk is that the abstract and opening discussion overclaim by presenting the N^{-1} law as a universal property of gradual adaptation, when in fact the same model makes s=∞ optimal for final-task population loss. A prominent caveat plus a short treatment of the target-endpoint objective should be sufficient to resolve the issue. The overlap with Xu and Ma (2026) is limited to a related-work citation and does not raise a novelty concern."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, this is a real theoretical result: for a path of overparameterized linear regression tasks sharing a zero-loss solution, and with progress measured as reduction in squared distance to the common solution set, the optimal per-task gradient-flow time scales as Θ(N^{-1}), i.e. total effective time Θ(1). The proof is self-contained, uses standard Kato transport, and the two-tail argument is convincing. Second, the abstract and the practical framing overstate the result. The same model makes infinite per-task training optimal if you care about final-task loss: fitting to convergence drives the error into the final null space, giving zero target loss. The paper's own figures show target loss still improving after whole-path mean loss has turned upward. So the N^{-1} law is objective-dependent, not a general gradual-adaptation principle.\n\nWhat's new: nobody in the cited gradual adaptation or continual learning theory treats per-task optimization time as a variable coupled to task resolution. The continuum limit (Theorem 1) is clean, and Theorem 2 gives explicit Θ(τ) and Θ(τ^{-1}) tails, which is more than a vague 'don't overtrain' statement. Proposition 3 that finite fitting beats exact fitting at fixed per-task time is a nice, non-obvious point. The experiments are honestly labeled qualitative; the bootstrap intervals contain 1 and the exact constant is not confirmed, which is fine for consistency checks.\n\nSoft spots: the caveat above is the main one. The practical recommendation (fewer updates per task as N grows) is tied to the worst-path loss objective, which the paper defines precisely in Proposition 1 but drops in the abstract. The stress-test note is right: for target-endpoint loss, s=∞ is optimal. The authors do mention in Section 5 that continued training can improve the target task while worsening full-path performance, but that admission belongs in the abstract and in any recommendation to practitioners. Minor issues: the appendix has some dense expansion details that are hard to verify, and the Yearbook bootstrap upper endpoint is grid-limited at N=4.\n\nBottom line: this deserves a serious referee. The math looks sound for what it claims, and the objective-dependence is a caveat, not an error in the proofs. I would accept for review and ask the authors to reframe the claims to state clearly which objective the scaling law applies to.","headline":"A sound and clean theory result for whole-path common-solution progress, but the abstract's unqualified Θ(N^{-1}) claim is misleading because the same model makes infinite per-task training optimal for final-task loss.","tokens_in":22892,"tokens_out":2812,"would_cite":true,"duration_ms":25696,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A gradual adaptation path divided into N tasks is best traversed by training each task for a time proportional to 1/N, because progress vanishes if the total budget is too small or too large.","keywords":["gradual domain adaptation","continual learning","training-time scaling","overparameterized linear regression","gradient flow","whole-path learning progress","effective training time","1/N scaling"],"falsifier":"On a smooth rank-one rotating regression path with known matrices, compute the exact progress $P_{N,s}(v_0)$ on a fine grid of $s$ for $N=16,32,\\ldots,256$ and locate the maximizer $N s_N^*$; Theorem 2 predicts it stays in a fixed bounded interval with all accumulation points maximizing the limiting curve, so observing $N s_N^* \\to 0$ or $N s_N^* \\to \\infty$ would refute the scaling law.","tokens_in":21777,"feed_emoji":"⏱️","tokens_out":7965,"duration_ms":73980,"temperature":0.7,"pith_summary":"The paper asks how the per-task training time should change when the same gradual adaptation path is sliced into more intermediate tasks. It analyzes overparameterized linear regression where tasks change smoothly along a path and all share a zero-loss solution, and it treats the time spent on each task as a tunable design variable. The central result is that learning progress depends on the total effective training time $\\tau = N s_N$: as $N$ grows with $N s_N \\to \\tau$, the discrete sequence converges to a limiting continuum curve, and the limiting progress is $\\Theta(\\tau)$ for small $\\tau$ and $\\Theta(\\tau^{-1})$ for large $\\tau$. Only a constant-order total budget $N s_N = \\Theta(1)$ preserves positive progress, so the optimal per-task time scales as $s_N^* = \\Theta(N^{-1})$. This gives a concrete rule for scheduling updates in gradual adaptation, and experiments on rotated MNIST and temporal Yearbook data are consistent with less per-task training as the path is divided more finely.","feed_headline":"Optimal per-task training shrinks as 1/N on a finer task path","feed_subtitle":"When a gradual path is divided into N tasks, only a fixed total training budget keeps progress positive.","key_machinery":"The central object is the effective training time $\\tau = N s_N$ together with the continuum evolution $\\dot{x}_\\tau = -\\tau H(t) x_\\tau$; its progress identity $P_\\tau = 2\\tau \\int x_\\tau^\\top H x_\\tau\\,dt$ turns the whole-path progress into an accumulated cost that can be compared across all discretizations. The argument combines exact exponential updates $v_j = e^{-s_N H(t_j)}v_{j-1}$, a Taylor expansion of these updates against the continuum ODE with uniform $O(N^{-1})$ approximation, and Kato transport $U(t)$, defined by $\\dot U = [\\dot P, P]U$, which rotates the moving zero-loss spaces into a fixed frame so the large-$N s_N$ regime can be expanded into the operators $G_s$ and $G_H$. Two tail expansions, $P_\\tau = 2\\tau\\langle v_0, A_H v_0\\rangle + O(\\tau^2)$ and $P_\\tau = 2\\tau^{-1}\\langle v_0, G_H v_0\\rangle + o(\\tau^{-1})$, deliver the $\\Theta(\\tau)$ and $\\Theta(\\tau^{-1})$ bounds.","core_discovery":"Under a $C^3$, constant-rank path of Hessians $H(t)$ with a common zero-loss parameter $w^*$, the learning progress $P_{N,s_N}(v_0) = \\|v_0\\|^2 - \\|v_N\\|^2$ (distance squared to the common solution set, after removing the component invisible to all tasks) has a sharp continuum limit. If $N s_N \\to \\tau \\in [0,\\infty)$, the endpoint converges to the solution of $\\dot{x}_\\tau(t) = -\\tau H(t) x_\\tau(t)$, and the progress converges to $P_\\tau(v_0)$, which satisfies the exact identity $P_\\tau(v_0) = 2\\tau \\int_0^1 x_\\tau(t)^\\top H(t) x_\\tau(t)\\,dt$. The limiting curve rises linearly for small $\\tau$ and decays as $\\tau^{-1}$ for large $\\tau$, so its maximum lies at a finite positive $\\tau$. The paper therefore proves that for every sufficiently large $N$, all optimizers of per-task training time are finite and satisfy $s_N^*(v_0)=\\Theta(N^{-1})$, equivalently $N s_N^* = \\Theta(1)$, meaning the total effective training budget stays constant as the path is refined. An explicit corollary is that training every task to convergence belongs to the vanishing-progress regime.","pith_inferences":["An implication the paper leaves implicit: in practical test-time or continual adaptation, the update budget should be set by the rate of distribution change rather than by a fixed number of steps per task, since the total effective training time along the path is what determines progress.","A testable extension is to measure the same scaling under an endpoint-weighted objective; the paper's own right-hand panels in Figures 3 and 4 show final-task loss still improving after whole-path mean loss rises, so a different optimum should appear.","A neighbouring problem it suggests: for a path with non-uniform task spacing, the analogous law would likely become a local update density proportional to the local rate of change of the Hessian, an extension the paper does not address."],"forward_implications":["When a fixed task path is divided into more steps, each task should be trained proportionally less; the optimal total budget $N s_N^*$ stays at the same finite scale rather than growing with $N$.","Training each intermediate task to convergence is not harmless: it is the overtraining regime and its progress falls like $N^{-1}$, tending to zero as the discretization is refined.","Task resolution and per-task optimization time cannot be chosen independently: keeping $s_N$ fixed while increasing $N$ sends the total effective time to infinity and moves the system to the wrong regime.","At any fixed nonzero per-task time, finite fitting gives a strictly larger first-order progress coefficient than exact fitting, and this ordering survives any strictly increasing transformation of squared distance.","Rotated MNIST and temporal Yearbook experiments show fitted exponents whose confidence intervals contain the predicted inverse scaling, supporting the qualitative rule of less per-task training under finer discretization."],"supporting_citations":[{"why":"supplies the gradual self-training setting that motivates studying intermediate distributions and their number.","marker":"(Kumar et al., 2020)"},{"why":"provides existing gradual-domain-adaptation bounds that treat each local step as ERM, the contrast for varying per-task optimization time.","marker":"(Wang et al., 2022a)"},{"why":"exact analysis of forgetting in linear regression, a baseline study of task order and training regime in this setting.","marker":"(Evron et al., 2022)"},{"why":"defines the transport used to put the rotating zero-loss spaces in a fixed coordinate frame.","marker":"(Kato, 1950)"},{"why":"provides the adiabatic contraction estimates used for the large-$\\tau$ expansion.","marker":"(Avron et al., 2012)"},{"why":"contributes the Yearbook portrait dataset used for the natural temporal-shift experiment.","marker":"(Ginosar et al., 2017)"},{"why":"supplies the Wild-Time benchmark preprocessing and data split used for the Yearbook experiment.","marker":"(Yao et al., 2022)"},{"why":"represents early-stopping theory whose mechanism the paper distinguishes from its own whole-path progress effect.","marker":"(Ali et al., 2019)"}],"fun_headline_variants":["Optimal per-task training time scales as 1/N in gradual adaptation","Fixed total budget is optimal for gradual adaptation progress","Training to convergence per task ruins gradual adaptation","Gradual adaptation: less time per task as path refines"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The $N^{-1}$ law depends on measuring progress as whole-path squared-distance reduction to a common zero-loss solution; the paper's own figures show it need not transfer to target-endpoint loss, because final-task loss can keep improving after whole-path progress starts to decline, and the proof also assumes this shared solution and a smooth constant-rank Hessian path.","fun_headline_variants_meta":{"raw":{"variants":["Optimal per-task training time scales as 1/N in gradual adaptation","Fixed total budget is optimal for gradual adaptation progress","Training to convergence per task ruins gradual adaptation","Gradual adaptation: less time per task as path refines"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000553,"raw_usage":{"total_tokens":2652,"prompt_tokens":980,"completion_tokens":1672,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":596,"completion_tokens_details":{"reasoning_tokens":1606}},"tokens_in":596,"tokens_out":1672,"duration_ms":12712,"temperature":1.0,"reasoning_tokens":1606,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T17:17:13.380040+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a smooth rank-one rotating regression path with known matrices, compute the exact progress $P_{N,s}(v_0)$ on a fine grid of $s$ for $N=16,32,\\ldots,256$ and locate the maximizer $N s_N^*$; Theorem 2 predicts it stays in a fixed bounded interval with all accumulation points maximizing the limiting curve, so observing $N s_N^* \\to 0$ or $N s_N^* \\to \\infty$ would refute the scaling law.","supporting_citations":[{"cited_title":"Understanding self-training for gradual domain adaptation","cited_arxiv_id":null,"evidence_quote":"supplies the gradual self-training setting that motivates studying intermediate distributions and their number."},{"cited_title":"Ward, Nathan Srebro, and Daniel Soudry","cited_arxiv_id":null,"evidence_quote":"exact analysis of forgetting in linear regression, a baseline study of task order and training regime in this setting."},{"cited_title":"a henb \\","cited_arxiv_id":null,"evidence_quote":"contributes the Yearbook portrait dataset used for the natural temporal-shift experiment."},{"cited_title":"Zico Kolter, and Ryan J","cited_arxiv_id":null,"evidence_quote":"represents early-stopping theory whose mechanism the paper distinguishes from its own whole-path progress effect."}],"review_version":2}