{"id":"f01f8091-c535-49eb-9f82-3d562c799a42","arxiv_id":"2505.12585","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"FreKoo splits evolving model parameters into low-frequency trends and high-frequency noise, extrapolates the trends with a learned Koopman operator, and regularizes the noise to improve temporal domain generalization.","lead":"This paper introduces FreKoo, a method that uses frequency analysis of model parameter trajectories to handle concept drift in temporal machine learning. It separates the trajectory into slow, predictable trends and fast noise, then forecasts the trends with a Koopman operator while smoothing the noise.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 1's generalization bound implicitly requires samples from the unseen target domain, so the theoretical grounding of FreKoo does not hold in the TDG setting.","rationale":"The reader's weakest assumption was the spectral-separation hypothesis in Section 3.2.1. That is a legitimate concern about whether low-frequency components always contain the predictable drift. However, the more decisive problem is that Theorem 1, one of the paper's three stated contributions, is not a valid generalization bound for the TDG setting because it assumes access to target-domain samples. Since the abstract and Section 3.3 explicitly claim a multiscale generalization bound connecting spectral dynamics to improved generalization, this flaw directly undercuts the theoretical grounding. The empirical results may still indicate a useful method, but they are weakened by baselines quoted from other papers rather than re-run, and the method trails Koodos on House (Table 1). Because the reader already assigned CONDITIONAL, my analysis reinforces rather than changes that verdict: the paper should be accepted only if the theory is corrected or its claims are narrowed, and if the empirical comparison is conducted under a shared protocol.","tokens_in":20034,"tokens_out":3990,"duration_ms":50376,"concrete_test":"Re-derive Eq. (31) after replacing the target-domain empirical risk in Eq. (37) with a source-only estimator or setting n = 0 (no target access). If the O(1/√n) estimation term vanishes or becomes vacuous, then Theorem 1 cannot support improved target-domain generalization. Additionally, inspect the learned Koopman matrix on Elec2: compute its spectral radius; if ρ(K) ≥ 1, Lemma 3's sharpened exponential-decay bound does not apply and E_low is not controlled.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central theoretical contribution, Theorem 1 (Section 3.3, Eq. (12)), is derived in Appendix B.4 using an empirical risk term on the target domain. In particular, Eq. (37) bounds the true target risk by E_n[ℓ(g(X; θ_hat_{T+1}),Y)] plus O(1/√n), where n is the number of samples drawn from D_{T+1}, and Assumption 2 (Eq. (15)) requires Rademacher complexity evaluated on a sequence drawn from D_{T+1}. But in Temporal Domain Generalization, D_{T+1} is an unseen future domain and no target samples are available during training. Consequently, E_n is undefined, the probability statement over 'the draw of n samples for D_{T+1}' is vacuous, and the theorem does not establish any bound on expected excess risk for the actual target domain. This is not a minor gap in constants; it removes the claimed connection between FreKoo's spectral-Koopman mechanism and improved target generalization. Lemma 1 also does not close the gap: it only records a norm inequality for K^h, and no training constraint enforces ρ(K) < 1, so the 'Koopman stability controls E_low' claim is not guaranteed by the proposed objective.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FreKoo, a method for temporal domain generalization (TDG) that analyzes the trajectory of model parameters in the frequency domain. It decomposes the parameter trajectory into low-frequency and high-frequency components via a discrete Fourier transform, models the low-frequency dynamics with a learned Koopman operator, and applies a smoothness regularization to the high-frequency components. The authors claim a novel multiscale generalization bound (Theorem 1) that connects the spectral decomposition, Koopman stability, and high-frequency regularization to target-domain excess risk. They report state-of-the-art results on six of seven TDG benchmarks, with additional experiments on periodicity modeling, ablations, and sensitivity analysis.","tokens_in":20293,"tokens_out":4136,"duration_ms":42445,"significance":"If valid, the frequency-domain perspective on parameter trajectories would be a meaningful contribution to TDG, offering a principled way to separate predictable drift from noise. The framework is novel and the experimental design covers a diverse set of drift types. However, the theoretical centerpiece, Theorem 1, is derived in a setting that is incompatible with the problem definition, and the bound is expressed in terms of unquantified errors. The empirical superiority is also not uniformly supported: on House the method is worse than Koodos, and on Rot-MNIST the difference is within one standard deviation. The method itself may have merit, but the central claims as presented are not established.","major_comments":[{"comment":"The proof of Theorem 1 bounds the true target risk by an empirical risk E_n[l(g(X;θ̂_{T+1}),Y)] plus O(1/√n), where E_n is over n samples drawn from the target domain D_{T+1}. In the TDG setting, D_{T+1} is an unseen future domain and no target samples are available during training. Therefore E_n is undefined, the probability statement over draws from D_{T+1} is vacuous, and the theorem does not establish any bound on the expected excess risk for the actual target domain. This is not a minor gap in constants; it removes the claimed connection between FreKoo's spectral-Koopman mechanism and improved target generalization.","section":"Section 3.3 / Appendix B.4, Eq. (37)"},{"comment":"The bound in Eq. (12) contains the terms Elow and Ehigh, but these are not bounded by the stated lemmas. Lemma 1 only bounds the propagation of an initial error under K; it does not bound the initial error itself, and no training constraint enforces ρ(K)<1. Lemma 2 establishes an equivalence between Rhigh and a Gaussian random-walk MAP estimate, but provides no bound on Ehigh. Consequently, the first term of the bound is unquantified, and the theorem does not yield a finite generalization guarantee for the proposed objective.","section":"Section 3.3 / Appendix B.2, B.3, Eq. (12)"},{"comment":"The Rademacher complexity bound in Assumption 2 is assumed for sequences drawn from the target domain D_{T+1}. As in the first major comment, this assumption references an inaccessible quantity in TDG, and the proof does not justify how a bound on source-domain complexity transfers to the target. Moreover, the estimation error term in the proof is asserted as a 'standard' bound without formal treatment of the β-mixing assumption or a clear derivation from the stated Rademacher bound to the specific predictor θ̂_{T+1}.","section":"Assumption 2 / Eq. (15) and Appendix B.4"}],"minor_comments":[{"comment":"The claim of a 'principled Bayesian justification' for Rhigh is overstated: Lemma 2 merely restates the Tikhonov form of the regularizer as a Gaussian random-walk prior, which is an equivalence, not an independent justification.","section":"Section 3.3, Lemma 2"},{"comment":"The claim of state-of-the-art performance on six of seven benchmarks is not fully supported: on House, FreKoo (9.0 MAE) is worse than Koodos (8.8), and on Rot-MNIST the difference between FreKoo (6.9±0.7) and Koodos (7.0±0.3) is within one standard deviation. The authors should provide statistical significance tests or a more nuanced discussion.","section":"Table 1"},{"comment":"There is a punctuation error in Eq. (9): the expression ends with a period instead of a closing parenthesis, and the opening parenthesis in the argument of φ^{-1} is not closed.","section":"Eq. (9)"},{"comment":"The P-Moons dataset used for the periodicity experiment is not formally defined in the main text; it appears to be the 37-domain extended 2-Moons benchmark, but this should be stated explicitly before presenting the results.","section":"Section 4.4"},{"comment":"The related-work section mentions several references that are not cited in the main text (e.g., Refs. [36]-[38] are only used in Appendix A). It would improve readability to either cite them in the main text or consolidate the related-work appendix.","section":"Section 2 / Appendix A"}],"recommendation":"reject","confidential_remarks":"The central theoretical result, Theorem 1, is invalid in the TDG setting because its proof requires samples from the unseen target domain. This is a load-bearing error that cannot be fixed by a local correction; it would require a substantially different analysis (e.g., a source-domain bound with drift terms), which is beyond the scope of a revision. The empirical results also do not consistently support the SOTA claim. Given that the theory is a stated main contribution and the empirical gains are partial, I do not see a path to acceptance at this venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"FreKoo is a plausible new combination: split parameter trajectories into low- and high-frequency bands via DFT, extrapolate the low band with a Koopman operator, and regularize the high band toward smoothness. The spectral view on parameter trajectories for TDG is genuinely new as far as I know, and the ablations show each component matters. The 37-domain periodic 2-Moons test is a nice stress test.\n\nBut the theory does not hold up. Theorem 1's proof in Appendix B.4 bounds the true target risk by the empirical risk on D_{T+1}, plus O(1/sqrt(n)). In TDG, D_{T+1} is an unseen future domain with no available samples, so that empirical risk is undefined and the probability statement is vacuous. This is not a missing constant; it removes the claimed link between FreKoo's spectral-Koopman mechanism and generalization to the actual target. Lemma 2 is a restatement: minimizing R_high is exactly MAP under a random-walk prior, so calling it a Bayesian justification is circular. Lemma 1 is just a Jordan-form norm inequality; nothing in the objective enforces rho(K) < 1, so the claim that stability controls E_low is not guaranteed. The core spectral-separation hypothesis, that predictable drift lives in the top-Q Fourier bins, is assumed, not derived.\n\nEmpirically, the results are decent but the 'significant superiority' is overstated. Baselines are quoted from other papers rather than re-run, making the comparison loose. On House, FreKoo (9.0 MAE) is worse than Koodos (8.8); on Rot-MNIST the difference is within one standard deviation. On Elec2 and Appliance the gains are real, and the ablations look consistent, so there is a genuine method here.\n\nWho should read this? Someone working on TDG or parameter-forecasting methods will get value from the idea and the experiments. The theory should be ignored or fixed. I think the paper deserves a serious referee: the method is novel enough and the experiments thorough enough that a peer-review round could clarify whether the spectral separation is robust, whether the baselines can be re-run under a shared protocol, and whether the generalization bound can be repaired, for example with a source-domain complexity term. Acceptance should follow only after those issues are addressed.","headline":"Novel spectral-Koopman combination for TDG with solid empirical promise, but Theorem 1's proof implicitly needs target-domain samples and the theoretical grounding is vacuous.","tokens_in":20825,"tokens_out":3070,"would_cite":false,"duration_ms":30337,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Splitting parameter drift into predictable low frequencies and noisy high frequencies gives state-of-the-art temporal domain generalization.","keywords":["temporal domain generalization","concept drift","parameter trajectory prediction","Fourier spectral decomposition","Koopman operator","frequency-domain learning","domain generalization"],"falsifier":"Construct a parameter trajectory that is a strong low-frequency sinusoid plus a weak linear drift whose frequency bin falls below the energy threshold, train FreKoo with $\\tau$ chosen to mask that bin, and test on the continuation; if the predicted $\\theta_{T+1}$ errors are no better than a method that forecasts all frequencies, the spectral-separation hypothesis is falsified. The same test can be run on real streams by computing oracle optimal parameters of future test domains and measuring the Fourier energy of their difference trajectory in the masked bins.","tokens_in":19827,"feed_emoji":"📈","tokens_out":7369,"duration_ms":70358,"temperature":0.7,"pith_summary":"FreKoo claims that a model's parameter trajectory over time, not just its data, carries the signal needed to generalize to future domains, and that this signal is spectrally separated: long-term incremental and periodic drift concentrates at low frequencies, while transient noise and domain-specific artifacts concentrate at high frequencies. The paper builds a training procedure that decomposes parameter trajectories with a Fourier mask, models the low-frequency part with a learned Koopman operator for stable linear extrapolation, and regularizes the high-frequency part toward smoothness instead of trying to predict it. It proves a multiscale generalization bound in which the excess risk on the future target domain is controlled by the latent prediction errors of the two spectral bands, plus a standard sample-complexity term. On seven temporal-domain-generalization benchmarks spanning synthetic rotations and real-world streams, the method reports the best error on six, with the largest gains on periodic-drift datasets. A sympathetic reader would take the paper's contribution to be a proof of concept that frequency-domain dynamics of parameters is a usable inductive bias for robust temporal generalization.","feed_headline":"Spectral drift split tops six of seven temporal generalization tests","feed_subtitle":"FreKoo keeps low-frequency drift in a Koopman model and smooths high-frequency noise to predict the next domain.","key_machinery":"The central machinery is the spectral decomposition of the parameter trajectory $\\Theta = [\\theta_1,\\dots,\\theta_T]$ via a temporal discrete Fourier transform. An energy proxy averages spectral magnitudes across parameter dimensions, a binary mask keeps the top-$Q$ frequencies ($Q = \\lceil \\tau N_{\\text{freq}} \\rceil$), and inverse DFT splits $\\Theta$ into $\\Theta_{\\text{low}} + \\Theta_{\\text{high}}$. The low-frequency part feeds an encoder $\\phi_{\\text{low}}$ into a latent space where a learnable matrix $K$ approximates the Koopman operator and propagates $z_{t,\\text{low}}$ linearly, with a reconstruction loss enforcing consistency; the high-frequency part feeds a separate encoder and is penalized by $\\sum_{t} \\|z_{t+1,\\text{high}} - z_{t,\\text{high}}\\|^2$, equivalent to MAP estimation under a random-walk prior. The predicted next parameters are $\\phi^{-1}(K\\phi_{\\text{low}}(\\theta_{t,\\text{low}}) + \\phi_{\\text{high}}(\\theta_{t,\\text{high}}))$. This construction carries the argument: $K$'s Jordan structure bounds low-frequency extrapolation error, the regularizer controls high-frequency bias, and Theorem 1 chains both to target-domain excess risk.","core_discovery":"The central claim is that temporal domain generalization reduces to predicting the next point of a parameter trajectory, and that this prediction is made reliable by separating the trajectory into a predictable low-frequency component and a noisy high-frequency component. FreKoo selects the dominant Fourier bins by average spectral magnitude, keeps those as $\\Theta_{\\text{low}}$ and models their evolution with a linear Koopman operator $K$ in a learned latent space, while the residual $\\Theta_{\\text{high}}$ is not extrapolated but constrained by a temporal-difference regularizer equivalent to a Gaussian random-walk prior. The predicted parameter for the next domain is decoded from the sum of the Koopman-extrapolated low-frequency latent and the regularized high-frequency latent. The theory states that the excess risk on the target domain is bounded by Lipschitz constants times $(E_{\\text{low}} + E_{\\text{high}})$ plus a Rademacher-complexity term, where $E_{\\text{low}}$ is controlled by Koopman stability and $E_{\\text{high}}$ by the smoothness regularizer; experiments report state-of-the-art error on six of seven benchmarks, with clear gains where drift is periodic or noisy.","pith_inferences":["Beyond the paper: the spectral-separation hypothesis implies a testable diagnostic—compute the oracle optimal parameters for future test domains and check whether their Fourier energy lies in the masked low-frequency bins; if it does not, no amount of Koopman tuning can rescue the mask.","The paper leaves the threshold $\\tau$ as a fixed hyperparameter; a natural extension is to learn $\\tau$ per dimension or per dataset from a validation future domain, since the sensitivity analysis shows the optimal value differs between 2-Moons (higher better) and Appliance (intermediate).","If concept drift contains sudden regime shifts, those shifts appear as broad-spectrum energy and will be partially smoothed away by the high-frequency regularizer; a testable extension is to compare FreKoo against detect-then-adapt methods on sudden-drift streams, where the method's assumptions are least favorable.","Another implication is that the parameter trajectory itself becomes a learned representation: after training, $\\Theta_{\\text{low}}$ can be inspected as a spectral signature of the environment, helping practitioners distinguish periodic market or usage cycles from noise in deployed models."],"forward_implications":["On benchmarks with recurring seasonal drift (Elec2 and Appliance energy), the method reports errors of 9.2% and 4.0 MAE, substantially below prior temporal domain generalization baselines, so periodicity does not have to be detected explicitly to be exploited.","Because the generalization bound is additive in $E_{\\text{low}}$ and $E_{\\text{high}}$, any future method that reduces either latent error directly improves the target-domain guarantee.","The frequency decomposition is applied to parameters rather than data, so the framework can be attached to any base model whose parameters are trained sequentially on ordered domains.","The ablation shows that removing the Koopman term degrades 2-Moons error from 1.0% to 15.3% and Elec2 from 9.2% to 28.9%, indicating the linear low-frequency extrapolation is the main driver of the gains."],"supporting_citations":[{"why":"Supplies the parameter-trajectory prediction paradigm and the seven-dataset experimental protocol that FreKoo builds on and compares against.","marker":"[8]"},{"why":"Represents the smooth-incremental temporal domain generalization baseline whose assumption of local monotone change FreKoo is designed to overcome.","marker":"[6]"},{"why":"Koopman-based continuous temporal domain generalization baseline; FreKoo's main comparison for periodic-drift benchmarks.","marker":"[12]"},{"why":"Provides the frequency-domain decomposition and Koopman predictor for non-stationary time series that FreKoo adapts to parameter trajectories.","marker":"[16]"},{"why":"Shows frequency-domain representations expose long-term trends and seasonal variation hidden in standard statistics, justifying the spectral separation.","marker":"[19]"},{"why":"Introduces the Koopman operator as a linear representation of nonlinear dynamics, the theoretical basis for the learned matrix $K$.","marker":"[20]"},{"why":"Supplies modern Koopman theory for dynamical systems, used to justify learning a finite-dimensional linear surrogate.","marker":"[21]"},{"why":"Provides the empirical Rademacher complexity generalization bound used in the proof of Theorem 1.","marker":"[44]"},{"why":"Supplies the Jordan-form norm bound that controls the propagated error in the Koopman stability lemma.","marker":"[46]"}],"fun_headline_variants":["FreKoo splits frequency to beat drift in six of seven tests","Koopman models low-freq drift, smooths high-freq noise","Spectral split predicts next domain, wins six of seven tests","Spectral band separation tops six benchmarks in TDG","High-freq noise smoothed, low-freq drift modeled: FreKoo wins"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the future-relevant part of parameter drift, including incremental trends and recurring cycles, occupies the few strongest Fourier frequencies selected by the energy mask, while everything discarded as high-frequency is transient noise; if real signal lives in the masked-out bins, the method both fails to extrapolate it and actively smooths it away.","fun_headline_variants_meta":{"raw":{"variants":["FreKoo splits frequency to beat drift in six of seven tests","Koopman models low-freq drift, smooths high-freq noise","Spectral split predicts next domain, wins six of seven tests","Spectral band separation tops six benchmarks in TDG","High-freq noise smoothed, low-freq drift modeled: FreKoo wins"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000788,"raw_usage":{"total_tokens":3510,"prompt_tokens":1013,"completion_tokens":2497,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":629,"completion_tokens_details":{"reasoning_tokens":2405}},"tokens_in":629,"tokens_out":2497,"duration_ms":20125,"temperature":1.0,"reasoning_tokens":2405,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:31:33.177813+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a parameter trajectory that is a strong low-frequency sinusoid plus a weak linear drift whose frequency bin falls below the energy threshold, train FreKoo with $\\tau$ chosen to mask that bin, and test on the continuation; if the predicted $\\theta_{T+1}$ errors are no better than a method that forecasts all frequencies, the spectral-separation hypothesis is falsified. The same test can be run on real streams by computing oracle optimal parameters of future test domains and measuring the Fourier energy of their difference trajectory in the masked bins.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the parameter-trajectory prediction paradigm and the seven-dataset experimental protocol that FreKoo builds on and compares against."},{"cited_title":"Thakur, V","cited_arxiv_id":null,"evidence_quote":"Represents the smooth-incremental temporal domain generalization baseline whose assumption of local monotone change FreKoo is designed to overcome."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the frequency-domain decomposition and Koopman predictor for non-stationary time series that FreKoo adapts to parameter trajectories."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows frequency-domain representations expose long-term trends and seasonal variation hidden in standard statistics, justifying the spectral separation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the Koopman operator as a linear representation of nonlinear dynamics, the theoretical basis for the learned matrix $K$."},{"cited_title":"Machine learning foundations","cited_arxiv_id":null,"evidence_quote":"Provides the empirical Rademacher complexity generalization bound used in the proof of Theorem 1."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Jordan-form norm bound that controls the propagated error in the Koopman stability lemma."}],"review_version":1}