{"id":"5b81925f-8c09-44af-9968-255e93238db6","arxiv_id":"2412.19727","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A Gaussian process forecaster that uses decayed random Fourier signature features to weight recent time series observations more heavily, beating GP baselines and matching diffusion models on standard benchmarks.","lead":"This paper introduces a new way to make signature-based Gaussian process models 'forget' old data points, using learned exponential decay factors inside random Fourier features. It applies the method to time series forecasting and reports speed and accuracy competitive with strong deep learning baselines on eight benchmark datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Missing no-decay control: all experiments use RFDSF, so the paper never isolates whether the novel forgetting mechanism (Eq. 15) drives the reported gains; the fixed global λ also contradicts the 'dynamically adapt' claim.","rationale":"The reader's verdict is CONDITIONAL, and I agree with its conditions. My pass converges on the same verdict but identifies the strongest unaddressed gap as attribution: the paper never compares decayed features against the identical model with decay disabled. The formal mechanism in Eq. 15 is coherent, and the recursion in Eq. 16 correctly implements exponential decay; I am not alleging that the equations are wrong or that the empirical results are fabricated. The concern is that the central framing of the paper ('Learning to Forget', 'dynamically adapt its context length') is not supported by any experiment that varies the forgetting mechanism. The fixed-global-λ limitation is explicitly conceded in the conclusion, which further undercuts the dynamic-adaptation wording. If the no-decay control performs comparably, the novelty claim would be substantially weakened, although the model might still be a useful scalable GP baseline. The β calibration asymmetry noted by the reader is also real: only GP methods receive per-series test-prefix scaling, so the deep-baseline comparison should be re-run with the same calibration. Neither issue is fatal to the paper as a conditional accept, but both must be addressed before the headline claims can be taken at face value. My recommendation is therefore UNCHANGED relative to the reader's CONDITIONAL verdict, with the added explicit requirement of a λ=1 ablation and a fair calibration protocol.","tokens_in":21739,"tokens_out":15819,"duration_ms":176622,"concrete_test":"Run VRS3GP and RS3GP with all decay factors fixed to λ=1 (i.e., the undecayed RFSF map) on the same eight GluonTS datasets and the synthetic multi-sinusoidal task, keeping all other hyperparameters (D=200, M=5, fractional-differencing window W, β calibration) identical. Compare CRPS to the decayed variant in Table 1. If the λ=1 control matches or beats VRS3GP on most datasets, the reported gains are not attributable to the forgetting mechanism. In addition, on a synthetic series with an abrupt regime shift, compare fixed-λ VRS3GP to a sliding-window RFSF with an oracle or adaptive window length; if the fixed-λ model degrades substantially, the 'dynamically adapt its context length' claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that the decayed signature features (RFDSF, Eq. 15) provide a novel, data-driven forgetting mechanism that is responsible for the model's performance. This is never directly tested. Every experiment, synthetic and real, uses the decayed feature map; RS3GP vs VRS3GP varies only whether the RFF frequencies are learned variationally, not whether decay is present. The natural control, RFSF with λ fixed to 1 (no decay) in the same GP framework, is absent from Tables 1 and 2 and from Figure 1. Consequently, the reported advantage over SVGP, DKLGP, and TSDiff could plausibly come from the random-feature architecture, the learned fractional differencing (Appendix A.1), or the per-series β calibration (Appendix B.4) rather than from the forgetting mechanism. The abstract's 'dynamically adapt its context length' is also stronger than what Eq. 15 delivers: λ is a single channel-wise vector learned globally, so the decay rate is constant within a sequence. The paper's Conclusion concedes this, noting that 'more sophisticated forgetting mechanisms might be warranted for capturing complex dependencies in non-stationary time series.' A control with λ=1 is the minimal experiment that would determine whether the central novelty contributes at all; if it performs comparably, the contribution reduces to a hyperparameterization of RFSF rather than a new forgetting mechanism.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Random Fourier Decayed Signature Features (RFDSF), a modification of Random Fourier Signature Features (RFSF) in which each increment contributing to the signature feature map is multiplied by a channel-wise exponential decay factor. These features are embedded in a Gaussian process model, leading to RS3GP and its variational counterpart VRS3GP, with variational inference over the readout weights, RFF frequencies, and phases, plus an objective based on the PPGPR idea. The authors claim that this provides a data-driven forgetting mechanism, one-pass recurrent processing, sublinear-time computations, and favorable performance on forecasting benchmarks: superiority over GP baselines and competitiveness with diffusion-based deep learning models such as TSDiff. The main technical development is the recurrence for RFDSF and its use inside a scalable GP for sequence-to-sequence probabilistic forecasting. The paper also reports experiments on eight GluonTS datasets and a scalability study of runtime and memory against sequence length.","tokens_in":22036,"tokens_out":8269,"duration_ms":72174,"significance":"If the empirical and complexity claims held, the contribution would be practically valuable: RFDSF gives an interpretable, recurrent, finite-dimensional feature map that can be computed in one pass over a sequence, and the GP formulation provides calibrated probabilistic forecasts at lower training cost than competing GP and diffusion baselines. The paper is strongest in its explicit recurrences, the algorithmic presentation in Appendix A.2, the range of benchmark datasets, and the clear discussion of ELBO pathologies in Appendix A.4. However, the central novelty is not isolated experimentally: all reported models use the decayed feature map, so the measured gains cannot be attributed to the forgetting mechanism. The \"sublinear time\" claim is also contradicted by the paper's own O(L) complexity expression. These issues, together with an overstatement of the model's ability to adapt its context length within a sequence, prevent me from recommending acceptance at this stage.","major_comments":[{"comment":"The experimental section never isolates the proposed forgetting mechanism. Every reported model uses RFDSF; the comparison RS3GP versus VRS3GP varies only whether the RFF parameters are treated variationally, not whether decay is present. A minimal control is RFDSF with λ fixed to 1, which reduces to RFSF (with fractional differencing) in the same GP framework. Without this control, the gains over SVGP, DKLGP, and TSDiff could equally be attributed to the random-feature architecture, the learned fractional differencing, or the per-series β calibration. Please add a no-decay ablation to Tables 1 and 2 and to the synthetic experiment in Figure 1, or state explicitly that the empirical contribution is the whole RS3GP pipeline rather than the forgetting mechanism.","section":"Section 4, Table 1; Section 3.1, Eq. (15)"},{"comment":"The displayed unrolling in Eq. (16) is not the unrolling of the recurrence that precedes it. In the k-sum, the factor Φ_{m−p}(x_{0:l−1}) is constant in k, whereas the correct unrolling requires an expression such as Φ_{m−p}(x_{0:k−1}) or an equivalent level-specific incremental term. As printed, the right-hand side does not reduce to the recurrence and does not establish the claimed exponential decay over time steps. Algorithm 2 appears to implement the intended recurrence correctly, so the issue is likely fixable, but the displayed formula must be corrected or proved by induction before the central derivation is acceptable.","section":"Section 3.1, Eq. (16)"},{"comment":"The paper repeatedly claims \"sublinear time computations\" and that long time series can be processed in \"sublinear time due to parallelizability,\" but the stated complexity O((M+W)MLD + MLDd) is linear in the sequence length L. A work-efficient scan has logarithmic depth on a GPU, but the total work remains linear in L. Please either give a formal parallel-complexity statement that supports the claim or replace \"sublinear\" with language such as \"parallelizable with log-depth\" and adjust the abstract, Section 4, and the Conclusion accordingly.","section":"Section 3.1, Eq. (16); Section 4; Appendix A.2"},{"comment":"The abstract's claim that the model \"can dynamically adapt its context length\" is stronger than what Eq. (15) delivers. The decay factor λ is a fixed channel-wise vector learned globally from the training set; it does not vary within a sequence or depend on the input. The model can therefore select one global exponential decay rate, but it cannot adapt to a regime shift or to locally varying context lengths. Section 5 implicitly concedes this when it states that \"more sophisticated forgetting mechanisms might be warranted\" for non-stationary series. Please temper the wording of the abstract and introduction, or add an experiment with a time-varying or input-dependent decay mechanism.","section":"Abstract; Section 3.1; Section 5"}],"minor_comments":[{"comment":"In Eq. (11), the product inside the expectation is written with x_{i_1} and y_{j_1} for every p; it should be x_{i_p} and y_{j_p} to match the definition of the signature kernel in Eq. (6).","section":"Section 3.1, Eq. (11)"},{"comment":"Eq. (15) defines Φ_m(x_{0:l}) but the summation is over Δ_m(K); the index K is inconsistent with the prefix length l and should be l.","section":"Section 3.1, Eq. (15)"},{"comment":"In the related-work paragraph, \"Random Fourier Fignature Features\" should read \"Random Fourier Signature Features\".","section":"Section 1.1"},{"comment":"Table 2 reports training times in hours without standard deviations or a statement of whether these are wall-clock or GPU times; since the comparison against TSDiff and the GP baselines is a headline result, please clarify the measuring procedure and, if possible, report multiple runs.","section":"Table 2"},{"comment":"The β calibration step and the scalability figure are described only briefly; it would help to state explicitly whether β is selected on the observed portion of each test series and to label the axes and hardware in Figure 3 more completely.","section":"Appendix B.4 and B.5"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read the paper. The contribution is a simple, clean trick: put channel-wise exponential decay into Random Fourier Signature Features (RFDSF, Eq. 15) and use the resulting features in a variational GP forecaster. The recursion (16) correctly implements the decayed features, the variational/PPGPR machinery is standard, and the empirical story is largely credible: training is 4-10x faster than TSDiff, and CRPS is competitive on four of eight datasets. That part deserves credit. The problems are in how the contribution is framed and tested. First, the abstract says the model \"can dynamically adapt its context length,\" but λ is a single channel-wise vector learned on the training set. Within a sequence the decay rate is constant. The model can't locally forget after a regime shift. The conclusion quietly concedes this. Second, there is no ablation with λ=1. Every experiment uses RFDSF; a no-decay RFSF baseline in the same GP framework is absent. So the reported gains over SVGP/DKLGP/TSDiff could come from the random-feature architecture, fractional differencing, or the β calibration, not from the new forgetting mechanism. That control is the minimum needed to support the paper's main claim. Third, \"sublinear time computations\" (Section 3.1) is not what the complexity in Appendix A.2 says: it's O(L) work with a log-depth scan. That's an overstatement. Fourth, the per-series β calibration on the observed part of the test set is applied to GP baselines but not to TSDiff/DeepAR etc.; that's favorable treatment of your own baselines and should at least be acknowledged, or applied uniformly. None of these are fatal. The recursion is right, the framework is standard, and the speed-ups are real. But the paper currently sells the forgetting mechanism harder than the evidence supports. A referee should ask for the λ=1 ablation, a corrected complexity statement, and a uniform calibration protocol. The paper deserves peer review — the underlying idea is worth engaging with — but it will need those fixes before the \"dynamic forgetting\" headline can be trusted.","headline":"A useful exponential-decay extension of signature features in a fast GP forecaster, but the forgetting claim is under-tested and oversold.","tokens_in":22667,"tokens_out":3490,"would_cite":true,"duration_ms":36203,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["60L70","62M10","60G15"],"pacs":[],"model":"deepseek-v4-flash","headline":"By inserting channel-wise exponential decay into Random Fourier Signature Features, the paper gives Gaussian-process time-series models a learned forgetting mechanism that processes a full series in one pass and matches diffusion-model…","keywords":["time series forecasting","signature kernels","random Fourier features","Gaussian processes","variational inference","forgetting mechanism","exponential decay","recurrent feature maps"],"falsifier":"Train VRS3GP on a long synthetic series that switches from a slow, long-memory regime to a fast, short-memory regime, and compare post-switch forecasts with a no-forgetting signature baseline. Since the learned decay vector $\\lambda$ is constant across time steps within a sequence, any true within-series adaptation in this experiment would have to come from another component; if VRS3GP does not beat the baseline after the switch, the claim that the decay mechanism lets the model dynamically adapt its context length to recent information is not supported.","tokens_in":21465,"feed_emoji":"📈","tokens_out":9515,"duration_ms":83933,"temperature":0.7,"pith_summary":"This paper proposes a way to make signature-based time-series models forget the distant past. Signature features normally summarize an entire path, which helps capture global structure but hurts forecasting, where recent observations matter most; existing practice handled this by arbitrarily slicing the series into chunks. The authors insert a learned exponential decay factor into each feature channel of random Fourier signature features, so older increments contribute less, and wrap the result in a variational Gaussian process that learns the decay rates from data. The resulting model, VRS3GP, processes a whole series in one recurrent pass and, on eight benchmark datasets, beats other GP baselines and matches the diffusion forecaster TSDiff while training substantially faster. If correct, this replaces ad-hoc context-length choices with a principled, scalable forgetting mechanism for signature methods.","feed_headline":"Signature GPs learn to forget, matching diffusion forecasters","feed_subtitle":"Channel-wise decay rates set context length in one pass; training takes hours instead of days on long series.","key_machinery":"The load-bearing object is the Random Fourier Decayed Signature Feature map, Eq. (15): over ordered time indices it forms products of random Fourier feature differences $\\delta\\varphi^{(p)}(x_{i_p})$, normalizes them by the factorial of the multi-index, and multiplies each factor by $\\lambda^{\\odot(l-i_p)}$ for a channel-wise decay vector $\\lambda\\in\\mathbb{R}^D$. Its recurrence makes the feature vector at every time step available from the previous time step, so the model is recurrent in the sense of a gated RNN while remaining a random feature map for a Gaussian process. The channel-wise decay plays the role of a gating mechanism: channels with $\\lambda$ near one keep long memory, channels with small $\\lambda$ act like short windows, and variational inference learns the balance from data. The same recurrence is implementable with a work-efficient scan, which is what gives the reported near-log-linear GPU behavior and the claim that context length is adapted in a data-driven way.","core_discovery":"The paper's central claim is that a signature's inability to forget is not an inherent limitation but a missing ingredient: if each signature increment is weighted by a learned, channel-wise exponential decay factor, the resulting Random Fourier Decayed Signature Features (RFDSF) give the model an effective context length set by data rather than by an arbitrary slice. The defining recursion has the form $$\\Phi_m(x_{0:l})=\\$lambda^{{\\odot m}}$\\odot\\Phi_m(x_{0:l-1})+\\sum_{p=1}^{m}\\frac{1}{p!}\\$lambda^{{\\odot(m-p)}}$\\odot\\Phi_{m-p}(x_{0:l-1})\\odot\\bigodot_{q=m-p+1}^{m}\\delta\\$varphi^{{(q)}}$(x_l),$$ so the decayed feature at time $l$ is built from the decayed feature at time $l-1$ plus a contribution from the newest increment, and the whole series can be processed in one recurrent, parallelizable pass. On top of these features the authors place a Gaussian-process likelihood with a linear readout and learn the decay factors, frequency lengthscales, and fractional differencing orders by variational inference, using a predictive-variance-aware objective for better calibration. The paper reports that this model, VRS3GP, beats other GP-based forecasters, matches the CRPS of the diffusion model TSDiff, and trains considerably faster on long sequences.","pith_inferences":["The learned $\\lambda$ is a global per-channel parameter, so the 'dynamic' context length is dynamic across datasets and channels, not within a single series; a genuinely time-varying or input-dependent decay would be needed to react to regime shifts in real time.","The exponential-decay assumption is one member of a family; a natural test is whether attention-style weighting over past increments or a state-dependent forget gate improves non-stationary benchmarks, since the current mechanism has no way to re-remember information once it has decayed.","Reported CRPS gains may partly come from the calibration machinery (the modified objective, variance penalty, and per-series uncertainty scaling) rather than from the decay itself; ablating those components would reveal how much of the improvement is attributable to forgetting."],"forward_implications":["Context selection no longer requires hand-slicing: the learned decay vector sets an effective context length per feature channel, replacing ad-hoc windowing in signature-based models.","Long sequences become practical for a Bayesian GP forecaster: one pass over $10^4$ steps runs in about $10^{-2}$ seconds in under 1 GB of GPU memory, with benchmark training times measured in hours instead of days.","The model meets or beats the diffusion forecaster TSDiff on parts of the benchmark suite and outperforms other GP baselines on most datasets, putting a kernel method on par with deep probabilistic forecasters on standard CRPS evaluations.","Because the forgetting mechanism is defined inside the signature feature map, it transfers beyond GPs: the same decayed features could be plugged into other signature-based learning pipelines."],"supporting_citations":[{"why":"Supplies the Random Fourier Signature Features whose recursion the paper modifies; the theoretical guarantee that RFSF approximates the signature kernel motivates the decayed variant.","marker":"Tóth et al. (2023)"},{"why":"Provides random Fourier features and the Bochner-spectral approximation that the signature feature map is built from.","marker":"Rahimi & Recht (2007)"},{"why":"Defines the exact signature kernel for sequential data and the quadratic-complexity recursion that the paper's random features bypass.","marker":"Király & Oberhauser (2019)"},{"why":"Introduces the sparse-spectrum GP treatment with variational distributions over frequency inputs that the Bayesian formulation adopts.","marker":"Gal & Turner (2015)"},{"why":"Establishes the fixed-random-outcome training scheme and reparameterization used to learn the frequency and phase variational parameters.","marker":"Cutajar et al. (2017)"},{"why":"Provides the PPGPR objective whose covariance-aware data-fit term the paper modifies with a variance penalty for calibration.","marker":"Jankowiak et al. (2020)"},{"why":"Defines the TSDiff diffusion baseline and the benchmark protocol whose CRPS results the paper compares against.","marker":"Kollovieh et al. (2024)"}],"fun_headline_variants":["Signature GPs forget on demand, matching diffusion forecasters","Learned decay rates give signature GPs adaptive context lengths","One-pass Bayesian forecaster with learned forgetting rivals TSDiff","Recurrent decayed signature features: fast, memory-lean forecasting"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that one learned exponential decay rate per feature channel, fixed across the whole training set, is enough to capture how much history each forecasting task should remember; if the relevant context length changes within a single series, the model cannot locally adapt.","fun_headline_variants_meta":{"raw":{"variants":["Signature GPs forget on demand, matching diffusion forecasters","Learned decay rates give signature GPs adaptive context lengths","One-pass Bayesian forecaster with learned forgetting rivals TSDiff","Recurrent decayed signature features: fast, memory-lean forecasting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000693,"raw_usage":{"total_tokens":3201,"prompt_tokens":1078,"completion_tokens":2123,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":694,"completion_tokens_details":{"reasoning_tokens":2053}},"tokens_in":694,"tokens_out":2123,"duration_ms":24291,"temperature":1.0,"reasoning_tokens":2053,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:56:03.156798+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train VRS3GP on a long synthetic series that switches from a slow, long-memory regime to a fast, short-memory regime, and compare post-switch forecasts with a no-forgetting signature baseline. Since the learned decay vector $\\lambda$ is constant across time steps within a sequence, any true within-series adaptation in this experiment would have to come from another component; if VRS3GP does not beat the baseline after the switch, the claim that the decay mechanism lets the model dynamically adapt its context length to recent information is not supported.","supporting_citations":[],"review_version":1}