{"id":"eb0ba240-8628-49bf-b23d-84a34d3f759a","arxiv_id":"2505.08158","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"FFDCI trains a feature-based quantile error predictor and then applies an online adaptive offset, claiming asymptotic coverage and shorter intervals than existing conformal baselines on 12 datasets.","lead":"This paper proposes an add-on called FFDCI that wraps a deep time series forecasting model and adds prediction intervals using the model's internal features, without retraining the model. If it works as claimed, users of large forecasting models could get uncertainty bands with coverage guarantees at nearly no extra cost.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1 as printed is not what Theorem 3.2 proves: its update uses the wrong sign and lookahead data, so the central coverage guarantee does not apply to the described method.","rationale":"Read in good faith, the intended algorithm is clear from prose and Eq. (11): when an interval covers, shorten it; when it misses, lengthen it. The sign in Algorithm 1 is the opposite, and the timing in Algorithm 1 is infeasible online. The proof of Theorem 3.2 is built on Eq. (11) and a lagged indicator, so it does not certify the printed pseudocode. This is not a matter of reviewer taste or an omitted baseline; it is an internal inconsistency in the artifact the paper asks readers to implement. If the sign and lag are fixed, the main coverage bound is plausible and the empirical results may be reproducible, which is why this is a fixable rejection rather than a claim that the idea is wrong. The MACE theorem has a separate, also load-bearing gap: the lower density bound in Eq. (52) is absent from Assumption 3.3, and the Step 1 telescoping mixes l'(a_t) with l'(a_{t-j}); consequently the claimed bound (13) is not established as written. I agree with the reader's REJECT and would not lower the correctness-risk assessment; the concrete synthetic check above would settle whether the printed algorithm's failure is as severe as the sign analysis indicates.","tokens_in":48590,"tokens_out":7613,"duration_ms":74832,"concrete_test":"Run a one-step (j=1) univariate synthetic experiment with i.i.d. N(0,1) errors, α=0.1, γ=0.002, T=5000, flat q̂: (1) execute Algorithm 1 literally (correcting only the impossible lookahead by using the most recent observed indicator I(y_{t−1}∈C_{t−1}) while keeping the printed sign); (2) execute Eq. (11) with the same lagged indicator. Compare rolling average coverage. If Algorithm 1's sign is used, coverage should fall far below 90% while Eq. (11) tracks ≈90%, confirming that the printed algorithm is not the algorithm Theorem 3.2 bounds.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's coverage guarantee (Theorem 3.2, Eq. 12) is proved for update Eq. (11): a_{t+1}=a_t+γ((1−α)−I(·)), which shrinks after a cover and grows after a miss. Algorithm 1 prints a_{t+1}=a_t+γ(I(·)−α), which does the opposite: it grows after a cover and shrinks after a miss. Moreover, Algorithm 1 evaluates I(y_{t,i,j}∈C_{t,i,j}) at time t, but y_{t,i,j} is not observed until t+j (stated in §2.4); the proof instead uses I(y_{t−j,i,j}∈C_{t−j,i,j}) (Appendix D.1), introducing an additional lag that the pseudocode does not implement. Therefore the theorem as written applies to a different, delay-corrected update, not to Algorithm 1. A second issue: Theorem 3.4's Step 2 (Eq. 52) assumes a lower bound p1 on the conditional density of β near the quantile, which is not stated in Assumption 3.3; without it the MACE bound does not follow. The paper is internally inconsistent on the core update, so the central claim is not supported as submitted, even though the intended update is plausibly fixable.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"FFDCI is a conformal wrapper for frozen deep multivariate multi-horizon point forecasters. A quantile model is fit once on validation-set residuals, using features extracted by the point forecaster as inputs; during deployment the interval for each dimension and horizon is ŷ ± (q̂ + a), where a is an adjustment term updated online from realized coverage. The paper claims Theorem 3.2, per-dimension and per-horizon empirical coverage converging to 1−α at rate O((M+γ)/(Tγ) + j/T), and Theorem 3.4, Mean Absolute Coverage Error bounded by O(√(σ(q*−q̂) + M(j+1)/T)), where σ(q*−q̂) is the RMSE of the feature-based quantile fit. Experiments on 12 datasets with three base forecasters (iTransformer, Leddam, SOFTS) compare coverage, interval length, worst-dimension, and worst-horizon coverage against ECI, TQA-E, LPCI, CF-SST, ACI, PID, SPCI, FCP, and others, with ablations and sensitivity analyses.","tokens_in":48823,"tokens_out":26617,"duration_ms":224427,"significance":"The paper addresses a real need: converting a pretrained deep time series forecaster into one that emits per-dimension, per-horizon confidence intervals without retraining. The design is sensible, combining the ACI update with offline feature-based quantile regression; if the theorems were correct, the MACE bound would provide a clean statement that interval efficiency inherits the quality of the quantile fit. The empirical effort is substantial and a strength of the paper: 12 datasets × 3 base models with ablations, learning-rate and hidden-size sensitivity studies, MC-dropout and quantile-regression comparisons, and a released code repository. The weakness is the theoretical core, which as submitted is internally inconsistent: the coverage theorem is not proved for the algorithm printed in the paper, and the MACE proof rests on an unstated density assumption and contains inequality-direction errors. The central idea is defensible, but the current version does not support its stated claims.","major_comments":[{"comment":"The update rule is inconsistent across Algorithm 1, Eq. (11), Lemma D.1, and the proof of Theorem 3.2, and the printed pseudocode is the wrong one. Algorithm 1 sets a_{t+1,i,j} = a_{t,i,j} + γ(I(y_{t,i,j}∈C_{t,i,j}) − α), which grows the interval after a cover and shrinks it after a miss, while Eq. (11) and the prose specify a_{t+1,i,j} = a_{t,i,j} + γ(1 − I(y_{t,i,j}∈C_{t,i,j}) − α), the opposite direction. In addition, §2.4 correctly states that I(y_{t,i,j}∈C_{t,i,j}) is unobservable until t+j, and Lemma D.1 works with the delayed indicator I(y_{t−j,i,j}∈C_{t−j,i,j}), but Algorithm 1 uses the undelayed indicator. The telescoping identity in the proof of Theorem 3.2, I(y_{t,i,j}∈C_{t,i,j}) = (a_{t+j+1,i,j} − a_{t+j,i,j})/γ + (1−α), has yet another sign relative to Eq. (11) and corresponds to an update of the form a_{t+1} = a_t + γ(I_{t−j} − (1−α)). Under Algorithm 1's sign, the boundedness argument in Lemma D.1 fails: a term that keeps growing after every covered step and shrinking after every miss is not bounded, so the coverage guarantee cannot hold for the pseudocode as printed. The theorem as proved applies to a delay-corrected update with the sign of Eq. (11), not to Algorithm 1; the central claim is therefore unsupported for the described method, although the intended rule is identifiable and presumably fixable.","section":"§2.4, Algorithm 1, Eq. (11), §D.1"},{"comment":"The proof of Theorem 3.4 relies on a condition that is not stated in Assumption 3.3. In Step 2 (Appendix D, Eq. (52)), the inequality ∫_a^{a*} (β−a)p(β)dβ ≥ (p1/2)(a−a*)² requires a positive lower bound p1 on the conditional density p(β) over the whole interval between a and a*; for residual distributions with atoms or near-zero density at the quantile, this fails and the bound in Eq. (53) does not follow. Step 2 is also internally inconsistent: Eq. (47) claims an upper bound E[l(β,a) − l(β,a*)] ≤ (p2/2)(a−a*)², while Eq. (52) derives a lower bound ≥ (p1/2)(a−a*)², and Eq. (53) then substitutes the lower bound while using the wrong constant (p1/2 where the direction requires 2/p1). The proof further inserts this conditional-expectation inequality into the sum of realized losses in Eq. (46) without taking an expectation, so Theorem 3.4 as stated is not established.","section":"Theorem 3.4, Assumption 3.3, §D.2 (Eqs. 47–53)"},{"comment":"There are additional algebra and indexing errors in the coverage proof. Substituting Eq. (26) into Eq. (25) gives |(a_T − a_{j+1})/(Tγ) − (1−α)(j+1)/T| + (j+1)/T, which is bounded by 2(M+γ)/(Tγ) + (2−α)(j+1)/T; the move from Eq. (27) to Eq. (28) discards the (1−α)(j+1)/T term without justification, so the stated constant is too small on the j/T term (the asymptotic O((M+γ)/(Tγ) + j/T) claim itself survives). In Lemma D.1, Case 2 bounds a_{t+1} by a_t + γ(1−α) ≤ M + γ(1−α), implicitly using a_t ≤ M whereas the induction hypothesis only gives a_t ≤ M + γ, and Case 1 concludes a lower bound for a_{t+1} by comparing with a_{t−j} without justifying a_t ≥ a_{t−j}; the lemma is likely true for the delayed update, but the proof as printed is not.","section":"Theorem 3.2 proof, §D.1, Eqs. (24)–(28), Lemma D.1"},{"comment":"The definition of experimental validity is ad hoc and affects the headline empirical claim. Table 1 declares methods 'valid' when overall coverage exceeds 88%, although the nominal target is 90%; FFDCI itself falls to 87.3% on ETTh2 and 88.6% on traffic, while several baseline entries at 88.4% are counted as valid, and the paper reports neither standard errors nor the per-dataset spread of coverage around the 88% line. In addition, Table 10 contains implausible entries (ETTm2/leddam ACI = 0.011 and electricity/leddam ACI = 0.756, against neighboring values near 0.10 and 0.07), which weakens the claim that FFDCI has the smallest approximate MACE in most cases.","section":"§4.2, Tables 1 and 10"}],"minor_comments":[{"comment":"Notation for the dimensions is inconsistent: the prediction horizon is s in §2.2 but d1 elsewhere, the feature dimension is d2 in §2.2 but d1 in §2.3 and Algorithm 1, and Eq. (4) divides by T×s×p while intervals are p×d1 matrices; please harmonize these symbols.","section":"§2.2–§2.3, Algorithm 1"},{"comment":"The definition of σ(q*_{i,j} − q̂_{i,j}) in Eq. (14) averages over t but carries only the (i,j) subscript, and the summand in Eq. (1) uses P(y_t∈C_t) without indicating the dependence on t; the notation should be made explicit.","section":"Eqs. (1), (14)"},{"comment":"The claimed advantage over a constant quantile (q̂ constant) is not formalized: the text asserts that the MACE bound would then involve σ(q*) and that a better-fit q̂ gives an improvement, but no theorem states or proves this comparison under the same assumptions.","section":"§3.2, last paragraph"},{"comment":"The MACE in Eq. (6) is defined through the true conditional coverage probability P(y_t∈C_t), which the paper acknowledges is unobservable; however, no argument connects the 100-step local-coverage proxy of Appendix H to this probability, so the experimental support for Theorem 3.4 is indirect.","section":"§4.2, Appendix H"},{"comment":"Theorem 3.4 bounds deployment-time MACE by σ(q* − q̂), where q* is the unobservable true conditional quantile; the paper's suggestion that σ 'could be regarded as the RMSE of quantile regression' refers to a validation-fit quantity, which need not equal the deployment-time σ under distribution shift, so the practical content of the bound is limited.","section":"Theorem 3.4"},{"comment":"There are numerous typos and naming inconsistencies: 'DDFCI' vs 'FFDCI' in Table 1 and Appendix I, 'week assumptions' (§3), 'donated as' (§2.1), 'holp' (Conclusion), 'last raw' (Appendix I), 'competitve' (Appendix F.1), and the garbled display of Eq. (12).","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The two theorems are the paper's stated theoretical contribution, and both need substantive repair rather than copy-editing: the coverage theorem must be proved for the actual pseudocode (or the pseudocode changed to the delayed, correct-sign rule), and Theorem 3.4 needs explicit density assumptions, a corrected Step 2, and an expectation step. The empirical part is extensive and the method appears to work reasonably, but it cannot by itself support the theoretical claims. I recommend major revision with the expectation that acceptance will require a fixed theoretical core; if the authors cannot repair the MACE proof, the contribution would reduce to an empirical study of a feature-based ACI variant."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this paper has a plausible practical idea and a serious internal inconsistency in the core update rule. As printed, the algorithm is not the thing the theorems prove, and that has to be fixed before the paper is acceptable.\n\nWhat's genuinely useful: the composition of feature-based quantile residual fitting with an online adaptive offset for multivariate multi-step deep forecasting is a sensible extension, and the experimental setup is thorough — 12 datasets, three base forecasters, ablations isolating the two components, sensitivity analysis for gamma and hidden size, plus a GitHub link. The ablation story is coherent: the adaptive offset maintains coverage and the feature-fitted quantile reduces interval width. If the intended update were implemented correctly, the method could be a practical tool.\n\nThe soft spots are load-bearing. Algorithm 1 prints a_{t+1}=a_t+gamma(I-alpha). Equation (11) defines a_{t+1}=a_t+gamma(1-I-alpha). Those drive the interval in opposite directions after coverage. The proof of Theorem 3.2 uses yet another relation (a_{t+j+1}-a_{t+j}=gamma(I-(1-alpha))). So the coverage guarantee stated in Theorem 3.2 applies to a different update than the pseudocode. A reader who implements Algorithm 1 will not get the adaptive behavior described in the prose. Also, the lag — the indicator for time t is only observed at t+j — is addressed in the text but not in the pseudocode. This is fixable, but the paper as submitted doesn't support its central claim.\n\nThe MACE theorem has a separate, smaller issue: Step 2 of the proof assumes a lower bound on the conditional density of the residual near the quantile, but that assumption is never stated. Without it, the quadratic bound on the pinball loss difference fails. This is a missing assumption rather than a fatal error, but it needs to be added and justified.\n\nMinor: the experiments declare 88% coverage as 'valid' when the target is 90%. The full tables make the actual numbers transparent, but the threshold softens the story a bit.\n\nWho it's for: researchers working on conformal prediction for deep forecasting. It deserves a serious referee because the idea is worth pursuing, but the current text needs correction — at minimum, align Algorithm 1 and Eq (11), fix the lag in the pseudocode, and state the density assumption. I'd engage with a revised version.","headline":"Plausible feature-based conformal method for deep forecasting, but the printed update rule contradicts the theorem's update, so the coverage guarantee doesn't apply as submitted.","tokens_in":49382,"tokens_out":3813,"would_cite":false,"duration_ms":34735,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62G15","62M10","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper introduces FFDCI, a conformal method that adds adaptive corrections to feature-based quantile estimates, proving coverage converges to the target level and that interval tightness tracks the quality of the quantile model.","keywords":["conformal prediction","time series forecasting","online conformal inference","deep learning features","quantile regression","adaptive coverage","prediction intervals","distribution shift"],"falsifier":"On a dataset with many repeated or discrete residual values, compute the local coverage of FFDCI over time and compare with the target 90%. If the residual distribution has a point mass at the quantile, the MACE may not shrink as predicted by Theorem 3.4. Additionally, checking whether the actual coverage gap exceeds the bound from Theorem 3.2 for small $T$ on a simple synthetic autoregressive process would test the coverage theorem directly.","tokens_in":48322,"feed_emoji":"📈","tokens_out":3059,"duration_ms":24779,"temperature":0.7,"pith_summary":"For any pretrained deep time series forecaster, FFDCI provides valid and relatively tight prediction intervals without retraining the point model. The core idea is to fit a small quantile-prediction model on the frozen model's internal features to estimate error quantiles, then run a lightweight online adjustment term on top to keep coverage near the target even under distribution shift. Two theorems state that average coverage converges to the nominal level and that the mean absolute coverage error shrinks with the RMSE of the fitted quantiles. If this is right, calibrated uncertainty quantification becomes a cheap add-on to existing deep forecasting systems. Experiments on 12 datasets with three base forecasters report coverage near 90% with shorter intervals than several baselines.","feed_headline":"Pretrained forecasters get valid intervals without retraining","feed_subtitle":"New conformal method fits error quantiles to deep features and adjusts online, with coverage that converges to the target level.","key_machinery":"The central object is the adjustment term $a_{t,i,j}$, updated by online gradient descent on the pinball (quantile) loss, using a lagged coverage indicator because the outcome at step $j$ is only observed $j$ time steps later. The predicted quantile $\\hat{q}_{t,i,j}$ from a feature-fitted quantile model serves as a time-varying offset, so the online algorithm only needs to track the residual between $\\hat{q}$ and the true quantile. The key identity used in the proof is the quantile-loss gradient identity: the derivative of the pinball loss with respect to $a$ is $(1-\\alpha) - I(y \\in \\text{interval})$, which links the update rule directly to the coverage indicator.","core_discovery":"The paper claims that the problem of maintaining valid online prediction intervals can be separated into two easier pieces: predicting the quantile of the absolute error from the deep model's features, and learning only the residual between that predicted quantile and the true quantile through a simple online update. Its Theorem 3.2 shows that for every dimension and horizon, the average coverage over time converges to $1-\\alpha$ at a rate of $O((M+\\gamma)/(T\\gamma) + (j+1)/T)$, relying on bounded errors. Its Theorem 3.4 further claims that the mean absolute coverage error over time is bounded by a constant times the square root of the RMSE of the true minus predicted quantiles plus a horizon-dependent term. The upshot is that the interval length is determined by deep features where they are informative, while the online adjustment guarantees validity without model retraining.","pith_inferences":["A natural extension would be to use more flexible quantile models (e.g., recurrent networks) or to replace the quantile loss with a smoothed surrogate, possibly yielding tighter interval lengths while retaining the same type of coverage bound.","The theorem's dependence on RMSE of the quantile residual suggests that one could actively select features or quantile models by minimizing this RMSE on a validation set, which may provide a principled way to choose among candidate quantile predictors.","The bound also suggests that a quantile model that is periodically updated online could further reduce MACE, although this would require additional analysis beyond the current fixed-model assumption.","The method's lack of explicit handling of regime shifts suggests that incorporating change-point detection or adaptive resetting of the adjustment term could improve robustness in highly non-stationary environments."],"forward_implications":["If the coverage theorem holds, any pretrained deep forecasting model can be wrapped with FFDCI to produce valid intervals without retraining, with the same coverage guarantee for every dimension and every horizon.","The MACE bound implies that better feature-based quantile models lead to tighter intervals while maintaining coverage, so improvements in representation quality directly translate into less uncertainty waste.","Because the update only needs the lagged coverage indicator, the method applies to multi-step forecasting with a fixed horizon and does not require a streaming error signal at every step.","The framework's coverage guarantee holds under distribution shift, provided the error process remains bounded, making it applicable to non-stationary forecasting settings.","The ablation suggests that the two components play complementary roles: the online adjustment maintains coverage when the quantile model is misspecified or the distribution shifts, while the feature-fitted quantile model shortens intervals."],"supporting_citations":[{"why":"Provides the Adaptive Conformal Inference (ACI) update rule that FFDCI extends, and the basic coverage guarantee that Theorem 3.2 builds upon.","marker":"Gibbs & Candes (2021)"},{"why":"Supplies the online conformal prediction framework with arbitrary distribution shifts and the Lipschitz coverage function assumption used in Theorem 3.4.","marker":"Gibbs & Candès (2024)"},{"why":"Defines the per-dimension and per-horizon coverage requirements (Equation 5) and the temporal quantile adjustment baseline TQA-E.","marker":"Lin et al. (2022)"},{"why":"Provides the iTransformer base model and the dataset splitting protocol used in the experiments.","marker":"Liu et al. (2024)"},{"why":"Motivates using features from the point prediction model to calibrate quantiles via representation learning.","marker":"Feldman et al. (2023)"},{"why":"Supplies the online convex optimization regret bound used in Step 1 of the MACE proof.","marker":"Hazan (2016)"}],"fun_headline_variants":["Online conformal intervals for deep forecasting without retraining","Adaptive coverage from deep features without model retraining","Tight prediction intervals via online conformal on deep features","No retraining needed for valid online intervals in forecasting"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The MACE bound assumes the residual distribution has a density that is bounded below by a positive constant near the quantile, so if the residual has atoms or near-zero density at that point, the claimed quadratic lower bound fails and the MACE guarantee does not follow.","fun_headline_variants_meta":{"raw":{"variants":["Online conformal intervals for deep forecasting without retraining","Adaptive coverage from deep features without model retraining","Tight prediction intervals via online conformal on deep features","No retraining needed for valid online intervals in forecasting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000356,"raw_usage":{"total_tokens":1912,"prompt_tokens":902,"completion_tokens":1010,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":518,"completion_tokens_details":{"reasoning_tokens":946}},"tokens_in":518,"tokens_out":1010,"duration_ms":8213,"temperature":1.0,"reasoning_tokens":946,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:03:12.936190+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a dataset with many repeated or discrete residual values, compute the local coverage of FFDCI over time and compare with the target 90%. If the residual distribution has a point mass at the quantile, the MACE may not shrink as predicted by Theorem 3.4. Additionally, checking whether the actual coverage gap exceeds the bound from Theorem 3.2 for small $T$ on a simple synthetic autoregressive process would test the coverage theorem directly.","supporting_citations":[],"review_version":1}