{"id":"b16057c1-0a80-4b55-b214-1dc1161ecdee","arxiv_id":"1908.06729","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Simple imputation-based online predictors (Yule-Walker and Kalman filter) outperformed a sampling-based no-imputation method, AERR, across synthetic and real time series with missing values.","lead":"This paper tests five algorithms for forecasting a time series online when some past readings are missing. In its experiments, simply filling missing values with the model's own predictions and then updating the model was the more reliable strategy, beating a sampling-based method that avoids imputation.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"AERR update in Algorithm 4 is not an unbiased gradient under random missingness; its poor MSE may be an artifact, undermining the central imputation conclusion.","rationale":"Reading in good faith, the paper is an empirical comparison whose central conclusion is that simple imputation methods (YW/KF) are reliable and that sampling-based imputation-absent AERR is not. The weakest link is not the MSE definition: the authors generated complete series and then masked values, so evaluating against original values at all positions is feasible and standard; the paper is ambiguous but not necessarily wrong. The duplicate row in Table 1 is a real reporting error but minor. The load-bearing point is that the only non-imputation method is described by pseudocode whose gradient estimator is biased under missingness. I derived the p=1 case directly from Algorithm 4: both sampling steps use the missing indicator, and the expectation of their product does not factor as the product of expectations; the resulting bias is αx²γ/(1−γ) plus label-dependent terms. Thus the reported AERR failure could be a bug, not a property of the approach. A corrected implementation could change the ranking. This is a falsifiable, concrete concern: rerun the experiments with a corrected unbiased estimator. I therefore cannot accept the current version's central comparative claim without that check; 'UNVERDICTED' reflects that the key comparison is not interpretable as reported, while acknowledging the paper's useful algorithmic descriptions and the possibility that imputation may still be competitive.","tokens_in":23344,"tokens_out":19956,"duration_ms":222696,"concrete_test":"Implement Algorithm 4 exactly as written and a corrected AERR variant with a provably unbiased gradient estimator under MCAR missingness (e.g., resample missing coordinates until observed in both sampling steps and drop the 1/(1−γ) factors, or use an inverse-probability weighting that also corrects the label term). Rerun the Section 4.1 missing-rate sweep (standard AR(5), L=2000, σ²=0.32, miss=0..0.3) and the Stock data experiment (Section 4.6.1). Also compute the expected update for p=1 analytically to confirm the bias formula αx²γ/(1−γ). If corrected AERR's MSE moves substantially toward OGD or below, the paper's conclusion about imputation versus imputation-absent methods is not supported by the reported comparison.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"The paper's conclusion that imputation is 'simple but reliable' (Section 4.7) rests on AERR being the representative imputation-absent/sampling method and on its consistently worst MSE. But the AERR implementation in Algorithm 4 has a biased gradient estimator. In the second sampling step, if the chosen coordinate is missing the code sets φ_t=0; if observed it uses φ = ||α||² x_j/(α_j(1−γ)) − y. For p=1, with missing indicator probability γ, the expected product g_t=φ_t x_t is (αx − (1−γ)y)x/(1−γ), not the true gradient (αx−y)x; the bias is αx²γ/(1−γ), nonzero for α≠0 and γ>0. The same failure occurs for p>1 after marginalizing over the missing mask. Hence the update is not stochastic gradient descent on the stated least-squares loss, and the text's claim that γ is used to 'make up for missing values and obtain unbiased estimations' (Section 3.4) is contradicted by the pseudocode. AERR's poor results in Figures 1 and 6–9 may therefore be an artifact of this biased update rather than evidence about the sampling-based, imputation-absent approach.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents an experimental comparison of five methods for online prediction of autoregressive time series with missing values: Yule-Walker estimation (YW), Kalman filtering (KF), online gradient descent (OGD), attribute-efficient ridge regression (AERR), and an offline autoregressive least-squares imputation baseline (ARLS). The authors adapt each method to the online missing-value setting, describe them in Algorithms 1-5, and evaluate MSE under varying missing rates, series lengths, noise variances, AR coefficients, prediction model orders, and on four real datasets. The central conclusion, stated in Sections 4.7 and 5, is that imputation remains a simple, reliable, and effective strategy, whereas the sampling-based imputation-absent representative AERR performs worst overall and is especially sensitive to noise variance and rapidly changing series.","tokens_in":23657,"tokens_out":7580,"duration_ms":76033,"significance":"If the conclusions hold, the paper would provide practical guidance for choosing among online AR prediction methods under missing data, and it would support the common practice of prediction-filling imputation. The strengths of the paper are its systematic algorithmic exposition, the breadth of the experimental sweep (synthetic configurations plus four real datasets), and the inclusion of an offline baseline. However, the central claim is currently supported by a single imputation-absent representative whose implementation appears to have a biased gradient estimator, and several experimental choices (the MSE definition, the noise variance supplied to KF, and the absence of error bars) are not documented well enough to establish the reported rankings. The paper is a useful starting point for a comparison study, but the load-bearing evidence needs substantial clarification and re-analysis before the conclusions can be accepted.","major_comments":[{"comment":"The update rule in Algorithm 4 is not an unbiased stochastic gradient of the stated least-squares loss, contrary to the text's claim that the double sampling gives 'an unbiased estimation of the gradient' and that gamma is used 'to obtain unbiased estimations.' For p=1, let the observation be x and the missing indicator be m with P(m=1)=gamma. The first-stage estimator x_t equals x/(1-gamma) when observed and 0 when missing; the second-stage estimator phi equals alpha x/(1-gamma) - y when observed and 0 when missing. Because both estimators use the same missing indicator, E[phi x_t] = (1-gamma)(alpha x/(1-gamma) - y)(x/(1-gamma)) = alpha x^2/(1-gamma) - xy, whereas the true gradient is alpha x^2 - xy. The bias is alpha x^2 gamma/(1-gamma), which is nonzero for gamma>0 and alpha not equal to 0. The same correlation persists after marginalizing over the missing mask for p>1. Thus the poor MSE of AERR in Figures 1 and 6-9 may be an artifact of a biased update rather than evidence about sampling-based, imputation-absent methods. This is load-bearing because Section 4.7's conclusion that imputation is 'simple but reliable' depends on AERR being a fair representative of the imputation-absent class.","section":"Section 3.4, Algorithm 4"},{"comment":"The MSE definition, MSE(y1:T, ytilde1:T) = (1/T) sum_{t=1}^T (y_t - ytilde_t)^2, is undefined for missing positions because there is no observed y_t. The sentence that follows, saying y_i and ytilde_i denote the actual observation and prediction value, does not specify how missing indices enter the sum. If missing positions are included with y_t replaced by the imputed value, then for imputation-based methods such as YW, KF, and OGD the filled value is exactly their own prediction by construction, so those positions contribute zero error and mechanically favor imputation. If missing positions are excluded, the comparison is different. The paper must state exactly which convention is used, and the reported numbers in Tables/Figures should be recomputed or confirmed under that explicit convention.","section":"Section 4, MSE definition"},{"comment":"Algorithm 2 uses the observation noise variance sigma^2 inside the Kalman gain, G_t <- P_{t|t-1} H_t^T (H_t P_{t|t-1} H_t^T + sigma^2)^{-1}, and the synthetic experiments generate data with a known noise variance sigma^2 (the default theta in Section 4 sets sigma^2 = 0.32). If the experiments pass the true generating sigma^2 to KF, then KF receives oracle information about the noise that YW, OGD, and AERR do not have, which could explain why its MSE is so close to the offline ARLS baseline in Figure 1. The paper should disclose what value of sigma^2 was used for KF in each experiment, and ideally should include a variant in which sigma^2 is estimated online rather than given.","section":"Section 3.2 and Section 4.1"},{"comment":"All conclusions are based on point estimates of average MSE over 20 replications, but Figures 1-9 report no standard errors, confidence intervals, or significance tests. Some of the claimed separations are small, for example the gap between YW and KF in Figure 1, and Section 4.7 asserts that 'KF and YW have generally better performance' and that 'AERR is more sensitive to the noise variance.' Without measures of variability it is impossible to tell whether these orderings are stable across replications or particular to the reported runs. The paper should add error bars or intervals, or at least a table of mean and standard deviation for the main missing-rate experiment.","section":"Section 4.1-4.6"},{"comment":"Algorithm 4 has mandatory parameters B, eta, and k, but the experimental section reports that k=10 was chosen by 'testing on several numbers' and gives no values for B or eta, no tuning grid, and no sensitivity analysis. Since AERR is the only imputation-absent method in the comparison, an under-tuned AERR would invalidate the conclusion that imputation-absent methods are inferior. The paper should report the full parameter settings used for AERR, describe the tuning procedure, and show how the results vary with B and eta.","section":"Section 4.6.1, AERR parameters"}],"minor_comments":[{"comment":"Rows 4 and 5 of Table 1 list the identical coefficient vector [0.1, 0.7, 0.7, 0.0, -0.5], so Figure 4(e) is a duplicate of Figure 4(d) rather than a fifth distinct setting. The intended fifth coefficient vector should be provided and the experiment re-run.","section":"Table 1"},{"comment":"Many figure labels are corrupted into strings such as '/s48/s46/s48/s48' instead of readable text, making legends and axes difficult or impossible to interpret. These figures need to be regenerated with proper fonts and labels.","section":"Figures 1-9"},{"comment":"No code, data, or parameter configuration files are provided. For an experimental evaluation paper, releasing the implementation and exact experimental scripts would substantially improve reproducibility and would also help verify the KF sigma^2 and MSE-convention questions raised above.","section":"General reproducibility"},{"comment":"The notation is inconsistent, with X_t used in Sections 2 and 3.3 while y_t is used elsewhere, and there are typos including 'Bayesain' (Section 3.2), 'overally' (Section 5), and 'F oundations' in reference [36]. Also, Section 4.5 introduces p_fit and p_gen but p_gen is not defined at that point; please define both explicitly.","section":"Notation and typos"},{"comment":"References [4] and [5] appear to be the same paper by Anava, Hazan, and Zeevi cited in two different venues; this duplication should be cleaned up.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The AERR gradient-bias issue is the most serious concern: as written, Algorithm 4 does not implement the unbiased estimator claimed in the text, and this is load-bearing for the paper's main conclusion. I would ask the authors to correct the update or replace AERR with a properly implemented imputation-absent baseline, and to clarify the MSE convention and the KF noise variance, before the manuscript can be considered for publication. If the conclusions remain unchanged after those corrections, the paper could be a useful contribution; in its current form, the experimental evidence is not yet convincing."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis paper is a useful, systematic experimental comparison of five online AR prediction methods under missing data, but its headline conclusion does not actually survive contact with the algorithm. The AERR implementation used as the representative \"no-imputation\" method is biased, so the paper's main claim that imputation is the reliable strategy is weaker than it looks.\n\nWhat's new: it is the first head-to-head of YW, KF, OGD, AERR, and offline ARLS in this online streaming setting, and the online adaptation of AERR (from Hazan-Koren) is not in the original paper. The experimental scope is decent: synthetic series varying length, noise, coefficients, order, missing rate, plus four real datasets. The method descriptions are clear and the prose is readable.\n\nThe soft spots are real, and one is load-bearing. In Algorithm 4, the gradient estimator is not unbiased. The correction (1−γ) compensates for missingness in the first sampling, but when the second-sampling coordinate is missing the code sets φ=0. For p=1, the expected update is αx²/(1−γ) − yx, not the gradient αx² − yx; the bias is γαx²/(1−γ). The same problem appears for p>1. Section 3.4 claims the missing rate is used to \"obtain unbiased estimations,\" but the pseudocode contradicts that. Since AERR is the only semi-imputation method in the comparison, its consistently worst MSE in Figures 1 and 6–9 may be an artifact of this biased update rather than evidence about sampling-based methods. The authors actually note the zeroing in Section 4.3, but they do not seem to realize it breaks unbiasedness.\n\nBeyond that, the experimental reporting is too thin: no error bars or significance tests across the 20 runs, no code or parameter values, Table 1 lists the same coefficient vector twice, and the MSE definition never says how missing positions are handled in the sum. If imputed values are included in the MSE, imputation methods get credit for their own fills, which would bias the comparison. KF is also handed the noise variance in the synthetic experiments, which gives it an unfair advantage.\n\nNone of this kills the paper as an exploratory benchmark. The relative ordering of YW, KF, OGD, and ARLS is probably robust, and the practitioners' takeaway that YW/KF are strong defaults may hold. But the central imputation-vs-sampling conclusion is not supported without fixing AERR or dropping the claim. Worth a serious referee, but only with major revisions.","headline":"Useful benchmark, but the AERR baseline is biased and the headline imputation conclusion does not follow.","tokens_in":24127,"tokens_out":7146,"would_cite":false,"duration_ms":71215,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62M10","62-07"],"pacs":[],"model":"deepseek-v4-flash","headline":"For online autoregressive prediction with missing values, filling gaps with the model's own forecast is reliable and beats sampling-based methods.","keywords":["autoregressive model","online time series prediction","missing values","imputation","Yule-Walker equations","Kalman filter","online gradient descent","attribute-efficient ridge regression"],"falsifier":"Re-run the five methods on the same synthetic AR data and missing masks, but at each missing time $t$ score the imputation $\\tilde{y}_t$ against the true value $y_t$ that was deleted; if AERR then matches or beats YW and KF, the paper's main conclusion that imputation is reliably effective would be contradicted.","tokens_in":23174,"feed_emoji":"📈","tokens_out":12233,"duration_ms":110706,"temperature":0.7,"pith_summary":"This paper tries to establish that, in online one-step-ahead prediction of autoregressive time series with missing observations, the plain strategy of filling each missing value with the current forecast remains the most reliable option. The authors adapt five methods to the online AR setting—Yule-Walker, Kalman Filter, Online Gradient Descent, Attribute-Efficient Ridge Regression, and an offline least-squares baseline—and compare their mean squared error on synthetic series with missing rates from 0 to 30 percent and on four real datasets. Their headline conclusion is that imputation is a simple but reliable and effective way to cope with missing values in online prediction tasks. A reader should care because the result gives concrete guidance: in this common streaming setting, elaborate sampling-based missing-data machinery does not beat straightforward prediction-filling.","feed_headline":"Filling in missing values beats skipping them in online forecasting","feed_subtitle":"Yule-Walker and Kalman with forecast-filling match an offline baseline up to 30% missing data.","key_machinery":"The mechanism under comparison is the missing-value handling strategy, wrapped around the AR model $X_t=\\sum_{i=1}^{p}\\alpha_i X_{t-i}+\\epsilon_t$. The three imputation-based methods (YW, KF, OGD) fill each missing $y_t$ with the current prediction $\\tilde{y}_t$ and then feed the completed sequence into their standard estimator: YW solves the Yule-Walker equations for the autocorrelation coefficients, KF runs the predict-update Kalman recursion on a state-space model whose hidden state is the AR coefficient vector, and OGD updates coefficients by gradient descent on the squared one-step loss. AERR, adapted from attribute-efficient ridge regression, samples from the past $p$ observations to form an unbiased gradient estimate and discards sampled missing values, keeping imputation out of the coefficient update; ARLS is an offline EM-style least-squares method that iterates coefficient estimation and missing-value imputation and serves as the baseline. The comparison is designed to isolate what prediction-filling imputation, versus sampling around missingness, buys in online AR forecasting.","core_discovery":"The paper's central claim is that, for AR-model time series with missing values in an online setting, imputation remains a simple but reliable and effective strategy. Concretely, the experiments show that missing-value-tolerant YW and KF—which replace each missing observation with their own one-step-ahead prediction before updating the model—keep MSE close to the offline ARLS baseline and to the white-noise variance of the generating process, while AERR, the sampling-based semi-imputation method, has the largest MSE under nearly all settings and on stock data reaches over ten times ARLS's error at a 30 percent missing rate. OGD also performs comparatively poorly and is sensitive to missing rates on real data. The claimed implication is that the imputation-absent, sampling-based route is not ideal in this online AR setting, and the practical choice among imputation methods should be driven by data properties: on some real series KF tolerates missingness better than YW, while YW struggles on non-seasonal or trending series and KF suffers when the noise is non-Gaussian.","pith_inferences":["A natural extension would test whether forecast-filling remains competitive when missingness is non-random, such as bursts of consecutive losses, since the paper only generates missing values independently with a fixed rate.","A direct robustness check is to score imputed positions against the true values that the synthetic generator later reveals; the paper does not report this, and it could change the ranking.","The results suggest a neighbouring question: whether the imputation advantage persists for ARMA or ARIMA models or for nonstationary series, which the experiments do not cover.","The online AR benchmark could serve as a cheap testbed for neural-network-based missing-value methods, which the paper cites only as background."],"forward_implications":["Practitioners doing online AR forecasting with missing rates up to 30 percent can reasonably expect KF or YW with simple forecast-filling imputation to approach the accuracy of an offline baseline.","Sampling-based imputation-absent methods such as AERR are unlikely to pay off in online AR settings unless their parameter-tuning and convergence issues are addressed.","The best prediction order is the true AR order; increasing the fitted order beyond the generating order does not improve YW, KF, OGD, or ARLS, and it hurts AERR.","Method choice should depend on the series: avoid YW for non-seasonal or strongly trending series, and avoid KF when the noise is clearly non-Gaussian.","Longer training series shrink the performance gaps, particularly for the gradient-descent methods."],"supporting_citations":[{"why":"Defines the autoregressive model that all five methods assume and that generates the synthetic data.","marker":"[2]"},{"why":"Supplies the direct imputation-absent online method with which AERR is compared and positioned.","marker":"[4]"},{"why":"Provides the offline ARLS algorithm that serves as the experimental baseline.","marker":"[8]"},{"why":"Introduces the Yule-Walker equations that the YW method solves.","marker":"[13]"},{"why":"Introduces attribute-efficient ridge regression, the method AERR adapts.","marker":"[18]"},{"why":"Introduces the Kalman filter recursions used by the KF method.","marker":"[22]"},{"why":"Supplies the bias-correction idea for sampling with missing values that AERR's implementation uses.","marker":"[34]"},{"why":"Discusses solution methods for the Yule-Walker equations used in the YW implementation.","marker":"[37]"}],"fun_headline_variants":["Imputation beats sampling in online AR prediction","Forecast-filling tames missing values in AR forecasting","YW and Kalman fill the gap; sampling falls short","Online AR prediction: impute, don't sample","Missing values? Imputation wins in AR prediction"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison assumes that missing positions should not be counted as prediction errors; if the true values at those positions were scored against the imputed fill-ins, the imputation methods' apparent advantage could shrink or disappear.","fun_headline_variants_meta":{"raw":{"variants":["Imputation beats sampling in online AR prediction","Forecast-filling tames missing values in AR forecasting","YW and Kalman fill the gap; sampling falls short","Online AR prediction: impute, don't sample","Missing values? Imputation wins in AR prediction"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000558,"raw_usage":{"total_tokens":2676,"prompt_tokens":989,"completion_tokens":1687,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":605,"completion_tokens_details":{"reasoning_tokens":1612}},"tokens_in":605,"tokens_out":1687,"duration_ms":12331,"temperature":1.0,"reasoning_tokens":1612,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:02:42.059841+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the five methods on the same synthetic AR data and missing masks, but at each missing time $t$ score the imputation $\\tilde{y}_t$ against the true value $y_t$ that was deleted; if AERR then matches or beats YW and KF, the paper's main conclusion that imputation is reliably effective would be contradicted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the autoregressive model that all five methods assume and that generates the synthetic data."},{"cited_title":"Anava, E","cited_arxiv_id":null,"evidence_quote":"Supplies the direct imputation-absent online method with which AERR is compared and positioned."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the offline ARLS algorithm that serves as the experimental baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the Yule-Walker equations that the YW method solves."},{"cited_title":"Hazan and T","cited_arxiv_id":null,"evidence_quote":"Introduces attribute-efficient ridge regression, the method AERR adapts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the Kalman filter recursions used by the KF method."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the bias-correction idea for sampling with missing values that AERR's implementation uses."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Discusses solution methods for the Yule-Walker equations used in the YW implementation."}],"review_version":1}