{"id":"3d6e62b6-c4bd-472f-9d54-024dd4e07c8e","arxiv_id":"2608.08174","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"A prespecified predictive correction can be confirmed or refuted sequentially by the product of corrected-to-source likelihood ratios, which is an anytime-valid e-process.","lead":"This paper builds a sequential monitor that tests whether a pre-specified modification of a predictive model really predicts new outcomes better than the original model. The monitor uses the likelihood ratio between the modified and original predictive distributions, and it stays statistically valid even if you watch it continuously and stop whenever you like.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Source-predictive exactness is the load-bearing assumption; the anytime-valid guarantee is conditional on p0 and does not automatically extend to misspecified or estimated source predictives.","rationale":"The reader's weakest_assumption and my load-bearing concern coincide: exact specification of the source predictive null. The paper is transparent about this limitation, explicitly stating in Section 6 that guarantees are conditional on the fitted source predictive and do not account for source-model misspecification. Table 4 already demonstrates the failure mode. Since the central theorem is a correct application of Ville's inequality and the limitation is openly scoped, the concern does not invalidate the accepted verdict; it merely clarifies that the anytime-valid guarantee is conditional on p_0. The proposed test would quantify the practical impact of estimating p_0 from finite data, but the paper's own framing and experiments already establish the issue. Therefore I recommend keeping the reader's ACCEPT verdict unchanged.","tokens_in":39446,"tokens_out":9766,"duration_ms":104136,"concrete_test":"Run the variance-correction e-process of Section 5.5 with p_0 estimated from a finite training sample rather than oracle: generate D_tr of size n (e.g., n = 50, 200, 1000) from the true Gaussian source, fit the source variance, then generate target outcomes under the same true source (no correction) and record the rate of crossing log(1/α), averaged over repeated draws of D_tr. If the averaged false-confirmation rate exceeds α for moderate n, the anytime-valid guarantee does not hold marginally over the training data and the paper's conditional claim should be flagged as such; if the rate remains at or below α, the conditional guarantee is practically robust in this regime.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Theorem 1 proves that (M_t) is a nonnegative martingale under every P in P_0^pred, but P_0^pred is defined by fixing the conditional law of Y_i given G_i to be exactly the fitted source predictive p_0(·|X_i,D_tr). The anytime-valid false-confirmation guarantee therefore holds only conditionally on this exact specification. If the deployed conditional law is not p_0—for example because the source predictive variance is underestimated by a factor c_mis, or because p_0 was estimated from finite D_tr—the null class is misspecified and the e-process can cross at a rate far above α. Table 4 shows confirmation rates jumping from 0.027 to 0.892 as c_mis goes from 1 to 1.5. This is the load-bearing condition: the practical claim 'anytime-valid evidence for a predictive correction' inherits all the uncertainty in p_0. The paper explicitly states this limitation (Section 6) and does not provide a reference-calibrated or conformal layer, so the guarantee is not unconditional over D_tr. The mathematical argument itself is sound; the soft spot is the gap between the conditional theorem and the unconditional interpretation that practitioners may attach to 'anytime-valid evidence.'","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper develops an anytime-valid framework for evaluating a prespecified predictive correction against a fixed source predictive distribution. A nonnegative tilt h transforms the source predictive p0 into a corrected predictive ph, and the normalized likelihood ratio h/Z_h is shown to be a conditional e-value whose running product is a nonnegative martingale under the source predictive null. This yields a Ville-type bound on false confirmation under continuous monitoring, optional stopping, and arbitrary or adaptively selected input sequences. The paper also derives a conditional drift decomposition for evidence growth under an arbitrary target, identifies a correction-dependent half-space in which the false-confirmation bound persists under misspecification, gives reciprocal refutation boundaries and an overshoot identity, and covers label-shift, Gaussian mean and variance corrections, exponential-family tilts, prespecified mixtures, predictable tilts, and beyond-tolerance comparisons. Synthetic experiments verify the analytic drift predictions and quantify failure modes under source miscalibration and cross-family targets.","tokens_in":39677,"tokens_out":6818,"duration_ms":73209,"significance":"If the results are taken as stated, the paper provides a clean and useful extension of e-process methodology from simple likelihood-ratio monitoring to prespecified predictive corrections, with explicit treatment of adaptive inputs, drift rates, and a well-characterized robustness region. The proofs are standard martingale and Ville arguments, and the experiments reproduce the analytic drift rates to Monte Carlo accuracy. The paper is unusually transparent about its central limitation: the anytime-valid guarantee is conditional on the fitted source predictive being the true conditional law, and Section 6 as well as the miscalibration experiments state this clearly. The contribution is incremental rather than revolutionary, but it is coherent, reproducible, and likely to be of practical interest for sequential model-monitoring and calibration-transfer problems.","major_comments":[],"minor_comments":[{"comment":"The abstract and introduction state the result as 'anytime-valid evidence' without immediately qualifying that the guarantee is conditional on the fitted source predictive p0 being the exact conditional law of Y_i given X_i and Dtr; Section 6 does acknowledge this, but the abstract should carry the same qualification to prevent overstatement in the paper's main public-facing claim.","section":"Abstract and Section 1"},{"comment":"The pseudocode line 'else if exact two-boundary mode and S_i < log α' would be clearer if it explicitly noted that the lower refutation boundary is available only in exact-normalization mode and when h is strictly positive p0-almost surely; the surrounding text states this, but a comment in the algorithm would help avoid misuse.","section":"Section 3.6, Algorithm 1"},{"comment":"The Monte Carlo standard error for the confirmation rates is correctly stated as at most 0.0071, but the paper does not report standard errors for the mean final log wealth values; adding them would make it easier to judge the agreement with the analytic drift predictions.","section":"Section 5.1"},{"comment":"The statement that the cumulative confirmation rate is 'identical at t = 1000 and t = 5000' is an empirical observation; the text could state the first time at which the rate stabilizes or note explicitly that all crossings occurred early, which the null drift implies.","section":"Section 5.10"},{"comment":"The four self-citations to Choi (2026a-d) are preprints or workshop papers; the published version should verify availability and provide arXiv identifiers or DOIs where applicable.","section":"References"}],"recommendation":"minor_revision","confidential_remarks":"The manuscript relies on several self-citations to preprints and workshop papers; the editor may want to confirm that these are publicly available before publication. The paper's scope is appropriate for a statistical methodology journal, and the technical content is sound; the main revision request is about explicitly qualifying the conditionality of the guarantees in the abstract and paper's framing."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core result is exactly what it looks like: a product of normalized tilts is a nonnegative martingale under the source predictive null, so Ville gives you an e-process with optional stopping and adaptive inputs. That is not new in itself, but the paper's contribution is the packaging, and the packaging is good. The arbitrary input-dependent tilt, the drift decomposition into a KL difference, the protected half-space for misspecified targets, the overshoot identity, and the composite-tolerance result are all genuinely useful and are worked out carefully. The experiments agree with the analytic drift predictions to Monte Carlo error, which is the right way to sanity-check this kind of paper. Credit where it is due: the authors are unusually transparent that a crossing supports the correction relative to the source but does not identify the mechanism, and the cross-family experiments demonstrate that limitation rather than hiding it.\n\nThe soft spot is the one the stress-test note flags, and it is real but proportionate. The anytime-valid guarantee is conditional on the fitted source predictive p0 being exactly the deployed conditional law. Table 4 shows what happens when the source variance is miscalibrated: false-confirmation rates jump from 0.027 to 0.892. The paper states this limitation in Section 6, but the abstract and intro could be read as promising an unconditional 'anytime-valid evidence' statement. A reader who takes the title too literally will overclaim. That is not a mathematical flaw; the theorem is valid under the stated null. It is a scope limitation that deserves to be on the first page, not just in the discussion.\n\nMinor points: no code is shipped, which is a shame for reproducibility but does not affect the theory. The self-citations are for special cases and are appropriate; they are not padding. The finite-horizon bounds are honest about being conservative.\n\nWho is this for? Any applied statistician or ML researcher doing calibration transfer, assay drift, or label-shift monitoring with a prespecified correction in mind. It will save them from reinventing this and gives them the right caveats up front. It deserves a serious referee: the math checks out, the limitations are acknowledged, and the experiments support the claims. I would send it to review. I would also bring it to reading group; it is a clean example of how to do sequential confirmation honestly.","headline":"A sound, honest packaging of likelihood-ratio monitoring for prespecified predictive corrections; the anytime-valid guarantee is real but strictly conditional on the source predictive being exactly right, and the paper says so itself.","tokens_in":40166,"tokens_out":1049,"would_cite":true,"duration_ms":14518,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62L10","62F03","60G42"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proves that a prespecified predictive correction can be confirmed sequentially by multiplying corrected-to-source likelihood ratios, with a martingale boundary crossing that gives anytime-valid relative confirmation under…","keywords":["e-values","e-processes","anytime-valid inference","predictive corrections","sequential likelihood ratio","distribution shift","optional stopping","predictive log score"],"falsifier":"Generate outcomes from a Gaussian with variance $c_{\\mathrm{mis}}\\sigma^2$ while monitoring the variance-correction e-process computed with model variance $\\sigma^2$ and correction factor $c=1.8$; under the paper's Table 4, the false-confirmation rate jumps from $0.027$ at $c_{\\mathrm{mis}}=1$ to $0.892$ at $c_{\\mathrm{mis}}=1.5$, so a reader can directly check whether the bound holds only when the source predictive null is true. Alternatively, simulate a target $q$ with $E_q[h(x,Y)]>Z_h(x,D_{\\mathrm{tr}})$ and verify whether the crossing probability exceeds $\\alpha$ when the moment condition of Proposition 3 fails.","tokens_in":39250,"feed_emoji":"📈","tokens_out":5227,"duration_ms":53111,"temperature":0.7,"pith_summary":"The paper shows that when a practitioner fixes a predictive correction before seeing target outcomes, each new outcome can contribute the conditional evidence factor $h(X_i,Y_i)/Z_h(X_i,D_{\\mathrm{tr}})$, where $h$ is the tilt defining the correction and $Z_h$ normalizes it under the source predictive. Under the null that each outcome follows the fixed source predictive $p_0(y\\mid x,D_{\\mathrm{tr}})$, this factor is a conditional e-value, so its running product is a nonnegative martingale. Crossing the boundary $1/\\alpha$ therefore controls the probability of falsely confirming the correction at every stopping time, including continuously monitored and data-dependent stops, for any input mechanism. The logarithm of the product equals the cumulative predictive log-score advantage of the corrected predictive over the source predictive. This matters for small-batch scientific and operational settings where a plausible correction is known in advance and one wants to decide whether to deploy it without fixing a monitoring horizon.","feed_headline":"Predictive corrections get anytime-valid confirmation","feed_subtitle":"Each outcome multiplies the corrected-to-source likelihood ratio; crossing the boundary controls false confirmation under optional stopping.","key_machinery":"The central object is the normalized tilt $e(x,y)=h(x,y)/Z_h(x,D_{\\mathrm{tr}})$, where $Z_h(x,D_{\\mathrm{tr}})=\\int h(x,y)p_0(y\\mid x,D_{\\mathrm{tr}})\\,dy$ is the normalizer of the tilt under the source predictive. Under the source predictive null, $e_i$ is a conditional e-value, and the product $M_t=\\prod_{i=1}^t e_i$ is a nonnegative martingale, so Ville's inequality supplies the anytime-valid crossing bound. The same ratio equals the corrected-to-source predictive likelihood ratio, making $\\log M_t$ exactly the cumulative predictive log-score advantage of the corrected predictive over the source predictive.","core_discovery":"The central discovery is that any fixed nonnegative tilt $h$ with finite positive normalizer turns the source predictive into a corrected predictive, and the corrected-to-source likelihood ratio $e_i=h(X_i,Y_i)/Z_h(X_i,D_{\\mathrm{tr}})$ is a conditional e-value under the source predictive null. Consequently, $M_t=\\prod_{i=1}^t e_i$ is a nonnegative martingale, and Ville's inequality gives $\\sup_{P\\in P_0^{\\mathrm{pred}}} P(\\sup_{t\\ge 0}M_t>1/\\alpha)\\le \\alpha$. This makes the stopping time $\\tau^*=\\inf\\{t:M_t>1/\\alpha\\}$ an anytime-valid test for relative confirmation of the correction, valid under continuous monitoring, optional stopping, and arbitrary or adaptively selected inputs. The paper further shows that the expected log-growth under a target predictive $q$ is $\\Gamma_h(x)=D_{\\mathrm{KL}}(q(\\cdot\\mid x)\\|p_0(\\cdot\\mid x,D_{\\mathrm{tr}}))-D_{\\mathrm{KL}}(q(\\cdot\\mid x)\\|p_h(\\cdot\\mid x,D_{\\mathrm{tr}}))$, so positive drift means the corrected predictive is closer to the target in conditional KL divergence. It also identifies a correction-dependent half-space of misspecified targets where the same $\\alpha$ bound persists, and derives a reciprocal boundary for refutation plus an overshoot identity explaining why the realized null crossing probability is often below $\\alpha$.","pith_inferences":["The conditional nature of the e-process makes it insensitive to pure covariate shift, so a practitioner who wants a full distribution-shift alarm should pair this conditional-outcome monitor with a separate input-distribution monitor; the paper itself notes this separation.","The overshoot identity suggests a practical calibration check: report the conditional mean overshoot $E[M_{\\tau^*}\\mid \\tau^*<\\infty]$ alongside the crossing, since the realized null crossing probability equals the corrected-predictive crossing probability divided by that mean overshoot.","The protected half-space criterion gives a pre-deployment robustness test: before monitoring, one can check whether plausible misspecified target distributions keep $E_q[h(x,Y)]\\le Z_h(x,D_{\\mathrm{tr}})$ at the inputs likely to be seen; if not, the anytime-valid bound is not guaranteed.","The mixture construction provides a family-level evidence claim, not a license to select the best component; a data-driven choice among corrections would require a separately prespecified error allocation or a different rule."],"forward_implications":["A boundary crossing means the corrected predictive has accumulated more than $\\log(1/\\alpha)$ nats of observed log-score advantage, so the procedure justifies relative confirmation of the prespecified correction without estimating the full target distribution.","Validity holds for arbitrary and adaptively selected input sequences, so an experimenter can actively choose informative inputs to accelerate evidence accumulation without changing the source-null error guarantee.","Under i.i.d. or stationary-ergodic sampling, $\\frac1t\\log M_t\\to \\Gamma(q;h)$ almost surely, so the correction is eventually confirmed whenever the corrected predictive has strictly smaller expected log loss, and is eventually refuted when the source predictive does.","Finite-horizon crossing bounds in Corollary 2 control the probability of delayed confirmation, translating the asymptotic growth rate into explicit lower bounds on confirmation by a given time.","For a correctly specified correction, the asymptotic growth rate is the expected KL divergence between the corrected and source predictives, and eventual confirmation is almost sure whenever that divergence has positive expectation over the input distribution."],"supporting_citations":[{"why":"Supplies Ville's inequality, the core time-uniform martingale bound that yields the $\\alpha$ crossing guarantee.","marker":"(Ville, 1939)"},{"why":"Places the two-boundary monitoring procedure as a sequential probability ratio test for a simple predictive pair.","marker":"(Wald, 1945)"},{"why":"Provides the definition and calibration properties of e-values used for the per-observation factor.","marker":"(Vovk and Wang, 2021)"},{"why":"Interprets testing by betting and supports the e-variable / e-process viewpoint of the construction.","marker":"(Shafer, 2021)"},{"why":"Establishes the e-process framework and optional-stopping validity that the paper relies on.","marker":"(Ramdas et al., 2023)"},{"why":"Provides the prequential predictive log-score evaluation that identifies $\\log M_t$ as cumulative log-score advantage.","marker":"(Dawid, 1984)"},{"why":"Motivates tolerated-risk monitoring, which the beyond-tolerance construction extends to comparing full predictive distributions.","marker":"(Podkopaev and Ramdas, 2022)"},{"why":"Prior label-shift confirmation construction that this paper generalizes to input-dependent tilts.","marker":"(Choi, 2026b)"}],"fun_headline_variants":["E-process for predictive corrections: valid under optional stopping","Anytime-valid evidence for predictive corrections via tilt","Corrected-to-source likelihood ratio: anytime-valid test","Predictive corrections: e-process supports early stopping"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The result hinges on the null that each outcome is generated exactly from the fixed source predictive $p_0(y\\mid x,D_{\\mathrm{tr}})$; if the source predictive is miscalibrated, the same false-confirmation bound can fail, as the paper's own sweep illustrates with confirmation rates rising from $0.027$ to $0.892$ when the true variance is $1.5$ times the modeled variance.","fun_headline_variants_meta":{"raw":{"variants":["E-process for predictive corrections: valid under optional stopping","Anytime-valid evidence for predictive corrections via tilt","Corrected-to-source likelihood ratio: anytime-valid test","Predictive corrections: e-process supports early stopping"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00121,"raw_usage":{"total_tokens":5068,"prompt_tokens":1115,"completion_tokens":3953,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":731,"completion_tokens_details":{"reasoning_tokens":3889}},"tokens_in":731,"tokens_out":3953,"duration_ms":27797,"temperature":1.0,"reasoning_tokens":3889,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T00:18:29.048690+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate outcomes from a Gaussian with variance $c_{\\mathrm{mis}}\\sigma^2$ while monitoring the variance-correction e-process computed with model variance $\\sigma^2$ and correction factor $c=1.8$; under the paper's Table 4, the false-confirmation rate jumps from $0.027$ at $c_{\\mathrm{mis}}=1$ to $0.892$ at $c_{\\mathrm{mis}}=1.5$, so a reader can directly check whether the bound holds only when the source predictive null is true. Alternatively, simulate a target $q$ with $E_q[h(x,Y)]>Z_h(x,D_{\\mathrm{tr}})$ and verify whether the crossing probability exceeds $\\alpha$ when the moment condition of Proposition 3 fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies Ville's inequality, the core time-uniform martingale bound that yields the $\\alpha$ crossing guarantee."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Places the two-boundary monitoring procedure as a sequential probability ratio test for a simple predictive pair."},{"cited_title":"and Wang, R","cited_arxiv_id":null,"evidence_quote":"Provides the definition and calibration properties of e-values used for the per-observation factor."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Interprets testing by betting and supports the e-variable / e-process viewpoint of the construction."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the e-process framework and optional-stopping validity that the paper relies on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the prequential predictive log-score evaluation that identifies $\\log M_t$ as cumulative log-score advantage."},{"cited_title":"and Ramdas, A","cited_arxiv_id":null,"evidence_quote":"Motivates tolerated-risk monitoring, which the beyond-tolerance construction extends to comparing full predictive distributions."}],"review_version":1}