{"id":"84753f75-6ca5-47b7-a3bd-5eca3f0f793a","arxiv_id":"2611.17059","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Capacity-aware OCRC uses randomized proxy delays and inverse probability weighting to make online conformal risk control work with delayed, partially lost feedback under a fixed tracking budget.","lead":"This paper builds an online risk-control algorithm that keeps prediction sets reliable even when feedback arrives late and only a limited number of pending labels can be tracked. It gives convergence rates and a capacity scaling rule, tested on medical imaging, question answering, image classification, and wireless beam selection.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"Corollary 2's proof chooses an infeasible proxy admission probability in the K>E[dt] regime, so the mean-delay scaling law is not established as stated.","rationale":"I read the paper as establishing a worst-case guarantee (Theorem 1, Corollary 1) and a distribution-dependent refinement (Theorem 2, Corollary 2). The arbitrary-delay chain is solid: the saturation Chernoff bound, the K+1 threshold-range argument, and the Bernoulli optimization in Proposition 1 are internally consistent, and the experiments corroborate the qualitative trade-off. The load-bearing soft spot is Corollary 2's proof, not the i.i.d./known-G assumption, which the paper acknowledges in Sec. VII-D. Appendix E-B constructs p>1 in the K>E[dt] regime, violating the survival-function monotonicity constraints. Thus the mean-delay scaling law for \"any capacity K\" is not proven. This matches the reader's conditional verdict and the feasibility concern in their rationale, so I recommend no change: UNCHANGED. A concrete numerical regime can settle whether the statement needs a condition or whether a feasible F in Proposition 2's family attains the bound.","tokens_in":37370,"tokens_out":31143,"duration_ms":216922,"concrete_test":"Numerical check of the proof, not of the theorem: take K=20, E[dt]=1, dmax=T=10^6, α=0.1. First verify that the p in Appendix E-B is ≈3.8, violating 1=F(-1)≥F(0)≥F(1). Then search the feasible family (36) from Proposition 2 (varying θ so that ℓFθ,G≤K) and evaluate the Theorem 2 bound (32) at its optimum. If the best feasible bound exceeds Bη/T + 2ᾱ/T + Cα(E[dt]/T)^{K/(K+1)}, Corollary 2 as stated needs the extra hypothesis K≤E[dt] or a new argument for K>E[dt]; if some feasible F matches the bound, restore the proof with that F.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Corollary 2 is the paper's stochastic-delay scaling law: risk gap ≤ Bη/T + 2ᾱ/T + Cα(E[dt]/T)^{K/(K+1)}. Its proof in Appendix E-B bounds Theorem 2 with the constant proxy law F(0)=1, F(a)=p for a∈{1,...,dmax}, choosing p = K/(e E[dt]) (E[dt]/T)^{1/(K+1)}, and asserts p≤1 \"since K≤E[dt]≤T\". But Corollary 2's hypotheses do not include K≤E[dt]. When K>E[dt], p can exceed 1; e.g., K=20, E[dt]=1, T=10^6 gives p≈3.8. A survival function must satisfy 1=F(-1)≥F(0)≥...≥F(dmax)≥0, as in (27c)/(35c), so this p does not define a valid distribution and the inverse weights 1/F(dt) are not well-defined probabilities. The feasible fallback F≡1 gives, through Theorem 2, only the K QF,G term in (33), i.e., a K/T-type bound, which is not shown to lie below Cα(E[dt]/T)^{K/(K+1)} in this regime. Hence the advertised \"any capacity K\" mean-delay rate is not proved for K>E[dt]; a condition such as K≤E[dt], or a separate large-capacity argument, is needed. This is a proof gap, not an attack on Theorem 1/Corollary 1.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"The paper studies online conformal risk control (OCRC) under delayed feedback when only K pending feedback signals can be tracked. It proposes CA-OCRC, whose tracking policy samples a random proxy delay for each round and whose threshold update uses two-level inverse probability weighting to compensate for feedback that is lost through delayed or untracked rounds. The main theoretical results are a finite-horizon risk bound for arbitrary delays (Theorem 1), a distribution-dependent bound for i.i.d. stochastic delays (Theorem 2), and two scaling laws: O((dmax/T)^{K/(K+1)}) for arbitrary delays (Corollary 1) and O((E[dt]/T)^{K/(K+1)}) for stochastic delays (Corollary 2). The authors also optimize the proxy delay distribution in both settings and validate the method on tumor segmentation, open-domain question answering, hierarchical image classification, and beam selection.","tokens_in":37859,"tokens_out":18642,"duration_ms":118381,"significance":"If the results hold as stated, the paper makes a useful contribution: it extends the delayed-feedback literature for online conformal prediction from the unlimited-capacity regime to a realistic limited-tracking-capacity regime, and it gives an explicit rate trade-off between capacity and risk convergence. The arbitrary-delay part appears carefully derived: Theorem 1 combines a threshold-telescoping argument with a Chernoff-type saturation bound, and Corollary 1 follows from a feasible Bernoulli proxy law. The stochastic-delay refinement, replacing dmax by E[dt], is natural and, if corrected, would strengthen the paper considerably. The experiments cover diverse tasks and compare against reasonable baselines, and the authors are explicit that the correlated-delay experiments are covered only by the worst-case arbitrary-delay guarantee. However, one load-bearing proof step in the stochastic scaling law is not supported as written, so the paper needs revision before the advertised claims can be accepted.","major_comments":[{"comment":"The proof of Corollary 2 bounds Theorem 2 with the constant proxy law F(0)=1 and F(a)=p for a>=1, and chooses p = K/(e E[dt]) (E[dt]/T)^{1/(K+1)}. It asserts that p<=1 and p E[dt]<=K 'since K<=E[dt]<=T'. However, Corollary 2's hypotheses do not include K<=E[dt] or E[dt]<=T. When K>E[dt], the chosen p can exceed 1; for example, with K=20, E[dt]=1 and T=10^6, p is approximately 3.8, which violates the feasible-survival-function constraints in (35c) and makes the inverse weight 1/F(dt) ill-defined. Because the reduction 'it suffices to bound (32) for ... F(a)=p' requires p to be a feasible proxy distribution, the proof does not establish the advertised mean-delay scaling for all K. The feasible choice p=1 gives, through Theorem 2, a bound of order (K+E[dt])/T together with a non-vanishing saturation term, not the claimed C_alpha (E[dt]/T)^{K/(K+1)} in this regime. Corollary 2 therefore needs an explicit hypothesis such as K<=E[dt] (and E[dt]<=T, also used), or a separate large-capacity argument; the 'any capacity K' statements in Sec. I-C and Sec. VIII must be revised accordingly. For fixed K and E[dt] the chosen p eventually becomes feasible for sufficiently large T, so the asymptotic rate may be recoverable, but that qualifier is absent from the theorem as stated.","section":"Appendix E-B, Corollary 2 (Eq. (38))","authors_acknowledge":null,"authors_acknowledge_p":null}],"minor_comments":[{"comment":"The proof of Corollary 2 also divides by E[dt] when defining p; the degenerate case E[dt]=0, where all delays are zero, should be excluded or handled separately in the statement.","section":"Sec. II-C, Eq. (10)","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The closed-form solution (E-11) for the Bregman isotonic regression step is presented very tersely; a brief derivation or an explicit reference for the isotonic solution formula would improve reproducibility.","section":"Appendix E, Proposition 2","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"There is a typo in the motivation paragraph: 'it may lost' should read 'it may be lost.'","section":"Sec. I-A","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The definition of R_F,G uses the conditional expectation E_G[dt | dt <= dmax]; the text should state explicitly that the theorem is used only when q_G(dmax)>0, although in the main setting this follows from dmax being the maximum possible delay.","section":"Sec. VI-A, Eq. (33)","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"major_revision"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"The paper delivers something genuinely useful: the first risk-control guarantees for online conformal prediction under both delayed feedback and a finite tracking budget. The arbitrary-delay analysis (Theorem 1, Corollary 1) is coherent and, as far as I can tell, correct. The threshold telescoping, the saturation bound via Chernoff, and the scaling law K = O(log(T/dmax)) all hang together. The two-level inverse probability weighting is a natural and clean extension of prior intermittent-feedback work, and the paper is honest about where distribution-aware designs need G to be known and i.i.d.\n\nThe soft spot is real and it is in Corollary 2. The proof of that corollary chooses an admission probability p = K/(e E[dt]) (E[dt]/T)^{1/(K+1)} and claims p <= 1 using an assumption K <= E[dt] that never appears in the statement. When K > E[dt] — which nothing rules out, and which is actually the practically interesting case when delays are short relative to capacity — that p exceeds 1, so it is not a valid survival function and the inverse weights 1/F(dt) are not probabilities. The paper's own underload constraint (35b) is not what is being violated; the feasibility of the specific p is. Without that step, the stochastic scaling law O((E[dt]/T)^{K/(K+1)}) is not established for K > E[dt]. It is a gap in one corollary's proof, not in Theorem 2 or in the arbitrary-delay results, and the paper's own worst-case Theorem 1 still covers that regime via dmax. But as written, the claim in the abstract and Sec. VI-C that the mean-delay rate holds for any capacity K is unsupported.\n\nThe experiments are consistent with the qualitative story — more capacity converges faster, worst-case design is conservative, the F=G heuristic saturates — but there is no code or data, so independent verification is limited. That is a minor issue, not a blocker. The paper's limitation statements about correlated delays and unknown G are accurate and appropriately scoped.\n\nThis deserves a serious referee. The fix is straightforward: either add K <= E[dt] to Corollary 2's hypotheses, or give a separate argument for the large-capacity regime. I would accept to review it and my verdict would be conditional on that correction.","headline":"A solid capacity-aware OCRC framework whose arbitrary-delay results are rigorous, but the stochastic-delay scaling law in Corollary 2 has a real proof gap in the K > E[dt] regime.","tokens_in":38981,"tokens_out":595,"duration_ms":41258},"pith_extraction":null,"created_at":"2026-11-13T10:37:24.590143+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium","input_kind":null,"input_chars":null,"input_total_chars":null,"passes_missing":null,"published_standing":null,"published_confidence":null,"objection_audit":null,"run_record":null,"basis":{"kind":"not_recorded","source":null,"chars_given":null,"chars_total":null},"receipt":null,"falsifier":null,"supporting_citations":[],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":null},"url":"https://pith.science/accuracy"},"review_version":1}