{"id":"66036219-ab13-4149-adfb-06faad8cb3e9","arxiv_id":"2411.18569","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A new confidence-interval method, the zoom correction, provides valid post-selection inference on the winner under arbitrary dependence and without tuning parameters.","lead":"This paper introduces the zoom correction, a method for building valid confidence intervals for the performance of the best-looking candidate in a noisy comparison. The method works under weak assumptions, adapts to how close the competition is, and is demonstrated on leaderboard, climate, and language data.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Validity hinges on a correctly specified tail bound S; real-data applications estimate S, so the finite-sample coverage guarantee is not delivered as stated.","rationale":"I traced the proof of Lemma 3.1 and Theorem 3.1. Lemma B.1 and the monotonicity lemmas B.2-B.3 are valid; B.3's proof phrases an argument in terms of P(X in A) rather than S, but the conclusion follows from S-monotonicity alone, so the theorem is sound as a mathematical statement. Grid search is an acknowledged approximation to computing min and max of C; it does not invalidate Theorem 3.1 but should be bounded or replaced by a certified search in the implemented method. The variance-adaptive extension is not central, but its 'valid for any choice of sigma_i^2' claim needs qualification when sigma_i are data-dependent, since the tail bound S is unconditional. The reader's CONDITIONAL verdict is appropriate: the main claim is supported under the stated assumptions, but the assumptions and the implementation guarantees should be stated more carefully before the method is used as an off-the-shelf defense against the winner's curse.","tokens_in":28798,"tokens_out":33728,"duration_ms":300371,"concrete_test":"Run a calibration experiment: draw X_1,...,X_m ~ N(theta,Sigma) with m=50, theta=0, and unknown Sigma. On each replicate, estimate Sigma_hat from the same sample, set S via the plug-in multivariate normal, and compute the step-down zoom interval (Algorithm 1) at alpha=0.1. Over 10,000 replicates, measure the empirical coverage of theta_hat_i. If coverage is materially below 0.9, the plug-in-S route fails; repeating with S built from an independent half-sample should restore coverage near 0.9, confirming that the method is only as good as the input tail bound.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central Theorem 3.1 is correct conditional on S(v) being a valid tail bound, because every guarantee, including the step-down Theorem 4.1, uses r_alpha defined through S: if P(exists i: |xi_i|>v_i) exceeds S(v) for any v, the acceptance probability can fall below 1-alpha and the projection interval can undercover. This is not a proof error, but it is the most load-bearing point of the argument. In nonparametric settings with arbitrary dependence, a valid S is not available except by a union bound over marginals, which weakens the method and may be far from tight. In the real-data sections (climate, leaderboards, politeness), S is constructed from normal approximations and plug-in covariance estimates, so the reported intervals inherit an approximation error that the finite-sample guarantee does not cover. Estimating S from the same data is explicitly outside the assumption, yet the paper's broad 'nonparametric, arbitrary dependence' framing makes this input condition easy to miss. The theorem itself stands; the advertised flexibility is only as strong as the user-supplied S.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes the \"zoom correction,\" a finite-sample confidence-interval method for the mean of the empirically selected best candidate. The idea is to invert a \"zoom test\" whose acceptance rectangle is deliberately widened around suboptimal coordinates, forming a simultaneous confidence region, and then to project this region onto the winning coordinate. Lemma 3.1 characterizes the projection exactly as {t : |X_hat - t| <= r_alpha(theta^t)}, which yields the main coverage theorem (Theorem 3.1). Under a union-bound tail bound, a step-down implementation is given in Theorem 4.1, with extensions to top-k winners, the population winner's value and identity, near-winners, and a variance-adaptive variant. The paper includes proofs, synthetic experiments, and applications to climate extremes, model leaderboards, and language devices.","tokens_in":28991,"tokens_out":16962,"duration_ms":152906,"significance":"If Theorem 3.1 stands, this is a valuable contribution: it provides valid inference on the empirical winner under arbitrary dependence and with no tuning parameters, requiring only a known tail bound on the errors. I checked the central proof steps (Lemmas 3.1, B.1-B.3, subsequent lemmas, and the step-down arguments) and found the mathematical core sound. The method strictly dominates Bonferroni and locally simultaneous inference in the examples, and the code is provided. The principal caveat is that the finite-sample guarantee is conditional on a correctly specified tail bound S; the real-data sections estimate this input, so the reported intervals do not automatically inherit the stated coverage. This caveat is manageable but needs to be made prominent and addressed.","major_comments":[{"comment":"The validity statement in Section 1.1 is explicit that the method requires a valid tail bound S(v) with P(exists i: |xi_i| > v_i) <= S(v), and the coverage theorem is conditional on that object. In the real-data analyses, however, S is not known and is instead estimated: Section 6.2 uses a covariance matrix estimated from older years, Section 6.3 uses normal approximations for logistic regression, and Section 6.4 uses plug-in covariance estimates for logistic coefficients and Fisher z-transformed correlations. The resulting intervals therefore inherit an approximation error that Theorem 3.1 and Theorem 4.1 do not cover. This is the load-bearing input condition of the method, so the paper should either construct S in a way that supplies finite-sample guarantees (for example by using nonparametric tail bounds with a valid uncertainty correction), or explicitly reframe the empirical sections as approximate and study sensitivity of the endpoints to misspecification of S. Without this, the advertised 'nonparametric, arbitrary dependence' claim is stronger than what is delivered.","section":"Sections 1.1 and 6.2-6.4"},{"comment":"The theorem defines tl and tu as the minimum and maximum of the exact set C_hat^alpha = {t : |X_hat - t| <= r_alpha(theta^t)}, but the implementation used in all experiments is a grid search over t in [X_hat - r_alpha(0), X_hat + r_alpha(0)]. The paper acknowledges this 'technically allows for some inaccuracy' and says it expects the inaccuracy to be negligible. As written, a finite grid can miss the true endpoints, so the plotted intervals are not the exact zoom-correction intervals and could undercover relative to 1 - alpha. Please provide an error-controlled implementation, such as conservative bounds on the discretization error or a root-finding based method, or at least report a numerical study of grid sensitivity to demonstrate that the practical coverage is not compromised.","section":"Section 3, paragraph after Theorem 3.1"}],"minor_comments":[{"comment":"The abstract promises validity under 'arbitrary dependencies' without mentioning the known-tail-bound condition. Consider adding a qualifier in the abstract so that readers are immediately aware that the guarantee is conditional on a user-supplied S(v).","section":"Section 1.1 and Abstract"},{"comment":"The proof invokes 'the continuity of the noise distribution' to assert the boundary condition tl = X_hat - r_alpha(theta^tl), but the paper assumes continuity of S, not of the error distribution. The step-down proof in Theorem 4.1 actually only needs inequalities Sl(rl) >= alpha and Su(ru) >= alpha; please rephrase to avoid an unstated distributional assumption.","section":"Lemma 4.1 proof"},{"comment":"The phrase 'known estimate of the variance' is internally contradictory, and the text immediately says the procedure is valid for any choice of sigma_i. Please describe sigma_i as user-specified positive scale parameters (or known fixed constants), not as estimated variances, to avoid confusion.","section":"Section 5.4"},{"comment":"The update alpha_{j+1} = alpha_j - S((bDelta_(j) + S^{-1}(alpha))/3) uses the fixed uncorrected quantile S^{-1}(alpha) rather than a quantity depending on the current step. The proof shows this is conservative, but a short intuitive explanation of why this particular update is chosen would improve readability.","section":"Algorithm 2"},{"comment":"When saying that older data are used to estimate the covariance, please specify the estimator (e.g., sample covariance) and the number of years available, since the quality of the plug-in S is relevant to the first major comment.","section":"Section 6.2"}],"recommendation":"major_revision","confidential_remarks":"This is a strong paper with a correct and elegant central theorem. My recommendation of major revision is driven by the gap between the finite-sample guarantee and the way the method is applied in the empirical sections: the latter estimate the tail bound S, and the paper does not quantify the resulting approximation error. This is fixable by reframing the claims and adding a sensitivity analysis or a valid nonparametric construction of S. The grid-search implementation is a second concern but is also local and fixable. I do not see a fatal flaw in the mathematical core."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The zoom correction is a real advance in selective inference, not a repackaging. The key idea is to build a simultaneous confidence region whose coordinate radii are max{active radius, suboptimality} and then project onto the empirical winner. The computational shortcut in Lemma 3.1, using the worst-case vector θ^t, is the technical core, and it is genuinely new. The step-down implementation in Section 4 is a nice practical addition, and the extensions to top-k winners, the population winner, and near-winners are nontrivial. The method recovers uncorrected inference when the winner is obvious, which LSI and hybrid do not.\n\nThe central coverage theorem holds up. I checked Lemmas 3.1, B.1–B.3, and the step-down arguments in Theorem 4.1; I found no gaps. The method is tuning-free in the main version, and the code and data are shipped, so the experiments are reproducible. This is careful work.\n\nThe soft spots are real but proportionate. The entire guarantee is conditional on a valid tail bound S(v) with P(∃i : |ξ_i|>v_i) ≤ S(v). If S is misspecified or estimated from the same data, coverage can fail. The paper states this, but the framing 'nonparametric, arbitrary dependence' makes it easy to miss, and the real-data sections construct S from normal approximations and plug-in covariance estimates. Those intervals are useful, but they do not carry the finite-sample guarantee of Theorem 3.1. The stress-test note is right: this is the most load-bearing point in the paper. The grid-search implementation is acknowledged to be approximate, and the variance-adaptive extension in Section 5.4 is proven only for fixed σ_i, not data-dependent estimates. Those are minor relative to the main argument; the tail-bound issue is the one that needs to be stated clearly at the front.\n\nThis paper deserves a serious referee. It is a substantial methodological contribution with sound proofs and broad applicability. In review, I would ask the authors to make the S-input condition prominent, bound the grid-search error or slow down the claim, and state explicitly which results hold with estimated variances. The step-down version, which uses only marginal tail bounds, is the fully rigorous version and deserves emphasis.","headline":"A genuinely new, mostly sound method for inference on the winner; its advertised flexibility is exactly as strong as the user-supplied tail bound, and the real-data intervals go beyond that guarantee.","tokens_in":29510,"tokens_out":2241,"would_cite":true,"duration_ms":20560,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F25","62F07"],"pacs":[],"model":"deepseek-v4-flash","headline":"The zoom correction gives valid confidence intervals for the winner's mean under arbitrary dependence and without Gaussian assumptions.","keywords":["winner's curse","zoom correction","post-selection inference","simultaneous inference","confidence intervals","top-k winners","arbitrary dependence","selection bias"],"falsifier":"Run a large simulation (say 100,000 trials) with $m=1000$, $X_i \\sim N(0,1)$ independent, all $\\theta_i=0$, and $\\alpha=0.1$; compute the Theorem 3.1 interval using the exact joint tail bound and check whether the empirical coverage of $\\theta_{\\hat\\imath}$ is at least $0.9$ overall. A reproducible coverage deficit under the exact $S$ would refute the paper's central claim, while under-coverage only when $S$ is deliberately misspecified would confirm that the tail-bound input, not the method, is the fragile part.","tokens_in":28598,"feed_emoji":"🎯","tokens_out":9313,"duration_ms":78228,"temperature":0.7,"pith_summary":"This paper introduces the zoom correction, a method for building confidence intervals for the mean of the empirically best candidate, the winner, after the data have selected it. The method guarantees at least $1-\\alpha$ coverage under arbitrary dependence between candidates and without assuming Gaussianity or independence; the only required input is a known tail bound on the estimation errors. The correction works by constructing a simultaneous confidence region for all candidates that spends its error budget only on those that could plausibly win, then projecting that region onto the selected coordinate. Because the projection is computed through a 'worst-case' mean vector, the interval can be evaluated with a one-dimensional search, and a step-down version gives explicit endpoint formulas. The same machinery extends to top-$k$ winners, the population-best value and identity, and near-winners.","feed_headline":"Zoom correction yields valid confidence intervals for the winner","feed_subtitle":"It needs only a known tail bound on the noise and works under arbitrary dependence among candidates.","key_machinery":"The load-bearing object is the zoom test's acceptance region. At a candidate mean $\\theta$, the region is $\\prod_{j=1}^m [\\theta_j - \\max\\{r_\\alpha, \\Delta_j/2\\}, \\theta_j + \\max\\{r_\\alpha, \\Delta_j/2\\}]$, where $\\Delta_j = \\max_\\ell \\theta_\\ell - \\theta_j$ is the suboptimality of coordinate $j$ and the active radius $r_\\alpha$ is the smallest $r$ with $S(\\max\\{r, \\Delta_j/2\\}_{j=1}^m) \\le \\alpha$, so the region has coverage $1-\\alpha$ under the tail bound $S$. Inverting this test produces the simultaneous confidence region, and Lemma 3.1 reduces projection onto the winner coordinate to a one-dimensional condition: for the worst-case vector $\\theta^t$ with $\\theta^t_{\\hat\\imath}=t$ and $\\theta^t_j = \\min\\{2X_j/3 + t/3, t\\}$ for $j\\ne \\hat\\imath$, the point $t$ lies in the projected set exactly when $|X_{\\hat\\imath} - t| \\le r_\\alpha(\\theta^t)$.","core_discovery":"The central claim is Theorem 3.1: if $\\hat\\imath = \\arg\\max_i X_i$ is the empirical winner and $r_\\alpha(\\theta)$ is the smallest radius such that the tail bound $S$ is at most $\\alpha$ around a candidate mean $\\theta$, then the set $\\hat C^\\alpha_{\\hat\\imath}=\\{t: |X_{\\hat\\imath}-t|\\le r_\\alpha(\\theta^t)\\}$, with $\\theta^t$ the worst-case vector that makes $t$ as easy as possible to include, satisfies $P(\\theta_{\\hat\\imath}\\in[\\min\\hat C^\\alpha_{\\hat\\imath},\\max\\hat C^\\alpha_{\\hat\\imath}])\\ge 1-\\alpha$. The interval comes from inverting a pointwise hypothesis test, the zoom test, whose acceptance region is a rectangle centered at $\\theta$ with half-width $\\max\\{r_\\alpha,\\Delta_j/2\\}$ in coordinate $j$, where $\\Delta_j$ is how far $\\theta_j$ sits below the population maximum. This construction devotes the error budget to 'active' candidates, so the interval automatically widens when many candidates are close and narrows to essentially uncorrected inference when one candidate is clearly best.","pith_inferences":["The worst-case-vector shortcut is a general template: any data-dependent query for which a dominating parameter vector exists can inherit a scalar projection check, so the construction may extend beyond argmax winners to other selection rules.","Since the proof only consumes the tail bound $S$ inside the acceptance region, users could plug in sharper dependence-aware bounds, such as those from martingale concentration, and obtain shorter intervals without changing the method.","If a user cannot supply a trustworthy prior tail bound and instead estimates $S$ from the same data, the coverage guarantee has no automatic rescue; the method's flexibility does not remove the need for an external or pre-specified error bound.","The paper treats the grid-search computation as practically negligible, but in the non-monotone equation for the lower endpoint $r_l$, a coarse grid could in principle miss the true boundary; an exact root-finding implementation would be a worthwhile robustness check."],"forward_implications":["With only a known marginal tail bound and a union bound, the step-down zoom correction gives valid winner intervals even when the candidate estimates are dependent in unknown ways.","The interval width adapts to the data: it matches the fully simultaneous correction when candidates are tied, and approaches the uncorrected marginal interval when one candidate is far ahead.","The same inverted-region construction provides valid confidence sets for the top $k$ winners, for the population-best value $\\theta^*$, and for the identity of the population winner.","In Gaussian settings, the paper's experiments show the method is competitive with hybrid inference and strictly tighter than locally simultaneous and fully simultaneous inference, with no covariance estimate required.","The variance-adaptive extension keeps the coverage guarantee when candidates have different known scales, by scaling each coordinate's radius by $\\sigma_j$."],"supporting_citations":[{"why":"Defines locally simultaneous inference, the closest prior paradigm that focuses power around the selected candidate; the zoom correction is developed and compared against it.","marker":"[21]"},{"why":"Introduces hybrid inference on winners, a Gaussian-covariance method the paper compares with and proposes to improve by using the zoom correction as its initial step.","marker":"[2]"},{"why":"Provides simultaneous inference over selected parameters, an independent-observations baseline for top-$k$ inference used in the experiments.","marker":"[3]"},{"why":"Supplies the standard simultaneous-inference framework and the projection argument that any simultaneous region can be projected onto a data-dependent coordinate.","marker":"[14]"},{"why":"Gives earlier confidence intervals for selected population means under independent observations, which the paper positions its method against.","marker":"[10]"},{"why":"Provides the polyhedral lemma underlying conditional post-selection inference, the main Gaussian baseline to which the zoom correction is compared.","marker":"[12]"}],"fun_headline_variants":["Zoom correction tames the winner's curse","One tail bound fixes winner selection bias","Valid intervals for the best of many","Zoom test adapts to arbitrary dependence","Winner's curse? Not with zoom correction"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The guarantee assumes the user can supply a valid tail bound $S(v)$ with $P(\\exists i : |\\xi_i| > v_i) \\le S(v)$, or valid marginal tail bounds for a union bound; if that input is too small or is calibrated on the same data, the coverage claim can fail.","fun_headline_variants_meta":{"raw":{"variants":["Zoom correction tames the winner's curse","One tail bound fixes winner selection bias","Valid intervals for the best of many","Zoom test adapts to arbitrary dependence","Winner's curse? Not with zoom correction"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000403,"raw_usage":{"total_tokens":2108,"prompt_tokens":961,"completion_tokens":1147,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":577,"completion_tokens_details":{"reasoning_tokens":1084}},"tokens_in":577,"tokens_out":1147,"duration_ms":11502,"temperature":1.0,"reasoning_tokens":1084,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:05:24.906932+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a large simulation (say 100,000 trials) with $m=1000$, $X_i \\sim N(0,1)$ independent, all $\\theta_i=0$, and $\\alpha=0.1$; compute the Theorem 3.1 interval using the exact joint tail bound and check whether the empirical coverage of $\\theta_{\\hat\\imath}$ is at least $0.9$ overall. A reproducible coverage deficit under the exact $S$ would refute the paper's central claim, while under-coverage only when $S$ is deliberately misspecified would confirm that the tail-bound input, not the method, is the fragile part.","supporting_citations":[{"cited_title":"Locally simultaneous inference.The Annals of Statistics, 52(3): 1227–1253, 2024","cited_arxiv_id":null,"evidence_quote":"Defines locally simultaneous inference, the closest prior paradigm that focuses power around the selected candidate; the zoom correction is developed and compared against it."},{"cited_title":"Miller Jr.Simultaneous Statistical Inference","cited_arxiv_id":null,"evidence_quote":"Supplies the standard simultaneous-inference framework and the projection argument that any simultaneous region can be projected onto a data-dependent coordinate."},{"cited_title":"Confidence intervals for the means of the selected populations","cited_arxiv_id":null,"evidence_quote":"Gives earlier confidence intervals for selected population means under independent observations, which the paper positions its method against."},{"cited_title":"Exact post-selection inference, with application to the lasso.The Annals of Statistics, 44(3):907–927, 2016","cited_arxiv_id":null,"evidence_quote":"Provides the polyhedral lemma underlying conditional post-selection inference, the main Gaussian baseline to which the zoom correction is compared."}],"review_version":1}