{"id":"9d3c9c6b-8b5c-4ca2-b70d-8de74b4b9ef7","arxiv_id":"2502.05127","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A self-supervised conformal prediction method uses SURE estimates of reconstruction error, instead of ground truth, to calibrate prediction sets for linear image restoration problems.","lead":"Conformal prediction normally needs ground-truth images to calibrate uncertainty bounds. This paper replaces that calibration signal with Stein's Unbiased Risk Estimator, computed from noisy measurements alone, and shows near-perfect coverage in denoising and deblurring experiments.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central coverage claim is false as stated: with A=I and the identity estimator, SURE is the constant σ² while true scores are χ²_m/m, so the calibrated sets cover only ~50% at α=0.1, not 90%.","rationale":"The reader's conditional accept rests on the empirical success in denoising and deblurring. I agree those experiments are encouraging; the paper deserves credit for a clean idea and for reporting empirical coverage curves and calibration histograms. But the central theoretical premise—that SURE-based quantiles inherit the conformal coverage property—is not only unproved, it is false in the full generality in which the method is stated. The identity-estimator case is a minimal, fully specified instance of the paper's framework: full-rank A, Gaussian noise, differentiable estimator. Here SURE is perfectly accurate for the conditional mean risk (σ²), yet the realized nonconformity scores have nontrivial dispersion, so calibrating to the SURE constant produces severe undercoverage. This isolates the essential mechanism: conformal prediction needs the distribution of the scores, and SURE supplies only the conditional mean of those scores. The paper's own wording in Section 3 ('we expect close agreement') is an assertion, not a theorem, and the later claim that the bias is 'small' is anecdotal and unquantified. Because the central coverage guarantee is what makes the method a conformal method at all, a false universal claim warrants rejection as stated; a revision would need to restrict the estimator class, add a conservative correction, or prove a quantile-dominance bound. The proposed toy test settles the issue quickly and unambiguously.","tokens_in":7881,"tokens_out":12783,"duration_ms":136802,"concrete_test":"Run Algorithm 1 in the minimal setting A=I_m, x̂(y)=y, σ²=1, m=100, M=1000 calibration draws, α=0.1, then evaluate coverage on 10,000 fresh draws. The SURE values are all 1, so the calibrated threshold is 1; nominal coverage would require the true score ∥ε∥²/100 to be ≤1 for 90% of draws, but P(χ²_100≤100)≈0.5. If observed coverage is ~50% as theory predicts, the paper's universal coverage claim is falsified. For contrast, supervised conformal prediction using true scores on the same data should give ~90% coverage, isolating the SURE substitution as the cause.","verdict_should_be":"REJECT","load_bearing_attack":"The paper's central claim (Section 3, after Eq. (5)) is that conformal quantiles obtained from {SURE(y_i)} will be in close agreement with true quantiles of s(X,Y), so coverage is nearly maintained. The load-bearing gap is not just missing tightness: SURE(Y) is unbiased for the conditional mean E[s(X,Y)|X], not for the realized score s(X,Y). Replacing each realized score by an estimate of its conditional mean shifts the upper quantile of the calibration distribution downward whenever the conditional variance of s(X,Y)|X is positive.\n\nA concrete failure satisfies all stated assumptions (full-rank A, Gaussian noise, differentiable estimator): take A=I_m and x̂(y)=y. Then Eq. (5) gives SURE(y)=σ² exactly, while the true score is s(x,y)=∥x-y∥²/m ∼ σ²χ²_m/m. With M calibration samples, Algorithm 1 returns threshold σ², and new-test coverage is P(χ²_m ≤ m)≈1/2 for m large, far below 1-α for α=0.1. The failure persists even though SURE is perfectly accurate, because SURE estimates the conditional mean, not the realized score. The paper's informal 'expect close agreement' language is therefore false as stated; the experiments use well-trained estimators, but the method is presented as estimator-agnostic and the formal problem statement contains no condition excluding this case.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a self-supervised conformal prediction method for image restoration. The key idea is to replace the usual calibration scores s(x_i, y_i) = ||A x_i - A xhat(y_i)||^2/m, which require ground-truth images, with SURE estimates computed from noisy measurements alone. The authors argue that because SURE is unbiased and accurate in high dimensions, the quantiles of the SURE estimates will be close to the quantiles of the true scores, so the resulting conformal prediction sets nearly maintain the desired coverage. The method is evaluated on image denoising with a SURE-trained DRUNet and on non-blind deblurring with the Polyblur estimator, reporting empirical coverage close to the supervised conformal baseline on the DIV2K dataset.","tokens_in":8149,"tokens_out":4349,"duration_ms":46023,"significance":"If the central claim were correct, the method would provide a valuable practical tool: conformal calibration without any ground-truth data for a broad class of linear Gaussian inverse problems. The experimental setup is sensible, the empirical comparison with supervised conformal prediction is informative, and the use of SURE with a Hutchinson trace approximation is computationally attractive. However, the paper's theoretical foundation is the load-bearing weakness: the assertion that quantiles of SURE estimates approximate quantiles of true scores is not merely unproven; it is false in general, as demonstrated by a simple estimator satisfying all of the paper's assumptions. Because the abstract and Section 3 promise 'remarkably accurate' and 'nearly exact' coverage, the unsupported and actually false claim undermines the paper's main contribution.","major_comments":[{"comment":"The central claim that 'the conformal calibration quantiles obtained from SURE will be in close agreement with the true quantiles of s(X,Y)' is false as stated. SURE(Y) is an unbiased estimator of the conditional mean risk E[s(X,Y)|X=x], not of the realized score s(X,Y). Replacing each realized score by an estimate of its conditional mean can shift the upper quantile of the calibration distribution downward whenever the conditional variance of s(X,Y)|X is positive. This is not a mere technicality: take A = I_m and xhat(y) = y, which satisfies the stated assumptions (full-rank A, differentiable estimator). Then Eq. (5) gives SURE(y) = sigma^2 exactly, while the true scores are s(x,y) = ||x - y||^2/m, which under the Gaussian noise model follow (sigma^2/m) chi^2_m. With M calibration samples, Algorithm 1 returns the threshold sigma^2, and for a new test point the coverage is P(chi^2_m <= m), which is approximately 0.5 for large m, far below 0.9 for alpha = 0.1. Thus the method does not achieve the advertised coverage even in the infinite-calibration-sample limit. To make the central claim valid, the authors would need a condition on the estimator (e.g., small conditional variance of s(X,Y)|X) or a quantitative bound on the quantile error; no such condition or bound is provided.","section":"Section 3, after Eq. (5)"},{"comment":"The paper does not provide any finite-sample or asymptotic theorem that would justify the coverage property of the proposed self-supervised calibration. The only formal guarantee in the paper, Eq. (2), applies to split conformal prediction with the true scores {s(X_i,Y_i)}; Algorithm 1 instead uses {SURE(Y_i)}, and the finite-sample correction used for supervised conformal prediction is not shown to carry over. Moreover, the text states that each prediction set C(y_i) uses a leave-one-out quantile with the i-th SURE estimate removed, but Algorithm 1 line 4 computes a single quantile over all M estimates without any leave-one-out step. The pseudocode should be reconciled with the text, and a rigorous statement of what the algorithm actually guarantees (if anything) must be supplied.","section":"Section 3, Algorithm 1 and surrounding text"}],"minor_comments":[{"comment":"Equation (8) appears to have a typo: the Hutchinson approximation is written as (1/K) n^T J n^T h(y), which does not have consistent dimensions; it should likely be (1/K) n^T J h(y) n or the vector form should be made explicit.","section":"Section 3, Eq. (8)"},{"comment":"The figures report empirical coverage and histograms, but there are no error bars or standard deviations over multiple test folds; given the small test size (200 images), error bars would help assess the variability of the reported coverage values.","section":"Section 4, Figures 2-4"},{"comment":"The display in Eq. (3) uses both a norm notation with subscript Sigma(y) and an explicit quadratic form; please use one consistent notation to avoid confusion.","section":"Section 3, Eq. (3)"},{"comment":"The quantile level in Algorithm 1 is written as top ceil((M+1)(1-alpha))/M, while the text describes a leave-one-out quantile over M-1 values; please clarify which finite-sample correction is actually used in the experiments.","section":"Algorithm 1"}],"recommendation":"reject","confidential_remarks":"The counterexample in my major comment is not based on an exotic failure mode; it uses the identity estimator, which is a perfectly valid differentiable estimator under the paper's assumptions. The paper's informal language ('expect close agreement', 'bias is small') cannot be repaired by a small adjustment because the method estimates conditional means rather than realized scores, and the resulting coverage gap is systematic. Even if the authors added a heuristic caveat, the abstract's promise of 'nearly exact marginal coverage' would still be misleading. The paper is not suitable for publication in its current form; a new methodological idea or a substantially restricted claim with a supporting theorem would be needed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper combines Stein's unbiased risk estimator with split conformal prediction to build prediction sets for linear inverse problems without any ground-truth calibration data. That specific combination is new, and the experiments on denoising and deblurring with DRUNet and Polyblur show coverage almost identical to supervised conformal prediction. The SURE identity is standard and used correctly, the Hutchinson trace approximation is sensible, and the presentation is mostly clear. This is a natural idea that could matter for imaging problems where ground truth is expensive or unavailable.\n\nThe soft spot is load-bearing. The paper asserts that quantiles of SURE values will be in close agreement with quantiles of true scores, but this is not proven, and as stated it is false. SURE is unbiased for the conditional mean E[s(X,Y)|X], not for the realized score s(X,Y). Replacing each realized score by a conditional-mean estimate shifts the upper quantile downward whenever there is conditional variance. A concrete counterexample satisfies all the paper's assumptions: take A=I and x̂(y)=y. Then SURE(y)=σ² exactly, while the true scores are σ²χ²_m/m. The calibrated threshold becomes σ², and for large m the prediction sets cover roughly 50% of test points at α=0.1 instead of 90%. This is not a tightness issue; the gap is structural. The experiments work because the estimators are well trained and the conditional variance of the score is small, but the paper presents the method as estimator-agnostic and gives no condition that excludes the failure case.\n\nThere is also a smaller inconsistency: the text describes a leave-one-out quantile with M−1 samples, but Algorithm 1 uses the ⌈(M+1)(1−α)⌉/M quantile of all M values. And while the experiments are reproducible in principle, no code is provided.\n\nThe paper deserves a serious referee. The idea is novel and the empirical demonstration is promising, but the central coverage claim needs either a real proof under stated conditions or an explicit restriction to estimators where the conditional variance of the score is controlled. As it stands, the headline guarantee is not established. I would send it to peer review and ask for a major revision, not desk-reject it.","headline":"SURE-based self-calibration for conformal prediction is a genuinely useful idea with encouraging experiments, but the paper's central quantile-agreement claim is false without extra conditions, so the coverage guarantee is unproven.","tokens_in":8672,"tokens_out":1782,"would_cite":false,"duration_ms":19079,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proposes a conformal prediction method for imaging that calibrates uncertainty from noisy measurements alone, using Stein's unbiased risk estimator in place of ground-truth scores.","keywords":["conformal prediction","Stein's unbiased risk estimate","uncertainty quantification","image restoration","self-supervised learning","inverse problems","image denoising","image deblurring"],"falsifier":"A concrete check: set $A=I_m$, fix $\\sigma$, and use the do-nothing estimator $\\hat{x}(y)=y$. Then $SURE(y)=\\sigma^2$ is constant, while the true scores $\\|x-y\\|_2^2/m$ follow a scaled chi-squared distribution with mean $\\sigma^2$. For confidence levels whose true score quantile differs from $\\sigma^2$, the SURE-calibrated quantile gives coverage that deviates from $1-\\alpha$ by an exactly computable amount, showing where unbiasedness alone is insufficient and how large a gap remains in more realistic cases.","tokens_in":7652,"feed_emoji":"🖼️","tokens_out":10554,"duration_ms":98754,"temperature":0.7,"pith_summary":"Image restoration problems with a linear, full-rank, possibly ill-conditioned forward operator and Gaussian noise can be given principled prediction sets even when no ground-truth images are available for calibration. The paper's proposal is to replace each true reconstruction-error score, which depends on the unknown image, with Stein's Unbiased Risk Estimator (SURE) computed from the noisy measurement alone, and then to run split conformal prediction on these SURE values. The central assertion is that the quantile of SURE values is close enough to the true score quantile that the resulting sets still have coverage close to the nominal $1-\\alpha$ level. Demonstrations on denoising with a self-supervised neural estimator and on deblurring with a model-based estimator report coverage curves nearly indistinguishable from supervised conformal prediction.","feed_headline":"No ground truth needed to calibrate image uncertainty sets","feed_subtitle":"A SURE-based conformal score turns noisy measurements into prediction sets with near-perfect coverage.","key_machinery":"The load-bearing object is the SURE estimate of the measurement-space reconstruction error. For the score $s(x,y)=\\|Ax - A\\hat{x}(y)\\|_2^2/m$, the corresponding SURE value is $SURE(y)=\\|y - A\\hat{x}(y)\\|_2^2/m - \\sigma^2 + (2\\sigma^2/m)\\operatorname{div}(A\\hat{x}(y))$, computed from the measurement alone; the divergence term is approximated by Hutchinson's stochastic trace estimator using automatic differentiation. Calibration then proceeds exactly as split conformal prediction, but with $S_i=SURE(y_i)$ in place of $s(x_i,y_i)$, and the final prediction set is the sublevel set of the same score below the SURE-calibrated quantile. The full-rank assumption on $A$ keeps this sublevel set bounded and the method applicable to any differentiable estimator.","core_discovery":"The paper claims that in the linear Gaussian model $Y|X=x \\sim \\mathcal{N}(Ax,\\sigma^2 I_m)$ with full-rank $A$, the quantity $SURE(y) = \\|y - A\\hat{x}(y)\\|_2^2/m - \\sigma^2 + (2\\sigma^2/m)\\operatorname{div}(A\\hat{x}(y))$ is an unbiased estimate of the score's conditional expectation, and that when $m$ is large this estimate is accurate enough that the empirical $\\lceil M(1-\\alpha)\\rceil/M$-quantile of $\\{SURE(y_i)\\}$ can serve as a conformal calibration threshold. The prediction set is $\\{x \\in \\mathbb{R}^n: \\|Ax - A\\hat{x}(y)\\|_2^2/m \\le \\hat{Q}_\\alpha\\}$, with full rank of $A$ keeping the set bounded. If correct, the method transfers conformal prediction's near-exact marginal coverage guarantee to settings where only noisy measurements are observed, without any ground-truth calibration data.","pith_inferences":["A direct way to stress-test the method beyond the paper's experiments would be to bootstrap the SURE calibration sample and measure the spread of $\\hat{Q}_\\alpha$; when that spread is large relative to the score quantile's intrinsic scale, coverage will degrade, giving practitioners a cheap diagnostic.","The same measurement-only calibration idea should transfer to other noise models with unbiased risk estimators, such as Poisson or Poisson-Gaussian noise; the paper lists this as future work but does not claim a result.","The full-rank requirement on $A$ is what keeps prediction sets bounded; for rank-deficient operators one could project scores onto the measurement range or add equivariance constraints, an extension the paper mentions as a perspective rather than a demonstrated method.","An immediate testable refinement of the paper's evaluation would be to compare expected set volumes as well as coverage, since a coverage guarantee alone does not indicate how informative the resulting prediction regions are."],"forward_implications":["Any differentiable image-restoration estimator for a linear Gaussian inverse problem with full-rank $A$ can be equipped with conformal prediction sets using only exchangeable noisy measurements, with no paired ground-truth images needed for calibration.","Because calibration happens in the measurement domain on data of the same type encountered at deployment, the method sidesteps the calibration-to-deployment distribution shift that supervised conformal prediction can suffer.","The same SURE principle can be combined with self-supervised training objectives, so an entire uncertainty-quantified restoration pipeline can be built from measurements alone, as demonstrated by the denoising experiment.","In the large-$m$ regime where SURE's variance is small, the coverage gap between self-supervised and supervised conformal prediction should vanish; the paper reports coverage curves for denoising and deblurring that are close to the supervised baseline across confidence levels."],"supporting_citations":[{"why":"Supplies the conformal prediction framework whose finite-sample coverage guarantee the paper aims to reproduce without ground truth.","marker":"[9]"},{"why":"Motivates the normalized non-conformity measure whose measurement-space version the method uses.","marker":"[11]"},{"why":"Provides the unbiased risk estimate that makes calibration possible from noisy measurements alone.","marker":"[12]"},{"why":"Gives the theoretical analysis of SURE variance that the paper invokes to expect accurate quantile agreement for large m.","marker":"[13]"},{"why":"Supplies empirical evidence that SURE is accurate in imaging settings.","marker":"[14]"},{"why":"Provides the Monte-Carlo SURE approach whose divergence approximation underpins the estimator-agnostic implementation.","marker":"[16]"},{"why":"Provides the Hutchinson stochastic trace approximation used for efficient divergence computation.","marker":"[17]"},{"why":"Supplies the DIV2K dataset used in both numerical experiments.","marker":"[18]"},{"why":"Provides the SURE-based self-supervised training loss used for the denoising estimator.","marker":"[20]"},{"why":"Provides the model-based Polyblur restoration method used in the deblurring experiment.","marker":"[21]"}],"fun_headline_variants":["Self-supervised conformal prediction cuts need for ground truth","SURE-based conformal sets for imaging without ground truth","No ground truth? Self-supervised conformal still calibrates","Imaging uncertainty sets self-calibrated via SURE","Conformal prediction that self-calibrates from noisy data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument depends on the empirical distribution of the SURE values sitting close to the distribution of the true scores, so that a quantile of one is a valid conformal threshold for the other; SURE's unbiasedness for the mean does not by itself control the quantiles.","fun_headline_variants_meta":{"raw":{"variants":["Self-supervised conformal prediction cuts need for ground truth","SURE-based conformal sets for imaging without ground truth","No ground truth? Self-supervised conformal still calibrates","Imaging uncertainty sets self-calibrated via SURE","Conformal prediction that self-calibrates from noisy data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000165,"raw_usage":{"total_tokens":1289,"prompt_tokens":1024,"completion_tokens":265,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":640,"completion_tokens_details":{"reasoning_tokens":183}},"tokens_in":640,"tokens_out":265,"duration_ms":2811,"temperature":1.0,"reasoning_tokens":183,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T20:09:06.784193+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete check: set $A=I_m$, fix $\\sigma$, and use the do-nothing estimator $\\hat{x}(y)=y$. Then $SURE(y)=\\sigma^2$ is constant, while the true scores $\\|x-y\\|_2^2/m$ follow a scaled chi-squared distribution with mean $\\sigma^2$. For confidence levels whose true score quantile differs from $\\sigma^2$, the SURE-calibrated quantile gives coverage that deviates from $1-\\alpha$ by an exactly computable amount, showing where unbiasedness alone is insufficient and how large a gap remains in more realistic cases.","supporting_citations":[{"cited_title":"Conformal uncertainty sets for robust optimiza- tion,","cited_arxiv_id":null,"evidence_quote":"Motivates the normalized non-conformity measure whose measurement-space version the method uses."},{"cited_title":"Estimation of the mean of a multivariate normal distribution,","cited_arxiv_id":null,"evidence_quote":"Provides the unbiased risk estimate that makes calibration possible from noisy measurements alone."},{"cited_title":"Second-order stein: Sure for sure and other applications in high-dimensional inference,","cited_arxiv_id":null,"evidence_quote":"Gives the theoretical analysis of SURE variance that the paper invokes to expect accurate quantile agreement for large m."},{"cited_title":"Unbiased risk estimates for singular value thresholding and spectral estimators,","cited_arxiv_id":null,"evidence_quote":"Supplies empirical evidence that SURE is accurate in imaging settings."},{"cited_title":"Monte-Carlo SURE: A black-box op- timization of regularization parameters for general denoising algorithms,","cited_arxiv_id":null,"evidence_quote":"Provides the Monte-Carlo SURE approach whose divergence approximation underpins the estimator-agnostic implementation."},{"cited_title":"On divergence ap- proximations for unsupervised training of deep denoisers based on stein’s unbiased risk estimator,","cited_arxiv_id":null,"evidence_quote":"Provides the Hutchinson stochastic trace approximation used for efficient divergence computation."},{"cited_title":"Ntire 2017 challenge on single image super- resolution: Dataset and study,","cited_arxiv_id":null,"evidence_quote":"Supplies the DIV2K dataset used in both numerical experiments."},{"cited_title":"Poly- blur: Removing mild blur by polynomial reblurring,","cited_arxiv_id":null,"evidence_quote":"Provides the model-based Polyblur restoration method used in the deblurring experiment."}],"review_version":1}