{"id":"68b02fd4-9beb-416f-b8a7-6bfaaf735711","arxiv_id":"2506.16189","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Canonicalizing inputs before conformal prediction preserves coverage and shrinks prediction sets under rotation shifts, without retraining the underlying model.","lead":"Researchers combined conformal prediction, a way to give AI models uncertainty ranges, with pose-correcting neural networks so the uncertainty guarantees survive when test images are rotated. The approach is plug-and-play for existing models and could make AI predictions more trustworthy in cases where data arrives at unexpected angles, such as medical scans or 3D sensor data.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The coverage guarantee is not formally supported because the canonicalization network is trained on the same calibration data used for conformal calibration (Tab. 2, §4), violating the split-conformal requirement of a fixed score function; imperfect equivariance (Tab.","rationale":"The reader's weakest assumption focuses on the canonicalizer's imperfect equivariance and 87% pose accuracy breaking exchangeability. That is a real quantitative concern, but the more fundamental issue is the training-data pipeline: the split-conformal theorem requires the score function to be independent of the calibration set. The paper explicitly says the CN is trained on Dcal, and its §3 theoretical motivation treats c_theta as a fixed equivariant function. If the CN is genuinely trained on Dcal, the coverage guarantee is invalid regardless of how accurate or equivariant the CN is, because the calibration scores are used to fit the very function that produces them. This is a formal, structural problem rather than a tuning or accuracy problem, and it directly attacks the 'guarantees' language in the abstract and §3. The proposed concrete test is a single experiment with a disjoint Dcan, which would either restore the formal validity or expose the reliance on calibration-set training. I set the verdict to CONDITIONAL rather than REJECT because the empirical results may survive a proper split, and the paper's §2.3 hints at a holdout Dcan; the needed change is largely a matter of experimental protocol and textual clarity, but it is essential before the coverage claim can be accepted. Agreement with the reader is partial because both of us identify the canonicalizer as the weak link, but our proposed fix and the precise failure mode differ.","tokens_in":19715,"tokens_out":5019,"duration_ms":63744,"concrete_test":"Rerun the §4.1 image and point-cloud experiments with an explicit train/calibration split: train the canonicalizer on a holdout set Dcan disjoint from Dcal (e.g., half of the originally used calibration data for CN training, reserving the other half exclusively for conformal score computation), and report coverage and set sizes exactly as in Tabs. 3, 4, and 7. If coverage remains at the nominal 1 − α level with set sizes close to the reported values, the concern is resolved and the method is valid under a standard split-conformal protocol. If coverage drops or set sizes inflate, the reported results depend on training the canonicalizer on Dcal, and the formal coverage claim must be revised.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that CP2 reinstates conformal coverage guarantees under geometric shifts via canonicalization. For split conformal prediction, the score function must be fixed before calibration: the calibration scores and test scores must be exchangeable, and the score function must not depend on Dcal. The paper states in Tab. 2 that 'the CN is trained on Dcal', and Fig. 3 says the canonicalizer is trained on Dcal with C4/C8 knowledge. If so, the canonicalizer is fit to the same samples whose scores are used to compute the conformal quantile. Even with a perfectly equivariant canonicalizer, the fitted c_theta is a function of Dcal, so the calibration scores are in-sample while the test scores are out-of-sample; the split-conformal coverage theorem in §2.1 does not apply. The §3 exchangeability argument silently treats c_theta as a fixed, exactly equivariant map, which is inconsistent with training c_theta on Dcal. The empirical coverage near 1−α in Tabs. 3–4 may still hold approximately if overfitting is mild, but the claimed formal 'guarantee' is not established. This is more load-bearing than the 87% pose accuracy in Tab. 5: a separate, disjoint training set for the canonicalizer would address the formal issue, whereas accuracy is a quantitative robustness concern. The paper is internally inconsistent on this point: §2.3 suggests Dcan 'e.g. a hold-out data split', but Tab. 2 and the experimental text specify Dcal. The manuscript must either clarify that a disjoint Dcan was used, or the theoretical claim must be weakened and the method re-verified under a valid split.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CP^2, a framework that integrates a canonicalization network (CN) with split conformal prediction to maintain valid coverage and improve prediction-set efficiency under geometric shifts such as rotations. The authors motivate the approach by arguing that canonicalization restores exchangeability under the action of a symmetry group, and they present three uses: standard split conformal prediction with a canonicalized predictor, a diagnostics tool that constructs partition-conditional group maps to guide mondrian conformal prediction, and a geometric weighting scheme for weighted conformal prediction in double-shift settings. Experiments on CIFAR-10, CIFAR-100, and ModelNet-40 with C4, C8, and SO(3) rotation shifts compare against non-equivariant, augmented, and equivariant baselines, reporting empirical coverage near the target level and improved set sizes.","tokens_in":19942,"tokens_out":4841,"duration_ms":63964,"significance":"If the formal coverage claim were established, the paper would be a valuable bridge between geometric deep learning and conformal prediction, offering a post-hoc, model-agnostic way to handle geometric shifts without retraining the predictor. The empirical study is well structured: it covers two data modalities, multiple rotation groups, several baselines, and reports means and standard errors over 10 calibration/test splits. The code is publicly available, and the limitations section candidly discusses imperfect canonicalization and unknown group elements. However, the central theoretical claim, namely that CP^2 reinstates conformal coverage guarantees under geometric shifts, is not actually proven as stated: the CN is trained on the calibration set, which breaks the split-conformal condition that the score function be fixed before calibration, and the exchangeability argument assumes exact equivariance that the trained network does not satisfy. These issues are load-bearing for the paper's main claim, so a major revision is needed.","major_comments":[{"comment":"The split-conformal coverage theorem invoked in §2.1 requires the nonconformity score function to be fixed before the calibration scores are computed, independent of D_cal. Table 2 states explicitly that 'the CN is trained on D_cal', and §4 confirms that the canonicalizer is trained using the canonicalization prior. Since the canonicalizer is part of the score function, the calibration scores are in-sample with respect to the CN parameters while the test scores are out-of-sample, so the calibration and test scores are not exchangeable and Eq. (1) does not apply. The §3 argument silently treats c_theta as a fixed map, which is inconsistent with training it on D_cal. This is a formal gap in the central claim that CP^2 'guarantees well-calibrated nonconformity scores'. The authors should either train the CN on a disjoint hold-out set D_can and fix it before calibration, as suggested in §2.3, or provide a theorem that accounts for the data-dependent score function; the current manuscript does neither.","section":"§3, §2.3, Tab. 2"},{"comment":"The theoretical motivation assumes exact canonicalization, specifically c_theta(g·x)^{-1} = c_theta(x)^{-1}·g^{-1}, to conclude invariance of the joint distribution under G^n. Table 5 reports that the trained canonicalizer predicts the correct group element for only about 87% of samples, so the exact equivariance condition is not met. The paper acknowledges miscanonicalization only qualitatively in §6 and in Appendix A, but the exchangeability argument in §3 has no error term or robustness bound. As a result, the claimed formal guarantee is not established even if the CN were trained on a separate dataset; the empirical coverage near 1-α may still hold approximately, but the paper should state this as an approximate or heuristic claim, or provide a quantitative sensitivity analysis linking canonicalization accuracy to coverage deviation.","section":"§3, Tab. 5, §6"},{"comment":"The weighted conformal variant in the double-shift setting uses weights w_i(x_{n+1}) computed from the canonicalizer's per-sample group distributions, which in turn depend on D_cal via training and on the test point. The cited weighted conformal results (Barber et al. 2023, Guan 2023) assume either fixed weights or specific conditions on data-dependent weights that are not verified here. Additionally, the modulation parameter p is selected on the calibration data. The paper does not claim a formal guarantee for this WCP variant, but the presentation in §3.1 and Tab. 2 suggests it as a robustness mechanism; the authors should clarify the formal status and specify the assumptions under which Eq. (3) with their geometric weights would be valid.","section":"§3.1, §4.3, Eq. (3)"}],"minor_comments":[{"comment":"The word 'exchangability' is misspelled; it should be 'exchangeability'.","section":"§3"},{"comment":"'skewdness' should be 'skewness'.","section":"§3.1"},{"comment":"The symbol 'oo' appears to be a rendering artifact for '∞'; please ensure the infinity symbol is typeset correctly.","section":"Tabs. 3, 6, 9"},{"comment":"'caputed' should be 'captured'.","section":"Fig. 7 caption"},{"comment":"The notation switches between using ρ explicitly and omitting it in Eq. (6); please clarify when ρ is suppressed for readability.","section":"§2.3"},{"comment":"The table lists three shift settings, but the text says only the first and third rows are empirically examined; please state this directly in the table caption or footnote.","section":"Tab. 2"}],"recommendation":"major_revision","confidential_remarks":"The paper is well within the scope of UAI and the empirical contribution is potentially useful. The main concern is not the quality of the experiments but the mismatch between the claimed formal coverage guarantee and the actual procedure, specifically the use of D_cal to train the canonicalizer. This is fixable within the scope of a revision by using a separate hold-out set and by softening or re-proving the theoretical claim. I do not see any citation or novelty concerns beyond the usual related-work self-citations."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things worth knowing about this paper. The first is that it is genuinely the first to bolt learned canonicalization onto conformal prediction, and the empirical exploration is solid: three integration modes, image and point-cloud benchmarks, discrete and continuous rotations, two scoring rules, ten splits, public code. The second is that the central formal claim – that canonicalization reinstates conformal coverage guarantees – is not established by the paper's own setup, because the canonicalizer is trained on the calibration data.\n\nThe experiments show a real effect: under C4/C8 and SO(3) rotation shifts, canonicalization keeps coverage near the target and set sizes small, while the unadjusted base predictors blow up. The group-map diagnostic is a nice by-product, and the geometric weighting for double shifts is a reasonable first pass, even if its p=2.0 is tuned on the evaluation curves.\n\nThe load-bearing weakness is not the ~87% pose accuracy that the reader flagged. It is that Table 2 and the experimental text say the canonicalization network is trained on Dcal, the same data used to compute conformal scores. Split-conformal theory requires the score function to be fixed before looking at calibration data; here the score function includes a network fit to those very samples. Calibration scores are in-sample while test scores are out-of-sample, so the exchangeability argument in Section 3 silently treats c_theta as a fixed, exactly equivariant map that the experiments never supply. Even a perfect canonicalizer would not fix this: the fitted parameters still depend on Dcal. Section 2.3 defines Dcan as, for example, a hold-out data split, which conflicts with the experiments. This is a fixable but essential issue: use a genuinely disjoint split for the canonicalizer, or give up the formal guarantee language and present the method as an empirical heuristic.\n\nThe minor issues are minor: the weighting parameter p is selected on the evaluation data, and the diagnostics threshold is post hoc. Both are worth sensitivity checks, but they do not change the picture.\n\nWho this is for: anyone working on conformal prediction under shift, and anyone combining learned equivariance with UQ. It deserves a serious referee, but I would push for major revisions and an explicit statement of whether a disjoint Dcan was used. If the authors clarify or weaken the claim, it becomes a solid contribution; as is, the headline overstates what is proven.","headline":"First learned-canonicalization-conformal-prediction combination with solid empirical support, but the coverage guarantee is undercut by training the canonicalizer on the calibration data; fixable with a disjoint split or a weakened claim.","tokens_in":20599,"tokens_out":4508,"would_cite":false,"duration_ms":45443,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F25","62G15"],"pacs":[],"model":"deepseek-v4-flash","headline":"Canonicalization restores conformal prediction guarantees under geometric shifts, keeping black-box predictors untouched.","keywords":["conformal prediction","geometric shift","canonicalization","group equivariance","exchangeability","uncertainty quantification","rotation invariance","weighted conformal prediction"],"falsifier":"Train a canonicalizer on C4-rotated CIFAR-10, then evaluate conformal prediction on C8-rotated test images (a group misspecification the paper reports as unstable in Table 3) and record whether empirical coverage drops below the target by more than the tolerated margin across many random splits; a sustained drop would falsify the claim that canonicalization alone reinstates coverage under geometric shifts.","tokens_in":19406,"feed_emoji":"🔄","tokens_out":1507,"duration_ms":17822,"temperature":0.7,"pith_summary":"Conformal prediction gives prediction sets with formal coverage guarantees, but those guarantees break when test data undergo geometric shifts such as rotations or flips. This paper argues that the breakdown can be repaired by feeding test samples through a learned canonicalization network that maps them back to a standard orientation before the conformal score is computed. The key claim is that canonicalization restores exchangeability between calibration and test data, so that coverage guarantees hold again and prediction sets stay small. The paper shows this works across image and point-cloud classification, and that the geometric information extracted along the way can also be used for diagnostics and for weighting calibration samples under more complex shifts.","feed_headline":"Canonicalization brings conformal prediction back under geometric shifts","feed_subtitle":"A bolt-on network that rotates data upright restores coverage guarantees and keeps prediction sets small, without touching the black-box…","key_machinery":"The central object is the canonicalization network, a learned map c_theta: X -> G that predicts the (inverse) group element needed to bring an input back to a canonical pose, so that the invariance relation f_theta(x) = f_theta(c_theta(x)^{-1}·x) holds. The canonicalization prior loss L_prior = E[D_KL(P_G|x || P_hat_G|x)] trains this network with a prior concentrated on the identity, forcing it to learn the pose distribution of the calibration data. The paper's argument hinges on the equivariance identity c_theta(g·x)^{-1} = c_theta(x)^{-1}·$g^{{-1}}$, which, when satisfied, makes the distribution of canonicalized samples invariant under G^n and thereby restores exchangeability for conformal prediction.","core_discovery":"The paper's central claim is that integrating a canonicalization network into the conformal prediction pipeline—canonicalizing each test sample before scoring it with a frozen, non-equivariant predictor—reinstates well-calibrated nonconformity scores and efficient prediction sets under geometric shifts. Formally, if the canonicalizer c_theta satisfies c_theta(g·x)^{-1} = c_theta(x)^{-1}·$g^{{-1}}$, then the joint distribution of canonicalized samples is invariant under the product group G^n, extending exchangeability from permutation invariance alone to include sample-wise geometric transformations. This makes the canonicalized scores exchangeable, so split conformal prediction's coverage guarantee holds even when the original data are shifted. Empirically, on CIFAR-10/100 with C4/C8 rotation shifts and on ModelNet-40 with SO(3) shifts, the approach keeps coverage near its target while prediction set sizes stay close to the no-shift baseline, and it does so without retraining the predictor.","pith_inferences":["The paper's exchangeability argument implicitly assumes the canonicalizer is equivariant in the exact sense of the identity; since real canonicalizers make errors (about 87% correct in the paper's own Table 5), the guarantees are approximate, and the method's practical reliability depends on how gracefully the conformal quantile tolerates those errors.","The canonicalization prior, which anchors the network to the identity pose of the calibration distribution, suggests a direct connection to covariate-shift reweighting: the group distribution can be viewed as a ratio between shifted and unshifted data densities, potentially unifying geometric reweighting with existing weighted conformal methods.","Because the canonicalizer is trained on calibration data only, it may fail when the test-time shift is much larger than anything seen during training; a testable extension is to update the canonicalizer online as new samples arrive, mirroring adaptive conformal procedures for time series.","The diagnostic use of group maps could be extended to detect spurious correlations between geometric pose and sensitive attributes in classification tasks, providing a tool for auditing model behavior beyond coverage."],"forward_implications":["Coverage guarantees of split conformal prediction are maintained under geometric shifts when the canonicalizer is accurate, with prediction set sizes comparable to the no-shift setting.","The canonicalizer's group distribution provides a per-sample geometric descriptor that can be aggregated over class or feature partitions to expose hidden geometric structure in the data.","When a partition's group map shows a clear pattern, running mondrian conformal prediction on the geometric partition yields more balanced conditional coverage by proxy for the true partition of interest.","In double-shift settings where the test shift introduces group elements unseen in calibration, geometric weighting of calibration scores based on the canonicalizer's output delays coverage breakdown.","A broader group definition (e.g., C8 rather than C4) makes the method more robust when the exact shift group is unknown, at modest cost."],"supporting_citations":[{"why":"Establishes the equivariance identity for canonicalization networks that the paper's exchangeability argument relies on.","marker":"[Kaba et al., 2023]"},{"why":"Introduces the canonicalization prior objective and the group-distribution extension that the paper trains its canonicalizer with.","marker":"[Mondal et al., 2023]"},{"why":"Supplies the definition of exchangeability and the foundational coverage guarantee of conformal prediction.","marker":"[Shafer and Vovk, 2008]"},{"why":"Provides the APS nonconformity score used in the paper's main experiments.","marker":"[Romano et al., 2020b]"},{"why":"Provides the weighted conformal prediction framework that the paper's geometric weighting scheme is built on.","marker":"[Barber et al., 2023]"}],"fun_headline_variants":["Canonicalizing poses restores conformal coverage under shifts","Upright data keeps conformal prediction honest","Geometric canonicalization locks in conformal guarantees","Rotate data, keep coverage: canonicalization for CP","Canonicalization reinstates conformal coverage amid rotations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The canonicalization network must be accurate enough that shifting a test sample and then canonicalizing it is equivalent to canonicalizing an unshifted sample, so that the distribution of canonicalized samples is the same for calibration and test data.","fun_headline_variants_meta":{"raw":{"variants":["Canonicalizing poses restores conformal coverage under shifts","Upright data keeps conformal prediction honest","Geometric canonicalization locks in conformal guarantees","Rotate data, keep coverage: canonicalization for CP","Canonicalization reinstates conformal coverage amid rotations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000522,"raw_usage":{"total_tokens":2495,"prompt_tokens":883,"completion_tokens":1612,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":499,"completion_tokens_details":{"reasoning_tokens":1534}},"tokens_in":499,"tokens_out":1612,"duration_ms":12283,"temperature":1.0,"reasoning_tokens":1534,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:43:53.560607+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a canonicalizer on C4-rotated CIFAR-10, then evaluate conformal prediction on C8-rotated test images (a group misspecification the paper reports as unstable in Table 3) and record whether empirical coverage drops below the target by more than the tolerated margin across many random splits; a sustained drop would falsify the claim that canonicalization alone reinstates coverage under geometric shifts.","supporting_citations":[{"cited_title":"A tutorial on conformal prediction","cited_arxiv_id":null,"evidence_quote":"Supplies the definition of exchangeability and the foundational coverage guarantee of conformal prediction."},{"cited_title":"Conformal prediction beyond exchangeability","cited_arxiv_id":null,"evidence_quote":"Provides the weighted conformal prediction framework that the paper's geometric weighting scheme is built on."}],"review_version":1}