{"id":"956952cc-11d3-45d3-93fb-40809cb524c1","arxiv_id":"2501.18426","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A conformal prediction framework using zonotopes and SVD dimension reduction produces finite-sample-guaranteed uncertainty sets for functional surrogate models.","lead":"The paper builds prediction sets for neural PDE surrogates that output entire functions, using a low-dimensional error model (SVD) and zonotopes, and calibrates the sets with conformal prediction. The result is a method that aims to guarantee, with user-set probability, that the true solution function lies inside the predicted uncertainty band.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section 4.1's truncation argument is invalid: the min-max box E is built from the calibration data, so a new exchangeable error can fall outside it, and coverage of R_alpha = Z_alpha x E is not guaranteed to be at least 1 - alpha.","rationale":"The central thesis is a finite-sample, distribution-free conformal guarantee for functional surrogates. The zonotope calibration in Sec. 3.2 follows standard nested-set conformal reasoning, and the SVD projection is linear, so the core idea is coherent. The only step that bridges the truncated lower-dimensional guarantee and the full functional prediction set is the Cartesian product with E in Sec. 4.1. That step is not valid as stated. The argument offered, that I_E always returns true during calibration, only shows that calibration scores are unaffected; it says nothing about a new point. Since E is a function of the calibration data, the new point has nontrivial probability of falling outside it, and in the many-truncated-dimensions regime this probability can dominate the coverage shortfall. This is an internal correctness gap, not a disagreement with a consensus; it can be demonstrated on a simple exchangeable Gaussian example. The fix is conceptually straightforward, for example by accounting for E-membership probability in calibration or by building E on an independent split with a conservative correction, but without it the theorem's statement is unsupported. The reader's conditional recommendation is the right level; my reading does not change that verdict.","tokens_in":18546,"tokens_out":9812,"duration_ms":98158,"concrete_test":"Simulate n=500 error vectors from N(0, I_1000), treating the first 32 coordinates as retained and the remaining 968 as truncated. Build E as the coordinate-wise min/max of the truncated coordinates, calibrate Z_alpha with the Sec. 3.2 membership procedure at target alpha=0.1, and measure empirical coverage of R_alpha = Z_alpha x E on 10^4 fresh samples. If the coordinates are independent, the expected full coverage is roughly ((499/501)^968) x coverage_Z, about 1.8% x 90% = 1.6%, far below the nominal 90%. Reproducing this failure would falsify the Sec. 4.1 claim; if the implementation does not fail, it must be doing something other than the construction described in the text.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The Sec. 4.1 justification is that I_E always returns 'true' during calibration, so the truncated dimensions may be ignored in Eq. (4). This conflates calibration-time membership with test-time membership. For a continuous scalar coordinate, P(X_{n+1} in [min_i X_i, max_i X_i]) = (n-1)/(n+1), not 1. With d truncated coordinates, the new point can be outside E with substantial probability; for independent coordinates, P(inside E in all d dims) = ((n-1)/(n+1))^d. For n=500 and d about 968 this is roughly 1.8%. Consequently, e_{n+1} in R_alpha requires both that the retained coordinates fall in Z_alpha and that the truncated coordinates fall in E. The effective nonconformity scores are not exchangeable: every calibration point is inside E by construction, while a new point has extra probability mass at score zero whenever any truncated coordinate leaves E. The claimed guarantee P(F_{n+1} in C_alpha) >= 1 - alpha is therefore not proven by the text. Table 1 may look acceptable because the discarded modes have tiny variance, but a distribution-free guarantee cannot rely on that property.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes a conformal prediction method for functional surrogate models. The authors map prediction residuals to a low-dimensional SVD space, fit an enclosing zonotope with a data-depth core, define a nested family of zonotopes, calibrate the nesting parameter by rank/membership scores, and propagate the calibrated sets back through the SVD to obtain a functional prediction set. To compensate for SVD truncation, they take a Cartesian product with the min/max box of the truncated residuals and claim the conformal guarantee is preserved. The main positive contribution is the use of zonotopes for exact linear propagation and the nested-set calibration. The central validity claim, however, is not established: the Section 4.1 truncation argument is incorrect, and the fit/calibration protocol needs clarification.","tokens_in":18793,"tokens_out":16433,"duration_ms":156890,"significance":"Zonotope-based conformal sets for high-dimensional functional outputs would be a useful contribution if the validity gap is repaired: the method is model-agnostic, the set family is nested, propagation is linear and exact, and the experiments are run on standard PDE benchmarks and compared against functional conformal baselines. The paper's strengths are the explicit Proposition 1 family, the use of conformal quantiles rather than fitted parameters to set alpha, and the practical computational pipeline. The advertised guarantee-preserving truncation bound is not currently proven, and the empirical coverage in Table 1 does not by itself establish the finite-sample claim.","major_comments":[{"comment":"The argument that the Cartesian product with the bounding box E preserves the conformal guarantee is incorrect. The statement that IE always returns true during calibration, and thus would not change the scoring in Eq. (4), only holds for the calibration points; a new exchangeable point can fall outside E in the truncated coordinates. For continuous data, the per-coordinate probability of lying inside the calibration min/max interval is (n-1)/(n+1), so with d truncated coordinates the probability of lying inside E is ((n-1)/(n+1))^d under independence, which can be far below 1. The test membership score for R_alpha = Z^alpha_pZ x E then has extra mass at zero compared with the calibration scores, and the conformal quantile in Eq. (3) does not imply P(F_{n+1} in C_alpha) >= 1-alpha. A separate calibration of E (for example, per-dimension conformal intervals with a Bonferroni split of alpha) is needed to restore the guarantee.","section":"Section 4.1"},{"comment":"Eq. (3) is asserted with the usual exchangeability argument, but the paper does not state whether the same data are used to fit the enclosing zonotope Z and core pZ and to compute the calibration scores alpha_i. As written, Section 3.1 fits Z on some data and Section 3.2 immediately calibrates on that same data, which is a double use: the score function then depends on the very calibration scores to which it is compared, breaking the exchangeability required for Eq. (3). The experiments in Section 4.2 mention a separate 1025-point set used to train the SVD, but the method section contains no such split and no validity proof for any split. The authors should either prove validity for the data-dependent family (e.g., by a full-conformal argument that includes the test point in the construction) or explicitly adopt a split-conformal protocol with distinct fitting and calibration data and prove Eq. (3) for that protocol.","section":"Sections 3.2 and 4"},{"comment":"The empirical validation should not be taken as establishing the guarantee. At alpha = 0.1 the rotated-zonotope coverage is 88.57 for Burgers (FNO) and 87.33 for Navier Stokes (FNO), and for validation sizes on the order of 500 to 1000 the latter is several standard errors below the nominal 90 percent. While finite-sample fluctuation is possible, this pattern is consistent with the Section 4.1 defect, and the paper should provide error bars and a discussion before claiming empirical support.","section":"Table 1"}],"minor_comments":[{"comment":"Please clarify the ordering used in sorted alpha and replace alpha_{ceil(epsilon n)} by the finite-sample quantile with n+1, since the standard split-conformal quantile uses ceil((1-alpha)(n+1)); as written, the notation is ambiguous.","section":"Section 3.2, Eq. (3)"},{"comment":"The half-space containment condition a1 <= lambda a2 should be a1 = lambda a2 with lambda >= 0; coordinatewise inequality is neither necessary nor sufficient for parallelism.","section":"Appendix A.2"},{"comment":"The phrase the data is totally bounded (and remains so as alpha changes) in the extra dimensions is tautological for a finite calibration set and says nothing about a future draw; it should be replaced by an explicit probabilistic statement.","section":"Section 4.1"},{"comment":"The claim that the guarantee holds irrespective of the grid discretisation is supported only by a Gaussian example; please provide the argument in the text and state the data-generation protocol for the 200 million coverage points.","section":"Figure 6"},{"comment":"The column header Calibration data dimensions is ambiguous: for Burgers the entry 2048 1024 does not match the 500 calibration points stated in Section 4.2, so the table should state which quantities are training size, calibration size, and output dimension.","section":"Table 2"}],"recommendation":"major_revision","confidential_remarks":"I believe the core zonotope conformal idea is salvageable, but the current version's main theorem is not proven. The Section 4.1 truncation issue is fixable with a proper conformal treatment of the truncated box, and the fit/calibration split needs a clear protocol; if those points are addressed, the contribution is likely publishable. I do not see evidence of intentional circularity; the issue is a mathematical gap in the proof."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nHere is my read of arXiv:2501.18426.\n\nThe genuinely useful contribution is the zonotope-based conformal prediction machinery for functional surrogates. Fitting a nested family of zonotopes to SVD-reduced errors, calibrating the nesting parameter with conformal quantiles, and propagating the sets back through the linear map is a clean and practical idea. Proposition 1 is correct, the calibration procedure is standard split-conformal logic, and the experiments on Burgers, Wave, and Navier-Stokes surrogates show the method scales to 1024-4096 dimensional outputs with competitive tightness. The authors also position the work properly against copula, supremum, and ellipsoidal conformal methods. That part is solid and deserves referee time.\n\nThe soft spot is Section 4.1. The claim that the Cartesian product with the bounding box E preserves the coverage guarantee is not true as stated. E is the element-wise min-max box of the calibration data in the truncated SVD coordinates, so every calibration error lies in E by construction. But a new exchangeable error can fall outside E. For a single continuous coordinate, P(X_{n+1} in [min_i, max_i]) = (n-1)/(n+1), not 1; with d truncated coordinates there is no distribution-free lower bound better than something like 1 - 2d/(n+1), which is negative for their settings. The augmented nonconformity scores used in the argument are therefore not exchangeable: calibration scores always see IE = 1, test scores see IE = 0 with positive probability. The statement that the data is totally bounded in the truncated dimensions does not give a probability guarantee. So P(F_{n+1} in C_alpha) >= 1-alpha is not established. The empirical coverages in Table 1 are close to nominal, and a few are below it (e.g., 88.57 for alpha=0.1 on Burgers FNO), which is consistent with the flaw being small in practice because the truncated modes have tiny variance. But the paper sells a distribution-free guarantee, and that guarantee has a hole. It is fixable: include the truncated dimensions in the conformal score, or calibrate E with its own conformal level and combine conservatively, or state a weaker claim. The current text does none of those.\n\nMinor points: the quantile formula in Section 3.2 should be written with explicit ascending order and the (n+1) term; the discretized membership calculation is conservative, so it is fine; and the comparison tables would benefit from standard errors, though that is secondary.\n\nWho is it for: anyone building uncertainty sets for neural PDE surrogates. It deserves a serious referee, but I would not let the paper pass until Section 4.1 is fixed or the guarantee statement is weakened. Send it to review rather than desk reject.\n\nBest,","headline":"Solid new conformal machinery for functional surrogates, but the SVD-truncation coverage claim in Section 4.1 is unproven and needs fixing before the headline guarantee is trustworthy.","tokens_in":19360,"tokens_out":7515,"would_cite":true,"duration_ms":72293,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper constructs conformal prediction sets for function-valued surrogates as nested zonotopes, preserving the coverage guarantee when SVD dimensions are truncated.","keywords":["conformal prediction","prediction sets","zonotopes","functional surrogate models","neural operators","uncertainty quantification","PDE emulators","SVD dimension reduction"],"falsifier":"Generate many exchangeable test points and check the truncated-coordinate indicator $I_E(x_{n+1})$ directly: if $E$ is the min-max box of $n$ calibration points in $k$ truncated dimensions, the frequency of $x_{n+1} \\in E$ is approximately $(1-2/(n+1))^k$; for $k$ large relative to $n$, this empirical coverage drops below $1-\\alpha$, and the final prediction set $C_\\alpha$ will miss the true function even though the low-dimensional zonotope contains its projection.","tokens_in":18306,"feed_emoji":"🎯","tokens_out":6329,"duration_ms":52931,"temperature":0.7,"pith_summary":"The paper's goal is to give pre-trained functional surrogate models—maps between function spaces, such as neural operators that emulate PDE solvers—a set-valued prediction with a finite-sample statistical guarantee: for a new input, the true output function lies inside the predicted set with probability at least $1-\\alpha$. The construction works on the surrogate's calibration errors: it reduces them by SVD, encloses the projected errors in a zonotope, calibrates a nested family of zonotopes with conformal prediction, and propagates the sets back through the linear SVD map. A Cartesian product with a bounding box over the discarded SVD modes is introduced so the guarantee survives the dimension reduction. If correct, this gives PDE emulators and other scientific ML models a practical, model-agnostic uncertainty quantification with a formal coverage guarantee, at a small computational cost once calibration is done.","feed_headline":"Nested zonotopes yield guaranteed prediction sets for PDE surrogates","feed_subtitle":"Conformal calibration on SVD-compressed errors gives finite-sample coverage that survives truncation.","key_machinery":"The load-bearing object is the nested zonotope family $Z^\\alpha_{p_Z} = \\langle c_Z(1-\\alpha) + p_Z\\alpha,\\; G_Z(1-\\alpha)\\rangle$, where $c_Z$ is a centre, $G_Z$ a generator matrix, and $p_Z$ a data-depth core inside the enclosing zonotope. As $\\alpha$ goes from 0 to 1 the sets shrink and contract toward $p_Z$, and the paper proves $Z^{\\alpha_1} \\supseteq Z^{\\alpha_2}$ for $\\alpha_1 \\le \\alpha_2$ via half-space inclusion. Calibration turns the abstract parameter $\\alpha$ into a conformal quantile: each calibration point is scored by the largest $\\alpha$ at which it remains inside the family, and the prediction level is the appropriate empirical quantile of these scores. Zonotopes matter because they are closed under Cartesian products and pass exactly through linear maps, which lets the method append the truncation bounding box and then project the whole set back to the output space in one matrix operation.","core_discovery":"The central claim is that one can construct a prediction set $C_\\alpha = \\hat{f}(X_{n+1}) + U\\Sigma(Z^\\alpha_{p_Z} \\times E)$ that satisfies $P(F_{n+1} \\in C_\\alpha) \\ge 1-\\alpha$ under exchangeability. Here $U\\Sigma$ is the SVD reconstruction map, $Z^\\alpha_{p_Z}$ is a nested family of zonotopes enclosing the projected calibration errors, and $E$ is a bounding box on the truncated SVD coordinates. The paper proves that the zonotope family is nested, calibrates it by ranking calibration points according to the largest membership level at which each point stays inside the family (the conformal score), and argues that the Cartesian product with $E$ does not disturb calibration because $E$ contains all calibration data in the truncated dimensions and remains fixed as $\\alpha$ changes. Numerical experiments on Burgers, wave, and Navier-Stokes surrogates show empirical coverage close to the nominal level with tighter sets than a supremum-based baseline.","pith_inferences":["The truncated-dimension guarantee could be strengthened by replacing the empirical min-max box $E$ with a separately calibrated set, for instance a family of boxes with its own conformal quantile; the paper currently uses $E$ only through its containing the calibration data and staying fixed as $\\alpha$ changes.","Since calibration relies only on membership scores, the same pipeline works with any parametric nested family closed under the propagation map, so ellipsoidal or non-symmetric families are a natural comparison.","A testable extension is an input-dependent calibration step that conditions the enclosing zonotope or the core $p_Z$ on $X$, which would improve conditional coverage at the cost of a more complex calibration procedure.","A users could choose the SVD truncation rank by balancing the tightness gained from keeping more modes against the extra volume contributed by the bounding box in the truncated coordinates."],"forward_implications":["Functional surrogates such as Fourier neural operators can report a set of plausible solution fields, not just a point prediction, with a user-chosen coverage level.","The calibration cost is paid once (75 seconds in the Burger's example), while a new prediction set is computed in about 0.02 seconds, making the method usable as an online wrapper.","Because the construction is model agnostic and needs only a held-out calibration set, the same procedure applies to any surrogate whose output is a finely discretised function.","The SVD truncation bound means users do not need to keep every output dimension in the expensive set-calibration step; a modest number of modes plus a bounding box preserves the formal guarantee.","Compared with supremum-based functional conformal bands, the zonotope sets are tighter on the PDE benchmarks at similar empirical coverage levels."],"supporting_citations":[{"why":"Supplies the conformal prediction framework and the exchangeability assumption that the calibration step inherits.","marker":"[Vovk et al., 2005, Shafer and Vovk, 2008]"},{"why":"Provides the nested-set construction for conformal prediction that the zonotope calibration directly adapts.","marker":"[Gupta et al., 2022]"},{"why":"Gives the ellipsoidal multivariate conformal method used as comparison and the Mahalanobis-style depth metric used to choose the core $p_Z$.","marker":"[Messoudi et al., 2022]"},{"why":"Supplies the Fourier neural operator architecture and the Navier-Stokes data used in the PDE experiments.","marker":"[Li et al., 2020]"},{"why":"Supplies the PDE surrogate models, datasets, and training configurations used for validation.","marker":"[Gopakumar et al., 2024a]"},{"why":"Provides the set-representation library used to construct, manipulate, and propagate the zonotopes.","marker":"[Forets and Schilling, 2021]"},{"why":"Provides the algorithm for overapproximating a polytope by a zonotope, used in one of the fitting variants.","marker":"[Guibas et al., 2003]"},{"why":"Is the supremum-based functional conformal baseline whose coverage and efficiency are compared in the benchmarks.","marker":"[Diquigiovanni et al., 2022]"}],"fun_headline_variants":["Conformal zonotopes guarantee PDE surrogate prediction sets","Finite-sample coverage for PDE surrogates via zonotopes","Zonotope sets with conformal guarantees for functional surrogates","Nested zonotopes prove coverage for PDE surrogates"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The guarantee in the truncated SVD dimensions rests on the assumption that a new point's discarded coordinates, not just the calibration points' coordinates, fall inside the min-max bounding box $E$; with many truncated dimensions this becomes substantially less likely than the nominal $1-\\alpha$.","fun_headline_variants_meta":{"raw":{"variants":["Conformal zonotopes guarantee PDE surrogate prediction sets","Finite-sample coverage for PDE surrogates via zonotopes","Zonotope sets with conformal guarantees for functional surrogates","Nested zonotopes prove coverage for PDE surrogates"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000636,"raw_usage":{"total_tokens":2913,"prompt_tokens":907,"completion_tokens":2006,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":523,"completion_tokens_details":{"reasoning_tokens":1934}},"tokens_in":523,"tokens_out":2006,"duration_ms":12976,"temperature":1.0,"reasoning_tokens":1934,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T23:33:11.759726+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate many exchangeable test points and check the truncated-coordinate indicator $I_E(x_{n+1})$ directly: if $E$ is the min-max box of $n$ calibration points in $k$ truncated dimensions, the frequency of $x_{n+1} \\in E$ is approximately $(1-2/(n+1))^k$; for $k$ large relative to $n$, this empirical coverage drops below $1-\\alpha$, and the final prediction set $C_\\alpha$ will miss the true function even though the low-dimensional zonotope contains its projection.","supporting_citations":[],"review_version":1}