{"id":"fce3cc06-ec6a-4ff8-8629-79ecef5da6fd","arxiv_id":"2502.09985","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Training the base regressor to minimize the (1−α)-quantile absolute error, as in EffOrt, empirically and theoretically yields shorter conformal prediction intervals without losing coverage.","lead":"This paper shows that in split conformal regression, the base predictor should be trained to minimize the (1−α)-quantile of absolute residuals rather than the usual squared error, and proves finite-sample bounds on the resulting interval length. It introduces EffOrt and Ad-EffOrt, which produce shorter prediction intervals at the same coverage, with experiments on synthetic and real data.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central excess-volume bound is only as strong as Assumption 2's uniform deviation phi: for the NN class used in the appendix it is neither instantiated nor verified, so the claimed learning-error separation may be vacuous in practice.","rationale":"The reader's weakest assumption is Assumption 2, and I agree. The central claim is the finite-sample excess-volume bound for EffOrt; without a concrete, nonvacuous phi for the function classes actually used, the theorem is conditional on an assumption whose practical reach is unknown. This concern is load-bearing because the paper explicitly frames the function-class complexity term as the main theoretical message and uses NN experiments to demonstrate the method. I note two further issues: Assumption 1's Holder constants are not stated uniformly over F, and the gradient algorithm solves a smoothed objective rather than the exact empirical QAE; both are patchable but would strengthen the paper. These do not change the verdict: the manuscript's contribution is genuine, and conditional acceptance with a request to instantiate and verify phi, and to align the abstract with the proved statements, is appropriate. Hence the verdict remains unchanged.","tokens_in":25103,"tokens_out":15979,"duration_ms":171823,"concrete_test":"For the linear function class of Section 5.1 and the ReLU network class of Appendix D.1, with n_l=1000, alpha=0.1, delta=0.05, instantiate the Appendix B.2 VC/Rademacher bound to obtain a numerical phi, and estimate the local Holder constants L,gamma from fitted residuals. Then evaluate the learning term 4L phi^gamma in Eq. (13) and check whether it is small relative to the observed oracle interval length and whether phi<=r. If 4L phi^gamma exceeds a small fraction (e.g., 10%) of the oracle length, or if no finite phi<=r can be established for the NN class, the finite-sample excess-volume guarantee does not cover the experimental settings; if it is small, Assumption 2 is not a practical obstruction.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The main theorem (Theorem 1, Eq. (13)) decomposes the excess volume of EffOrt into a calibration term decaying as n_c^{-gamma} and a learning term 4L phi(F,delta,n_l)^gamma. The entire learning term is controlled by Assumption 2 (Section 3.4), which requires a uniform high-probability bound on sup_{t>=0, f in F} |P(|Y-f(X)|<=t) - empirical CDF|. This is not pointwise DKW: it must hold simultaneously for every threshold t and every f in F, and phi must also satisfy phi(F,delta,n_l)<=r for Theorem 1 to apply. The paper gives a finite-class bound (Prop. 2) and a generic VC/Rademacher bound (Appendix B.2), but no concrete phi is derived for the one-hidden-layer ReLU networks used in the Appendix D.1 experiments. For such a class the VC dimension can be large in the number of parameters, so with n_l=1000 the bound may be vacuous (phi not small, or phi>r), making Eq. (13) uninformative and the claimed separation between calibration and learning error untestable. The main-text linear-class experiments are closer to being covered, but the theoretical claim is stated for general F.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies volume (interval length) optimality in split conformal regression. It first observes that, for a fixed base predictor, the calibration step of split conformal prediction solves an empirical version of a minimum-volume-set problem, and it derives finite-sample excess-volume bounds for the calibrated interval (Proposition 1 and Corollary 1). It then argues that the learning step should minimize the (1-alpha)-quantile of absolute error rather than the mean squared error, and introduces EffOrt, a split conformal method that does this. Theorem 1 gives a coverage guarantee and an excess-volume bound that separates a calibration error term depending on n_c from a learning error term involving a uniform deviation phi(F,delta,n_l) (Assumption 2). The paper also introduces Ad-EffOrt for covariate-adaptive interval widths, with additional theoretical results in Appendix B.3, and reports synthetic and real-data experiments in which EffOrt and Ad-EffOrt are compared with standard split conformal, locally weighted conformal, and conformalized quantile regression.","tokens_in":25349,"tokens_out":23463,"duration_ms":201344,"significance":"If Theorem 1 is correct, the paper provides a genuinely finite-sample analysis of interval volume in conformal regression, a topic that is usually treated only empirically or asymptotically. The decomposition of excess volume into calibration and learning terms is a useful conceptual contribution, and the proposal to replace least-squares fitting with quantile-of-absolute-error fitting is well motivated and clearly explained. The paper also ships code and its experimental section compares several heavy-tailed and asymmetric settings, which is a strength. The central derivation is not circular: it uses DKW and Scott-Nowak style MVS theory as external tools. The main risks are that Assumption 2 is not instantiated for the function classes used in the experiments, and that Assumption 1 is stated per f rather than uniformly, while the proof of Theorem 1 uses common constants for the data-dependent predictor f_hat and the oracle f*. These issues are fixable but affect the main theorem as written.","major_comments":[{"comment":"The learning-error term in Eq. (13) is entirely controlled by Assumption 2, which requires a uniform high-probability bound on sup_{t>=0, f in F} |P(|Y-f(X)|<=t) - empirical CDF|. This is substantially stronger than pointwise DKW because it must hold simultaneously over all thresholds and all functions in F. The paper gives a finite-class bound (Proposition 2) and generic VC/Rademacher bounds (Appendix B.2), but it never instantiates phi(F,delta,n_l) for the function classes used in the experiments. In particular, for the one-hidden-layer ReLU networks used in Appendix D.1, the VC dimension of the induced class can be large relative to n_l=1000, so the condition phi(F,delta,n_l)<=r may fail and Eq. (13) becomes uninformative. Since Theorem 1 is stated for a general F and the separation between calibration and learning errors is a central claim, the authors should either instantiate phi for the linear class of Section 5.1 and for the neural-network class of Appendix D.1, or explicitly restate Theorem 1 only for classes for which such a phi is shown to satisfy the required smallness condition, and verify that condition in the experiments.","section":"Section 3.4, Theorem 1 and Assumption 2"},{"comment":"Assumption 1 is stated as: for every f in F and every alpha, there exist r, gamma, L such that the quantile function of |Y-f(X)| is locally (gamma,L)-Holder. This allows the constants L and gamma to depend on f. In the proof of Theorem 1 (Appendix A.2, Step 2), the same L and gamma are applied to both Q(.;|Y-f_hat(X)|) and Q(.;|Y-f*(X)|), and the statement of Eq. (13) uses a single pair L,gamma. Since f_hat is data-dependent and F is an arbitrary class, the per-f statement of the assumption does not provide the common constants needed by the proof. The assumption should be strengthened to require uniform constants over F (or the theorem should explicitly define L and gamma as suprema/infima over F and state the additional conditions needed for those to be finite and attained). As written, the proof of the central excess-volume bound has a gap.","section":"Section 3.4 and Appendix A.2, Assumption 1"},{"comment":"There are two inconsistencies in the adaptive-size theory. First, Lemma 2 states in Eq. (23) a bound against the oracle lambda(C^{1-alpha}_{f*,s*,t*}), but in the proof the bad event Theta_lambda is defined with lambda(C^{1-alpha+phi}_{f*,s*,t*}), and after the first implication the proof switches to s*_{1-alpha+phi}. These objects do not match, so the proof as written does not establish the stated lemma. Second, the proof of Theorem 2 applies Assumption 1 on an interval whose lower endpoint is 1-alpha-2phi(F,S,delta,n_l), which requires 2phi(F,S,delta,n_l)<=r, whereas the theorem only assumes phi(F,S,delta,n_l)<=r. Both issues appear local and fixable, but they must be corrected before the appendix results provide the 'strong theoretical grounding' that the paper claims for Ad-EffOrt.","section":"Appendix B.3, Lemma 2 and Theorem 2"}],"minor_comments":[{"comment":"Proposition 1 assumes that (n_c+1)(1-alpha) is not an integer, but Theorem 1, which invokes Proposition 1, does not restate this condition. Please add the condition or explain why the atomless assumption makes the integer case harmless.","section":"Section 3.1 and Theorem 1"},{"comment":"Proposition 3 uses sqrt(log(1/delta)/(2n_c)) in the quantile argument, whereas Proposition 1 uses sqrt(log(2/delta)/(2n_c)). Please align the constants or explain the different confidence accounting.","section":"Appendix B.1, Proposition 3"},{"comment":"The gradient formula in Eq. (27) and Algorithm 1 divide by sum_i B_i, which can be zero if no training loss falls within epsilon of the current quantile. Please state the safeguard used in practice or note the degeneracy explicitly.","section":"Section 3.3.1 and Algorithm 1"},{"comment":"The empirical QAE objective is nonconvex, but the paper does not specify the initialization theta_1 used by the gradient descent. Since the reported interval lengths may depend on this choice, please provide the initialization for the linear and neural-network experiments.","section":"Section 5.1 and Appendix D.1"},{"comment":"The abstract and introduction refer to 'the shortest oracle prediction interval' without the qualifier that the oracle is restricted to the interval class C^{const}_F (and C^{adap}_{F,S} for Ad-EffOrt). Please make this restriction explicit to avoid overclaiming.","section":"Introduction and Section 3.1"}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea and Theorem 1 are worth publishing after the assumptions are clarified and the appendix is corrected. The most important risk is whether the uniform deviation phi in Assumption 2 can be made small enough for realistic neural-network classes; if not, the paper's practical claims should be re-scoped to linear or low-complexity classes where phi can be instantiated. The per-f formulation of Assumption 1 and the stated-proof mismatch in Lemma 2 are local but must be fixed. The self-citation in the proof of Theorem 2 is acceptable only if the cited proposition is stated or the preprint is readily available; otherwise please ask the authors to include the lemma in the appendix."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Rough take: this paper delivers a real theoretical result. Theorem 1 is a finite-sample excess-volume bound for split conformal regression that cleanly separates calibration error (terms in n_c) from learning error (the phi term). That separation is new as far as I know, and it supports the core prescription: train the base predictor to minimize the (1-alpha)-quantile of absolute error rather than MSE if you want short intervals. The proof strategy is sound, using the Scott-Nowak MVS bound for the learning step and a DKW-based quantile bound for calibration, joined by a union bound.\n\nThe paper is also honest about its limits. Ad-EffOrt is presented without the same theoretical guarantees, and the authors say so explicitly. The experiments cover asymmetric and heavy-tailed noise, with linear models in the main text and NN results in the appendix. The empirical gains from EffOrt are real but modest, which is expected when MSE is near-optimal.\n\nThe main soft spot is Assumption 2. It requires a uniform high-probability bound on sup over f in F and t of the absolute deviation between the true CDF of |Y-f(X)| and the empirical CDF. That is stronger than pointwise DKW, and the paper does not instantiate it for the one-hidden-layer ReLU network class used in the appendix. The finite-class bound (Prop 2) and the generic VC/Rademacher bounds are fine, but they do not tell us whether phi is small at n_l = 1000 for that class. So the learning term in Eq (13) may be vacuous for the experiments that are supposed to show the method working. This is not fatal, but it should be addressed by either specializing phi for the classes actually used or rephrasing the theorem as conditional on a specific phi.\n\nOther issues are minor: Prop 1 requires (n_c+1)(1-alpha) non-integer, the Holder constants in Assumption 1 may depend on f, and the smoothing procedure in Section 3.3.1 means the implemented algorithm does not exactly minimize the empirical quantile analyzed in Theorem 1. These are the usual theory-practice gaps and do not sink the result.\n\nBottom line: the central theorem is a genuine contribution and deserves serious refereeing. I would send it to review, asking for the abstract to be aligned with what is proved and for a concrete phi for the actual model classes, even if only for linear functions. The paper is readable, the citations are honest, and the central claim holds up.","headline":"A genuine finite-sample excess-volume bound for split conformal regression, with the main soft spot being an uninstantiated uniform deviation assumption that may make the learning term vacuous for the neural-network experiments.","tokens_in":25936,"tokens_out":2221,"would_cite":true,"duration_ms":22576,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62G05","62G08","62G20"],"pacs":[],"model":"deepseek-v4-flash","headline":"A finite-sample bound shows conformal intervals shrink when the base predictor minimizes the (1−α)-quantile of absolute error.","keywords":["conformal prediction","split conformal regression","prediction interval length","quantile absolute error","minimum volume sets","excess volume bound","adaptive prediction intervals","robust regression"],"falsifier":"Measure the realized excess volume of EffOrt on a heavy-tailed asymmetric noise model with a neural-network class $\\mathcal{F}$, and compare it with Theorem 1 using an empirically estimated $\\varphi(\\mathcal{F},\\delta,n_\\ell)$. If the gap between EffOrt and the oracle interval does not shrink as $n_\\ell$ grows, or if the uniform deviation $\\varphi$ cannot be bounded even for moderate $n_\\ell$, the theorem's learning-rate term is not the right description.","tokens_in":24868,"feed_emoji":"📏","tokens_out":6788,"duration_ms":63596,"temperature":0.7,"pith_summary":"Split conformal regression is easy to certify for coverage, but its prediction intervals can be far longer than the shortest possible interval. This paper shows that the calibration step of split conformal prediction is exactly an empirical volume-minimization problem, and that the right learning objective for short intervals is not the mean squared error but the (1−α)-quantile of the absolute residual. Building on that, the authors prove a finite-sample excess-volume bound that separates the error due to calibration (in the number of calibration points $n_c$) from the error due to learning (in a uniform deviation term $\\varphi(\\mathcal{F},\\delta,n_\\ell)$ over the function class). The result motivates two new methods, EffOrt and Ad-EffOrt, which train the base predictor to minimize the quantile of absolute error and produce shorter intervals on asymmetric and heavy-tailed data while keeping marginal coverage.","feed_headline":"Train on the residual quantile for shorter conformal intervals","feed_subtitle":"New theory splits interval-length error into calibration and learning terms; EffOrt exploits both.","key_machinery":"The load-bearing object is the $(1-\\alpha)$-QAE problem: minimize the $(1-\\alpha)$-quantile of the absolute residual $|Y-f(X)|$ over the function class $\\mathcal{F}$, because that quantile is exactly the half-width of the optimal constant-size oracle interval. To make the argument quantitative, the paper treats the learning step as an empirical minimum-volume-set estimator and applies the Dvoretzky–Kiefer–Wolfowitz inequality to control empirical quantile deviations. Two assumptions carry the rates: a Hölder continuity condition on the quantile function of the residuals (Assumption 1), which converts probability differences into length differences, and a uniform high-probability bound $\\varphi(\\mathcal{F},\\delta,n)$ on the sup over $f\\in\\mathcal{F}$ and $t\\ge 0$ of $|P(|Y-f(X)|\\le t) - $ the empirical counterpart $|$ (Assumption 2), which measures the complexity of $\\mathcal{F}$. For optimization, the empirical quantile is smoothed with a differentiable approximation $\\Gamma_\\varepsilon$ so that gradients can be computed.","core_discovery":"With a fixed base predictor $f$, the optimal constant-width interval is $[f(x)-t_*, f(x)+t_*]$ where $t_*$ is the $(1-\\alpha)$-quantile of $S=|Y-f(X)|$, and the split conformal calibration step estimates that quantile from a calibration set, giving an excess length of order $O(n_c^{-\\gamma})$ under a Hölder condition. The paper's central claim is that when $f$ is also learned, the whole procedure is a minimum-volume-set problem, and the correct objective is to minimize $Q(1-\\alpha; |Y-f(X)|)$ over $f$ in the class $\\mathcal{F}$. EffOrt approximately solves that problem with a gradient-based smooth quantile solver. Theorem 1 states that, with high probability, the interval length of EffOrt is at most the oracle length plus $2L(1/n_c + \\sqrt{\\log(2/\\delta)/(2n_c)})^\\gamma + 4L\\varphi(\\mathcal{F},\\delta,n_\\ell)^\\gamma$, where the first terms come from calibration and the last from learning. This is the first finite-sample excess-volume bound that explicitly separates the two split steps and shows the learning term, governed by the complexity of $\\mathcal{F}$, dominates the rate.","pith_inferences":["The additive decomposition suggests an optimal data-split ratio could be derived by balancing $1/n_c$ against $\\varphi(\\mathcal{F},\\delta,n_\\ell)^\\gamma$, a calculation the paper does not perform.","The QAE objective may be useful beyond conformal prediction, for example as a robust regression loss that targets a prespecified quantile rather than the mean, especially under asymmetric noise.","For adaptive intervals, a similar separation between learning and calibration errors could be obtained via Assumption 4; verifying that assumption for neural-network classes would be a natural next step.","The bound implies that controlling the capacity of $\\mathcal{F}$ (for instance through regularization) directly reduces prediction interval length, connecting interval efficiency to classical generalization bounds."],"forward_implications":["Split conformal calibration is asymptotically volume-optimal for a fixed base predictor, with rate $O(n_c^{-\\gamma})$ under Hölder regularity.","The learning objective for short intervals is the $(1-\\alpha)$-quantile of absolute residuals, not squared error; MSE can be suboptimal under asymmetry and heavy tails.","Excess volume decomposes additively into a calibration term ($n_c$) and a learning term ($\\varphi(\\mathcal{F},\\delta,n_\\ell)^\\gamma$), so more learning data than calibration data is preferable.","EffOrt provides a practical gradient-based solver via a smooth quantile approximation, and Ad-EffOrt extends the approach to covariate-adaptive interval widths.","Marginal coverage at level $1-\\alpha$ is preserved almost surely after the calibration step."],"supporting_citations":[{"why":"Introduces split conformal regression, whose calibration step the paper reinterprets as an empirical volume-minimization problem.","marker":"(Papadopoulos et al., 2002)"},{"why":"Supplies the minimum-volume-set estimation theory used in Step 1 of the proof of Theorem 1 to compare the learned interval with a coverage-increased oracle.","marker":"(Scott and Nowak, 2005)"},{"why":"Provides the classical coverage guarantee and the Hölder regularity context that converts probabilistic deviations into interval-length deviations.","marker":"(Lei et al., 2018)"},{"why":"Gives the DKW inequality used to control the sup-norm distance between the true and empirical distribution functions of the absolute residuals.","marker":"(Dvoretzky et al., 1956)"},{"why":"Supplies the tight constant in the DKW inequality, which is applied in the proof of Proposition 1.","marker":"(Massart, 1990)"},{"why":"Provides the smooth approximation of the empirical quantile that makes the QAE objective differentiable for gradient-based optimization in EffOrt.","marker":"(Pena-Ordieres et al., 2020)"}],"fun_headline_variants":["Volume-optimal conformal regression: EffOrt shrinks intervals","First finite-sample bound on conformal interval excess length","EffOrt: Learn the quantile to minimize conformal volume","Conformal regression: minimize interval length, not just coverage","EffOrt closes the gap between conformal calibration and learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The results assume a uniform high-probability bound $\\varphi(\\mathcal{F},\\delta,n)$ on how well any function in the class $\\mathcal{F}$ approximates the true probability that $|Y-f(X)|\\le t$, simultaneously for all thresholds $t$; if the function class is complex and the learning sample is small, this bound may be large or unverified, and the main excess-volume guarantee becomes vacuous.","fun_headline_variants_meta":{"raw":{"variants":["Volume-optimal conformal regression: EffOrt shrinks intervals","First finite-sample bound on conformal interval excess length","EffOrt: Learn the quantile to minimize conformal volume","Conformal regression: minimize interval length, not just coverage","EffOrt closes the gap between conformal calibration and learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000697,"raw_usage":{"total_tokens":3172,"prompt_tokens":988,"completion_tokens":2184,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":604,"completion_tokens_details":{"reasoning_tokens":2099}},"tokens_in":604,"tokens_out":2184,"duration_ms":23217,"temperature":1.0,"reasoning_tokens":2099,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T19:50:35.122825+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the realized excess volume of EffOrt on a heavy-tailed asymmetric noise model with a neural-network class $\\mathcal{F}$, and compare it with Theorem 1 using an empirically estimated $\\varphi(\\mathcal{F},\\delta,n_\\ell)$. If the gap between EffOrt and the oracle interval does not shrink as $n_\\ell$ grows, or if the uniform deviation $\\varphi$ cannot be bounded even for moderate $n_\\ell$, the theorem's learning-rate term is not the right description.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces split conformal regression, whose calibration step the paper reinterprets as an empirical volume-minimization problem."},{"cited_title":"and Nowak, R","cited_arxiv_id":null,"evidence_quote":"Supplies the minimum-volume-set estimation theory used in Step 1 of the proof of Theorem 1 to compare the learned interval with a coverage-increased oracle."},{"cited_title":"J., and Wasserman, L","cited_arxiv_id":null,"evidence_quote":"Provides the classical coverage guarantee and the Hölder regularity context that converts probabilistic deviations into interval-length deviations."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the tight constant in the DKW inequality, which is applied in the proof of Proposition 1."},{"cited_title":"R., and Waachter, A","cited_arxiv_id":null,"evidence_quote":"Provides the smooth approximation of the empirical quantile that makes the QAE objective differentiable for gradient-based optimization in EffOrt."}],"review_version":1}