{"id":"7750cc90-b92f-4111-bef8-1248c8e70e37","arxiv_id":"2411.13479","paper_version":4,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"Projecting non-conformity scores onto the coherent subspace of a hierarchy makes split-conformal prediction regions smaller while preserving finite-sample coverage, under i.i.d. data and, for the efficiency results, elliptical residual assumptions.","lead":"This paper inserts a reconciliation step, a projection onto the hierarchy's consistency constraints, into split conformal prediction, and proves the resulting prediction regions are never larger while keeping coverage guarantees. This matters because hierarchical data, such as national and regional electricity demand, are common, and smaller valid uncertainty sets make forecasts more useful.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The component-wise efficiency theorems depend essentially on the elliptical-residual assumption; without Assumption 2, the common-L_alpha scaling in the proof of Theorem 3 breaks and P_{Sigma^{-1}} can fail to dominate other projections component-wise.","rationale":"The reader's weakest_assumption already identifies Assumption 2, and I agree. I checked the proofs of Theorems 3 and 4 and Lemma 1; they are internally consistent: Lemma 1 is correctly derived, the projection characterization is valid, and the weighted-sum to component-wise limiting argument is sound. The only genuinely load-bearing step is the reduction to a single L_alpha via Lemma 3, which is exactly the ellipticity assumption. I do not see an internal contradiction in the theorems as stated; they are honest conditional statements. The experimental section and Section 6 also acknowledge that practical MinT is unstable and that the i.i.d./elliptical setting is favorable. There is a minor presentational overclaim in the abstract (no mention of ellipticity), and the sentence in Section 2.3.2 about signed scores being coherent is imprecise, but neither threatens the theorem. The concrete test above would determine whether the concern lands empirically; if non-elliptical failures appear, the efficiency result should be advertised as assumption-dependent rather than a general conformal property. Since the reader's CONDITIONAL verdict already reflects this limitation, I leave the verdict unchanged.","tokens_in":34041,"tokens_out":26116,"duration_ms":283658,"concrete_test":"Take a fixed hierarchy, e.g., H = [[1,0],[0,1],[1,1]], and draw calibration residuals of size T_calib from a non-elliptical distribution with a specified positive-definite covariance Sigma, for instance \\hat s_t = M z_t with independent components z_{t,1} ~ skew-Normal, z_{t,2} ~ t_5, z_{t,3} ~ N(0,1), and M chosen so Cov(\\hat s_t)=Sigma. Run Algorithm 4 with P_{Sigma^{-1}} and with one alternative projection, e.g., P_OLS, and compute the empirical E[L1(~C_i)^2] for each component over many Monte Carlo replications of calibration and test residuals. If for at least one component the P_{Sigma^{-1}} interval has larger expected squared length than the alternative, then Theorem 4's component-wise dominance does not extend beyond elliptical residuals, confirming that the assumption is load-bearing rather than a harmless technical condition.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central efficiency claim (Theorem 4, Section 4.3) rests on Assumption 2 (Section 4.1): calibration residuals are i.i.d. elliptical. The proof of Theorem 3 (Appendix A) uses Lemma 3 to represent every marginal of every projected residual as sqrt(Lambda_ii) v for one common random variable v; consequently all component interval lengths are scalar multiples of the same order-statistic random variable L_alpha (Eq. (6)). This is what reduces expected squared lengths to traces of P Sigma P^T and lets Lemma 1 (minimum-trace projection) imply component-wise dominance. Without ellipticity, this proportionality fails: marginal distributions of projected residuals can have different quantile spreads even when their variances are equal. The trace inequality then bounds a quantity (P Sigma P^T)_ii that is no longer proportional to E[L1(~C_i)^2]. Thus the headline 'provably smaller' component-wise intervals is not a distribution-free conformal statement; it is a parametric efficiency theorem. The paper is explicit that Assumption 2 is restricted to efficiency, so coverage survives, but any practical use of the size reduction must verify or assume elliptical scores. Experiments in Appendix F generate Gaussian noise through a linear smoother, so they cannot probe robustness to non-elliptical residuals.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a projection (forecast-reconciliation) step into split conformal prediction for hierarchical multivariate data, where observations satisfy y=H y_{1:n}. For joint coverage, it modifies ellipsoidal conformal prediction by replacing the base regressor with its orthogonal projection onto the coherent subspace in the chosen norm and shows, deterministically, that the resulting ellipsoids have no larger volume. For component-wise coverage, it proposes signed-score split conformal prediction with a projection matrix P satisfying P H=H, proves finite-sample marginal coverage for every component, and then derives efficiency guarantees: under i.i.d. residuals with an elliptical distribution, the projection P_w=H(H^\\top diag(w)H)^{-1}H^\\top diag(w) achieves no larger expected weighted sum of squared interval lengths than the unprojected benchmark; under a second-order moment, the minimum-trace projection P_{\\Sigma^{-1}} dominates every projection onto Im(H) component-wise. The theoretical claims are supported by self-contained proofs in the appendices, and numerical experiments on synthetic hierarchies illustrate the gains and the practical instability of the estimated MinT projection.","tokens_in":34232,"tokens_out":26243,"duration_ms":296915,"significance":"If the stated assumptions are met, the paper contributes one of the few finite-sample efficiency results for conformal prediction and builds a useful bridge between conformal prediction and forecast reconciliation. Its strengths are the transparent decomposition of coverage and efficiency results, the self-contained proofs of the trace inequalities (including an elementary proof of the minimum-trace projection lemma), and the explicit statement of the assumptions under which the efficiency claims hold. The main limitation is also clear: the headline component-wise efficiency result is not distribution-free, since it relies on elliptical residuals and, for Theorem 4, on knowledge of the true covariance matrix. The experiments use Gaussian noise only and show that the estimated MinT projection can underperform the unprojected benchmark in high-dimensional hierarchies, so the empirical evidence does not probe behavior outside Assumption 2.","major_comments":[{"comment":"The statement that residuals are i.i.d. 'in particular when data are i.i.d.' is not correct for the split conformal pipeline. In Algorithms 3-5 the residuals y_t - \\hat{\\mu}(x_t) for t in D_calib share the regressor \\hat{\\mu} fitted on D_train, so unconditionally they are exchangeable but generally not independent; a simple counterexample is Gaussian data with a random common mean. The coverage proof of Theorem 2 only needs exchangeability, but the efficiency proofs of Theorems 3 and 4 use order statistics of i.i.d. draws and the common random variable L_alpha, so the i.i.d. assumption is load-bearing for the efficiency claims. The authors should either state Assumption 1 conditionally on the training and estimation data, or separate the exchangeability assumption needed for coverage from the stronger conditional i.i.d. assumption needed for efficiency.","section":"Section 4, Assumption 1"},{"comment":"Assumption 2 does not require the elliptical residual distribution to have a first-order moment, yet the proof of Theorem 3 centers the residuals by E[\\hat{s}_{1,i}], which need not exist; the proof of Theorem 4 even states that Theorem 3 did not rely on a first-order moment, which is contradicted by the written argument. The fix is straightforward: replace the expectation by the location vector c appearing in Definition 3, since interval lengths are translation-invariant and the order-statistic range does not depend on c. The proof should also handle the small-alpha case where the lower quantile index is zero (so L_alpha is infinite) and the case where some diagonal entry of \\Gamma is zero, because the factorization E[L_1(C_i)^2] = \\Gamma_{i,i} E[L_alpha^2] is not meaningful in those cases without an extended-convention qualification.","section":"Appendix A, proof of Theorem 3 and Lemma 3"}],"minor_comments":[{"comment":"The sentence claiming that signed non-conformity scores between coherent observations and forecasts are also coherent is only true for reconciled forecasts; for an arbitrary base forecast \\hat{y}_t, the vector y_t - \\hat{y}_t need not lie in Im(H). Please restrict the sentence to the projected scores used in Algorithm 4.","section":"Section 2.3.2"},{"comment":"The statement says that Algorithm (2) outputs ellipsoids \\check{E} that are more efficient than the ellipsoids \\check{E} output by Algorithm (1); the first of these should be \\mathring{E}.","section":"Appendix D, Theorem 5"},{"comment":"The claim that MinT performs worse than the Direct benchmark for the largest hierarchies is clearly supported by Configuration 5 but not by Configuration 6, where the confidence intervals overlap; the wording should be made more cautious.","section":"Section 5 and Appendix F.3.3"},{"comment":"All synthetic residuals are generated as Gaussian, so the experiments do not probe robustness to non-elliptical residual distributions; a sentence stating that the simulations are intended to illustrate the elliptical case and do not validate efficiency outside Assumption 2 would help.","section":"Appendix F"},{"comment":"The no-tie condition for the upper coverage bound is stated for the vector-valued scores, but the proof needs component-wise distinctness of the univariate scores; full-vector distinctness does not prevent ties in a single component. Please state the condition component-wise.","section":"Theorem 2, proof in Appendix E.1"}],"recommendation":"major_revision","confidential_remarks":"The core ideas are sound and the central coverage result is correct, but the manuscript currently mixes an exchangeability-style conformal setting with an i.i.d. assumption on residuals that is not implied by i.i.d. data, and the efficiency proof contains a small but real moment-existence gap. Both issues are fixable without changing the main theorems, so I recommend major revision rather than rejection. The paper would also be strengthened by making the elliptical-residual dependence of the headline efficiency result more prominent in the abstract and introduction."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper is better than the usual \"we applied method A to problem B\" piece. It actually proves something: adding a reconciliation projection to split conformal prediction yields smaller prediction intervals for hierarchical data, in a precise weighted-squared-length sense. Theorems 3 and 4 are new results, not repackaged old ones, and the proofs are self-contained and readable. The elementary proof of the minimum-trace projection lemma (their Lemma 1) is a nice contribution by itself.\n\nThe coverage guarantees are standard but handled carefully with signed scores, and the paper is explicit that the efficiency theorems rest on Assumption 2: i.i.d. residuals from an elliptical distribution. The stress-test note is correct that without ellipticity the common-L_alpha scaling in the proof of Theorem 3 breaks, so the trace inequality no longer controls expected interval lengths. But the paper never hides this—it labels Theorem 4 as an oracle result and the practical MinT implementation as unstable on large hierarchies. The experiments, however, only generate Gaussian noise, so they cannot probe how much the efficiency claim degrades under non-elliptical residuals. That is the main soft spot: the headline \"provably smaller\" intervals are a parametric efficiency theorem, not a distribution-free conformal statement. Coverage remains distribution-free, which is correctly stated.\n\nThe other limitations are minor. The experiments use synthetic data only, with no public code, and the claim that \"three algorithms perform uniformly better\" than Direct is immediately qualified by the MinT instability, so it is not really an overclaim. The theoretical contribution stands regardless.\n\nWho benefits: readers in probabilistic hierarchical forecasting, conformal prediction researchers working on efficiency guarantees, and forecast reconciliation people curious about conformal connections. The paper deserves a serious referee. It is not a desk reject. A referee should push on the ellipticity assumption—maybe ask for a small non-elliptical experiment or a discussion of when elliptical residuals are plausible—but the core is solid and honestly presented. I would cite the component-wise theorems and the Lemma 1 proof.","headline":"Genuinely new link between conformal prediction and forecast reconciliation, with honest limitations: the efficiency gains are real but depend on elliptical residuals, not distribution-free.","tokens_in":34810,"tokens_out":1566,"would_cite":true,"duration_ms":19003,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62G15","62H10","62M20"],"pacs":[],"model":"deepseek-v4-flash","headline":"Projecting predictions onto a hierarchy's coherent subspace makes conformal sets provably smaller while preserving coverage.","keywords":["conformal prediction","hierarchical data","forecast reconciliation","component-wise coverage","elliptical distributions","minimum-trace projection","prediction sets","efficiency"],"falsifier":"Generate i.i.d. residuals from a distinctly non-elliptical distribution—for instance, a centered log-normal or a mixture of two Gaussians with different scales—and run Algorithm 4 with the covariance-based projection P_{$Σ^{{-1}}$} against the plain component-wise SCP; if for some component the expected squared interval length is not smaller, or the empirical ratio of lengths departs from the predicted $\\sqrt$((P Σ P^T)_{i,i}/Σ_{i,i}) scaling, the elliptical assumption behind Theorem 4 is violated.","tokens_in":33818,"feed_emoji":"📊","tokens_out":5748,"duration_ms":58462,"temperature":0.7,"pith_summary":"The paper shows that adding a single projection step—reconciling base forecasts with the linear constraints of a data hierarchy—to split conformal prediction yields prediction sets that are valid and provably more efficient than the unprojected benchmark. For joint coverage, the projected ellipsoids are uniformly smaller in volume under no assumptions at all. For the harder component-wise coverage objective, the paper proves that a specific projection based on the error covariance (the minimum-trace projection) dominates every other coherent projection and the plain method, shrinking expected squared interval lengths component by component. The catch is that the component-wise efficiency theorem assumes the residuals follow an elliptical distribution; the coverage guarantees themselves remain distribution-free. The paper thereby connects two post-hoc forecasting procedures—conformal prediction and forecast reconciliation—and gives the first efficiency guarantees for hierarchy-aware conformal prediction.","feed_headline":"A projection step makes conformal sets provably smaller","feed_subtitle":"Reconciling forecasts with the hierarchy's linear structure cuts interval lengths without sacrificing coverage guarantees.","key_machinery":"The load-bearing object is the projection matrix onto the coherent subspace Im(H), the set of vectors consistent with the hierarchy's summation constraints. The paper singles out the minimum-trace projection P_{$Σ^{{-1}}$} = H(H^T $Σ^{{-1}}$ H)^{-1} H^T $Σ^{{-1}}$, which is the orthogonal projection in the norm induced by the inverse error covariance, and shows it minimizes the expected weighted sum of squared interval lengths. Two trace inequalities carry the argument: a Pythagorean-type projection inequality for W-norms, and the minimum-trace lemma from forecast reconciliation, which the paper reproves elementarily. Signed vector-valued non-conformity scores are what make the translation from interval lengths to traces work, since centering shifts cancel in the quantile spread.","core_discovery":"The central discovery is that the reconciliation step translates directly into a scaling law for interval lengths. Because residuals are assumed elliptical, each component's non-conformity score is a scaled version of a single common random variable; after projecting the scores with a matrix P onto the coherent subspace Im(H), the length of component i's interval is $\\sqrt$((P Γ P^T)_{i,i}) times a common quantile spread L_α. Expected squared lengths therefore reduce to a trace, and the trace inequality Tr(W P Σ P^T) ≥ Tr(W P_{$Σ^{{-1}}$} Σ P_{$Σ^{{-1}}$}^T) identifies P_{$Σ^{{-1}}$} = H(H^T $Σ^{{-1}}$ H)^{-1} H^T $Σ^{{-1}}$ as the optimal projection for all weight vectors simultaneously. Under i.i.d. residuals with an elliptical distribution admitting a second-order moment, the reconciled procedure run with P_{$Σ^{{-1}}$} gives, for every component, expected squared interval length no larger than that of any other projection onto the coherent subspace, and no larger than the plain component-wise SCP benchmark.","pith_inferences":["The trace-reduction perspective suggests the same projection approach could be extended to other structured linear constraints beyond summation hierarchies, such as balances in compositional data or linear panel restrictions, with the same proof template.","The efficiency gain concentrates at the most disaggregated level of the hierarchy in the simulations; a practitioner with primary interest in aggregate nodes might prefer a simpler projection, since aggregated forecasts are already close to coherent.","The ellipticity assumption, while used only for efficiency, could be tested directly: it predicts that after projection, each component's scores are homothetical—the same distribution up to scale—so a multivariate symmetry test on residuals could flag settings where the claimed dominance may fail.","Adapting the projection step to the adaptive-conformal-inference framework for non-exchangeable time series, as the authors suggest, would make the size reduction operational in the very applications (energy forecasting) that motivate the hierarchy setup; the coverage side is already covered by ACI, leaving the efficiency side open."],"forward_implications":["For any hierarchy and any base regressor, the reconciled ellipsoidal procedure returns prediction regions no larger than the plain ellipsoidal procedure, with no assumptions on the data.","Component-wise coverage at level 1−α holds for every projection satisfying P H = H, so the efficiency gains never come at the cost of the distribution-free coverage guarantee.","Under elliptical i.i.d. residuals, the weighted least-squares projection P_w with weights w provably beats the plain component-wise benchmark for the weighted criterion.","Under additionally bounded second moments, the oracle minimum-trace projection P_{Σ^{-1}} dominates every other projection onto Im(H) and every plain component-wise procedure, simultaneously for all positive weight vectors.","In simulations, the data-based WLS variant is the most robust improvement over the benchmark, while the MinT variant can degrade for very large hierarchies due to near-singular covariance estimation."],"supporting_citations":[{"why":"Supplies the split conformal prediction procedure used as the benchmark and the standard coverage argument.","marker":"Lei et al., 2018"},{"why":"Introduces signed-error conformal regression, the scores whose translation-invariance makes the trace reduction work.","marker":"Linusson et al., 2014"},{"why":"Origin of the minimum-trace projection lemma, the key inequality behind Theorem 4.","marker":"Wickramasuriya et al., 2019"},{"why":"Geometric view of reconciliation and the W-norm projection that inspire the trace-reduction lemma (Lemma 5).","marker":"Panagiotelis et al., 2021"},{"why":"Alternative proof of minimum-trace reconciliation, simplified in the appendix to provide an elementary proof of Lemma 1.","marker":"Ando & Narita, 2024"},{"why":"Provides the ellipsoidal conformal prediction method that the paper extends to hierarchical data for joint coverage.","marker":"Johnstone & Cox, 2021"},{"why":"Confirms the practical efficiency of ellipsoidal prediction regions and supplies the A-norm formulation used in Algorithms 1–2.","marker":"Messoudi et al., 2022"},{"why":"Background on elliptical distributions and the homothetic marginal property used to represent interval lengths as scaled copies of a common random variable.","marker":"Kollo & von Rosen, 2005"}],"fun_headline_variants":["Reconciliation makes conformal regions provably smaller","Optimal projection shrinks conformal intervals","Hierarchical reconciliation guarantees tighter conformal sets","Projection step provably reduces conformal set size","Elliptical residuals enable optimal conformal projections"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The component-wise efficiency results hold only if the residuals are i.i.d. and follow some elliptical distribution (with unknown shape and parameters); if the residuals are skewed, multimodal, or otherwise non-elliptical, the claimed size reduction can fail, though the coverage guarantees still hold.","fun_headline_variants_meta":{"raw":{"variants":["Reconciliation makes conformal regions provably smaller","Optimal projection shrinks conformal intervals","Hierarchical reconciliation guarantees tighter conformal sets","Projection step provably reduces conformal set size","Elliptical residuals enable optimal conformal projections"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000196,"raw_usage":{"total_tokens":1330,"prompt_tokens":885,"completion_tokens":445,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":501,"completion_tokens_details":{"reasoning_tokens":375}},"tokens_in":501,"tokens_out":445,"duration_ms":5884,"temperature":1.0,"reasoning_tokens":375,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:22:08.558203+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate i.i.d. residuals from a distinctly non-elliptical distribution—for instance, a centered log-normal or a mixture of two Gaussians with different scales—and run Algorithm 4 with the covariance-based projection P_{$Σ^{{-1}}$} against the plain component-wise SCP; if for some component the expected squared interval length is not smaller, or the empirical ratio of lengths departs from the predicted $\\sqrt$((P Σ P^T)_{i,i}/Σ_{i,i}) scaling, the elliptical assumption behind Theorem 4 is violated.","supporting_citations":[{"cited_title":"Signed-error conformal regression","cited_arxiv_id":null,"evidence_quote":"Introduces signed-error conformal regression, the scores whose translation-invariance makes the trace reduction work."},{"cited_title":"(2021) relying onPW–projections","cited_arxiv_id":null,"evidence_quote":"Geometric view of reconciliation and the W-norm projection that inspire the trace-reduction lemma (Lemma 5)."}],"review_version":1}