REVIEW 1 major objections 1 minor 1 cited by
Post-selection inference for quantifying uncertainty in changes in variance
T0 review · 1 major / 1 minor · reviewed 2026-05-24 · grok-4.3
Pith's one-line read Post-selection inference yields valid p-values for detected variance changes.
desk verdict Claims first post-selection p-values for variance changepoints but the abstract alone gives no way to check whether the conditioning actually produces uniform nulls. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Conditioning on the data information used to select which changes to test, which restores uniformity of the resulting p-values under the null.
What would settle it
A Monte Carlo experiment on data with constant variance in which the constructed post-selection p-values deviate systematically from uniformity.
Extended reading notes
Core claim
Currently such methods have been developed for detecting changes in mean only. This paper presents two approaches for constructing post-selection p-values for detecting changes in variance. These vary depending on the method used to detect the changes, but are general in terms of being applicable for a range of change-detection methods and a range of hypotheses that we may wish to test.
Load-bearing premise
The conditioning argument shown for mean changes transfers directly to variance hypotheses while preserving exact or asymptotic uniformity of the p-values under the null of no change.
Editorial extensions
If this is right
- Valid p-values become available for hypotheses about variance changes after detection.
- The constructions apply to multiple existing change-detection algorithms.
- A range of null hypotheses about the size or location of variance shifts can be tested.
Reading between the lines
- The same conditioning logic may extend to other parameters such as autocorrelation or skewness once the selection event is defined.
- In applications like financial volatility monitoring, these p-values could reduce false discovery rates when scanning many candidate change locations.
- Computational cost will depend on how easily the selection event can be characterized for each detection method.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript claims to present two approaches for constructing post-selection p-values for detecting changes in variance. These vary depending on the method used to detect the changes, but are general in terms of being applicable for a range of change-detection methods and a range of hypotheses that we may wish to test. By conditioning on the selection event, the resulting p-values are asserted to be uniform under the null of no change, extending prior post-selection inference work from mean changes to variance.
Significance. If the conditioning arguments can be shown to deliver uniform p-values, the work would address a clear gap by enabling valid uncertainty quantification after variance changepoint detection, with potential utility in applications involving second-moment shifts.
major comments (1)
- [Abstract] Abstract: the central claim that the two approaches produce p-values uniform under the null of constant variance rests on the transfer of a conditioning argument previously developed for means; no derivation, explicit construction, or simulation result is supplied to confirm that the relevant quadratic forms or ratios of sums of squares remain uniformly distributed after conditioning on the selection event, which is load-bearing for the contribution.
minor comments (1)
- [Abstract] Abstract: the two approaches are referenced but neither named nor briefly characterized, which limits immediate assessment of their scope and differences.
Simulated Author's Rebuttal
We thank the referee for their review. We address the concern about supporting the uniformity claim for the post-selection p-values.
read point-by-point responses
-
Referee: [Abstract] Abstract: the central claim that the two approaches produce p-values uniform under the null of constant variance rests on the transfer of a conditioning argument previously developed for means; no derivation, explicit construction, or simulation result is supplied to confirm that the relevant quadratic forms or ratios of sums of squares remain uniformly distributed after conditioning on the selection event, which is load-bearing for the contribution.
Authors: We agree the abstract, being a summary, does not contain the derivation. The manuscript develops the two approaches by expressing the variance selection events as functions of quadratic forms (ratios of sums of squares) and shows that the conditioning argument carries over because, under the global null of constant variance, these forms follow scaled chi-squared distributions whose conditional distribution after selection remains uniform. To make this explicit and address the load-bearing step, we will revise by adding a dedicated subsection with the full conditioning argument plus a small simulation study confirming uniformity of the resulting p-values under the null. revision: yes
Circularity Check
No circularity: abstract describes extension of prior post-selection framework without exhibited self-referential steps
full rationale
The provided abstract presents two approaches for post-selection p-values applicable to variance changes, noting that such methods already exist for mean changes. No equations, fitted parameters, or derivation chain appear in the text, precluding any identification of self-definitional reductions, fitted inputs renamed as predictions, or load-bearing self-citations. The work is framed as a generalization of an existing conditioning argument rather than a closed loop that equates outputs to inputs by construction.
Assumptions & free parameters
assumptions (1)
- domain assumption Conditioning on the changepoint selection event produces valid p-values for variance hypotheses
Cite this review
Pith. "Pith review of Post-selection inference for quantifying uncertainty in changes in variance." pith.science (2026). https://pith.science/paper/2405.15670
@misc{pith2026240515670,
author = {Pith},
title = {Pith review of: Post-selection inference for quantifying uncertainty in changes in variance},
year = {2026},
howpublished = {\url{https://pith.science/paper/2405.15670}},
note = {Machine review of arXiv:2405.15670}
}
read the original abstract
Quantifying uncertainty in detected changepoints is an important problem. However it is challenging as the naive approach would use the data twice, first to detect the changes, and then to test them. This will bias the test, and can lead to anti-conservative p-values. One approach to avoid this is to use ideas from post-selection inference, which conditions on the information in the data used to choose which changes to test. As a result this produces valid p-values; that is, p-values that have a uniform distribution if there is no change. Currently such methods have been developed for detecting changes in mean only. This paper presents two approaches for constructing post-selection p-values for detecting changes in variance. These vary depending on the method use to detect the changes, but are general in terms of being applicable for a range of change-detection methods and a range of hypotheses that we may wish to test.
Forward citations
Cited by 1 Pith paper
-
Post Hoc Inference for Component Attribution in Multivariate Change-Point Detection
After a multivariate change-point is detected, a grid-based or sample-splitting two-sample test determines whether a pre-specified block of coordinates changed, with Type I error bounded by α0+α1.
Reviewed May 24, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.