Quantifying uncertainty in spikes estimated from calcium imaging data
Pith reviewed 2026-05-24 12:59 UTC · model grok-4.3
The pith
A selective inference algorithm produces finite-sample p-values and confidence intervals with correct coverage for spikes estimated from calcium imaging data.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
We describe an efficient algorithm to compute finite-sample p-values that control selective Type I error, and confidence intervals with correct selective coverage, for spikes estimated using a recent proposal from the literature.
What carries the argument
A selective inference procedure that conditions on the data-dependent event of estimating a spike at a given timepoint, applied to the exponential-decay calcium model.
If this is right
- Finite-sample p-values control selective Type I error when testing the null of no spike at an estimated time.
- Confidence intervals achieve correct selective coverage for the calcium jump size at estimated spike times.
- The algorithm runs efficiently enough to apply to real datasets such as those in the spikefinder challenge.
- The approach directly corrects the invalidity of classical tests that ignore the selection step.
Where Pith is reading between the lines
- The same conditioning strategy could be adapted to other event-detection settings where a point process is first estimated from noisy continuous observations.
- If the exponential-decay assumption holds only approximately, one could examine how robust the coverage remains under small model perturbations.
- The method highlights a general template for post-selection inference that may apply to spike sorting pipelines beyond calcium imaging.
Load-bearing premise
The calcium imaging observations are generated exactly by the stated model in which calcium decays exponentially between spikes and jumps instantaneously at each spike.
What would settle it
Generating calcium traces from a model with gradual spike rise times or non-exponential decay and checking whether the reported selective p-values and intervals retain their nominal coverage.
Figures
read the original abstract
In recent years, a number of methods have been proposed to estimate the times at which a neuron spikes on the basis of calcium imaging data. However, quantifying the uncertainty associated with these estimated spikes remains an open problem. We consider a simple and well-studied model for calcium imaging data, which states that calcium decays exponentially in the absence of a spike, and instantaneously increases when a spike occurs. We wish to test the null hypothesis that the neuron did not spike -- i.e., that there was no increase in calcium -- at a particular timepoint at which a spike was estimated. In this setting, classical hypothesis tests lead to inflated Type I error, because the spike was estimated on the same data used for testing. To overcome this problem, we propose a selective inference approach. We describe an efficient algorithm to compute finite-sample p-values that control selective Type I error, and confidence intervals with correct selective coverage, for spikes estimated using a recent proposal from the literature. We apply our proposal in simulation and on calcium imaging data from the spikefinder challenge.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a selective inference procedure for testing the null hypothesis of no spike at times estimated from calcium imaging data under a model with exponential calcium decay between spikes and instantaneous jumps at spikes. It claims to provide an efficient algorithm that computes finite-sample p-values controlling selective Type I error and confidence intervals with exact selective coverage, with demonstrations on simulations and spikefinder challenge data.
Significance. If the algorithm and its selective error control are correctly derived, the work would fill an important gap by enabling rigorous, finite-sample uncertainty quantification for spike estimation in a well-studied generative model. The emphasis on exact selective coverage rather than asymptotic approximations is a notable strength for this application area.
major comments (1)
- [Model description and selective inference derivation] The selective Type I error control and coverage guarantees are derived under the exact generative model (exponential decay, instantaneous jumps, and the noise distribution used for the conditional law). This assumption is load-bearing for the central claim, as any mismatch (e.g., finite rise times, correlated noise, or baseline drift) alters the conditional distribution given the selection event and invalidates the reported p-values and intervals. The manuscript should include a dedicated discussion or simulation study of robustness to such misspecification.
minor comments (2)
- [Algorithm description] Clarify the precise definition of the selection event induced by the spike estimator and how the conditional distribution is computed in the algorithm; a short pseudocode or complexity statement would aid reproducibility.
- [Abstract] The abstract states that the method 'controls selective Type I error' but provides no equation or theorem reference; adding a pointer to the main result (e.g., Theorem X) would strengthen the summary.
Simulated Author's Rebuttal
We thank the referee for their positive evaluation and recommendation of minor revision. We address the single major comment below.
read point-by-point responses
-
Referee: [Model description and selective inference derivation] The selective Type I error control and coverage guarantees are derived under the exact generative model (exponential decay, instantaneous jumps, and the noise distribution used for the conditional law). This assumption is load-bearing for the central claim, as any mismatch (e.g., finite rise times, correlated noise, or baseline drift) alters the conditional distribution given the selection event and invalidates the reported p-values and intervals. The manuscript should include a dedicated discussion or simulation study of robustness to such misspecification.
Authors: We agree that the exact finite-sample selective Type I error control and coverage properties are derived under the assumed generative model and that deviations such as finite rise times, temporally correlated noise, or baseline drift would alter the conditional distribution given the selection event, thereby invalidating the exact guarantees. In the revised manuscript we will add a dedicated subsection to the Discussion that (i) restates the modeling assumptions required for the conditional law, (ii) discusses the plausibility of these assumptions for typical calcium imaging data, and (iii) qualitatively describes how common forms of misspecification could affect the reported p-values and intervals. We will also note that the selective-inference framework can be extended to richer generative models provided the relevant conditional distribution can be characterized. Because a comprehensive simulation study of robustness under multiple misspecification regimes would substantially expand the scope of the work, we will not include such simulations; the added discussion will nevertheless give readers the necessary context for interpreting the results. revision: partial
Circularity Check
No circularity; selective p-values and CIs derived from conditional distribution under explicit generative model
full rationale
The paper applies the established selective inference framework to a fixed spike estimator under an explicit generative model (exponential decay + instantaneous jumps). The algorithm computes the exact conditional law given the selection event defined by the estimator; the resulting p-values and intervals therefore control selective error by the mathematics of conditioning, not by any data-dependent fitting or renaming. No equations reduce a reported quantity to its own inputs by construction, no self-citation chain is load-bearing for the central claim, and the method is externally falsifiable by simulation under the stated model. This matches the normal non-circular case (score 0-2).
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Calcium concentration decays exponentially in the absence of a spike and increases instantaneously when a spike occurs.
Lean theorems connected to this paper
-
IndisputableMonolith/Cost/FunctionalEquation.leanwashburn_uniqueness_aczel unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
We consider a simple and well-studied model for calcium imaging data, which states that calcium decays exponentially in the absence of a spike, and instantaneously increases when a spike occurs... p = P(ϕ0 ⩾ ν⊤y | ˆτj(y)∈M(y′(ϕ0)), ϕ0>0) with ϕ0∼N(0,σ2∥ν∥22)
-
IndisputableMonolith/Foundation/AlphaCoordinateFixation.leanalpha_pin_under_high_calibration unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
Cost(y1:s,α;γ) admits a recursion... piecewise quadratic functions of α
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
Reference graph
Works this paper leans on
-
[1]
There is no changepoint at the (ˆτj − h)th time step. In this case, Cost ( y′ 1:(ˆτj− h+1) (φ) , α ; γ ) equals Cost ( y′ 1:(ˆτj− h)(φ), α/γ ; γ ) + 1 2 ( y′ ˆτj− h+1(φ) − α )2 , where α/γ accounts for the exponential calcium decay
-
[2]
There is a changepoint at the (ˆτj − h)th time step. In this case, Cost ( y′ 1:(ˆτj− h+1)(φ), α ; γ ) Supplementary Materials 7 equals min α′⩾0 { Cost ( y′ 1:(ˆτj− h)(φ), α ′; γ )} + λ + 1 2 ( y′ ˆτj − h+1(φ) − α )2 , where the changepoint incurs a penalty of λ, and there can be an arbitrary change in the calcium from timepoint ˆτj − h to ˆτj − h + 1. The...
-
[3]
Therefore, the first term in Cˆτj − h+1 is a piecewise quadratic function of φ and α according to Observation 2. As for the second term, we note that min α⩾0 { Cost(y′ 1:(ˆτj− h)(φ), α ; γ) } is a piecewise quadratic function of φ according to Observation 3, so its sum with λ + 1 2 (y′ ˆτj− h+1(φ) − α )2 is piecewise quadratic in φ and α . □ Lemma A.7 Supp...
-
[4]
Compute the collection of functions Cˆτj using Proposition 4
-
[5]
Compute the collection of functions ˜Cˆτj +1 using Proposition 7
-
[6]
Compute C(φ) using (3.25)
-
[7]
Compute C′(φ) using (3.26)
-
[8]
Compute S = {φ : C(φ) ⩽ C′(φ)}. 12 Y. T. CHEN AND OTHERS A.8 Proof of Proposition 5 Throughout the proof, we assume that the number of pieces in the piecewise quadratic functions under consideration is a constant that does not depend on h and T . Moreover, we will leverage the toolkit from Maidstone and others (2017); Rigaill (2015); Jewell and others (20...
work page 2017
-
[9]
, ˆτj}, assuming that we have computed Cs− 1
Step 1: We first consider the time to compute Cs for some s ∈ { ˆτj − h + 1, . . . , ˆτj}, assuming that we have computed Cs− 1. (a) We first compute ⋃ f ∈C s− 1{f (α/γ, φ ) + 1 2 (y′ s(φ) − α )2}, which takes O(|Cs− 1|) = O(s − ˆτj + h) operations. (b) We then compute gs(φ) using (3.24): the inner minimization over α ⩾ 0 takes O(1) operations for each f ∈ ...
-
[10]
Step 2: Applying the same logic used in analyzing Step 1 to the secon d step of Algorithm 1, we conclude that computing ˜Cˆτj+1 takes O(h2) operations using Proposition 7. Supplementary Materials 13
-
[11]
According to (3.25), computing C(φ) requires minf ∈C ˆτj {minα⩾0 f (α, φ )} and minf ∈ ˜Cˆτj +1 {minα′⩾0 f (α ′, φ )}. Both terms can be computed in O(|Cˆτj |) = O(h) operations using Observation 1; moreover, the summation will take O(1) operations according to Observation 2. Hence Step 3 takes O(h) operations in total
-
[12]
According to (3.26), C′(φ) = min f ∈C ˆτj , ˜f ∈ ˜Cˆτj +1 { min α⩾0 { f (α, φ ) + ˜f (γα, φ ) }} . (a) Computing the set { f (α, φ ) + ˜f (γα, φ ) ⏐ ⏐ ⏐ f ∈ C ˆτj , ˜f ∈ ˜Cˆτj+1 } takes O(|Cˆτj | · |˜Cˆτj +1|) = O(h2) operations, since each addition takes O(1) operations (Observation 2) and there are |Cˆτj | · |˜Cˆτj +1| such sums. (b) Minimizing over α ⩾...
-
[13]
To carry out Step 5, we first compute min {C(φ), C ′(φ)}, the minimum of two piecewise quadratic functions of φ only, which takes O(1) operations by Observation 1. In O(1) oper- ations, we can obtain S in (2.13) by computing the set of φ such that min {C(φ), C ′(φ)} = C(φ). To summarize, computing S defined in (2.13) using Algorithm 1 takes O(h2) operations...
-
[14]
C1 has only one function C1 = Cost(y′ 1(φ), α ; γ) = 1 2 (8 − α )2
-
[15]
To compute C2, we apply (3.23): C2 = { 1 2 (8 − α/ 0.5)2 + 1 2 (5.6 − 0.4φ − α )2, 1 2 (5.6 − 0.4φ − α )2 + g2(φ) } , where g2(φ) = min α⩾0 Cost(y1, α ; γ) + λ = 0 + λ = 1. Supplementary Materials 15 This completes the calculation Cost ( y′ 1:ˆτj (φ), α ; γ ) = Cost(y′ 1:2(φ), α ; γ) = min f ∈C 2 f (α, φ ). For the reverse direction, we will apply Proposi...
-
[16]
C4 consists of a single function: C4 = Cost(y′ 4(φ), α ; 1/γ ) = 1 2 (3 − α )2
-
[17]
Applying (A.21), we get C3 = min { 1 2 (3 − α/ 2)2 + 1 2 (2.8 + 0.8φ − α )2, min α′⩾0 { 1 2 (3 − α ′/ 2)2 } + λ + 1 2 (2.8 + 0.8φ − α )2 } , which yields Cost ( y′ T :ˆτj +1(φ), α ; 1/γ ) = Cost(y′ 4:3(φ), α ; 1/γ ) = min f ∈C 3 f (α, φ ). According to (3.18), C(φ) = min α⩾0 { Cost(y′ 1:2(φ), α ; γ) } + min α⩾0 { Cost(y′ 4:3(φ), α ; 1/γ ) } + λ, where min...
work page 2020
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.