REVIEW 3 major objections 6 minor 12 references
e-GAI: e-value-based Generalized $\alpha$-Investing for Online False Discovery Rate Control
T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper claims that online FDR can be controlled under arbitrary dependence by spending a fixed α-budget through e-value tests, using an oracle FDP estimate that always overestimates the truth.
desk verdict A useful oracle FDP theorem wrapped around an algorithm whose allocation fraction escapes (0,1) under the paper's own recommended settings, so the advertised provable FDR guarantee does not hold as stated. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the oracle e-value FDP estimate $\mathrm{FDP}_e^*(t) = \sum_{j \in H_0(t)} \alpha_j/(R_{j-1}+1)$, whose denominator $R_{j-1}+1$ serves as a predictable lower bound for the final rejection count $R_t \vee 1$; because the bound sits in the denominator, the estimate overestimates the true FDP. The companion mechanism is the RAI update $\omega_{t+1} = \omega_t + \omega_1 \varphi_{t-R_t}(1-\delta_t) - \omega_1 \psi_{R_t} \delta_t$, which sets the fraction of remaining wealth spent on the next test. Together they turn online FDR control into a budget-conservation inequality, with both e-LORD and e-SAFFRON defined by the form of the upper bound they keep below $\alpha$.
What would settle it
Check whether the procedure maintains $\omega_t<1$ under the paper's suggested constants: with $\varphi=\psi=0.5$, $\omega_1=0.005$, and an all-null sequence, the update (9) gives $\omega_t \approx \omega_1(1+t/2)$, which passes 1 near $t=400$; in that regime equation (8) spends more than the remaining wealth, and a simulation of the FDR should exceed $\alpha$.
Extended reading notes
Core claim
The core claim is Theorem 3.1: for conditionally valid e-values, $E[\mathrm{FDP}_e^*(t)] \le \alpha$ implies $\mathrm{FDR}(t) \le \alpha$, where $\mathrm{FDP}_e^*(t) = \sum_{j \in H_0(t)} \alpha_j / (R_{j-1}+1)$. The denominator $R_{j-1}+1$ is a predictable lower bound on the eventual number of rejections, which makes the oracle estimate conservative under arbitrary dependence. e-LORD and e-SAFFRON are built to keep this estimate below $\alpha$: at each time they spend a fraction $\omega_t$ of the remaining $\alpha$-wealth multiplied by $R_{t-1}+1$, with $\omega_t$ raised after acceptances and lowered after rejections, a "risk aversion investing" rule. The paper proves the same FDP estimator works for conditionally super-uniform p-values and for decaying-memory FDR, and shows e-LOND is a special case of e-LORD. The advertised consequence is provable FDR control at all stopping times with higher power than e-LOND.
Load-bearing premise
The FDR guarantee assumes the allocation fraction $\omega_t$ stays strictly between 0 and 1 at every time step, so that the FDP overestimate never spends more than the remaining $\alpha$-wealth.
Editorial extensions
If this is right
- e-LORD and e-SAFFRON provide valid online FDR control at every time under arbitrary dependence among e-values, needing only the conditional validity condition $E[e_t \mid \mathcal{F}_{t-1}] \le 1$ for nulls.
- Because testing levels adapt to the observed history, both procedures can have substantially higher power than e-LOND, whose levels follow a fixed decay sequence.
- The same e-GAI logic applies to conditionally super-uniform p-values, giving pL-RAI and pS-RAI procedures that do not require independence or PRDS.
- The mem-e-LORD and mem-e-SAFFRON variants control decaying-memory FDR and postpone $\alpha$-death, so the framework remains usable on very long streams.
- Since e-LOND is recovered as a special case, the framework generalizes the existing e-value method rather than replacing it.
Reading between the lines
- The paper's own suggested constant values $\varphi=\psi=0.5$ can violate the $\omega_t\in(0,1)$ premise on a long null stretch; a safe implementation would need smaller $\varphi,\psi$ or an explicit clipping rule.
- Because the proof only uses $E[e_t\mid\mathcal{F}_{t-1}]\le 1$, the same allocation layer can wrap any predictable e-value stream, making the framework a plug-in for likelihood-ratio, betting, or calibrated e-values.
- With known dependence, the denominator $R_{j-1}+1$ in the oracle estimate could be tightened toward $R_t$, which the paper identifies as an open direction and which would directly improve power.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes e-GAI, an e-value-based generalization of the generalized α-investing framework for online FDR control. The central idea is to use a new oracle FDP estimator, FDP_e^*(t)=∑_{j∈H0(t)} α_j/(R_{j-1}+1), and to enforce an overestimate of it to remain below α. Two algorithms, e-LORD and e-SAFFRON, are introduced, together with decaying-memory versions (mem-e-LORD, mem-e-SAFFRON) and an extension to conditionally super-uniform p-values (pL-RAI, pS-RAI). The paper claims provable online FDR control under arbitrary dependence for e-values satisfying the conditional validity condition, with higher power than the existing e-LOND procedure. Theoretical results are supported by simulation and real-data experiments (NYC taxi anomaly detection and financial bubble dating).
Significance. If the algorithmic guarantees were valid, the paper would offer a useful advance: it proposes a class of e-value-based online procedures that adaptively allocate testing levels while maintaining FDR control under arbitrary dependence, going beyond the fixed-sequence e-LOND. The oracle estimator and Theorem 3.1 are mathematically correct and provide a clean proof template; the paper also ships reproducible code. However, the central algorithmic guarantee is undermined by an incorrect assertion about the allocation parameter ω_t staying in (0,1). With the paper's own recommended parameters, ω_t can exceed 1 on a null stream, causing the remaining α-wealth to become negative and the procedure to reject every hypothesis. This invalidates the claimed provable FDR control for the proposed algorithms and makes the empirical power gains not backed by the stated theory.
major comments (3)
- [§3.2, Eq. (9) and Remark 3.3] The assertion in Remark 3.3 that any ω1∈(0,0.5), φ∈[0,0.5], ψ∈[0,0.5] keeps ω_t∈(0,1) for all t is false. On a fully null stream with δ_t=0 and constant φ=0.5, equation (9) gives ω_{t+1}=ω1(1+0.5t). For ω1=0.005 (the value used in all simulations), ω_t exceeds 1 for t≥401. Once ω_t>1, the remaining wealth S_t defined by the bracketed term in (8) evolves as S_t=(1−ω_t)S_{t−1} and changes sign, so α_t in (8) becomes negative. The decision rule δ_t=1{e_t≥1/α_t} then rejects every hypothesis, yielding FDP=1 on an all-null stream. This violates the condition [FDP_LORD_e(t)≤α required by Proposition 3.2 and directly contradicts the paper's central claim of provable online FDR control for e-LORD and e-SAFFRON under the recommended parameter range.
- [§5.1 and Appendix C.1, parameter choices] The simulation section uses ω1=0.005, φ=ψ=0.5, which lies inside the range asserted in Remark 3.3 but, as shown above, does not keep ω_t in (0,1) even for the simulation horizon T=500. Thus the empirical FDR control displayed in Figures 1, 4–9 is not supported by the theoretical guarantee. Moreover, the parameter recommendation is internally inconsistent: the main text (§5.1) sets φ=ψ=0.5, while the Appendix C.1 text states 'φ=ψ=0.05' and the captions of Figures 4–7 refer to φ=0.5. The authors need to resolve this discrepancy and, more importantly, either modify the RAI update so that ω_t is provably bounded in (0,1) or provide a separate argument that FDR control holds when ω_t exits this interval.
- [§3.2, Remark 3.3, last sentence] The statement 'Note that the choice of ωt satisfying the above conditions does not affect the guarantee of the FDR control' is only true if the conditions actually ensure ω_t∈(0,1). Since the asserted conditions do not have this property, the statement is misleading. The paper does not provide a proof of the claimed range, and no such proof can exist for constant φ>0, ψ>0 with an infinite horizon because ω_t grows linearly with the number of consecutive acceptances. The authors must either restrict φ and ψ to sequences whose cumulative sums are bounded, cap ω_t at 1 (with an analysis of the resulting procedure), or otherwise re-derive the FDR guarantee under a corrected condition.
minor comments (6)
- [§3.2, Eq. (9)] The notation φ_{t−R_t} and ψ_{R_t} suggests that φ and ψ are indexed sequences, but the text and simulations treat them as scalar parameters (e.g., 'φ>0, ψ>0 are user-defined parameters'). Please clarify whether they are constants or sequences, and align the notation accordingly.
- [Algorithm 1 and Remark 3.3] Algorithm 1 specifies the input as ω1∈(0,1), while Remark 3.3 restricts ω1 to (0,0.5). This inconsistency should be resolved.
- [Table 2] The quantity labeled '[FDP' is defined as the proportion of rejected points that fall inside known anomalous regions; this is a coverage measure, not a false discovery proportion. Using the FDP acronym for this quantity is misleading and should be replaced with a different term, such as 'anomaly coverage rate'.
- [Appendix C.1 vs. Section 5.1] The recommended parameter setting is given as φ=ψ=0.5 in the main text and in the captions of Figures 4–7, but the body of Appendix C.1 states φ=ψ=0.05. Please correct this inconsistency and ensure that the reported results correspond to the stated parameter values.
- [Appendix B.6] The claim that e-LOND is a special case of e-LORD when γ_t=ω_t∏_{j=1}^{t−1}(1−ω_j) is plausible, but the derivation in (18) is abbreviated. A short direct verification of the equality of the testing levels would improve readability.
- [Proof of Theorem 4.1] In the displayed inequality (i), the expression dRd_{j-1} should be d R^d_{j-1} to avoid confusion with the product of d and the rejection count; adding a space or parentheses would improve clarity.
Circularity Check
No significant circularity: the FDR theorems are self-contained; the flawed Remark 3.3 range is a correctness bug, not a circular derivation.
full rationale
The central FDR guarantee is derived, not assumed. Proposition 3.2 follows from Theorem 3.1 because the estimator in (7) termwise dominates the oracle estimator in (6), and Theorem 3.1 itself is proved from the valid e-value inequality 1{e_j >= 1/alpha_j} <= alpha_j e_j and iterated expectations. The update rule (8) is a construction that keeps the cumulative FDP bound below alpha whenever omega_t is in (0,1); no fitted parameter is hidden inside the theorem, and the e-SAFFRON bound in Proposition 3.4 uses the valid property P(e_j < 1/lambda_j | F_{j-1}) >= 1 - lambda_j. The relationship to e-LOND is an explicit algebraic equivalence (Appendix B.6), not a renamed prediction, and the comparisons to LORD++/SAFFRON/e-LOND are external benchmarks rather than self-citations. The empirical power comparison does tune (omega_1, phi, psi, lambda) on the same simulation family used for the power plots, which weakens the claim of superior power, but this tuning does not enter or disguise the FDR derivation. The substantive internal problem flagged in the paper is not circularity: Remark 3.3 asserts that any omega_1 in (0,0.5), phi in [0,0.5], psi in [0,0.5] keeps omega_t in (0,1), but on an all-null stream equation (9) with constant phi gives omega_{t+1} = omega_t + omega_1 phi, so with the paper's own recommended omega_1 = 0.005 and phi = 0.5, omega_t exceeds 1 near t = 400; thereafter the remaining wealth in (8) can go negative and the FDP bound breaks. This is an unproven and apparently false range condition, hence a correctness risk to the 'provable' guarantee, but the derivation chain itself does not reduce to its inputs.
Assumptions & free parameters
free parameters (4)
- ω1 =
0.005 (experiments), O(1/T) recommended
- φ =
0.5 (experiments)
- ψ =
0.5 (experiments)
- λ =
0.1 (e-SAFFRON default)
assumptions (3)
- domain assumption Conditional validity of e-values: E[e_t | F_{t-1}] ≤ 1 if θ_t=0.
- domain assumption Conditionally super-uniform p-values: P(p_t ≤ u | F_{t-1}) ≤ u under the null.
- standard math Standard probability calculus and iterated expectations (law of total expectation).
Cite this review
Pith. "Pith review of e-GAI: e-value-based Generalized $\alpha$-Investing for Online False Discovery Rate Control." pith.science (2026). https://pith.science/paper/ETNETRBW
@misc{pith2026250601452,
author = {Pith},
title = {Pith review of: e-GAI: e-value-based Generalized $\alpha$-Investing for Online False Discovery Rate Control},
year = {2026},
howpublished = {\url{https://pith.science/paper/ETNETRBW}},
note = {Machine review of arXiv:2506.01452}
}
abstract
Online multiple hypothesis testing has attracted a lot of attention in many applications, e.g., anomaly status detection and stock market price monitoring. The state-of-the-art generalized $\alpha$-investing (GAI) algorithms can control online false discovery rate (FDR) on p-values only under specific dependence structures, a situation that rarely occurs in practice. The e-LOND algorithm (Xu & Ramdas, 2024) utilizes e-values to achieve online FDR control under arbitrary dependence but suffers from a significant loss in power as testing levels are derived from pre-specified descent sequences. To address these limitations, we propose a novel framework on valid e-values named e-GAI. The proposed e-GAI can ensure provable online FDR control under more general dependency conditions while improving the power by dynamically allocating the testing levels. These testing levels are updated not only by relying on both the number of previous rejections and the prior costs, but also, differing from the GAI framework, by assigning less $\alpha$-wealth for each rejection from a risk aversion perspective. Within the e-GAI framework, we introduce two new online FDR procedures, e-LORD and e-SAFFRON, and provide strategies for the long-term performance to address the issue of $\alpha$-death, a common phenomenon within the GAI framework. Furthermore, we demonstrate that e-GAI can be generalized to conditionally super-uniform p-values. Both simulated and real data experiments demonstrate the advantages of both e-LORD and e-SAFFRON in FDR control and power.
Figures
Figures from the paper (8 more)
Reference graph
Works this paper leans on
-
[1]
Thus, we finish the whole proof
for every j ∈ {j ≤ t : δj = 1} by definition, the inequality (ii) follows after taking iterated expectations by conditioning on Fj−1 and then applying the conditionally super-uniform property of p-values, and inequality (iii) holds by condition. Thus, we finish the whole proof. B. Deferred Discussions B.1. Definition of PRDS Fisher (2024) introduced an on...
work page 2024
-
[4]
Average runtime of different algorithms under an AR(1) model over 100 repetitions. e-LORD e-SAFFRON e-LOND LORD++ SAFFRON SupLORD Runtime (×10−4s) 9.7 18.8 18.0 9.9 8.1 72.1 B.4. Extension of the e-GAI Framework to p-values In this section, we adapt the e-GAI framework to p-values and analyze the corresponding algorithms for the long-term performance foll...
work page 2018
-
[5]
and (Ramdas et al., 2018, Lemma
work page 2018
-
[7]
can be regarded as the e-SAFFRON algorithm as it satisfies [FDP SAFFRON e-ind (t) ≤ α. However, the conclusion on the equivalence between e-SAFFRON and SAFFRON differs from that between e-LORD and LORD++. When the independent e-values {et} are available at each time, applying SAFFRON to valid p-values {pt = 1/et} is equivalent to applying e-SAFFRON to {et...
work page 2018
-
[8]
The situation changes when the independent p-values are available
is a user-chosen parameter and Ci+(t) = Pt−1 j=τi+1 1 {pj ≤ λ}. The situation changes when the independent p-values are available. In addition to the conditions of independence, if the p-values further satisfy the conditionally uniformly distributed in (1), then there are no valid e-values in (2) such that applying SAFFRON to {pt} is equivalent to applyin...
work page 2018
-
[9]
p-value-based e-values. In case the p-values associated with each hypothesis are available, it is possible to convert them to e-values using a ‘p-to-e calibrator’ (Shafer et al., 2011), albeit with possible power loss (V ovk & Wang, 2021). A ‘p-to-e calibrator’ is a decreasing function f c : [0, 1] 7→ [0, ∞], such that R 1 0 f c(s)ds =
work page 2011
-
[10]
Note that an e-value et can be naturally transformed into a p-value pt by pt = min{1/et, 1}
(Shafer, 2021; V ovk & Wang, 2021). Note that an e-value et can be naturally transformed into a p-value pt by pt = min{1/et, 1}. This is because P(pt ≤ u | Ft−1) ≤ P(et ≥ 1/u | Ft−1) ≤ uE[et | Ft−1] ≤ u (23) for all u ∈ (0,
work page 2021
-
[11]
All procedures maintain FDR control
Empirical mem-FDR and mem-Power with standard error versus proportion of alternative hypotheses π1 for mem-e-LORD, mem-e-SAFFRON, mem-LORD++, mem-pL-RAI and mem-pS-RAI with µc = 3, 4, at T = 10000, 20000. All procedures maintain FDR control. Our e-GAI and RAI methods exhibit superior mem-power than mem-LORD++ when µc or π1 is small. at T ∈ {10000, 20000} ...
work page 2017
Show all 12 references
-
[12]
Proportion of points rejected out of anomalous regions. Method pL-RAI pS-RAI LORD++ SAFFRON SupLORD [FDP 0.197 0.195 0.261 0.361 0.217 Num Discovery 201 259 257 595 406 23 e-GAI: e-value-based Generalized α-Investing for Online False Discovery Rate Control 0 10000 20000 30000 ...
2014
-
[2017]
Therefore, LORD++ can be included within the e-GAI framework
can be regarded as the e-LORD algorithm, as it satisfies the condition [FDP LORD e-ind (t) ≤ α. Therefore, LORD++ can be included within the e-GAI framework. Furthermore, we can show the equivalence between e-LORD and LORD++. On the one hand, when the independent e-values {et}...
2017
-
[2018]
and Ahmad, S
Lavin, A. and Ahmad, S. Evaluating real-time anomaly detection algorithms–the numenta anomaly benchmark. In 2015 IEEE 14th international conference on machine learning and applications (ICMLA) , pp. 38–44. IEEE,
2015
-
[2024]
, pt) ∈ I | pj = u, Fj−1) is non-decreasing in u
The p-values are conditional PRDS if for each time t, any j ≤ t satisfying j ∈ H0(t), and increasing set I ⊂ Rt, the probability P ((p1, . . . , pt) ∈ I | pj = u, Fj−1) is non-decreasing in u. Wang & Ramdas (2022) discussed the PRDS condition on the studies of e-values in the ...
2022
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.