Pith. sign in

REVIEW 1 major objections 3 minor 1 cited by

Compound Selection Decisions: An Almost SURE Approach

T0 review · 1 major / 3 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper introduces ASSURE, an almost unbiased estimator of compound selection welfare, and proves that optimizing it over a class of threshold rules achieves regret O((log n)/√n) in general and O((log n)^5/n) under a margin condition—mat

desk verdict A genuine new estimator with a real minimax regret theory; the Gaussian known-sigma assumption is load-bearing and the empirical sections are thin, but the core is sound. read the letter →

arxiv 2511.11862 v2 pith:JIDZGZY3 submitted 2025-11-14 econ.EM math.STstat.MEstat.TH

Jiafeng Chen , Lihua Lei , Timothy Sudijono , Liyang Sun , Tian Xie This is my paper · ORCID
classification econ.EMmath.STstat.MEstat.TH MSC 62C1262F0762G20
keywords compoundselectionempiricalBayesSUREwelfaremaximizationsinckernelGaussiansequencemodelregretboundsthresholdrules
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

In a Gaussian sequence model where each unit has an unknown parameter and a known noise level, the decision maker wants to select a subset of units to maximize average payoff, the sum of (mu_i minus K_i). The paper introduces ASSURE, an almost unbiased estimator for the expected welfare of any threshold rule, and proves that maximizing this estimator over a user-chosen class of rules yields regret that vanishes at a minimax-optimal rate: O(log n / sqrt(n)) in general, and O((log n)^5 / n) under a margin condition. Because the estimator never models the distribution of the parameters, ASSURE offers a frequentist alternative to empirical Bayes selection that borrows strength across units while remaining robust to prior misspecification. The method is applied to selecting Census tracts for economic opportunity, choosing A/B tests to implement, and identifying discriminating firms.

What carries the argument

The ASSURE estimator (Eq. 2.5–2.6) is w_h(Y_i; Z_i, beta) = (Y_i - K_i) Csinc((Y_i - delta(Z_i;beta))/(sigma_i h)) - (sigma_i/h) sinc((Y_i - delta(Z_i;beta))/(sigma_i h)) with h = 1/sqrt(2 log n). The sinc kernel acts as a low-pass filter in Fourier space: convolving the Gaussian density with it leaves the target probability essentially unchanged, which drives the exponential bias decay in Proposition 1. Stein's identity supplies the derivative correction that makes the estimator almost unbiased, and the Gaussian Poincare inequality plus empirical process theory for independent but non-identically distributed data yield uniform control of the estimation error over the decision class.

What would settle it

Simulate n large with heavy-tailed errors, e.g., Y_i = mu_i + sigma_i t_5, holding sigma_i known. If ASSURE's pointwise bias exceeds C/(n log n) or its regret does not vanish at the predicted rate relative to the infeasible optimum, the Gaussian anchor of Proposition 1 is contradicted.

Watch

Extended reading notes

Core claim

The central claim is that compound selection decisions can be tuned nearly as well as if the true means were known. Proposition 1 constructs an estimator of the compound welfare W(beta) = (1/n) sum (mu_i - K_i) Phi((mu_i - delta_i)/sigma_i) from a sinc-kernel smoothing of the Heaviside function, with bandwidth h = 1/sqrt(2 log n); its bias decays like O(|mu-K| h^2 exp(-1/(2h^2))) = O(|mu-K|/(n log n)). Theorems 1 and 3 bound the regret—the gap between the welfare of the ASSURE-tuned rule and the best infeasible rule in a VC-subgraph class—by O(log n / sqrt(n)) and, under a curvature and margin assumption, O((log n)^5 / n). Theorems 2 and 4 give matching Omega(1/sqrt(n)) and Omega(1/n) lower

Load-bearing premise

The guarantees require that each estimate is exactly Gaussian with known variance; if the estimates are materially heavy-tailed or the standard errors are misspecified, the stated bias and regret bounds do not apply.

Editorial extensions

If this is right

  • ASSURE-tuned decisions are asymptotically no worse than the best infeasible threshold rule in the chosen class.
  • The O(log n / sqrt(n)) regret is minimax optimal up to logs: no decision rule can guarantee better than 1/sqrt(n) in the worst case.
  • Under a margin/curvature condition, regret improves to O((log n)^5 / n), and a matching lower bound shows this is essentially the best possible rate.
  • The welfare estimates can certify status-quo rules: in the A/B testing application, the industry-standard p<0.05 rule is shown to be conservative relative to the ASSURE optimum when costs are zero.
  • The same construction gives an exactly unbiased estimator for Poisson observations, with the same sqrt(n) minimax rate.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's formal guarantees rest on exact Gaussianity; the heavy-tailed simulations are encouraging, but the formal analysis of misspecification is deferred. An extension would be to prove the regret bound under approximate Gaussianity via Stein-type discrepancies.
  • The ensemble construction (Section 4.2) suggests a general recipe: any collection of monotone decision scores can be combined and tuned with ASSURE under stability conditions—this could make ASSURE a drop-in selection-aware tuner for off-the-shelf empirical Bayes and machine learning models.
  • The low-bias kernel idea likely transfers to other functionals of a Gaussian mean that are not unbiasedly estimable (e.g., probabilities of non-monotone events), offering a route for decision problems beyond threshold rules.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 3 minor

Summary. The paper studies compound selection decisions in a Gaussian sequence model Y_i ~ N(μ_i, σ_i^2) with known σ_i. A decision maker selects a subset of units to maximize average utility (1/n) Σ a_i (μ_i − K_i). The paper introduces ASSURE, an almost unbiased estimator of the welfare W(β) based on sinc-kernel smoothing (Definition 1), and proposes choosing β by maximizing this estimated welfare over a pre-specified VC-subgraph class of threshold rules. Theorem 1 establishes regret O(m_1/(n log n) + M log n/√n), and Theorem 2 gives a matching Ω(1/√n) lower bound. Under a curvature/margin assumption (Assumption 1), Theorem 3 gives a fast rate O((log n)^5/n) and Theorem 4 gives a matching Ω(1/n) lower bound. The paper also provides a Poisson-data extension, a leave-one-out extension to nonseparable decision rules, calibrated simulations, and three empirical applications. Proofs are in an extensive appendix, including i.n.i.d. empirical-process bounds and Gaussian Poincaré variance bounds.

Significance. If the bounds are correct, this is a valuable contribution to compound decision theory and empirical welfare maximization. The estimator is new, and the analysis is nonstandard: the exponential bias control via Fourier truncation (Proposition 1), the in-sample/infeasible-oracle regret decomposition, and the matching lower bounds are genuine contributions. The paper is careful about the distinction between in-sample utility and expected welfare, and the leave-one-out stability argument for fast rates is substantial. The Poisson and ensembling extensions broaden the applicability. The formal theory is strongest in the exact Gaussian, known-σ case; robustness outside that case is demonstrated only by simulation and explicitly deferred, which is the main scope limitation.

major comments (1)
  1. [§2.2, §2.3, §5, §6] The formal regret guarantees (Theorems 1 and 3) assume exact Gaussianity and known σ_i. The paper explicitly defers non-Gaussian analysis (Section 2.2: 'we leave a formal analysis to future work') and the Le Cam–Hajek justification (Section 2.3). Yet the abstract and empirical sections present ASSURE as broadly applicable, and the applications use CLT-based normality plus estimated standard errors. A t_10 simulation (Figure 11) does not transfer the theorems to heavy tails or estimated σ. Please qualify the headline claims in the abstract and introduction, or add a formal robustness result under an explicit perturbation model.
minor comments (3)
  1. [Theorem 1 and proof] The bias term in Theorem 1 is displayed as 'm1 nlogn'; Proposition 1 implies m_1/(n log n). The proof has a related inconsistent string ('m1 n2 logn'). Please correct the notation.
  2. [Theorem 4 proof, Appendix B.6] The displayed bound c*_{G0} ≥ (1/2) ln(1 + 8h/√n) ≥ 2/√n is not implied by the choice h=1/4; the correct scale is O(1/√n), which is sufficient for the Ω(1/n) conclusion. Also, the sentence that min(p_G0, p_G1) is 'bounded above' appears to need 'bounded below' for the displayed inequality to hold.
  3. [Theorem 1, definition of M] The definition of M includes '(E∥D∥_n + (E∥D∥_{4,n}^2)^{1/2})'. Please clarify the second term, presumably (E[∥D∥_{4,n}^2])^{1/2}, to avoid ambiguity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ASSURE regret bounds are derived from Gaussian Fourier/sinc identities and matching lower bounds; self-citations are benchmark-only.

full rationale

The derivation chain is self-contained. ASSURE is defined by the sinc-kernel weights in Eqs. (2.5)-(2.6); Proposition 1 derives the bias bound from the Gaussian Fourier identity (Lemma A.1) and a Mills-ratio tail bound, and the variance bound from the Gaussian Poincare inequality. The main regret bound (Theorem 1) uses the standard ERM decomposition Regret ≤ 2E sup|W_hat−W| + 2E sup|u−W| plus empirical-process bounds over a VC class; no fitted parameter or target quantity is reused as an input. Theorems 2 and 4 provide independent minimax lower bounds via two-point/Le Cam arguments, so the rates are not artifacts of ASSURE's construction. Self-citations to Chen (2025) concern the CLOSE-GAUSS decision class and simulation benchmarks, not the regret theorems, and are not load-bearing. The Gaussian/known-σ assumption is an explicit modeling assumption with non-Gaussian analysis deferred ('we leave a formal analysis to future work'; 'remains important future work'), which is a scope limitation, not circularity. The cost calibrations in the applications are illustrative choices of the decision primitive K and do not feed back into the regret derivation.

Assumptions & free parameters 3 free parameters · 7 assumptions · 1 invented entities

The central claim rests on (i) the Gaussian-known-sigma model (domain assumption; formal non-Gaussian analysis deferred), (ii) the restriction to a user-specified VC-subgraph class of separable threshold rules, (iii) standard VC/entropy machinery, and (iv), for the fast-rate theorem only, unverifiable curvature/separation conditions on W (Assumption 1). Free parameters are a hand-chosen, rate-optimal bandwidth and application costs calibrated to reproduce benchmark selection counts; the plug-in EB parameters in the comparisons are fitted by the benchmark methods themselves. No physical entities are postulated: the ASSURE estimator is listed as an invented entity with independent falsifiable support (its bias bound and matching regret lower bounds).

free parameters (3)
  • bandwidth h_n = 1/sqrt(2 log n)
    Smoothing parameter of the sinc-kernel ASSURE estimator (Eq. 2.6). Chosen by hand to balance exponential bias h^2 exp(-1/(2h^2)) against variance ~ h^{-4} sigma^2 / n; rate-optimal but not data-adaptive.
  • investigation cost K (discrimination application) = 0.025
    Section 6.3: set so that the nonparametric EB selection 'match[es] with the selection of 23 firms' from Kline et al. (2022); ASSURE is then compared with that benchmark.
  • cost K (Opportunity Atlas application) = 0.361
    Section D.2.1: chosen so that 'a plug-in empirical Bayes model chooses exactly 1/3 of units'; i.e., the 67th percentile of the EB posterior means. Another specification K=0.369 uses the 67th percentile of raw Y_i.
assumptions (7)
  • domain assumption Gaussian sequence model: Y_i | mu_i, sigma_i, X_i ~ N(mu_i, sigma_i^2), mutually independent, sigma_i known
    Section 2 setup; used in Proposition 1 (Fourier identity), the Poincare variance bound, and all theorems; the paper motivates it by CLT and empirically tests misspecification only in simulations.
  • domain assumption Decisions restricted to separable threshold rules delta(Z_i; beta) in a pre-specified class D
    Section 2 ('We also restrict to separable thresholds...'); the optimality guarantee is relative to D; admissibility of monotone thresholds is Lemma E.1 (Karlin-Rubin), but the class restriction is the user's choice.
  • standard math D is a VC subgraph class with index V(D) and envelope D; derivatives form VC classes for the fast-rate theorem
    Theorems 1 and 3 and Section B require uniform entropy bounds from van der Vaart and Wellner (2023); Examples 1-4 give concrete classes, with o-minimality (Wilkie 1996; Onshuus and Quiroz 2015) used in Prop. B.4.
  • ad hoc to paper Assumption 1 (A1)-(A4): boundedness, unimodality of W, curvature W''(beta*) < -kappa, well-separated maximizer
    Section 3.4; introduced solely for the fast-rate Theorem 3; conditions are on the unknown W(beta) and cannot be verified from data; Example 5 (L1)-(L3) gives lower-level sufficient conditions on mu_i.
  • domain assumption Boundedness of mu_i, sigma_i, sigma_i^{-1}, K_i (Corollary 1)
    Section 3.1; needed for the clean O(log n / sqrt(n)) rate; without it the regret depends on moment averages s_2, m_2, nu_4 that must be controlled.
  • standard math Monotone likelihood ratio implies optimal decisions are threshold rules (Karlin-Rubin; formal Lemma E.1)
    Section 1 footnote 3 and Appendix E.1; justifies restriction to threshold rules when comparing to the oracle within the class.
  • standard math No exactly unbiased estimator exists for the welfare functional (Stefanski 1989, Prop. E.1)
    Section 2.1 and Appendix E.1; motivates the 'almost unbiased' construction; the proof uses the Riemann-Lebesgue lemma.
invented entities (1)
  • ASSURE welfare estimator W_hat(beta) = (1/n) sum_i w_h(Y_i; Z_i, beta) independent evidence
    purpose: Nearly unbiased estimate of compound welfare W(beta), enabling welfare-maximizing rule selection within a class D
    A closed-form estimator, not a hidden degree of freedom: it has a proven exponential bias bound (Prop. 1), explicit variance bound (Eq. 2.7), matching minimax regret bounds (Thms. 1-4), and its finite-sample behavior is falsifiable in simulations; no new physical or modeling entity is introduced.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Compound Selection Decisions: An Almost SURE Approach." pith.science (2026). https://pith.science/paper/JIDZGZY3

@misc{pith2026251111862,
  author       = {Pith},
  title        = {Pith review of: Compound Selection Decisions: An Almost SURE Approach},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JIDZGZY3}},
  note         = {Machine review of arXiv:2511.11862}
}
abstract

This paper proposes methods for producing compound selection decisions in a Gaussian sequence model. Given unknown, fixed parameters $\mu_ {1:n}$ and known $\sigma_{1:n}$ with observations $Y_i \sim \textsf{N}(\mu_i, \sigma_i^2)$, the decision maker would like to select a subset of indices $S$ so as to maximize utility $\frac{1}{n}\sum_{i\in S} (\mu_i - K_i)$, for known costs $K_i$. Inspired by Stein's unbiased risk estimate (SURE), we introduce an almost unbiased estimator, called ASSURE, for the expected utility of a proposed decision rule. ASSURE allows a user to choose a welfare-maximizing rule from a pre-specified class by optimizing the estimated welfare, thereby producing selection decisions that borrow strength across noisy estimates. We show that ASSURE produces decision rules that are asymptotically no worse than the optimal but infeasible decision rule in the pre-specified class. We apply ASSURE to the selection of Census tracts for economic opportunity, the identification of discriminating firms, and the analysis of $p$-value decision procedures in A/B testing.

Figures

Figures reproduced from arXiv: 2511.11862 by the authors.

Figure 1
Figure 1. [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Semisynthetic simulation comparison on Opportunity Atlas dataset. Box and whisker plots summarize 40 Monte Carlo runs where only Yi are redrawn. Constants costs are taken with K = 0.361; the rationale behind this choice is discussed in Section 6.1 [PITH_FULL_IMAGE:figures/full_fig_p023_2.png] view at source ↗
Figure 3
Figure 3. Experimentation program semisynthetic simulation, with Xi = µi +σit10,(ρ = 0.7). Boxplots show 100 Monte Carlo comparisons where Yi is rerandomized. 6. Empirical Applications 6.1. The Opportunity Atlas. The Opportunity Atlas (Chetty et al., 2018) pro￾vides census-tract level estimates of economic mobility as measured by a suite of children’s outcomes in adulthood. For each tract, multiple economic indicators such as… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Visualization of Opportunity Atlas estimates from (Chetty et al., 2018; Bergman et al., 2024). Each point represents a census tract within the largest 20 Commuting Zones, measuring the household in￾come rank in adulthood for Black children, with genders pooled, whose p…
Figure 5
Figure 5. Figure 5: ASSURE estimate curves for linear shrinkage class with con￾stant costs on the OA dataset of [PITH_FULL_IMAGE:figures/full_fig_p027_5.png]
Figure 6
Figure 6. Figure 6: ASSURE decisions and empirical Bayes decisions overlayed on Opportunity Atlas data. K = 0.361, matching the middle panel of [PITH_FULL_IMAGE:figures/full_fig_p027_6.png]
Figure 7
Figure 7. Figure 7: Left: Treatment effect estimate vs. standard deviation estimate for a dataset of 330 feature experiments from a large online tech company. The orange line and orange dots highlight tests which pass the traditional p < 0.05 criteria for statistical significance. Right: …
Figure 8
Figure 8. Figure 8: Blue line shows welfare estimate Wc(β) for decisions of the form Yi ≥ βσi . Anonymized data on treatment effects from a single experimentation program. As such, the y-axis scale is obfuscated. optimal decision cannot directly be interpreted, though we hope this section…
Figure 9
Figure 9. Figure 9: Left: ASSURE estimated welfare curves for t-statistic de￾cision class Yi ≥ K + βσi with constant costs K corresponding to multiples of the median absolute value of treatment effect. Right: opti￾mal t-statistic decision threshold chosen by ASSURE. Grey dashed line corre…
Figure 10
Figure 10. Figure 10: ASSURE and linear shrinkage rule on data from Kline et al. (2022) As shown in [PITH_FULL_IMAGE:figures/full_fig_p031_10.png]
Figure 11
Figure 11. Figure 11: Same simulation setting as in [PITH_FULL_IMAGE:figures/full_fig_p087_11.png]
Figure 12
Figure 12. Figure 12: Comparison of all methods in four simulation settings for the Opportunity Atlas dataset. D.1.2. Calibrated Experimentation Program Data. To generate the calibrated dataset on experimentation programs, we first take the dataset described in Section 6.2. Given Y1:n, σ1:…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Assumption-Lean Shrinkage and Model Averaging for Spatial Parameters

    econ.EM 2026-06 unverdicted novelty 7.0 of 10

    SURE-based selection and averaging of flexible spatial shrinkage rules performs nearly as well as the best candidate and cuts estimated MSE by 27% versus non-spatial empirical Bayes in Opportunity Atlas mobility data.

Reference graph

Works this paper leans on

4 extracted references · 2 linked inside Pith · cited by 1 Pith paper

  1. [1]

    F.and Palmer, C.(2024)

    2, 5, 28, 29, 79 Bergman, P.,Chetty, R.,DeLuca, S.,Hendren, N.,Katz, L. F.and Palmer, C.(2024). Creating moves to opportunity: Experimental evidence on barriers to neighborhood choice.American Economic Review,114(5), 1281–1337. 2, 5, 22, 25, 26, 89 Berman, R.andV an den Bulte, C.(2022). False discovery in a/b testing.Man- agement Science,68(9), 6762–6782....

    arXiv 2024

  2. [2013]

    Z ηn 0 s 1 + logN ϵ Cλ 3 n(λn +B) ,D,L 2(Pn) dϵ # +E

    and (Van den Dries, 1998) for a textbook reference on this topic. Consider the decision class in (3.5) as functions of the costkand log standard deviationℓ: fβ(k, ℓ) =k+e 2ℓe−b1−b2ℓ(k−a 1 −a 2eℓ) indexed byβ= (a 1, a2, b1, b2).The subgraph sets Sβ := (k, ℓ, t) :t < k+e2ℓe−b1−b2ℓ(k−a 1 −a 2eℓ) are defined by a first-order formula in the structure (R,+,·,0,...

    1998

  3. [2036]

    1 n nX i=1 (µi −K i)1{Y i −ϵσ iωi ≥δ(Z i;β)} # =E[V n(β;ϵ)], 43 whereω i ∼N(0,1) andω i yY i, Zi. Define the welfare function of our interest. W(β; 0) =E

    2, 5, 30, 31 Kline, P. M.,Rose, E. K.andW alters, C. R.(2024). Discrimination report cards: An empirical bayes ranking approach. 4, 5, 11 Koenker, R.andGu, J.(2017). Rebayes: an r package for empirical bayes mixture methods.Journal of Statistical Software,82, 1–26. 86 —and—(2024). Empirical bayes for the reluctant frequentist.arXiv preprint arXiv:2404.034...

    arXiv 2024

  4. [2632]

    invidious comparisons: Ranking and selection as compound decisions

    2 Chou, W.,Gray, C.,Kallus, N.,Bibaut, A.andEjdemyr, S.(2025). Eval- uating decision rules across many weak experiments. InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pp. 4365–4374. 28 Crippa, F.(2025). Regret analysis in threshold policy design.Journal of Economet- rics,249, 105998. 4, 15, 18 Dano, K.,Honor ...

    arXiv 2025

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.