pith. sign in

arxiv: 2509.02376 · v2 · submitted 2025-09-02 · 📊 stat.ME · math.ST· stat.TH

Resampling-based multi-resolution false discovery exceedance control

Jesse Hemerik This is my paper

Pith reviewed 2026-05-18 19:54 UTC · model grok-4.3

classification 📊 stat.ME math.STstat.TH
keywords false discovery exceedancemultiple testingresamplingsimultaneous controlfalse discovery proportionfamilywise error rateconfidence envelopes
0
0 comments X

The pith

A resampling method outputs one threshold but bounds the false discovery proportion for all stricter thresholds at once.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper generalizes a popular resampling approach for multiple testing so that a single chosen rejection threshold comes with guarantees that hold simultaneously for every stricter threshold. Specifically, the false discovery proportion stays below a fixed level gamma with probability at least one minus alpha for the chosen threshold and all stricter ones alike. For very small sets of hypotheses this implies zero false discoveries with high probability. The method keeps power comparable to the strongest existing non-simultaneous false discovery exceedance procedures and requires only the same assumptions those procedures already use. A reader would care because the extra simultaneous layer supplies more robust error control in dependent data without an obvious power penalty.

Core claim

The proposed procedure outputs a single rejection threshold q but ensures that with probability 1-α, simultaneously over all stricter thresholds, the corresponding false discovery proportions are also below γ. In particular, for a small set of hypotheses the false discovery bound is 0. Despite these additional simultaneous guarantees the method has power comparable to the most powerful non-simultaneous false discovery exceedance procedure. The method remains valid under the same assumptions and can be viewed as an extension of prior simultaneous approaches that for the first time permits confidence envelopes whose shape depends on the data.

What carries the argument

Multi-resolution false discovery exceedance control, a resampling extension that enforces simultaneous false discovery proportion bounds across all stricter thresholds from one chosen cutoff.

If this is right

  • The procedure forces the familywise error rate to zero for small collections of hypotheses.
  • Power stays comparable to the strongest existing non-simultaneous false discovery exceedance methods.
  • Validity continues to rest on exactly the same assumptions required by earlier false discovery exceedance procedures.
  • The approach supplies confidence envelopes whose shape can depend on the observed data, removing a previous restriction on simultaneous methods.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Analysts could obtain layered error statements at several significance levels from a single computation.
  • The construction may transfer to other resampling-based error rates that currently lack simultaneous versions.
  • Fields that routinely explore data at varying stringency levels might adopt the procedure to reduce the chance of post-hoc threshold adjustments.
  • The result suggests that simultaneous properties can often be added to existing false discovery exceedance methods with little extra cost.

Load-bearing premise

The underlying resampling step must correctly capture the dependence structure among the test statistics.

What would settle it

Repeated simulations or real-data applications in which the observed false discovery proportion for at least one stricter threshold exceeds γ with frequency greater than α would show that the simultaneous guarantee fails.

Figures

Figures reproduced from arXiv: 2509.02376 by Jesse Hemerik.

Figure 1
Figure 1. Figure 1: The power of the single-step FDX method as depending on [PITH_FULL_IMAGE:figures/full_fig_p014_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Scatterplot showing the number of rejections of the new method versus the number [PITH_FULL_IMAGE:figures/full_fig_p015_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The power of the new method (“New”) versus Romano-Shaikh (“RS”), [PITH_FULL_IMAGE:figures/full_fig_p019_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: The power of the fast (“New”, “HSG”, “RW”) and computationally intensive [PITH_FULL_IMAGE:figures/full_fig_p020_4.png] view at source ↗
read the original abstract

MaxT is a highly popular resampling-based multiple testing procedure, which controls the Familywise Error Rate (FWER) and is powerful under dependence. This paper generalizes maxT to what we term ``multi-resolution'' False Discovery eXceedance (FDX) control. Basic FDX control means ensuring that the FDP -- the proportion of false discoveries among all rejections -- is at most $\gamma$ with probability at least $1-\alpha$. Here $\gamma$ and $\alpha$ are prespecified, small values between 0 and 1. The proposed method is in addition simultaneous, in the following way: the procedure outputs a single rejection threshold $q$, but ensures that with probability $1-\alpha$, simultaneously over all stricter thresholds, the corresponding FDPs are also below $\gamma$. In particular, for a small set of hypotheses, the FDP bound is 0, i.e., the FWER is 0. Despite these additional, simultaneous guarantees, our method has power comparable to Romano-Wolf, the most powerful non-simultaneous FDX method. Further, our method is valid under the same assumptions. Thus, this paper shows that FDX methods can often be made simultaneous almost for free. The proposed method can be formulated as an extension of simultaneous approaches such as Hemerik, Solari and Goeman (2019), for the first time allowing for confidence envelopes with a data-dependent shape -- thus resolving a major limitation of such methods.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes a resampling-based generalization of the MaxT procedure for multi-resolution false discovery exceedance (FDX) control. It outputs a single data-dependent threshold q such that, with probability at least 1-α, the false discovery proportion remains below a prespecified γ simultaneously for all stricter thresholds q' ≤ q (including FWER control when the number of rejections is small). The method is presented as an extension of Hemerik-Solari-Goeman simultaneous envelopes to data-dependent shapes via resampling, with validity claimed under the same assumptions as standard FDX procedures and power comparable to the Romano-Wolf method.

Significance. If the validity argument holds, the result would be significant for the multiple-testing literature: it shows that simultaneous FDX control can be obtained with little or no power loss relative to non-simultaneous resampling methods, while resolving the fixed-shape limitation of prior envelope approaches. This could be useful in dependent high-dimensional settings where both FDX control and interpretability across resolution levels matter. The paper correctly credits the underlying resampling framework and the Hemerik et al. (2019) envelopes.

major comments (2)
  1. [§3.2] §3.2, Theorem 1 (or the main validity result): the claim that the simultaneous FDP control over the entire path q' ≤ q follows directly from the marginal MaxT resampling quantiles is not fully supported by the standard subset-pivotality or exchangeability assumptions used for single-threshold FDX. An additional uniform convergence or monotonicity argument on the empirical FDP process appears necessary to guarantee that the resampling envelope dominates the joint tail; without it the multi-resolution property does not automatically inherit.
  2. [§4.1] §4.1–4.2 (simulation design): the reported power comparisons with Romano-Wolf are useful, but the tables do not separately tabulate the empirical probability that the simultaneous guarantee is violated (i.e., sup_{q'≤q} FDP(q') > γ). This quantity is load-bearing for the central claim and should be reported under both global-null and sparse-alternative regimes.
minor comments (2)
  1. [§2.3] Notation for the data-dependent shape function in the envelope construction is introduced without an explicit display equation; adding a displayed definition would improve readability.
  2. [Abstract] The abstract states that the method is 'valid under the same assumptions' as prior FDX methods, but the precise list of assumptions (e.g., whether positive regression dependence or only exchangeability is required) is only given later; moving a concise statement to the abstract or introduction would help.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their thoughtful and constructive comments on our manuscript. We address each major comment below and describe the revisions we plan to make.

read point-by-point responses

  1. Referee: [§3.2] §3.2, Theorem 1 (or the main validity result): the claim that the simultaneous FDP control over the entire path q' ≤ q follows directly from the marginal MaxT resampling quantiles is not fully supported by the standard subset-pivotality or exchangeability assumptions used for single-threshold FDX. An additional uniform convergence or monotonicity argument on the empirical FDP process appears necessary to guarantee that the resampling envelope dominates the joint tail; without it the multi-resolution property does not automatically inherit.

    Authors: We appreciate the referee's careful reading. The threshold q is defined as the infimum of values where the resampling envelope lies below γ, and the monotonicity of the rejection sets (stricter thresholds yield subsets of rejections) together with the marginal validity of the maxT quantiles ensures that the bound holds simultaneously over q' ≤ q. Nevertheless, we agree that an explicit supporting argument would strengthen the presentation. In the revision we will insert a short lemma in Section 3 establishing that the empirical FDP process is non-increasing in the threshold under the maintained exchangeability conditions; combined with the marginal controls this yields the joint guarantee without requiring additional uniform-convergence assumptions beyond those already used for single-threshold FDX. revision: yes

  2. Referee: [§4.1] §4.1–4.2 (simulation design): the reported power comparisons with Romano-Wolf are useful, but the tables do not separately tabulate the empirical probability that the simultaneous guarantee is violated (i.e., sup_{q'≤q} FDP(q') > γ). This quantity is load-bearing for the central claim and should be reported under both global-null and sparse-alternative regimes.

    Authors: We concur that reporting the finite-sample violation rate of the simultaneous guarantee is a valuable diagnostic. In the revised manuscript we will augment the simulation tables in Sections 4.1 and 4.2 with an additional column (or a new supplementary table) that records the Monte Carlo estimate of P(sup_{q' ≤ q} FDP(q') > γ) for our procedure, under both the global-null and sparse-alternative regimes. This will be computed at the nominal α level and compared against the theoretical bound. revision: yes

Circularity Check

1 steps flagged

Minor self-citation to prior simultaneous FDX work; core multi-resolution extension independently motivated and not reduced to fitted inputs

specific steps
  1. self citation load bearing [Abstract]
    "The proposed method can be formulated as an extension of simultaneous approaches such as Hemerik, Solari and Goeman (2019), for the first time allowing for confidence envelopes with a data-dependent shape -- thus resolving a major limitation of such methods."

    The simultaneous multi-resolution guarantee and validity under identical assumptions are positioned as following from the cited prior work by overlapping authors; while presented as a novel extension, the load-bearing justification for inheriting the full envelope control without additional uniformity arguments reduces to this self-citation chain rather than a standalone derivation.

full rationale

The paper presents a resampling-based generalization of maxT for simultaneous multi-resolution FDX control, claiming validity under the same assumptions as prior FDX methods and power comparable to Romano-Wolf. It explicitly formulates the method as an extension of Hemerik-Solari-Goeman (2019) to allow data-dependent shapes. This constitutes one minor self-citation (author overlap) that is not load-bearing for the central claim, as the multi-resolution property and simultaneous guarantee are motivated independently via the resampling construction rather than by redefining or fitting quantities inside the paper. No self-definitional steps, fitted inputs renamed as predictions, or ansatz smuggling appear in the provided abstract or description. The derivation remains self-contained against external resampling benchmarks.

Axiom & Free-Parameter Ledger

2 free parameters · 1 axioms · 0 invented entities

The central claim rests on the validity of the base resampling procedure for FWER control and on the prespecified parameters gamma and alpha; no new entities are introduced.

free parameters (2)
  • gamma
    Prespecified upper bound on the false discovery proportion
  • alpha
    Prespecified probability level for the exceedance control
axioms (1)
  • domain assumption Resampling procedure (as in MaxT) controls the relevant error rate under the dependence structure present in the data
    Invoked when the abstract states the new method is valid under the same assumptions as prior FDX methods

pith-pipeline@v0.9.0 · 5792 in / 1300 out tokens · 46790 ms · 2026-05-18T19:54:47.853545+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

76 extracted references · 76 canonical work pages

  1. [1]

    Permutation-based true discovery proportions for functional magnetic resonance imaging cluster analysis

    Andreella, A., Hemerik, J., Finos, L., Weeda, W., and Goeman, J. Permutation-based true discovery proportions for functional magnetic resonance imaging cluster analysis. Statistics in Medicine, 42 0 (14): 0 2311--2340, 2023

    work page 2023

  2. [2]

    and Vicente, P

    Batista, C. and Vicente, P. C. Is mobile money changing rural africa? evidence from a field experiment. Review of Economics and Statistics, pages 1--10, 2025

    work page 2025

  3. [3]

    Notip: Non-parametric true discovery proportion control for brain imaging

    Blain, A., Thirion, B., and Neuvial, P. Notip: Non-parametric true discovery proportion control for brain imaging. NeuroImage, 260: 0 119492, 2022

    work page 2022

  4. [4]

    Post hoc confidence bounds on false positives using reference families

    Blanchard, G., Neuvial, P., and Roquain, E. Post hoc confidence bounds on false positives using reference families. The Annals of Statistics, 48 0 (3): 0 1281--1303, 2020

    work page 2020

  5. [5]

    High-dimensional statistics with a view toward applications in biology

    B \"u hlmann, P., Kalisch, M., and Meier, L. High-dimensional statistics with a view toward applications in biology. Annual review of statistics and its application, 1 0 (1): 0 255--278, 2014

    work page 2014

  6. [6]

    A., Romano, J

    Canay, I. A., Romano, J. P., and Shaikh, A. M. Randomization tests under an approximate symmetry assumption. Econometrica, 85 0 (3): 0 1013--1030, 2017

    work page 2017

  7. [7]

    Chung, J. H. and Fraser, D. A. Randomization tests for a multivariate two-sample problem. Journal of the American Statistical Association, 53 0 (283): 0 729--735, 1958

    work page 1958

  8. [8]

    P., and Wolf, M

    Clarke, D., Romano, J. P., and Wolf, M. The romano--wolf multiple-hypothesis correction in stata. The Stata Journal, 20 0 (4): 0 812--843, 2020

    work page 2020

  9. [9]

    Cui, X., Dickhaus, T., Ding, Y., and Hsu, J. C. Handbook of multiple comparisons. CRC Press, 2021

    work page 2021

  10. [10]

    and Flachaire, E

    Davidson, R. and Flachaire, E. The wild bootstrap, tamed at last. Journal of Econometrics, 146 0 (1): 0 162--169, 2008

    work page 2008

  11. [11]

    J., Davenport, S., Hemerik, J., and Finos, L

    De Santis, R., Goeman, J. J., Davenport, S., Hemerik, J., and Finos, L. Permutation-based multiple testing when fitting many generalized linear models. Electronic Journal of Statistics, 19 0 (2): 0 3317--3332, 2025 a

    work page 2025

  12. [12]

    J., Hemerik, J., Davenport, S., and Finos, L

    De Santis, R., Goeman, J. J., Hemerik, J., Davenport, S., and Finos, L. Inference in generalized linear models with robustness to misspecified variances. Journal of the American Statistical Association, ``online first'' version, 2025 b

    work page 2025

  13. [13]

    and Roquain, E

    Delattre, S. and Roquain, E. New procedures controlling the false discovery proportion via romano--wolf’s heuristic. 2015

    work page 2015

  14. [14]

    Promotions and productivity: the role of meritocracy and pay progression in the public sector

    Deserranno, E., Kastrau, P., and Le \'o n-Ciliotta, G. Promotions and productivity: the role of meritocracy and pay progression in the public sector. American Economic Review: Insights, 7 0 (1): 0 71--89, 2025

    work page 2025

  15. [15]

    and Yu, M

    Dobriban, E. and Yu, M. Symmpi: predictive inference for data with group symmetries. Journal of the Royal Statistical Society Series B: Statistical Methodology, page qkaf022, 2025

    work page 2025

  16. [16]

    and Roquain, E

    D \"o hler, S. and Roquain, E. Controlling the false discovery exceedance for heterogeneous tests. 2020

    work page 2020

  17. [17]

    False Discovery Exceedance Controlling Multiple Testing Procedures, 2024

    Dohler, S., Junge, F., and Roquain, E. False Discovery Exceedance Controlling Multiple Testing Procedures, 2024. URL https://CRAN.R-project.org/package=FDX. R package version 2.0.2

    work page 2024

  18. [18]

    Informing mothers about the benefits of conversing with infants: Experimental evidence from ghana

    Dupas, P., Falezan, C., Jayachandran, S., and Walsh, M. Informing mothers about the benefits of conversing with infants: Experimental evidence from ghana. American Economic Journal: Economic Policy, 17 0 (2): 0 388--417, 2025

    work page 2025

  19. [19]

    A fast algorithm to compute a curve of confidence upper bounds for the false discovery proportion using a reference family with a forest structure

    Durand, G. A fast algorithm to compute a curve of confidence upper bounds for the false discovery proportion using a reference family with a forest structure. arXiv preprint arXiv:2502.03849, 2025

    work page arXiv 2025

  20. [20]

    E., and Knutsson, H

    Eklund, A., Nichols, T. E., and Knutsson, H. Cluster failure: Why fmri inferences for spatial extent have inflated false-positive rates. Proceedings of the national academy of sciences, 113 0 (28): 0 7900--7905, 2016

    work page 2016

  21. [21]

    Generalized augmentation to control the false discovery exceedance in multiple testing

    Farcomeni, A. Generalized augmentation to control the false discovery exceedance in multiple testing. Scandinavian Journal of Statistics, 36 0 (3): 0 501--517, 2009

    work page 2009

  22. [22]

    Genovese, C. R. and Wasserman, L. Exceedance control of the false discovery proportion. Journal of the American Statistical Association, 101 0 (476): 0 1408--1417, 2006

    work page 2006

  23. [23]

    Goeman, J. J. and Solari, A. The sequential rejection principle of familywise error control. The Annals of Statistics, 38: 0 3782--3810, 2010

    work page 2010

  24. [24]

    Goeman, J. J. and Solari, A. Multiple testing for exploratory research. Statistical Science, 26 0 (4): 0 584--597, 2011

    work page 2011

  25. [25]

    Goeman, J. J. and Solari, A. Multiple hypothesis testing in genomics. Statistics in medicine, 33 0 (11): 0 1946--1978, 2014

    work page 1946

  26. [26]

    J., Meijer, R

    Goeman, J. J., Meijer, R. J., Krebs, T. J., and Solari, A. Simultaneous control of all false discovery proportions in large-scale multiple hypothesis testing. Biometrika, 106 0 (4): 0 841--856, 2019

    work page 2019

  27. [27]

    J., Hemerik, J., and Solari, A

    Goeman, J. J., Hemerik, J., and Solari, A. Only closed testing procedures are admissible for controlling false discovery proportions. The Annals of Statistics, 49 0 (2): 0 1218--1238, 2021

    work page 2021

  28. [28]

    and Romano, J

    Guo, W. and Romano, J. A generalized sidak-holm procedure and control of generalized error rates under independence. Statistical applications in genetics and molecular biology, 6 0 (1), 2007

    work page 2007

  29. [29]

    K., et al

    Guo, W., He, L., Sarkar, S. K., et al. Further results on controlling the false discovery proportion. The Annals of Statistics, 42 0 (3): 0 1070--1101, 2014

    work page 2014

  30. [30]

    and Goeman, J

    Hemerik, J. and Goeman, J. J. False discovery proportion estimation by permutations: confidence for significance analysis of microarrays. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 80 0 (1): 0 137--155, 2018 a

    work page 2018

  31. [31]

    Permutation-based simultaneous confidence bounds for the false discovery proportion

    Hemerik, J., Solari, A., and Goeman, J. Permutation-based simultaneous confidence bounds for the false discovery proportion. Biometrika, 106 0 (3): 0 635--649, 2019

    work page 2019

  32. [32]

    and Goeman, J

    Hemerik, J. and Goeman, J. J. Exact testing with random permutations. TEST, 27 0 (4): 0 811--825, 2018 b

    work page 2018

  33. [33]

    J., and Finos, L

    Hemerik, J., Goeman, J. J., and Finos, L. Robust testing in generalized linear models by sign flipping score contributions. Journal of the Royal Statistical Society Series B: Statistical Methodology, 82 0 (3): 0 841--864, 2020

    work page 2020

  34. [34]

    The large-sample power of tests based on permutations of observations

    Hoeffding, W. The large-sample power of tests based on permutations of observations. The Annals of Mathematical Statistics, 23: 0 169--192, 1952

    work page 1952

  35. [35]

    A simple sequentially rejective multiple test procedure

    Holm, S. A simple sequentially rejective multiple test procedure. Scandinavian journal of statistics, pages 65--70, 1979

    work page 1979

  36. [36]

    Kashlak, A. B. Asymptotic symmetry and group invariance for randomization. arXiv preprint arXiv:2211.00144, 2022

    work page arXiv 2022

  37. [37]

    and Ramdas, A

    Katsevich, E. and Ramdas, A. Simultaneous high-probability bounds on the false discovery proportion in structured, regression and online settings. The Annals of Statistics, 48 0 (6): 0 3465--3487, 2020

    work page 2020

  38. [38]

    Evaluating the robustness of permutation-based multiple testing methods, 2022

    Kisoentewari, A. Evaluating the robustness of permutation-based multiple testing methods, 2022. Master's thesis, available at https://hdl.handle.net/1887/3485866

    work page 2022

  39. [39]

    Koning, N. W. More power by using fewer permutations. Biometrika, 111 0 (4): 0 1405--1412, 2024

    work page 2024

  40. [40]

    Koning, N. W. and Hemerik, J. More efficient exact group invariance testing: using a representative subgroup. Biometrika, 111 0 (2): 0 441--458, 2024

    work page 2024

  41. [41]

    L., Troendle, J

    Korn, E. L., Troendle, J. F., McShane, L. M., and Simon, R. Controlling the number of false discoveries: application to high-dimensional genomic data. Journal of Statistical Planning and Inference, 124 0 (2): 0 379--398, 2004

    work page 2004

  42. [42]

    L., Li, M.-C., McShane, L

    Korn, E. L., Li, M.-C., McShane, L. M., and Simon, R. An investigation of two multivariate permutation methods for controlling the false discovery proportion. Statistics in medicine, 26 0 (24): 0 4428--4440, 2007

    work page 2007

  43. [43]

    Lehmann, E. L. and Romano, J. P. Generalizations of the familywise error rate. volume 33, pages 1138--1154. 2005

    work page 2005

  44. [44]

    A., Shaikh, A

    List, J. A., Shaikh, A. M., and Xu, Y. Multiple hypothesis testing in experimental economics. Experimental Economics, 22: 0 773--793, 2019

    work page 2019

  45. [45]

    A., Shaikh, A

    List, J. A., Shaikh, A. M., and Vayalinkal, A. Multiple testing with covariate adjustment in experimental economics. Journal of Applied Econometrics, 38 0 (6): 0 920--939, 2023

    work page 2023

  46. [46]

    Contemporary album ratings and reviews, 2021

    Lucas, K. Contemporary album ratings and reviews, 2021. URL https://www.kaggle.com/datasets/kauvinlucas/30000-albums-aggregated-review-ratings

    work page 2021

  47. [47]

    Barnard's M onte C arlo tests: How many simulations? Applied Statistics, pages 75--77, 1979

    Marriott, F. Barnard's M onte C arlo tests: How many simulations? Applied Statistics, pages 75--77, 1979

    work page 1979

  48. [48]

    False discovery control for multiple tests of association under general dependence

    Meinshausen, N. False discovery control for multiple tests of association under general dependence. Scandinavian Journal of Statistics, 33 0 (2): 0 227--237, 2006

    work page 2006

  49. [49]

    and B \"u hlmann, P

    Meinshausen, N. and B \"u hlmann, P. Lower bounds for the number of false null hypotheses for multiple testing of associations under general dependence structures. Biometrika, 92 0 (4): 0 893--907, 2005

    work page 2005

  50. [50]

    H., B \"u hlmann, P., et al

    Meinshausen, N., Maathuis, M. H., B \"u hlmann, P., et al. Asymptotic optimality of the westfall--young permutation procedure for multiple testing under dependence. The Annals of Statistics, 39 0 (6): 0 3369--3391, 2011

    work page 2011

  51. [51]

    Nichols, T. E. and Holmes, A. P. Nonparametric permutation tests for functional neuroimaging: a primer with examples. Human brain mapping, 15 0 (1): 0 1--25, 2002

    work page 2002

  52. [52]

    Pollard, K. S. and van der Laan, M. J. Resampling-based multiple testing: Asymptotic control of type i error and applications to gene expression data. 2003

    work page 2003

  53. [53]

    A., Davaniah, S., Derache, A., Wilkinson, E., Tanser, F., Gupta, R

    Power, R. A., Davaniah, S., Derache, A., Wilkinson, E., Tanser, F., Gupta, R. K., Pillay, D., and de Oliveira, T. Genome-wide association study of hiv whole genome sequences validated using drug resistance. PLoS One, 11 0 (9): 0 e0163746, 2016

    work page 2016

  54. [54]

    Romano, J. P. On the behavior of randomization tests without a group invariance assumption. Journal of the American Statistical Association, 85 0 (411): 0 686--692, 1990

    work page 1990

  55. [55]

    Romano, J. P. and Shaikh, A. M. On stepdown control of the false discovery proportion. Lecture Notes-Monograph Series, pages 33--50, 2006 a

    work page 2006

  56. [56]

    Romano, J. P. and Shaikh, A. M. Stepup procedures for control of generalizations of the familywise error rate. The Annals of Statistics, pages 1850--1873, 2006 b

    work page 2006

  57. [57]

    Romano, J. P. and Wolf, M. Exact and approximate stepdown methods for multiple hypothesis testing. Journal of the American Statistical Association, 100 0 (469): 0 94--108, 2005 a

    work page 2005

  58. [58]

    Romano, J. P. and Wolf, M. Stepwise multiple testing as formalized data snooping. Econometrica, 73 0 (4): 0 1237--1282, 2005 b

    work page 2005

  59. [59]

    Romano, J. P. and Wolf, M. Control of generalized error rates in multiple testing. 2007

    work page 2007

  60. [60]

    Romano, J. P. and Wolf, M. Efficient computation of adjusted p-values for resampling-based stepdown multiple testing. Statistics & Probability Letters, 113: 0 38--40, 2016

    work page 2016

  61. [61]

    P., Shaikh, A

    Romano, J. P., Shaikh, A. M., and Wolf, M. Hypothesis testing in econometrics. Annu. Rev. Econ., 2 0 (1): 0 75--104, 2010

    work page 2010

  62. [62]

    Type i error rate control for testing many hypotheses: a survey with proofs

    Roquain, E. Type i error rate control for testing many hypotheses: a survey with proofs. Journal de la Soci \'e t \'e Fran c aise de Statistique , 152 0 (2): 0 3--38, 2011

    work page 2011

  63. [63]

    D., Finos, L., Weeda, W

    Rosenblatt, J. D., Finos, L., Weeda, W. D., Solari, A., and Goeman, J. J. All-resolutions inference for brain imaging. Neuroimage, 181: 0 786--796, 2018

    work page 2018

  64. [64]

    R., De Jong, W

    Rosyara, U. R., De Jong, W. S., Douches, D. S., and Endelman, J. B. Software for genome-wide association studies in autopolyploids and its application to potato. The plant genome, 9 0 (2): 0 plantgenome2015--08, 2016

    work page 2016

  65. [65]

    J., Zavattari, P., Moi, L., Columbano, A., Meaburn, E

    Saffari, A., Silver, M. J., Zavattari, P., Moi, L., Columbano, A., Meaburn, E. L., and Dudbridge, F. Estimation of a significance threshold for epigenome-wide association studies. Genetic epidemiology, 42 0 (1): 0 20--33, 2018

    work page 2018

  66. [66]

    Permory-mpi: a program for high-speed parallel permutation testing in genome-wide association studies

    Stei , V., Letschert, T., Sch \"a fer, H., and Pahl, R. Permory-mpi: a program for high-speed parallel permutation testing in genome-wide association studies. Bioinformatics, 28 0 (8): 0 1168--1169, 2012

    work page 2012

  67. [67]

    J., Tony Cai, T., Guindani, M., and Schwartzman, A

    Sun, W., Reich, B. J., Tony Cai, T., Guindani, M., and Schwartzman, A. False discovery control in large-scale spatial multiple testing. Journal of the Royal Statistical Society Series B: Statistical Methodology, 77 0 (1): 0 59--83, 2015

    work page 2015

  68. [68]

    High-speed westfall-young permutation procedure for genome-wide association studies

    Terada, A., Kim, H., and Sese, J. High-speed westfall-young permutation procedure for genome-wide association studies. In Proceedings of the 6th ACM Conference on Bioinformatics, Computational Biology and Health Informatics, pages 17--26, 2015

    work page 2015

  69. [69]

    G., Tibshirani, R., and Chu, G

    Tusher, V. G., Tibshirani, R., and Chu, G. Significance analysis of microarrays applied to the ionizing radiation response. Proceedings of the National Academy of Sciences, 98 0 (9): 0 5116--5121, 2001

    work page 2001

  70. [70]

    J., Dudoit, S., and Pollard, K

    van der Laan, M. J., Dudoit, S., and Pollard, K. S. Augmentation procedures for control of the generalized family-wise error rate and tail probabilities for the proportion of false positives. Statistical applications in genetics and molecular biology, 3 0 (1): 0 15, 2004

    work page 2004

  71. [71]

    Vesely, A., Finos, L., and Goeman, J. J. Permutation-based true discovery guarantee by sum tests. Journal of the Royal Statistical Society Series B: Statistical Methodology, 85 0 (3): 0 664--683, 2023

    work page 2023

  72. [72]

    Westfall, P. H. and Troendle, J. F. Multiple testing with minimal assumptions. Biometrical Journal: Journal of Mathematical Methods in Biosciences, 50 0 (5): 0 745--755, 2008

    work page 2008

  73. [73]

    Westfall, P. H. and Young, S. S. Resampling-based multiple testing: Examples and methods for p-value adjustment, volume 279. John Wiley & Sons, 1993

    work page 1993

  74. [74]

    R., Chan, C., Adepoju, T., Shinohara, R

    White, B. R., Chan, C., Adepoju, T., Shinohara, R. T., and Vandekar, S. Controlling the familywise error rate in widefield optical neuroimaging of functional connectivity in mice. Neurophotonics, 10 0 (1): 0 015004--015004, 2023

    work page 2023

  75. [75]

    M., Ridgway, G

    Winkler, A. M., Ridgway, G. R., Webster, M. A., Smith, S. M., and Nichols, T. E. Permutation inference for the general linear model. Neuroimage, 92: 0 381--397, 2014

    work page 2014

  76. [76]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

    work page