Pith. sign in

REVIEW 3 major objections 5 minor 37 references

Hypothesis testing for community structure in temporal networks using e-values

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Averaging per-snapshot e-values produces the first statistically valid test for community structure in temporal networks.

desk verdict Correct but modest: valid e-value averaging applied to temporal community testing, but an invalid calibrator is showcased as best and the firstness claim is overstated. read the letter →

arxiv 2507.23034 v3 pith:HCUQAFYC submitted 2025-07-30 stat.ME

Eric Yanchenko , Jonathan P. Williams , Ryan Martin This is my paper · ORCID
classification stat.ME MSC 62G1062H30
keywords e-valuestemporalnetworkscommunitydetectionhypothesistestingstochasticblockmodeldynamicp-valuecalibrationtypeIerrorcontrol
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes the first hypothesis test for the presence of community structure in a temporal network, where data arrive as a sequence of network snapshots over time. The procedure computes a valid p-value for static community structure on each snapshot, converts each p-value into an e-value using a calibrator, and averages the e-values. Because the arithmetic mean of e-values is itself an e-value, the resulting average preserves type I error control under arbitrary dependence among snapshots, which is exactly the complication that makes p-value combination intractable. The method is demonstrated on correlated and dynamic stochastic block models and on five real-world networks, with the test proposed as a modular extension any static community-structure test can be plugged into. A stated caveat is that averaging implicitly targets community structure whose strength is roughly constant over time, so the test does not capture intermittent, merging, splitting, or accumulating structures.

What carries the argument

The central object is the e-value, a nonnegative random variable whose expectation under the null hypothesis is at most one; large e-values indicate evidence against the null, and $1/E$ is a valid p-value. The argument turns on the combination property that the arithmetic mean of arbitrarily dependent e-values is again an e-value. Each snapshot p-value is transformed by a p-to-e calibrator, including $g_\kappa(p)=\kappa p^{\kappa-1}$, the averaged calibrator $g_{\mathrm{avg}}(p)=(1-p+p\log p)/(p(-\log p)^2)$, and the comparison 'max' function $g_{\max}(p)=-e^{-1}/(p\log p)$ for $p\le e^{-1}$, which the paper notes is not itself a calibrator. Theorem 1 then follows directly from the definition of an e-value, and the average is the test statistic.

What would settle it

Generate a temporal network with $T=10$ snapshots, where snapshots 1-5 are drawn from a stochastic block model with strong community structure and snapshots 6-10 from an Erdős–Renyi model. If the average of the calibrated e-values remains below the rejection threshold (say, 20) even though the per-snapshot p-values in the first half are tiny, then the claimed test does not detect the temporal community structure it is meant to test.

Watch

Extended reading notes

Core claim

Let $G^{(1)},\dots,G^{(T)}$ be snapshots of a temporal network, let $P_t$ be a valid p-value for the null hypothesis of no community structure on the $t$-th snapshot, and let $E_t = g(P_t)$ be the e-value obtained by applying a p-to-e calibrator such as $g_\kappa(p)=\kappa p^{\kappa-1}$. Theorem 1 asserts that $\bar E_T = \frac{1}{T}\sum_{t=1}^T E_t$ is an e-value, meaning $\sup_{P\in\mathcal{P}_0}\mathbb{E}_P(\bar E_T)\le 1$; hence a large average is evidence against the null, and the test's validity does not depend on how the snapshots are correlated. To the authors' knowledge, this is the first hypothesis test for community structure in temporal networks. The construction is modular: any static community-structure hypothesis test that yields valid p-values can be used, and the paper implements it both with an Erdős–Renyi null using Tracy-Widom asymptotics and with a Chung-Lu null using bootstrap.

Load-bearing premise

The test is meaningful only if temporal community structure can be represented by the average of per-snapshot static community evidence, so the strength of the structure must be roughly constant over time and the ordering of snapshots is treated as irrelevant.

Editorial extensions

If this is right

  • Any valid static community-structure test can be substituted into the framework, so the temporal test automatically inherits future improvements in static testing.
  • Weighted averaging, with weights summing to one, remains a valid e-value, allowing users to place more emphasis on recent snapshots without losing error control.
  • Using the rejection rule $\bar E_T > 20$ bounds the type I error at $0.05$, and the simulations report low rejection rates under the null and increasing power as community structure strengthens.
  • The same averaging scheme applies to multilayer networks, where each layer plays the role of a snapshot, extending the test beyond the temporal setting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the average is invariant to snapshot ordering, the test cannot distinguish communities that appear, merge, split, or disappear; detecting such phenomena would require a change-point or state-switching formulation rather than an averaging one.
  • A weighted average updated sequentially as new snapshots arrive would remain an e-value at every time, suggesting an anytime-valid monitoring procedure for emerging community structure without multiple-testing corrections.
  • The calibrator step inflates small p-values to maintain validity, which the paper identifies as a source of inefficiency; e-values constructed directly from a network model, rather than via p-value calibration, are the natural route to a more powerful temporal test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a hypothesis test for community structure in temporal networks. The test computes a static p-value for each snapshot, converts each p-value to an e-value using a p-to-e calibrator, and averages the e-values across snapshots. The central theoretical claim (Theorem 1) is that this average is a valid e-value under arbitrary temporal dependence. The method is illustrated on three simulated network processes (correlated SBM, dynamic SBM, dynamic DCBM) and five real-world networks, using the Bickel-Sarkar and Yanchenko-Sengupta static tests. The authors also candidly discuss limitations, including the implicit 'average community strength' interpretation, the sensitivity to zero p-values, and the difficulty of defining temporal community structure.

Significance. The elementary averaging argument is correct and gives a genuinely simple way to combine dependent snapshot-level evidence while preserving type I error guarantees. The paper also ships reproducible code and is unusually transparent about the meaning and limitations of the proposed null. If the novelty claim can be properly qualified relative to Wilson et al. (2017) and the invalid 'max' calibrator is removed from the main performance claims, the result would be a useful practical addition to the network testing toolkit. However, the current simulation narrative is partly based on a non-valid procedure, so the empirical support for the method's power comparisons must be reworked.

major comments (3)
  1. [Section 3.1, Section 4.1, Figures 1-3] The 'max' calibrator is explicitly defined in Section 3.1 as not a calibrator, yet it is included as one of the five compared calibrators in all simulation studies and is repeatedly described as giving the best performance (Sections 4.2.3, 4.3.3, 4.4.3). Because the integral of g_max over [0,1] diverges, a uniform p-value under the null can yield E[g_max(P)] = infinity, so the large e-values and high rejection rates reported for 'max' do not demonstrate valid power. The authors should remove 'max' from the main comparisons or clearly label it as an invalid oracle, and re-run the headline simulation summaries using only valid calibrators (kappa = 0.25, 0.5, 0.75, avg) to confirm that the qualitative conclusions change.
  2. [Section 1 and Section 3.2] The claim that there are 'no statistical methods available' to test for community structure in temporal networks and that the proposed test is 'the first hypothesis test for community structure in temporal networks' is difficult to reconcile with the authors' own description of Wilson et al. (2017) as deriving a hypothesis test for community structure in multilayer networks, of which temporal networks are a special case. The authors should either specify the precise difference in null hypotheses or objectives that makes their test novel, or soften the novelty claim.
  3. [Section 6 and Sections 4.3.3, 4.4.3] The paper acknowledges the infinite e-value issue in the Reality network, but the same phenomenon occurs in the dynamic SBM and dynamic DCBM simulations, where 'all p-values were 0... not plotted' (Sections 4.3.3 and 4.4.3). This means that a single snapshot with p=0 dominates the average, so the reported e-value is not a finite measure of 'average' community strength in exactly the settings used to claim power. The authors should report the frequency of zero p-values and either use a censored or regularized calibrator, or discuss this limitation more prominently in the simulation sections.
minor comments (5)
  1. [Section 3.2, Theorem 1] The statement 'Then \bar E_T is an e-value in the sense of (1)' refers to the p-value definition in equation (1); it should refer to the e-value definition in equation (2).
  2. [Section 4.1] There is a typo in 'We stress that he goal of these simulations' which should read 'the goal'.
  3. [Section 4.3.1] The transition probability notation '\pi_{k,k+1}' appears to be a typo for '\pi_{k,k'}' given the surrounding text; the current subscript is confusing.
  4. [Section 6] The phrase 'underling strength' should be 'underlying strength'.
  5. [Figures 1-3 and 8-10] The figure legends label 'max' as one of the 'Calibrator' series even though the text states it is not a proper calibrator; renaming the series to 'max (invalid)' or a similar label would avoid misleading readers.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Theorem 1 is a direct application of e-value definitions; the invalid 'max' calibrator is a validity concern, not a circular derivation.

full rationale

The derivation chain for Theorem 1 is self-contained: it uses the definition of a p-to-e calibrator and the known fact that an arithmetic mean of e-values is an e-value under arbitrary dependence. No fitted constants enter the construction; the calibrators (kappa = 0.25, 0.5, 0.75, avg) are fixed user choices, and the per-snapshot p-values come from external static tests (Bickel-Sarkar, and Yanchenko-Sengupta for the real-data CL null). The Yanchenko-Sengupta bootstrap test is independently published and is not constructed to force the temporal e-value conclusions, so citing it is not load-bearing circularity. The paper explicitly acknowledges that g_max is not a proper calibrator and still uses it in simulations; that is a correctness/validity concern about the experimental comparison, not a circular reduction. The paper also discloses in Section 3.2 and the Conclusion that averaging e-values implicitly assumes roughly constant community strength and order-invariance; this is a scope limitation, not a circular derivation. Thus no predicted quantity is equivalent by construction to its input, and the central claim is an honest application of e-value properties.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The e-value validity rests only on standard definitions and on the external validity of the chosen static tests. The practical recipe uses user-chosen parameters (kappa, threshold, bootstrap size) but none are fitted to the real-data conclusions. No new entities are postulated.

free parameters (3)
  • Calibrator parameter kappa = 0.25 (real data); 0.25, 0.50, 0.75, avg, max (simulations)
    User-selected parameter in the p-to-e calibrator; kappa=0.25 chosen post hoc for real data because it gave the largest simulated e-values.
  • Rejection threshold for e-value = E > 20
    Decision threshold corresponding to alpha <= 0.05 via Markov's inequality; arbitrary, not fitted.
  • Bootstrap iterations B in Chung-Lu test = 1000
    Number of bootstrap samples for real-data p-values; affects p-value resolution but not the e-value validity.
assumptions (6)
  • domain assumption Under an Erdos-Renyi null, the largest eigenvalue of the standardized adjacency matrix is asymptotically Tracy-Widom (Bickel and Sarkar, 2016).
    Used to compute p-values on each snapshot; the proposed test inherits this asymptotic null distribution.
  • domain assumption The bootstrap p-value of Yanchenko and Sengupta (2024) is valid under the Chung-Lu null model.
    Used in the real-data analysis to obtain snapshot p-values under a more flexible null.
  • standard math g_kappa(p)=kappa p^{kappa-1} is a valid p-to-e calibrator for kappa in (0,1).
    Ensures each converted snapshot statistic is an e-value when the raw p-value is valid.
  • standard math The arithmetic mean of e-values is an e-value under arbitrary dependence.
    Follows from linearity of expectation; this is the property that enables temporal combination.
  • domain assumption Under the null, each snapshot is marginally generated by the chosen static null model, while temporal dependence is arbitrary.
    The test defines no temporal null model beyond per-snapshot marginals; this is the main modeling limitation.
  • domain assumption Networks are undirected, binary, and have no self-loops.
    Standard assumptions used throughout the method and simulations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hypothesis testing for community structure in temporal networks using e-values." pith.science (2026). https://pith.science/paper/HCUQAFYC

@misc{pith2026250723034,
  author       = {Pith},
  title        = {Pith review of: Hypothesis testing for community structure in temporal networks using e-values},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HCUQAFYC}},
  note         = {Machine review of arXiv:2507.23034}
}
read the original abstract

Community structure in networks naturally arises in various applications. But while the topic has received significant attention for static networks, the literature on community structure in temporally evolving networks is more scarce. In particular, there are currently no statistical methods available to test for the presence of community structure in a sequence of networks evolving over time. In this work, we propose a simple yet powerful test using e-values, an alternative to p-values that is more flexible in certain ways. Specifically, an e-value framework retains valid testing properties even after combining dependent information, a relevant feature in the context of testing temporal networks. We apply the proposed test to synthetic and real-world networks, demonstrating various features inherited from the e-value formulation and exposing some of the inherent difficulties of testing on temporal networks.

Figures

Figures reproduced from arXiv: 2507.23034 by the authors.

Figure 1
Figure 1. Median e-value over 100 MC simulations for correlated SBM networks with [PITH_FULL_IMAGE:figures/full_fig_p011_1.png] view at source ↗
Figure 2
Figure 2. Median e-value over 100 MC simulations for dynamic SBM networks with [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗
Figure 3
Figure 3. Median e-value over 100 MC simulations for dynamic DCBM networks with [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Simulation rejection rates for correlated SBM networks with [PITH_FULL_IMAGE:figures/full_fig_p026_4.png]
Figure 5
Figure 5. Figure 5: Simulation rejection rates for dynamic SBM networks with [PITH_FULL_IMAGE:figures/full_fig_p027_5.png]
Figure 6
Figure 6. Figure 6: Simulation rejection rates for dynamic DCBM networks with [PITH_FULL_IMAGE:figures/full_fig_p028_6.png]
Figure 7
Figure 7. Figure 7: Simulation variability results for correlated SBM networks with [PITH_FULL_IMAGE:figures/full_fig_p029_7.png]
Figure 8
Figure 8. Figure 8: Median e-value over 100 MC simulations for correlated SBM networks with [PITH_FULL_IMAGE:figures/full_fig_p030_8.png]
Figure 9
Figure 9. Figure 9: Median e-value over 100 MC simulations for dynamic SBM networks with [PITH_FULL_IMAGE:figures/full_fig_p031_9.png]
Figure 10
Figure 10. Figure 10: Median e-value over 100 MC simulations for dynamic DCBM networks with [PITH_FULL_IMAGE:figures/full_fig_p032_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 29 canonical work pages

  1. [1]

    and Hochberg, Y

    Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological) , 57(1):289--300

    work page 1995

  2. [2]

    Bickel, P. J. and Sarkar, P. (2016). Hypothesis testing for automated community detection in networks. Journal of the Royal Statistical Society: Series B: Statistical Methodology , pages 253--273

    work page 2016

  3. [3]

    and Rossetti, G

    Cazabet, R. and Rossetti, G. (2023). Challenges in community discovery on temporal networks. In Temporal network theory , pages 185--202. Springer

    work page 2023

  4. [4]

    and Lu, L

    Chung, F. and Lu, L. (2002). The average distances in random graphs with given expected degrees. Proceedings of the National Academy of Sciences , 99(25):15879--15882

    2002

  5. [5]

    Dey, N., Martin, R., and Williams, J. P. (2024). Anytime-valid generalized universal inference on risk minimizers. arXiv preprint arXiv:2402.00202

    arXiv 2024

  6. [6]

    and Pentland, A

    Eagle, N. and Pentland, A. (2006). Reality mining: sensing complex social systems. Personal and ubiquitous computing , 10:255--268

    work page 2006

  7. [7]

    and Renyi, A

    Erd\"os, P. and Renyi, A. (1959). On random graphs. Publicationes Mathematicae Debrecen , pages 260--297

    work page 1959

  8. [8]

    Grünwald, P., de Heide, R., and Koolen, W. (2024). Safe Testing . Journal of the Royal Statistical Society Series B: Statistical Methodology , page qkae011

    work page 2024

Show all 37 references
  1. [9]

    W., Laskey, K

    Holland, P. W., Laskey, K. B., and Leinhardt, S. (1983). Stochastic block models: First steps. Social Networks , 5:109--137

    1983

  2. [10]

    and Saram \"a ki, J

    Holme, P. and Saram \"a ki, J. (2012). Temporal networks. Physics reports , 519(3):97--125

    2012

  3. [11]

    R., Stevens, N

    Jeske, D. R., Stevens, N. T., Tartakovsky, A. G., and Wilson, J. D. (2018). Statistical methods for network surveillance. Applied Stochastic Models in Business and Industry , 34(4):425--445

    2018

  4. [12]

    and Newman, M

    Karrer, B. and Newman, M. E. J. (2011). Stochastic blockmodels and community structure in networks. Physical Review E , 83:016107

    2011

  5. [13]

    C., Tizzoni, M., Kinyanjui, T

    Kiti, M. C., Tizzoni, M., Kinyanjui, T. M., Koech, D. C., Munywoki, P. K., Meriac, M., Cappa, L., Panisson, A., Barrat, A., Cattuto, C., et al. (2016). Quantifying social contacts in a household setting of rural kenya using wearable proximity sensors. EPJ data science , 5:1--21

    2016

  6. [14]

    Lancichinetti, A., Radicchi, F., and Ramasco, J. J. (2010). Statistical significance of communities in networks. Physical Review E , 81(4):046110

    2010

  7. [15]

    Larsson, M., Ramdas, A., and Ruf, J. (2025). The numeraire e-variable and reverse information projection. Annals of Statistics , 53(3):1015--1043

    2025

  8. [16]

    and Qi, Y

    Li, Y. and Qi, Y. (2020). Asymptotic distribution of modularity in networks. Metrika , 83(4):467--484

    2020

  9. [17]

    E., and Priebe, C

    Lyzinski, V., Fishkind, D. E., and Priebe, C. E. (2014). Seeded graph matching for correlated erdos-renyi graphs. Journal of Machine Learning Research , 15(108):3693--3720

    2014

  10. [18]

    Mastrandrea, R., Fournet, J., and Barrat, A. (2015). Contact patterns in a high school: a comparison between data collected using wearable sensors, contact diaries and friendship surveys. PloS one , 10(9):e0136497

    2015

  11. [19]

    and Miele, V

    Matias, C. and Miele, V. (2017). Statistical clustering of temporal networks through a dynamic stochastic block model. Journal of the Royal Statistical Society Series B: Statistical Methodology , 79(4):1119--1141

    2017

  12. [20]

    Newman, M. E. (2006). Modularity and community structure in networks. Proceedings of the National Academy of Sciences , 103(23):8577--8582

    2006

  13. [21]

    Palowitch, J., Bhamidi, S., and Nobel, A. B. (2017). Significance-based community detection in weighted networks. The Journal of Machine Learning Research , 18(1):6899--6946

    2017

  14. [22]

    Panzarasa, P., Opsahl, T., and Carley, K. M. (2009). Patterns and dynamics of users' behavior and interaction: Network analysis of an online community. Journal of the American Society for Information Science and Technology , 60(5):911--932

    2009

  15. [23]

    Ramdas, A., Ruf, J., Larsson, M., and Koolen, W. (2020). Admissible anytime-valid sequential inference must rely on nonnegative martingales. arXiv preprint arXiv:2009.03167

    2020 arXiv

  16. [24]

    and Wang, R

    Ramdas, A. and Wang, R. (2024). Hypothesis testing with e-values. arXiv preprint arXiv:2410.23614

    2024 arXiv

  17. [25]

    L., Flammini, A., and Menczer, F

    Sasahara, K., Chen, W., Peng, H., Ciampaglia, G. L., Flammini, A., and Menczer, F. (2021). Social influence and unfollowing accelerate the emergence of echo chambers. Journal of Computational Social Science , 4(1):381--402

    2021

  18. [26]

    Shafer, G., Shen, A., Vereshchagin, N., and Vovk, V. (2011). Test martingales, B ayes factors and p -values. Statist. Sci. , 26(1):84--101

    2011

  19. [27]

    L., Tang, M., Fishkind, D

    Sussman, D. L., Tang, M., Fishkind, D. E., and Priebe, C. E. (2012). A consistent adjacency spectral embedding for stochastic blockmodel graphs. Journal of the American Statistical Association , 107(499):1119--1128

    2012

  20. [28]

    Vanhems, P., Barrat, A., Cattuto, C., Pinton, J.-F., Khanafer, N., Regis, C., Kim, B.-A., Comte, B., and Voirin, N. (2013). Estimating potential infection transmission routes in hospital wards using wearable proximity sensors. PloS one , 8:e73970

    2013

  21. [29]

    and Wang, R

    Vovk, V. and Wang, R. (2021). E-values: Calibration, combination and applications. The Annals of Statistics , 49(3):1736--1754

    2021

  22. [30]

    Wang, R. (2023). A tiny review on e-values and e-processes

    2023

  23. [31]

    Wasserman, L., Ramdas, A., and Balakrishnan, S. (2020). Universal inference. Proceedings of the National Academy of Sciences , 117(29):16880--16890

    2020

  24. [32]

    D., Stevens, N

    Wilson, J. D., Stevens, N. T., and Woodall, W. H. (2019). Modeling and detecting change in temporal networks via the degree corrected stochastic block model. Quality and Reliability Engineering International , 35(5):1363--1378

    2019

  25. [33]

    H., Zhao, M

    Woodall, W. H., Zhao, M. J., Paynabar, K., Sparks, R., and Wilson, J. D. (2017). An overview and perspective on social network monitoring. IISE Transactions , 49(3):354--365

    2017

  26. [34]

    and Ramdas, A

    Xu, Z. and Ramdas, A. (2024). Online multiple testing with e-values. In International Conference on Artificial Intelligence and Statistics , pages 3997--4005. PMLR

    2024

  27. [35]

    and Sengupta, S

    Yanchenko, E. and Sengupta, S. (2023). Core-periphery structure in networks: A statistical exposition. Statistic Surveys , 17:42--74

    2023

  28. [36]

    and Sengupta, S

    Yanchenko, E. and Sengupta, S. (2024). A generalized hypothesis test for community structure in networks. Network Science , 12(2):122--138

    2024

  29. [37]

    Yuan, M., Liu, R., Feng, Y., and Shang, Z. (2022). Testing community structure for hypergraphs. The Annals of Statistics , 50(1):147--169

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.