REVIEW 2 major objections 6 minor 37 references
EC-SBM Synthetic Network Generator
T0 review · 2 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash
Pith's one-line read EC-SBM generates synthetic networks that match real communities' internal edge connectivity, with higher fidelity than current SBM-based simulators.
desk verdict A useful new synthetic-network generator with a clean theoretical core, but the headline accuracy comparison is undercut by selection on the test corpus. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the GEN-KECSSN procedure that generates a $k$-edge-connected spanning subnetwork on a cluster: it starts with a $(k+1)$-clique on the first $k+1$ vertices and then connects every later vertex to $k$ previously placed vertices. The theorem that every edge cut in the resulting graph has size at least $k$ is what converts the empirical minimum-cut values of the input clusters into a lower bound on synthetic cluster connectivity. Around this core, EC-SBM layers a stochastic block model (SBM) to fill in the remaining intra- and inter-cluster edges, a simplification step that collapses parallel edges and self-loops, a separate SBM pass for the outlier subnetwork, and a final degree-correction pass that adds edges so each vertex reaches its target degree.
What would settle it
Take any network from the 74-network corpus, generate an EC-SBM twin, and rerun the evaluation after deleting the GEN-KECSSN stage; if the eight statistics barely change, the connectivity mechanism is not load-bearing. Separately, run a standard community detection algorithm on both the real and synthetic networks and compare recovery of the planted clusters; if recovery patterns diverge sharply, the eight-statistic match is not sufficient for benchmarking.
Extended reading notes
Core claim
On its own terms, the paper's discovery is that realistic synthetic networks for community detection can be produced by a three-stage pipeline that treats cluster connectivity as a hard constraint rather than an emergent property. In Stage 1, for each cluster EC-SBM builds a spanning subgraph with edge connectivity at least the cluster's empirical minimum cut, using a procedure that starts from a $(k+1)$-clique and attaches each remaining vertex to $k$ earlier vertices; a theorem shows the result is $k$-edge-connected. SBM then adds the remaining edges, parallel edges and self-loops are removed, and a final degree-correction stage restores the target degree sequence. Over a corpus of 74 real networks clustered in several ways, EC-SBM with the SBM+WCC input clustering was the best overall at matching the eight statistics, beating RECCS on degree sequence, mixing parameters, and characteristic time, while being only slightly behind RECCS on cluster edge connectivity. On three networks with roughly 3 to 14 million nodes, EC-SBM completed in about one to five hours.
Load-bearing premise
The load-bearing premise is that matching the eight measured statistics, including edge connectivity and degree sequence, makes a synthetic network a valid stand-in for a real network when benchmarking community detection; the paper does not itself test any clustering algorithm on its generated networks.
Editorial extensions
If this is right
- Benchmark designers can generate synthetic graphs whose ground-truth communities are connected by construction, removing an artifact of plain SBM where 30 to 70 percent of clusters come out internally disconnected.
- The recommended recipe is to cluster the reference network with SBM followed by well-connected-clusters post-processing, since that input clustering gave EC-SBM its best fidelity.
- EC-SBM's runtime of hours on million-node networks makes it a feasible default where deep graph generators run out of memory or require excessive training data.
- For users whose priority is exact cluster edge connectivity, RECCS still has a slight edge, so the choice between generators depends on whether degree sequence and mixing fidelity or edge connectivity is more important.
Reading between the lines
- A natural next test is to use EC-SBM networks as benchmark data for community detection and check whether algorithms' relative rankings match those obtained on real networks; the paper leaves this unexplored.
- Because the $k$-edge-connected subgraph construction is independent of SBM, the same idea could be grafted onto other generators, such as LFR or ABCD, to enforce connected ground-truth clusters.
- The slight trade-off between EC-SBM and RECCS suggests a hybrid that runs EC-SBM and then applies RECCS's final connectivity refinement might dominate both on cluster edge connectivity without losing degree fidelity.
- If the eight-statistic proxy is validated, EC-SBM could also serve as a null model for studying when community detection fails: by varying cluster edge connectivity while holding other statistics fixed, one can isolate the role of internal connectivity.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes EC-SBM, a three-stage synthetic network generator that takes a clustered real-world network as input and produces a synthetic network with the same cluster assignment. Stage 1 constructs for each cluster a spanning subgraph that is guaranteed to have edge connectivity at least the empirical value, Stage 2 generates an outlier subnetwork via SBM, and Stage 3 adds edges to match the input degree sequence. The paper proves (Theorem 1) that the spanning-subgraph construction is k-edge-connected, and it reports an empirical comparison with SBM and RECCS on 74 networks, plus runtime results on three large networks, concluding that EC-SBM is generally more accurate than the alternatives and scales to millions of nodes.
Significance. If the empirical claims hold, EC-SBM would be a useful, scalable tool for generating synthetic networks with ground-truth clusters that preserve cluster edge connectivity and degree sequence, addressing a known weakness of SBM-based generators. The paper contributes a public code repository, a clear theorem with a sound proof for the core construction, and a substantial corpus (74 networks) for evaluation. However, the significance is tempered by two methodological issues discussed below: the comparison protocol selects configurations on the same test data used for the final evaluation, and several of the reported accuracy metrics are enforced by construction rather than independently validated. The paper is also candid in Section 5 that it does not test clustering algorithms on the generated networks, which limits the direct support for the stated benchmarking motivation.
major comments (2)
- [§4.1–4.3] The protocol for selecting input clusterings creates a selection-on-test-data problem. Experiment 1 uses the same 74 networks to choose SBM+WCC and Leiden-Mod+CM as the best clusterings (via a qualitative reading of Figure 5), and Experiment 2 then reports the generator comparison on those same 74 networks using exactly these clusterings (Figures 6–7). Because the choice of which clusterings to report was informed by the outcome on this corpus, the comparative claim that EC-SBM is 'generally more accurate' is not supported as a general statement; it is at best a description of this corpus under a post-hoc selected configuration. The paper reports only medians in Figures 6–7 and provides no error bars, confidence intervals, or significance tests, so the magnitude and reliability of the differences are unclear. A valid protocol would either select clusterings on a training subset and evaluate on a held-out subset, or report results across all ten clusterings with appropriate multiple-testing awareness.
- [§3, §6.2, Table 1, Figure 7] The evaluation treats degree sequence, outlier degree, and mincuts as measures of accuracy, but these are partly enforced by construction. Stage 1 (Section 6.2.2) explicitly constructs a spanning subgraph that is λ_i-edge-connected for each cluster, and Stage 3 adds edges to match the input degree sequence. Consequently, the near-zero RMSE values for degree and outlier degree, and the lower-bound direction of mincuts in Figure 7, are direct consequences of the algorithm's design rather than independent evidence of fidelity. The manuscript should state this explicitly and base its accuracy claims primarily on statistics that are not directly imposed—such as pseudo-diameter, characteristic time, global clustering coefficient, and mixing parameter—or otherwise justify why enforced properties should count as validation.
minor comments (6)
- [§2.1] The sentence 'We setmicro ers and micro degs to true' appears to have a formatting error; it should read 'We set micro_ers and micro_degs to true'.
- [§4.3] There is a duplicated passage in the text around Figure 5/6: the sentence beginning 'degree sequence. On the other hand, EC-SBM using SBM+WCC input clustering is a well-rounded pipeline...' appears twice, and the flow is broken.
- [References] References [12] and [13] list the same DOI (10.1017/nws.2020.45); the ABCD+o paper should have a distinct DOI.
- [Figure 5] The boxplots in Figure 5 are difficult to read because of the many categories and small labels; consider splitting into separate panels or increasing the font size.
- [§6.2.2] The text first says vertices are 'arbitrarily labeled' and later says the implementation processes them in decreasing order of degree; the proof of Theorem 1 holds for any order, but the exposition should clarify that the degree-ordered variant is an implementation choice, not a requirement of the theorem.
- [Table 2] The column header 'CEN orkut livejournal' lacks separators and is confusing; please format the table with clear per-column headers for each network.
Circularity Check
Degree-sequence and mincut accuracies are enforced by construction; remaining criteria provide independent content.
-
fitted input called prediction
[Section 3, Stage 3; Section 6.2.1]
"In the third stage, we add edges to ˜G′ so that the degree of each vertex matches that of the corresponding vertex in G. This process yields our final synthetic network, G′."
The desired degree sequence is an input parameter, and Stage 3 explicitly adds edges until each vertex degree equals that input. The reported degree-sequence RMSE and outlier-degree RMSE (Table 1; Figs. 6–7) are therefore computed on a quantity the algorithm is guaranteed to match by construction. Presenting these near-zero distances as evidence of accuracy or superiority over other generators is confusing a fitted construction step with an independent result; any generator with this exact degree-correction post-processing would achieve the same degree fidelity by definition.
-
fitted input called prediction
[Section 3, Stage 1; Section 6.2.2 (Theorem 1)]
"we propose a procedure to generate a spanning subgraph on the set of vertices assigned to the cluster with edge connectivity of at least k ... Theorem 1. Let λ(Gn) be the minimum edge-cut size of Gn. Then, λ(Gn) ≥ k."
The edge-connectivity sequence λ is computed from the empirical clustered network and then used as the parameter k to construct each synthetic cluster with edge connectivity at least λ_i. Thus the mincuts criterion is constrained by construction to never fall below the empirical target; only overshooting can contribute to RMSE. The reported accuracy on this cluster-specific statistic is not a test of whether EC-SBM can discover or predict edge connectivity, but a measure of how close an enforced lower bound happens to land. This makes the mincut comparison a fitted-input evaluation rather than an independent prediction.
full rationale
EC-SBM is a reconstruction-oriented generator, so some degree of matching to input statistics is expected; however, two of the headline accuracy criteria are guaranteed by the algorithm itself. The degree sequence (including outlier degrees) is explicitly matched by edge addition in Stage 3, and cluster edge connectivity is enforced to be at least the empirical λ_i by the k-edge-connected spanning subnetwork construction and Theorem 1. Consequently, the near-zero RMSE values for degree, o_deg, and partially mincuts are consequences of construction, not evidence of an independent generative ability. The paper does retain independent content: network-only statistics such as pseudo-diameter, characteristic time, and global clustering coefficient are not directly enforced, and EC-SBM's reported advantages there (and its scalability to millions of nodes) do not reduce to the same fitting steps. I do not count the Experiment 1/Experiment 2 selection-on-test-data protocol as circularity under the defined patterns, but it is a separate validity threat: the best input clusterings are chosen on the same 74 networks used for the final comparison. Overall, the central claim is partially circular because some of the 'high accuracy' evidence is built into the method by definition, while other criteria provide genuine, non-circular support.
Assumptions & free parameters
assumptions (4)
- domain assumption The graph-tool micro-canonical degree-corrected SBM (with micro_ers and micro_degs true) generates multigraphs whose degree sequence and edge count matrix are close to the requested parameters.
- standard math The empirical edge connectivity lambda_i of each cluster is realizable as a simple graph on that cluster's vertex set.
- domain assumption Matching the eight chosen statistics is a sufficient proxy for the usefulness of a synthetic network for evaluating community detection algorithms.
- domain assumption The input clustering produced by Leiden-CPM, Leiden-Mod, or SBM+WCC provides a meaningful ground-truth community structure.
Cite this review
Pith. "Pith review of EC-SBM Synthetic Network Generator." pith.science (2026). https://pith.science/paper/DZBZIVZJ
@misc{pith2026250203662,
author = {Pith},
title = {Pith review of: EC-SBM Synthetic Network Generator},
year = {2026},
howpublished = {\url{https://pith.science/paper/DZBZIVZJ}},
note = {Machine review of arXiv:2502.03662}
}
read the original abstract
Generating high-quality synthetic networks with realistic community structure is vital to effectively evaluate community detection algorithms. In this study, we propose a new synthetic network generator called the Edge-Connected Stochastic Block Model (EC-SBM). The goal of EC-SBM is to take a given clustered real-world network and produce a synthetic network that resembles the clustered real-world network with respect to both network and community-specific criteria. In particular, we focus on simulating the internal edge connectivity of the clusters in the reference clustered network. Our extensive performance study on large real-world networks shows that EC-SBM has high accuracy in both network and community-specific criteria, and is generally more accurate than current alternative approaches for this problem. Furthermore, EC-SBM is fast enough to scale to real-world networks with millions of nodes.
Figures
Reference graph
Works this paper leans on
-
[1]
Publicationes Mathematicae Debrecen 6, 290–297 (1959)
Erd¨ os, P., R´ enyi, A.: On random graphs I. Publicationes Mathematicae Debrecen 6, 290–297 (1959)
work page 1959
-
[2]
Chung, F.R., Lu, L.: Complex Graphs and Networks vol. 107. American Mathe- matical Soc., ??? (2006) 19
work page 2006
-
[3]
Newman, M.E.J., Girvan, M.: Finding and evaluating community structure in networks. Phys. Rev. E 69, 026113 (2004) https://doi.org/10.1103/PhysRevE.69. 026113
-
[4]
Fosdick, B.K., Larremore, D.B., Nishimura, J., Ugander, J.: Configuring random graph models with fixed degree sequences. SIAM Review 60(2), 315–355 (2018) https://doi.org/10.1137/16M1087175 https://doi.org/10.1137/16M1087175
-
[5]
Lancichinetti, A., Fortunato, S., Radicchi, F.: Benchmark graphs for testing com- munity detection algorithms. Phys. Rev. E 78, 046110 (2008) https://doi.org/10. 1103/PhysRevE.78.046110
work page 2008
-
[6]
Danon, L., D ´ ıaz-Guilera, A., Duch, J., Arenas, A.: Comparing community struc- ture identification. Journal of Statistical Mechanics: Theory and Experiment 2005(09), 09008 (2005) https://doi.org/10.1088/1742-5468/2005/09/P09008
-
[7]
Orman, G.K., Labatut, V., Cherifi, H.: On accuracy of community structure discovery algorithms. Journal of Convergence Information Technology 6(11), 283–292 (2011) https://doi.org/10.4156/jcit.vol6.issue11.32
-
[8]
Orman, G.K., Labatut, V., Cherifi, H.: Comparative evaluation of community detection algorithms: a topological approach. Journal of Statistical Mechan- ics: Theory and Experiment 2012(08), 08001 (2012) https://doi.org/10.1088/ 1742-5468/2012/08/P08001
work page 2012
Show all 37 references
-
[9]
International Journal of Web Based Communities 9(3), 349–370 (2013) https://doi.org/10.1504/IJWBC.2013.054908 https://www.inderscienceonline.com/doi/pdf/10.1504/IJWBC.2013.054908
Orman, G.K., Labatut, V., Cherifi, H.: Towards realistic artificial benchmark for community detection algorithms evaluation. International Journal of Web Based Communities 9(3), 349–370 (2013) https://doi.org/10.1504/IJWBC.2013.054908 https://www.inderscienceonline.com/doi/pdf...
2013
-
[10]
Hric, D., Darst, R.K., Fortunato, S.: Community detection in networks: Structural communities versus ground truth. Phys. Rev. E 90, 062805 (2014) https://doi. org/10.1103/PhysRevE.90.062805
2014 doi
-
[11]
Scientific Reports 6(1), 30750 (2016) https://doi.org/10.1038/srep30750
Yang, Z., Algesheimer, R., Tessone, C.J.: A comparative analysis of community detection algorithms on artificial networks. Scientific Reports 6(1), 30750 (2016) https://doi.org/10.1038/srep30750
2016 doi
-
[13]
Applied Network Science 8(1) (2023) https: //doi.org/10.1017/nws.2020.45 20
Kami´ nski, B., Pra lat, P., Th´ eberge, F.: Artificial benchmark for community detection with outliers (ABCD+o). Applied Network Science 8(1) (2023) https: //doi.org/10.1017/nws.2020.45 20
2023 doi
-
[14]
Lancichinetti, A., Radicchi, F., Ramasco, J.J.: Statistical significance of com- munities in networks. Phys. Rev. E 81, 046110 (2010) https://doi.org/10.1103/ PhysRevE.81.046110
2010
-
[15]
Bickel, P.J., Sarkar, P.: Hypothesis testing for automated commu- nity detection in networks. Journal of the Royal Statistical Society Series B: Statistical Methodology 78(1), 253–273 (2015) https: //doi.org/10.1111/rssb.12117 https://academic.oup.com/jrsssb/article- pdf/78/1/...
2015 doi
-
[16]
In: Matsatsinis, N.F., Marinakis, Y., Pardalos, P
Miasnikof, P., Prokhorenkova, L., Shestopaloff, A.Y., Raigorodskii, A.: A sta- tistical test of heterogeneous subgraph densities to assess clusterability. In: Matsatsinis, N.F., Marinakis, Y., Pardalos, P. (eds.) Learning and Intelligent Optimization, pp. 17–29. Springer, Cham (2020)
2020
-
[17]
International Journal of Data Science and Analytics 15(4), 379–390 (2023) https://doi.org/10.1007/ s41060-023-00389-6
Miasnikof, P., Shestopaloff, A.Y., Raigorodskii, A.: Statistical power, accuracy, reproducibility and robustness of a graph clusterability test. International Journal of Data Science and Analytics 15(4), 379–390 (2023) https://doi.org/10.1007/ s41060-023-00389-6
2023
-
[18]
Network Science 12(2), 122–138 (2024) https://doi.org/10
Yanchenko, E., Sengupta, S.: A generalized hypothesis test for community struc- ture in networks. Network Science 12(2), 122–138 (2024) https://doi.org/10. 1017/nws.2024.1
2024
-
[19]
Information Sciences 691, 121649 (2025) https: //doi.org/10.1016/j.ins.2024.121649
He, Z., Li, X., Hu, L., Jiang, M., Liu, Y.: Community structure testing by counting frequent common neighbor sets. Information Sciences 691, 121649 (2025) https: //doi.org/10.1016/j.ins.2024.121649
2025
-
[20]
Albert, R., Barab´ asi, A.-L.: Statistical mechanics of complex networks. Rev. Mod. Phys. 74, 47–97 (2002) https://doi.org/10.1103/RevModPhys.74.47
2002 doi
-
[21]
European Journal of Combinatorics 1(4), 311–316 (1980) https://doi.org/10.1016/S0195-6698(80)80030-8
Bollob´ as, B.: A probabilistic proof of an asymptotic formula for the number of labelled regular graphs. European Journal of Combinatorics 1(4), 311–316 (1980) https://doi.org/10.1016/S0195-6698(80)80030-8
1980 doi
-
[22]
Social Networks 5(2), 109–137 (1983) https://doi.org/10.1016/0378-8733(83) 90021-7
Holland, P.W., Laskey, K.B., Leinhardt, S.: Stochastic blockmodels: First steps. Social Networks 5(2), 109–137 (1983) https://doi.org/10.1016/0378-8733(83) 90021-7
1983 doi
-
[23]
Peixoto, T.P.: Bayesian Stochastic Blockmodeling, pp. 289–332. John Wiley & Sons, Ltd, ??? (2019). Chap. 11. https://doi.org/10.1002/9781119483298.ch11 . https://onlinelibrary.wiley.com/doi/abs/10.1002/9781119483298.ch11
2019 doi
-
[24]
New Journal of Physics 20(5), 052002 (2018) 21
Muscoloni, A., Cannistraci, C.V.: A nonuniform popularity-similarity optimiza- tion (nPSO) model to efficiently generate realistic complex networks with communities. New Journal of Physics 20(5), 052002 (2018) 21
2018
-
[25]
Frank, O., Strauss, D.: Markov graphs. Journal of the American Statistical Associ- ation 81(395), 832–842 (1986) https://doi.org/10.1080/01621459.1986.10478342 https://www.tandfonline.com/doi/pdf/10.1080/01621459.1986.10478342
1986
-
[26]
Holland, P.W., Leinhardt, S.: An exponential family of probability distri- butions for directed graphs. Journal of the American Statistical Associ- ation 76(373), 33–50 (1981) https://doi.org/10.1080/01621459.1981.10477598 https://www.tandfonline.com/doi/pdf/10.1080/01621459.1...
1981
-
[27]
In: Dy, J., Krause, A
You, J., Ying, R., Ren, X., Hamilton, W., Leskovec, J.: GraphRNN: Generating realistic graphs with deep auto-regressive models. In: Dy, J., Krause, A. (eds.) Proceedings of the 35th International Conference on Machine Learning. Proceed- ings of Machine Learning Research, vol. ...
2018
-
[28]
In: Learning on Graphs Conference, pp
Zhu, Y., Du, Y., Wang, Y., Xu, Y., Zhang, J., Liu, Q., Wu, S.: A survey on deep graph generation: Methods and applications. In: Learning on Graphs Conference, pp. 47–1 (2022). PMLR
2022
-
[29]
IEEE Transactions on Pattern Analysis and Machine Intelligence 45(5), 5370–5390 (2023) https://doi.org/10.1109/TPAMI.2022.3214832
Guo, X., Zhao, L.: A systematic survey on deep generative models for graph gen- eration. IEEE Transactions on Pattern Analysis and Machine Intelligence 45(5), 5370–5390 (2023) https://doi.org/10.1109/TPAMI.2022.3214832
2023
-
[30]
In: The Twelfth International Conference on Learning Representations (2024)
Jang, Y., Lee, S., Ahn, S.: A simple and scalable representation for graph gen- eration. In: The Twelfth International Conference on Learning Representations (2024). https://openreview.net/forum?id=nO344avRib
2024
-
[31]
Vaca-Ram ´ ırez, F., Peixoto, T.P.: Systematic assessment of the quality of fit of the stochastic block model for empirical networks. Phys. Rev. E 105, 054311 (2022) https://doi.org/10.1103/PhysRevE.105.054311
2022 doi
-
[32]
figshare (2014) https://doi.org/10
Peixoto, T.P.: The graph-tool python library. figshare (2014) https://doi.org/10. 6084/m9.figshare.1164194 . Accessed 2014-09-10
2014
-
[33]
https: //arxiv.org/abs/2502.02050
Anne, L., Vu-Le, T.-A., Park, M., Warnow, T., Chacko, G.: RECCS: Realistic Cluster Connectivity Simulator for Synthetic Network Generation (2025). https: //arxiv.org/abs/2502.02050
2025 arXiv
-
[34]
https://github.com/vtraag/leidenalg (2019)
Traag, V.: Leiden Algorithm: leidenalg. https://github.com/vtraag/leidenalg (2019)
2019
-
[35]
Scientific Reports 9(1), 5233 (2019) https://doi.org/10
Traag, V.A., Waltman, L., Eck, N.J.: From Louvain to Leiden: guaranteeing well- connected communities. Scientific Reports 9(1), 5233 (2019) https://doi.org/10. 1038/s41598-019-41695-z
2019
-
[36]
PLOS Complex Systems 1(3), 1–25 (2024) https: //doi.org/10.1371/journal.pcsy.0000009
Park, M., Tabatabaee, Y., Ramavarapu, V., Liu, B., Pailodi, V.K., Ramachan- dran, R., Korobskiy, D., Ayres, F., Chacko, G., Warnow, T.: Well-connectedness 22 and community detection. PLOS Complex Systems 1(3), 1–25 (2024) https: //doi.org/10.1371/journal.pcsy.0000009
2024 doi
-
[37]
https://arxiv.org/ abs/2408.10464
Park, M., Feng, D.W., Digra, S., Vu-Le, T.-A., Chacko, G., Warnow, T.: Improved Community Detection using Stochastic Block Models (2024). https://arxiv.org/ abs/2408.10464
2024 arXiv
-
[38]
Zen- odo (2023)
Peixoto, T.P.: The Netzschleuder network catalogue and repository. Zen- odo (2023). https://doi.org/10.5281/zenodo.7839981 . https://doi.org/10.5281/ zenodo.7839981 23
2023 doi
Reviewed August 9, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.