Pith. sign in

REVIEW 2 major objections 6 minor 6 references

Community Detection on a Randomly Growing Network

T0 review · 2 major / 6 minor · reviewed 2026-07-12 · grok-4.5

Pith's one-line read You cannot label every node correctly in a growing preferential-attachment network with noise, but you can perfectly recover the early hubs and cores that define the communities.

desk verdict Solid first rigorous local-recovery theory for planted preferential-attachment forests; impossibility plus SPAR are cleanly scoped and the proofs hold. read the letter →

arxiv 2606.07447 v2 pith:VAMHOTK4 submitted 2026-06-05 stat.ME math.STstat.TH

classification stat.MEmath.STstat.TH MSC 62H3005C8060C05
keywords communitydetectionpreferentialattachmentplantedforestgrowingnetworkslocalrecoverygraphpruninganchorcomponents
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Real networks grow over time and develop power-law degrees, hubs, and chains—features that static block models struggle to produce. This paper studies community detection under a planted-forest model: K independent preferential-attachment trees (the communities) glued together by sparse Erdős–Rényi noise. The authors prove that no algorithm can drive the global misclustering rate to zero once the noise is even moderately dense. The reason is that late-arriving peripheral nodes are information-theoretically ambiguous. They then show that the same growth process leaves recoverable structure among central nodes. Their two-stage procedure, SPAR, iteratively prunes low-degree nodes to isolate high-degree anchor components and then propagates labels by distance (or by model-based posterior). Under mild balance and noise conditions the method recovers the earliest arrivals of every community with zero error, recovers all sufficiently high-degree nodes perfectly, and achieves vanishing relative error on the children (and, for linear attachment, grandchildren) of the roots. Simulations and a hierarchical analysis of a statistician co-authorship network illustrate that the recovered cores align with recognized research communities.

What carries the argument

Selective Pruning with Anchor-based Recovery (SPAR): iterative removal of lowest-degree nodes that lie on cycles or whose deletion increases the number of high-degree components, isolating K anchor cores that seed distance- or posterior-based label propagation.

What would settle it

Generate planted forests with noise probability above the critical degree scale $n^{-(1+\alpha)/(2+\alpha)}$ and check whether SPAR still isolates the true community roots or whether high-degree nodes become randomly mixed across communities.

Watch

Extended reading notes

Core claim

Under the planted forest model with balanced community sizes and noise probability decaying slightly faster than the preferential-attachment degree scale, consistent recovery of every node is information-theoretically impossible, yet a simple pruning-plus-propagation algorithm recovers the labels of early-arriving, high-degree, and near-root nodes with zero or vanishing local misclustering error.

Load-bearing premise

The random noise edges must be sparse enough that the degrees of early nodes still track their arrival order; denser noise erases the signal that pruning relies on.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper studies community detection outside the SBM, under a planted forest model in which K independent affine preferential-attachment trees are superposed with Erdős–Rényi noise. It proves that no estimator can drive the global misclustering rate to zero when θ ≥ c/n (Theorem 1), then proposes SPAR: iterative degree-based pruning to extract K high-degree anchor components, followed by distance- or model-based label propagation. Under balanced community sizes and θ ≤ C0 n^{-(1+α)/(2+α)-δ}, SPAR recovers the first L arrivals of each community with zero local error w.h.p. (Theorem 2), recovers high-degree nodes perfectly (Corollary 1), and achieves vanishing local error on layer-1 nodes and, when α=0, layer-2 nodes (Theorem 3). Simulations across α and noise rates, plus a hierarchical analysis of the Ji–Jin coauthorship network, support the theory; code is public.

Significance. The work fills a genuine gap: community detection with rigorous guarantees for networks formed by sequential growth rather than independent edges. The impossibility result cleanly motivates local recovery, and the local guarantees (early arrivals, high degree, layers 1–2) are non-trivial and matched to preferential-attachment degree asymptotics. SPAR is simple, computationally efficient (O(|V|·|E|·deg(G))), and only loosely tied to the tree assumption, which is a methodological strength. Public code and a coherent real-data hierarchical clustering further raise the contribution. Limitations (known K, layer-2 only for α=0, noise-rate restriction) are scoped honestly and do not undercut the central claims as stated.

major comments (2)
  1. [Section 4, Remarks 4 and 10] Section 4 / Remark 4 and Remark 10: The main theorems treat K as known. Algorithm 5 gives a consistent estimator under the planted forest model, but the authors correctly note that τ2 is hard to tune in practice and that they fall back to hierarchical K=2 on real data. For a methods paper this is load-bearing for applicability: either strengthen the practical guidance for Algorithm 5 (or an alternative) with simulation evidence under misspecification, or state more clearly that hierarchical bipartition is the recommended operational procedure and report sensitivity of the coauthorship hierarchy to the leaf-size cutoff (currently 10).
  2. [Theorem 3, Remark 12, Section 2.4] Theorem 3 and Remark 12: Layer-2 recovery is proved only for α=0, and s≥3 is left open because misclassification accumulates across layers. The conjecture |Ls| ≍ n^{1/(2+α)} log^{s-1} n is plausible but unproved. This does not invalidate the stated theorems, but the abstract and introduction advertise recovery for “subsets of central nodes” including structural layers; a short, explicit statement of the α=0 restriction for layer 2 (and the barrier for s≥3) in the main-text overview (Section 2.4) would prevent over-reading of the guarantees.
minor comments (6)
  1. [Assumption 2, Table 4] Assumption 2: The noise threshold n^{-(1+α)/(2+α)} is well motivated by degree scales, and simulations (Table 4) suggest δ may be zero at the critical rate. A one-sentence conjecture in Section 4 or 7 would be useful for follow-up work.
  2. [Section 3.1, Algorithm 1] Algorithm 1: Degree is always computed w.r.t. the original Gn (not updated). This is justified for analysis (Remark after Algorithm 1); a brief note on whether updating degrees changes empirical performance would help practitioners.
  3. [Section 5, Figures 4–6] Figures 4–6 and S2–S4: Boxplots are informative but dense; ensuring consistent y-axis scales within each α panel (or adding a short caption note) would improve readability.
  4. [Definition 2, Theorem 2] Notation: The dual use of π for global and community-specific orderings (Definition 2) is clear once stated, but a single reminder when π1:L(T^i) first appears in Theorem 2 would help.
  5. [Section 1.1] Related work: The distinction from Ben-Hamou–Velona and Hajek–Sankagiri (class-dependent attachment vs. disjoint growth; known vs. unknown arrival times) is well drawn; a sentence on how SPAR relates to leaf-stripping root-finding (Addario-Berry et al. 2025) beyond the brief mention would complete the picture.
  6. [Throughout] Typos / polish: “affine” / “Erdös” spelling is inconsistent in places (e.g., “Erdös–Rényi” vs “Erdős–Rényi”); “difficult” appears with ligature issues in the PDF source. Standardize.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: local recovery guarantees are derived from the planted-forest generative assumptions plus external preferential-attachment degree asymptotics, not by construction from fitted targets or load-bearing self-citation.

full rationale

The central claims (global impossibility under θ=Ω(1/n); zero/vanishing local misclustering on early-arriving, high-degree and layer-1/2 nodes under the stated noise and balance assumptions) are proved from first principles inside the paper. Theorem 1 uses an information-theoretic indistinguishability argument via balanced-attachment events At (Appendix S1). Theorems 2–3 and Corollary 1 reduce the SPAR pruning-plus-distance procedure to known almost-sure degree-order limits for affine preferential attachment (Sénizergues 2021; Rudas et al. 2007; Lemmas 5–7, 14–17) together with elementary noise-control arguments under Assumption 2; the proofs never redefine the target error rates in terms of the algorithm outputs. Self-citations to Crane & Xu (2021, 2024) supply only the model name, the EM estimator for α, and the Gibbs sampler used in the optional model-based recovery stage; they are not invoked to justify uniqueness, to smuggle an ansatz, or to force the recovery rates. Free tuning parameters (τ, Q, τ′) appear in the algorithm statement and in simulations but are not used to define or fit the asymptotic error bounds. Consequently the derivation chain is self-contained against external benchmarks and exhibits only ordinary, non-load-bearing self-citation.

Assumptions & free parameters 4 free parameters · 6 assumptions · 3 invented entities

Theory rests on the planted-forest generative model (K APA trees + ER noise), balanced sizes, a noise-rate upper bound that preserves degree–arrival correlation, known K for main theorems, and classical APA degree/layer asymptotics. Algorithmic free parameters (τ, Q, τ′, α estimate) affect practice; the model itself invents the planted-forest community notion as the analysis object.

free parameters (4)
  • anchor-degree threshold τ (and recovery threshold τ′)
    Chosen via rank-calibrated simulation (Algorithm 4) or as degree of the 10th highest-degree node in experiments; theory only needs τ of order n^{1/(2+α)} below a constant C1(ε,H).
  • anchor-size threshold Q (and shrinkage ζ)
    Theory allows fixed Q (even Q=1); practice uses Q≈n/10 with ζ=0.8 shrinkage when pruning fails.
  • attachment parameter α and noise θ
    α estimated by EM from Crane & Xu; θ enters Assumption 2. In simulations they are set by design; on real data α̂=0 is plugged in.
  • community count K
    Main theorems assume K known; hierarchical K=2 is used on real data. Algorithm 5 gives a theoretical estimator needing carefully tuned τ1>τ2.
assumptions (6)
  • domain assumption Communities are generated as independent affine preferential-attachment trees (Definition 1–2), with labels inherited from the tree of attachment.
    Core modeling choice; tree shape is called a parsimony simplification (Section 2.2).
  • domain assumption Between-community edges are i.i.d. Erdős–Rényi with probability θ (Definition 2).
    Noise model used for both impossibility and recovery analyses.
  • domain assumption Assumption 1: each community size nk ≥ n/H for fixed H ≥ K (balanced communities).
    Required for Theorems 2–3; unbalanced simulations show worse error (Section S3).
  • domain assumption Assumption 2: θ ≤ C0 n^{-(1+α)/(2+α)-δ} for some δ>0.
    Noise must not erase preferential-attachment degree signal of early nodes.
  • standard math APA degree asymptotics: n^{-1/(2+α)} degrees of ordered nodes converge to a random sequence (Yj,α) in ℓq (Sénizergues; Lemmas 14–17).
    Used to link early arrivals to high-degree sets and set τ scales.
  • standard math For α=0, E|Ls(Tn)| ≍ √n (log n)^{s-1} (Lemma 11).
    Supports layer-2 error rates; general-α layer sizes for s≥2 are conjectural (Remark 12).
invented entities (3)
  • Planted forest model PF(α,θ,ℓ,π) / RC-PF
    purpose: Formal generative model for multi-community preferential-attachment growth plus ER noise.
    Named and fixed-label version of multi-root PAPER; communities are disjoint growth processes rather than class-dependent attachment probabilities.
  • SPAR (Selective Pruning with Anchor-based Recovery) independent evidence
    purpose: Two-stage algorithm: iterative cycle/split-aware degree pruning to K anchors, then distance or model-based label propagation.
    Main methodological contribution; guarantees are local to central node sets.
  • Anchor components / high-degree cores after pruning
    purpose: Operational seeds for community labels after removing low-degree bridges.
    Defined by Algorithm 1 outputs; theory shows they contain early high-degree nodes under the model.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Community Detection on a Randomly Growing Network." pith.science (2026). https://pith.science/paper/VAMHOTK4

@misc{pith2026260607447,
  author       = {Pith},
  title        = {Pith review of: Community Detection on a Randomly Growing Network},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VAMHOTK4}},
  note         = {Machine review of arXiv:2606.07447}
}
abstract

We study community detection on Markovian random networks outside of the Stochastic Block Model (SBM) framework. Specifically, we consider a random network growth process which generates $K$ separate preferential attachment trees and connects them with Erd\H{o}s--R\'enyi edges, so that each tree represents a community and each node inherits the label of the tree to which it belongs. This model is able to produce many features of real world networks that are improbable under SBM, such as power law degree distribution and the existence of chains and hubs. Given only the final graph, without any knowledge of the growth process, we seek to recover the unobserved community membership of the nodes. We first prove that it is impossible for any algorithm to consistently recover the community label of all the nodes. However, we design algorithms which are provably able to recover the community labels of subsets of central nodes, for several different notions of node centrality such as arrival time or degree. Our procedure consists of two stages where, in the first stage, we classify high degree nodes and then, in the second stage, extend the community assignments to the remaining vertices. Numerical experiments and a real data application on a coauthorship network demonstrate the effectiveness of our proposed approach.

Figures

Figures reproduced from arXiv: 2606.07447 by the authors.

Figure 1
Figure 1. Co-authorship network from Ji and Jin (2016). (a) Observed graph topology, where vertices correspond to authors and edges represent co-authorship relations. (b) Sub-network consisting of all nodes within graph distance two of Lawrence D. Brown or David Dunson. (c) Degree distribution of the network. For readability, authors with degrees exceeding 15 are omitted. The growth process of a network leaves distinctive top… view at source ↗
Figure 1
Figure 1. Distribution of adoption metrics for newer projects [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Forest community detection via Algorithm [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗
Figures from the paper (27 more)
Figure 2
Figure 2. Figure 2: Distribution of adoption metrics for older projects [PITH_FULL_IMAGE:figures/full_fig_p004_2.png]
Figure 3
Figure 3. Figure 3: Forest community detection results using Algorithm [PITH_FULL_IMAGE:figures/full_fig_p015_3.png]
Figure 3
Figure 3. Figure 3: Distribution of commit ratio by language for newer projects [PITH_FULL_IMAGE:figures/full_fig_p008_3.png]
Figure 4
Figure 4. Figure 4: Boxplots of misclassification rates for the first 50 arriving vertices in each tree, [PITH_FULL_IMAGE:figures/full_fig_p021_4.png]
Figure 4
Figure 4. Figure 4: Distribution of commit ratio by language for older projects [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Boxplots of misclassification rates for the 50 highest degree vertices in the network, [PITH_FULL_IMAGE:figures/full_fig_p022_5.png]
Figure 5
Figure 5. Figure 5: Topic-stratified analysis of commit adoption ratios, comparing adoption patterns across repository [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Boxplots of misclassification rates for the layer-1 vertices in the network, computed over [PITH_FULL_IMAGE:figures/full_fig_p022_6.png]
Figure 6
Figure 6. Figure 6: Topic-stratified analysis of commit adoption ratios, comparing adoption patterns across repository [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Community cores obtained from the co-authorship network of [PITH_FULL_IMAGE:figures/full_fig_p024_7.png]
Figure 7
Figure 7. Figure 7: Cumulative adoption of any tool across all 8 562 projects from 2025-01-05 to 2026-04-08. [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Communities obtained by the distance-based recovery procedure in Algorithm [PITH_FULL_IMAGE:figures/full_fig_p025_8.png]
Figure 8
Figure 8. Figure 8: Cumulative adoption of any tool across all 32 134 projects from 2025-01-01 to 2026-04-05. [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: Communities obtained by the Monte Carlo sample recovery procedure in Algorithm [PITH_FULL_IMAGE:figures/full_fig_p025_9.png]
Figure 9
Figure 9. Figure 9: Ridgeline plot showing adoption patterns for the 47 tools, with Claude [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Hierarchical dendrogram from iterative core pruning [PITH_FULL_IMAGE:figures/full_fig_p026_10.png]
Figure 10
Figure 10. Figure 10: Ridgeline plot showing adoption patterns for the 56 tools, with Claude [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: Communities obtained by distance recovery in the hierarchical procedure (Part I) 27 [PITH_FULL_IMAGE:figures/full_fig_p027_11.png]
Figure 11
Figure 11. Figure 11: Adoption patterns across the top 35 organizations, with dicklesworthstone leading at 110 adop [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]
Figure 12
Figure 12. Figure 12: Communities obtained by distance recovery in the hierarchical procedure (Part II) [PITH_FULL_IMAGE:figures/full_fig_p028_12.png]
Figure 12
Figure 12. Figure 12: Adoption patterns across the top 35 organizations, with microsoft leading at 740 adoptions. [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]
Figure 13
Figure 13. Figure 13: Forest community detection via Algorithm [PITH_FULL_IMAGE:figures/full_fig_p038_13.png]
Figure 13
Figure 13. Figure 13: Boxplot comparison of added lines, deleted lines, and total files across human, bot, and AI-assisted [PITH_FULL_IMAGE:figures/full_fig_p015_13.png]
Figure 14
Figure 14. Figure 14: Boxplots of misclassification rates for the first 50 arriving vertices in three unbalanced [PITH_FULL_IMAGE:figures/full_fig_p044_14.png]
Figure 14
Figure 14. Figure 14: Boxplot comparison of added lines, deleted lines, and total files across human, bot, and AI-assisted [PITH_FULL_IMAGE:figures/full_fig_p015_14.png]
Figure 15
Figure 15. Figure 15: Boxplots of misclassification rates for the 50 highest degree vertices in the network [PITH_FULL_IMAGE:figures/full_fig_p045_15.png]
Figure 16
Figure 16. Figure 16: Boxplots of misclassification rates for the layer-1 vertices in the network consists of [PITH_FULL_IMAGE:figures/full_fig_p046_16.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

6 extracted references · 2 linked inside Pith

  1. [1]

    Cerqueira and E

    A. Cerqueira and E. Levina. A pseudo-likelihood approach to community detection in weighted networks. arXiv preprint arXiv:2303.05909 ,

  2. [2]

    C. Xin, F. Xu, X. Ding, J. Gao, and J. Ding. Toping: Topologically interpretable graph learning via persistent rationale filtration. arXiv preprint arXiv:2510.05102 ,

  3. [3]

    Yanchenko, S

    E. Yanchenko, S. Sengupta, and D. Mukherjee. Statistical inference for core-periphery structures. arXiv preprint arXiv:2508.04730 ,

  4. [4]

    If the graph parameters (α, θ) are assumed to be unknown, the noise level θ can still be easy to estimate, e.g

    Algorithm 4 assumes the graph parameters (α, θ) are available. If the graph parameters (α, θ) are assumed to be unknown, the noise level θ can still be easy to estimate, e.g. by ˆθ = |E(Gn)|−n+1 |E(Gn)| . By contrast, α can be harder to estimate. A principled estimation procedure is given in the supplementary material of Crane and Xu (2024). In practice, ...

  5. [5]

    This completes the proof of ( S4.41)

    ε. This completes the proof of ( S4.41). S4.5 Technical Lemmas Lemma 13 (Theorem 2 in Rudas et al. (2007)). For Tn APA(α, n) with α >1, as n!1 , v : degTn (v) = 1 n P!p (1) > 0, that is, the proportion of degree- 1 vertices converges in probability to a positive constant p(1) that depends on α. Lemma 14 (Proposition 5.5 in Sénizergues (2021)). Let q > 2 +...

  6. [6]

    Let X be a random variable with Bin(n, θ) distribution

    65 Lemma 18 (Lemma S8 in Crane and Xu (2024)). Let X be a random variable with Bin(n, θ) distribution. For any q 1, θ2 [0, 1] and any n2 N, we have that EX q (2θn)q + Cq, where Cq > 0 is a constant that depends only on q. Proof. Write X as a random variable with the Bin(n, θ) distribution. Then, EX q = Z ∞ 0 P (X q t) dt (2θn)q + Z ∞ (2θn)q P (X q t) dt (...

Pith tools

Reviewed July 12, 2026 · model on record in the stance chip above.