REVIEW 4 major objections 7 minor 51 references
Experimental Analysis and Evaluation of Cohesive Subgraph Discovery
T0 review · 4 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read This paper claims the first systematic evaluation of 14 cohesive subgraph models, finding that truss-based models are denser, core-based models are larger and more interpretable, and SCAN performs best on community search.
desk verdict A genuinely useful first benchmark of 14 cohesive subgraph models, with a load-bearing parameter-calibration flaw that a serious revision can fix. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The argument is carried by a unified evaluation harness rather than by a single theorem. Fourteen algorithms are brought into one codebase with a common interface, and a default parameter grid (Table 6) assigns each model four cohesiveness levels so that every model is scored on the same thresholds. Quality is measured by seven metrics—average degree, cut ratio, clustering coefficient, edge density, inverse conductance, average connected component size, and modularity—computed in two ways: locally, averaging over each connected component, and globally, treating all returned nodes as one subgraph. Synthetic graphs are generated by varying five structural parameters, and real graphs range from 34 to roughly 4 million nodes, grounding the comparison in both controlled and natural settings. This machinery turns qualitative model categories (core, triangle, clique, connected-component, and other) into measurable rankings.
What would settle it
Re-run the same 14 models on the same datasets with per-dataset parameter tuning, for instance choosing each model's thresholds to maximize modularity or community-search accuracy on held-out ground truth, and check whether the qualitative ranking still holds: truss and combined models denser, core models larger, SCAN best for community search. If the ordering flips by dataset, the uniform-grid choice was the load-bearing decision.
Extended reading notes
Core claim
On its own terms, the paper claims to provide the first comprehensive evaluation of 14 existing cohesive subgraph models, all implemented in one project and run on 12 real-world networks plus 20 synthetic benchmark graphs with varied generative parameters. Across seven evaluation metrics computed at local and global levels, the experimentally supported finding is that truss-based models (k-truss, k-tripeak) and combined models (k-core-truss, (k,s)-core) consistently produce more cohesive subgraphs than core-based models, while core-based models (k-core and variants) produce larger subgraphs that are easier for users to interpret. The paper also reports that Alphacore and SCAN produce more cohesive results, and that for the community search downstream task, graph clustering algorithms such as SCAN perform best, with Alphacore, k-distance clique, k-core, and (k,s)-core giving comparable accuracy. A final recommendation is that when a cohesive subgraph model is used only as an initial solution, truss-based or combined approaches are the preferred lower bound, whereas core-based approaches are suitable when there is room for downstream refinement.
Load-bearing premise
The comparison's fairness rests on one hand-picked parameter grid (Table 6) being applied uniformly to all datasets, and the paper itself concedes that some results may not be well-tuned or may be missing when an algorithm returned nothing.
Editorial extensions
If this is right
- If the findings hold, users who value interpretability should start with core-based models such as k-core or (k,p)-core.
- Users who value cohesion should prefer truss-based or combined models such as k-truss, k-tripeak, k-core-truss, or (k,s)-core.
- For community search initialization, graph clustering models such as SCAN give the most accurate starting communities, while truss-based or combined models are the recommended lower-bound initial solution.
- Ratio-based models such as SCAN and (k,p)-core behave differently as network size grows, so conclusions drawn on small graphs should not be assumed to transfer to million-node networks.
- Parameter selection remains the main practical obstacle; the paper's uniform grid shows rankings but does not remove the need for per-dataset tuning.
Reading between the lines
- The same experimental harness could be extended to weighted, directed, signed, bipartite, or hypergraph settings; the core-versus-truss trade-off would likely reappear, but the specific rankings would need recalibration.
- A testable extension is to compare model rankings on a larger set of ground-truth community datasets with per-dataset tuned parameters, to see whether SCAN's community-search advantage persists outside the two datasets used here.
- The local-versus-global evaluation split suggests that any single-number benchmark can be misleading; rankings on edge density or conductance can change simply by averaging per component instead of aggregating, so future benchmarks should report both.
- The interpretability-versus-cohesion framing points toward a design space of hybrid models that interpolate between degree and support constraints, where the paper's comparison could serve as the baseline.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper presents an experimental benchmark of fourteen existing cohesive subgraph discovery models on simple undirected unweighted graphs. The models are organized into five families (core-based, triangle-based, clique-based, connected-component-based, and other), and are evaluated on twenty LFR synthetic graphs, twelve real-world networks, seven evaluation metrics, and a community-search downstream task. The paper's headline conclusions are that truss-based and combined models produce more cohesive subgraphs, core-based models produce larger and more interpretable subgraphs, and graph-clustering approaches such as SCAN perform best for community search. The authors also release unified implementations of the algorithms.
Significance. If the benchmark is reliable, this is a useful service contribution to the cohesive-subgraph community: it brings fourteen models under one implementation, covers a wide range of network scales and synthetic configurations, and attempts a task-based comparison rather than a purely structural one. The public code and the transparency about null results are commendable. However, the central comparative claims rest on hand-chosen parameter grids whose cross-model comparability is not established, and several metric formulas in Section 4.1 are garbled as typeset, so the quantitative basis for the rankings needs substantial repair before the conclusions can be taken at face value. The contribution is still valuable as a broad qualitative survey, but the 'first comprehensive evaluation' framing should be softened unless the authors can document the absence of comparable prior benchmarks.
major comments (4)
- [Section 4.1, Table 6] The parameter grids in Table 6 are not placed on any common scale, and this is load-bearing for the family-level rankings. For example, k-core runs at k=3,5,7,9 while k-truss runs at k=4,6,8,10, and (k,p)-core and SCAN use ratio thresholds at fixed k. Nothing in the paper establishes that q1 for one model corresponds to q1 for another in a comparable sense of 'cohesiveness level.' The observation that truss-based outputs are denser and smaller could simply mean that the selected truss thresholds were relatively stricter than the selected core thresholds. The paper itself concedes in Section 4.1 that 'some of our experimental results may not have been well-tuned or may not have been reported in cases where the algorithm returned null results,' which introduces a selection bias: a model that returns null at high q is silently absent from the cohesive end of the ranking. I recommend reporting the full result matrix including nulls, and adding a sensitivity analysis or a threshold-matching procedure (for example, matching outputs by size or density across model families) before drawing intrinsic-comparison conclusions.
- [Section 4.1, Global-level evaluation metrics] Several metric formulas are garbled as typeset, which is serious because these metrics are the basis for all quantitative conclusions. The Cut Ratio formula is written as 1 - sum_{i in H, j in \bar H} 1 / (|V[H]| * |V[G\H]|), which as written has no clearly defined summand and is dimensionally wrong; the standard cut ratio should be the size of the edge cut divided by |V[H]| * |V[G\H]|. The Inverse Conductance formula is written as 1 - sum_{i in H, j in \bar H} 1 / min(d_H, d_{\bar H}), which again omits the number of crossing edges and the correct volume denominators. The local-level Modularity expression l_c/|E| - d_c^2/(4|E|^2) does not match the standard Newman-Girvan modularity and is not derived or justified. The authors should provide correct, unambiguous formulas for every metric, preferably in a table with a reference to the standard definition.
- [Section 4.2, Synthetic networks and Table 4] The synthetic experiments use only one LFR instance per parameter combination: for each of the five parameter types, four values are tested, but there is no replication across seeds and no error bars or variance reporting in Figures 18-24. Given that the qualitative claims are about differences between model families, a single instance per setting cannot establish that observed gaps are not due to LFR sampling noise. The authors should either generate multiple LFR realizations per parameter point and report means with variance, or explicitly state that the figures show single-instance behavior and restrict the conclusions to those instances.
- [Section 4.2.4 and Section 4.3.2, Community search] The community-search evaluation treats every node in every identified cohesive subgraph as a query node and then measures NMI, ARI, and F1 on Karate and Polblogs. This protocol is fragile: on a 34-node graph and a 1,224-node graph, the scores are averaged over many overlapping queries, and the paper itself notes that a high accuracy score 'does not always indicate a good model for community search' because it may leave little room for improvement. The claim that SCAN 'performs best' for community search therefore needs a clearer baseline: what is the performance of simply returning the entire graph, and what is the performance of a dedicated community-search algorithm initialized by these cohesive subgraphs? Without such a baseline, the downstream-task conclusion is not yet supported.
minor comments (7)
- [Section 2] The sentence 'The notation decomposition is the process that obtaining information about all cohesive subgraphs in a graph' is ungrammatical and should be rewritten; also define 'decomposition' and 'computation' explicitly before contrasting them.
- [Section 3.1, Equation (1)] The Mahalanobis depth formula in Definition 21 is typeset with a stray superscript and an unclear covariance-matrix symbol: '(x - mu_F)^T -1 sum_F (x - mu_F)' should be '(x - mu_F)^T Sigma_F^{-1} (x - mu_F)' with Sigma_F defined as the covariance matrix.
- [Section 3.2, Example 5] The sentence 'with k = 3, the entire graph is returned by the k-truss returns the whole graph' contains a duplicated verb and should be corrected.
- [Section 3.5, Example 11] The colors are described as 'green-coloured and pick-coloured'; 'pick' should be 'pink', and the figure should be legible in grayscale if the journal version does not guarantee color printing.
- [Section 4.3.1] The text says 'we utilise the global metrics which are presented in Table 4.1', but there is no Table 4.1; the metrics are described in Section 4.1, not in a numbered table.
- [Throughout] The arXiv PDF retains ACM conference-formatting placeholders such as 'Conference acronym XX, June 03-05, 2018, Woodstock, NY' and a DOI placeholder; these should be cleaned up before public dissemination as a journal article.
- [Section 4.1, Table 4] The synthetic graph types are described clearly, but the default values (N=1,000, average degree=20, max degree=200, mu=0.2, t1=2, t2=1) are stated only in the caption of Table 4; consider making them part of the main text so the experimental design is self-contained.
Circularity Check
No significant circularity: the paper is an experimental benchmark of existing cohesive subgraph models against external metrics and ground truth, with no fitted quantity renamed as a prediction.
full rationale
This paper is a comparative evaluation of 14 existing cohesive subgraph models. No new model or theorem is derived, and no parameter is fitted to a target outcome and then reported as a prediction. The central findings — truss-based and combined models yield more cohesive subgraphs, core-based models yield larger and more interpretable subgraphs, and SCAN performs best on community search — are obtained by running published algorithms under the parameter grids of Table 6 and measuring them with standard external metrics such as modularity, edge density, inverse conductance, NMI, ARI, and F1-score. The ground-truth community search evaluation uses Karate and Polblogs, which are external datasets not constructed by this paper. None of the paper's claims reduces to its own definitions by construction: for example, the observation that truss-based outputs are denser is an empirical comparison, not an identity between a model definition and a metric. The paper does cite its own authors' prior work (e.g., references [20], [21], [22], [23]), but these citations appear only as related work on hypergraphs, bipartite networks, signed networks, and community search, and they are not load-bearing for the experimental conclusions. The acknowledged limitation that some results 'may not have been well-tuned or may not have been reported in cases where the algorithm returned null results' (Section 4.1) is a genuine concern about parameter fairness and external validity, but it is a correctness or robustness issue, not a circularity issue: tuning choices affect whether the comparison is fair, not whether the derivation is equivalent to its inputs. The skeptically noted mismatch between parameter scales across model families likewise affects whether the qualitative rankings are conclusive, but it does not make the benchmark circular, because the models are still evaluated against externally defined quality metrics. Therefore the appropriate finding is no significant circularity.
Assumptions & free parameters
free parameters (2)
- Algorithm parameter grids (q1-q4) for the 14 models =
Examples: k-core k=3,5,7,9; k-truss k=4,6,8,10; Alphacore alpha=0.2,0.4,0.6,0.8; SCAN (k,eps) in…
- LFR synthetic graph settings =
N=1000, average degree 10-40, max degree 100-400, mixing 0.2-0.8, degree exponent 1.5-3, community exponent 1-2.5
assumptions (4)
- domain assumption Input graphs are simple, undirected, and unweighted
- domain assumption Ground-truth communities in LFR, Karate, and Polblogs are valid references for quality
- domain assumption The referenced implementations (NetworkX, k-peak wrapper, Alphacore code) behave as documented
- ad hoc to paper Mahalanobis depth computed with the Moore-Penrose pseudo-inverse is a valid adaptation of Alphacore for undirected unweighted graphs
Cite this review
Pith. "Pith review of Experimental Analysis and Evaluation of Cohesive Subgraph Discovery." pith.science (2026). https://pith.science/paper/VZ7QBAN3
@misc{pith2026250710262,
author = {Pith},
title = {Pith review of: Experimental Analysis and Evaluation of Cohesive Subgraph Discovery},
year = {2026},
howpublished = {\url{https://pith.science/paper/VZ7QBAN3}},
note = {Machine review of arXiv:2507.10262}
}
read the original abstract
Retrieving cohesive subgraphs in networks is a fundamental problem in social network analysis and graph data management. These subgraphs can be used for marketing strategies or recommendation systems. Despite the introduction of numerous models over the years, a systematic comparison of their performance, especially across varied network configurations, remains unexplored. In this study, we evaluated various cohesive subgraph models using task-based evaluations and conducted extensive experimental studies on both synthetic and real-world networks. Thus, we unveil the characteristics of cohesive subgraph models, highlighting their efficiency and applicability. Our findings not only provide a detailed evaluation of current models but also lay the groundwork for future research by shedding light on the balance between the interpretability and cohesion of the subgraphs. This research guides the selection of suitable models for specific analytical needs and applications, providing valuable insights.
Figures
Figures from the paper (18 more)
Reference graph
Works this paper leans on
-
[1]
Lada A Adamic and Natalie Glance. 2005. The political blogosphere and the 2004 US election: divided they blog. In Proceedings of the 3rd international workshop on Link discovery. 36–43
work page 2005
-
[2]
Richard D Alba. 1973. A graph-theoretic definition of a sociometric clique. Journal of Mathematical Sociology 3, 1 (1973), 113–126
work page 1973
-
[3]
Albert-László Barabási. 2013. Network science. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 371, 1987 (2013), 20120375
work page 2013
-
[4]
Vladimir Batagelj, Andrej Mrvar, and Matjaž Zaveršnik. 1999. Partitioning approach to visualization of large graphs. In International Symposium on Graph Drawing. Springer, pringer, Berlin, Heidelberg, 90–97
work page 1999
-
[5]
V . Batagelj and M. Zaversnik. 2003. An O(m) Algorithm for Cores Decomposition of Networks. Advances in Data Analysis and Classification, 2011. V olume 5, Number 2, 129-145. (2003). arXiv:arXiv:cs/0310049
arXiv 2003
-
[6]
Francesco Bonchi, Arijit Khan, and Lorenzo Severini. 2019. Distance-generalized core decomposition. In SIGMOD. ACM, New York, NY , USA, 1006–1023
work page 2019
-
[7]
Coen Bron and Joep Kerbosch. 1973. Algorithm 457: finding all cliques of an undirected graph. Commun. ACM 16, 9 (1973), 575–577
work page 1973
-
[8]
Deming Chu, Fan Zhang, Xuemin Lin, Wenjie Zhang, Ying Zhang, Yinglong Xia, and Chenyi Zhang. 2020. Finding the best k in core decomposition: A time and space optimal solution. In ICDE. IEEE, 685–696
work page 2020
Show all 51 references
-
[9]
Shumo Chu and James Cheng. 2011. Triangle listing in massive networks and its applications. In SIGKDD. 672–680
2011
-
[10]
Jonathan Cohen. 2008. Trusses: Cohesive subgraphs for social network analysis. National security agency technical report 16, 3.1 (2008), 1–29
2008
-
[11]
Maximilien Danisch, Oana Balalau, and Mauro Sozio. 2018. Listing k-cliques in sparse real-world graphs. In WWW. 589–598
2018
-
[12]
Yixiang Fang, Reynold Cheng, Xiaodong Li, Siqiang Luo, and Jiafeng Hu. 2017. Effective community search over large spatial graphs. PVLDB 10, 6 (2017), 709–720
2017
-
[13]
Yixiang Fang, Reynold Cheng, Siqiang Luo, and Jiafeng Hu. 2016. Effective community search for large attributed graphs. PVLDB 9, 12 (2016), 1233–1244
2016
-
[14]
Yixiang Fang, Xin Huang, Lu Qin, Ying Zhang, Wenjie Zhang, Reynold Cheng, and Xuemin Lin. 2020. A survey of community search over big graphs. The VLDB Journal 29, 1 (2020), 353–392
2020
-
[15]
Nan Fu, Weiwei Ni, Lihe Hou, Dongyue Zhang, and Ruyu Zhang. 2024. Com- munity detection in decentralized social networks with local differential privacy. Information Sciences 661 (2024), 120164
2024
-
[16]
R George, K Shujaee, M Kerwat, Z Felfli, D Gelenbe, and K Ukuwu. 2020. A comparative evaluation of community detection algorithms in social networks. Procedia Computer Science 171 (2020), 1157–1165
2020
-
[17]
Priya Govindan, Chenghong Wang, Chumeng Xu, Hongyu Duan, and Sucheta Soundarajan. 2017. The k-peak decomposition: Mapping the global structure of graphs. In WWW. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 1441–1450
2017
-
[18]
2008.Exploring network structure, dynamics, and function using NetworkX
Aric Hagberg, Pieter Swart, and Daniel S Chult. 2008.Exploring network structure, dynamics, and function using NetworkX. Technical Report. Los Alamos National Lab.(LANL), Los Alamos, NM (United States)
2008
-
[19]
Yizhang He, Kai Wang, Wenjie Zhang, Xuemin Lin, and Ying Zhang. 2021. Exploring cohesive subgraphs with vertex engagement and tie strength in bipartite graphs. Information Sciences 572 (2021), 277–296
2021
-
[20]
Dahee Kim, Junghoon Kim, Sungsu Lim, and Hyun Ji Jeong. 2023. Exploring Cohesive Subgraphs in Hypergraphs: The(𝑘,𝑔)-core Approach. In CIKM. ACM, New York, NY , USA, 4013–4017
2023
-
[21]
Junghoon Kim, Kaiyu Feng, Gao Cong, Diwen Zhu, Wenyuan Yu, and Chunyan Miao. 2022. ABC: Attributed Bipartite Co-clustering. PVLDB 15, 10 (2022), 2134–2147
2022
-
[22]
Junghoon Kim, Hyun Ji Jeong, Sungsu Lim, and Jungeun Kim. 2023. Effective and efficient core computation in signed networks. Information Sciences 634 (2023), 290–307
2023
-
[23]
Junghoon Kim, Siqiang Luo, Gao Cong, and Wenyuan Yu. 2022. DMCS : Density Modularity Based Community Search. In SIGMOD. ACM, New York, NY , USA, 889–903
2022
-
[24]
Andrea Lancichinetti, Santo Fortunato, and Filippo Radicchi. 2008. Benchmark graphs for testing community detection algorithms. PRE 78, 4 (2008), 046110
2008
-
[25]
Guoliang Li, Shuo Chen, Jianhua Feng, Kian-lee Tan, and Wen-syan Li. 2014. Efficient location-aware influence maximization. In SIGMOD. 87–98
2014
-
[26]
Zhenjun Li, Yunting Lu, Wei-Peng Zhang, Rong-Hua Li, Jun Guo, Xin Huang, and Rui Mao. 2018. Discovering hierarchical subgraphs of k-core-truss. Data Science and Engineering 3, 2 (2018), 136–149
2018
-
[27]
Dandan Liu and Zhaonian Zou. 2023. gCore: Exploring Cross-Layer Cohesiveness in Multi-Layer Graphs. PVLDB 16, 11 (2023), 3201–3213
2023
-
[28]
R Duncan Luce and Albert D Perry. 1949. A method of matrix analysis of group structure. Psychometrika 14, 2 (1949), 95–116
1949
-
[29]
Fragkiskos D Malliaros, Christos Giatsidis, Apostolos N Papadopoulos, and Michalis Vazirgiannis. 2020. The core decomposition of networks: Theory, algo- rithms and applications. The VLDB Journal 29, 1 (2020), 61–92
2020
-
[30]
Robert J Mokken et al. 1979. Cliques, clubs and clans. Quality & Quantity 13, 2 (1979), 161–173
1979
-
[31]
James Moody and Douglas R White. 2002. Social cohesion and embeddedness: A hierarchical conception of social groups. Sociological Methodology 68, 1 (2002), 365–368
2002
-
[32]
Mark EJ Newman and Michelle Girvan. 2004. Finding and evaluating community structure in networks. PRE 69, 2 (2004), 026113
2004
-
[33]
François Rousseau and Michalis Vazirgiannis. 2015. Main core retention on graph- of-words for single-document keyword extraction. In ECIR. Springer, 382–393
2015
-
[34]
Stephen B Seidman. 1983. Network structure and minimum degree. Social networks 5, 3 (1983), 269–287
1983
-
[35]
Jung Hyuk Seo and Myoung Ho Kim. 2021. Finding influential communities in networks with multiple influence types. Information Sciences 548 (2021), 254–274
2021
-
[36]
Mauro Sozio and Aristides Gionis. 2010. The community-search problem and how to plan a successful cocktail party. In SIGKDD. ACM, New York, NY , USA, 939–948
2010
-
[37]
Gilbert Strang. 2006. Linear algebra and its applications.Belmont, CA: Thomson, Brooks/Cole
2006
-
[38]
Lei Tang and Huan Liu. 2010. Community detection and mining in social media. Synthesis lectures on data mining and knowledge discovery 2, 1 (2010), 1–137
2010
-
[39]
Etsuji Tomita, Akira Tanaka, and Haruhisa Takahashi. 2006. The worst-case time complexity for generating all maximal cliques and computational experiments. Theoretical computer science 363, 1 (2006), 28–42
2006
-
[40]
Jordi Torrents and Fabrizio Ferraro. 2015. Structural cohesion: Visualization and heuristics for fast computation. arXiv preprint arXiv:1503.04476 (2015)
2015 arXiv
-
[41]
Friedhelm Victor, Cuneyt G Akcora, Yulia R Gel, and Murat Kantarcioglu. 2021. Alphacore: Data Depth based Core Decomposition. In SIGKDD. ACM, New York, NY , USA, 1625–1633
2021
-
[42]
Tianhao Wang, Yong Zhang, Francis YL Chin, Hing-Fung Ting, Yung H Tsin, and Sheung-Hung Poon. 2015. A simple algorithm for finding all k-edge-connected components. PLOS One 10, 9 (2015), e0136264
2015
-
[43]
Yuxiang Wang, Chengjie Gu, Xiaoliang Xu, Xinjun Zeng, Xiangyu Ke, and Tianxing Wu. 2024. Efficient and effective (k, P)-core-based community search over attributed heterogeneous information networks. Information Sciences 661 (2024), 120076
2024
-
[44]
Stanley Wasserman, Katherine Faust, et al. 1994. Social network analysis: Meth- ods and applications. Cambridge university press
1994
-
[45]
Xudong Wu, Long Yuan, Xuemin Lin, Shiyu Yang, and Wenjie Zhang. 2019. Towards efficient k-tripeak decomposition on large graphs. InDASFAA. Springer, 604–621
2019
-
[46]
Xiaowei Xu, Nurcan Yuruk, Zhidan Feng, and Thomas AJ Schweiger. 2007. Scan: a structural clustering algorithm for networks. In SIGKDD. 824–833
2007
-
[47]
Jaewon Yang and Jure Leskovec. 2012. Defining and evaluating network commu- nities based on ground-truth. In SIGKDD Workshop on Mining Data Semantics. 1–8
2012
-
[48]
Wayne W Zachary. 1977. An information flow model for conflict and fission in small groups. Journal of anthropological research33, 4 (1977), 452–473
1977
-
[49]
Chen Zhang, Fan Zhang, Wenjie Zhang, Boge Liu, Ying Zhang, Lu Qin, and Xuemin Lin. 2020. Exploring finer granularity within the cores: Efficient(𝑘,𝑝)- core computation. In ICDE. IEEE, 181–192
2020
-
[50]
Fan Zhang, Long Yuan, Ying Zhang, Lu Qin, Xuemin Lin, and Alexander Zhou
-
[2018]
In DASFAA
Discovering strong communities with user engagement and tie strength. In DASFAA. Springer, 425–441
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.