REVIEW 3 major objections 5 minor 35 references
Intellectual and social similarity among scholarly journals: an exploratory comparison of the networks of editors, authors and co-citations
T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read This paper shows that a journal's intellectual neighbours are also its author and editor neighbours, and that the co-citation, authorship, and editorial-board networks are positively associated in all three fields studied.
desk verdict Solid empirical extension with deposited data, but the headline correlation claim needs a degree-preserving null before it can be fully trusted. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The comparison rests on three one-mode journal networks per field: nodes are journals, and an edge between two journals means they share at least one editor (interlocking editorship, IE), share at least one author (interlocking authorship, IA), or are cited together (co-citation, CC). Each network is converted into a matrix of pairwise Jaccard dissimilarities, and the matrices are compared with the generalized distance correlation ($\sqrt{R_d}$), a matrix-level analogue of a correlation coefficient with a permutation test for independence. The networks are then partitioned into communities by a modularity-optimising algorithm, and the resulting partitions are compared with Cramér's $V$, Rajski's coherence, and the adjusted Rand index. The recurring ordering of associations—co-citation with authorship strongest, co-citation with editorship weakest—is the pattern carrying the argument.
What would settle it
Rewire each network randomly while preserving every journal's degree, then recompute the generalized distance correlations. If values such as $\sqrt{R_d}=0.75$ for economics co-citation versus authorship appear as often in the rewired networks as in the real data, the reported similarity between intellectual and social maps would be refuted as an artifact of network density.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that the intellectual proximity encoded by co-citation is mirrored, to a statistically detectable degree, by the social proximity encoded by shared authors—and, more surprisingly, by shared editors. Across statistics, economics, and information and library sciences, the three distance matrices correlate above $0.5$ in every pairwise comparison; the strongest correlations are between co-citation and interlocking authorship ($\sqrt{R_d}=0.64$, $0.64$, and $0.75$ for the three fields). Communities detected in each network are not independent, and again the co-citation/author association is the strongest while co-citation/editor is the weakest. The paper reads these results as evidence that the three maps—editorial power, intellectual proximity, and author communities—are coherent across scholarly fields.
Load-bearing premise
The claim rests on the assumption that the measured correlations reflect genuine overlap in scholarly communities rather than an artifact of how dense each network happens to be; if density or shared degree structure alone produces the correlations, the similar maps could be coincidental.
Editorial extensions
If this is right
- In any field with sufficient data, the same three-map coherence should appear; the paper explicitly argues the method is generally applicable beyond statistics, economics, and information and library sciences.
- Journal rankings and bibliometric indicators cannot be treated as purely intellectual measurements, because they are entangled with who publishes in and who edits the journals.
- Because the strongest association is between co-citation and interlocking authorship, scholars' choices about where to publish are the social layer most tightly coupled to intellectual content.
- Editorial-board networks are more loosely tied to the other two but still predict them, suggesting gatekeeping positions and intellectual clusters are connected even when the connection is weaker.
Reading between the lines
- A testable extension would be to track editor appointments over time: if incoming editors begin publishing in their new journals, board overlap may partly cause author overlap rather than merely correlate with it.
- A stronger control than the paper's permutation test would be degree-preserving rewiring of each network; if the observed correlations survive that, the claim of substantive intellectual-social proximity would be on firmer ground.
- Comparing fields where editors rarely publish in the journals they oversee would reveal how much of the coherence is driven by the same people appearing as authors and as editors.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper compares, for three fields (statistics, economics, and information and library sciences), three types of journal-level networks: co-citation (CC), interlocking editorship (IE), and interlocking authorship (IA). The authors compute Jaccard dissimilarity matrices for each network, test the association between pairs of matrices using generalized distance correlation with permutation tests, and additionally partition each network into communities with the Louvain algorithm and compare the partitions using chi-square, Cramér's V, Rajski's coherence, and the adjusted Rand index. They report that all three networks are significantly associated in all three fields, with the strongest association between CC and IA, and conclude that intellectual proximity among journals is accompanied by proximity among authors and, more surprisingly, among editors.
Significance. The paper addresses a substantive question about the alignment between intellectual and social structures in scholarly communication, using original data and a comparative design across three fields. Its strengths include the use of generalized distance correlation rather than the classical Mantel test, the combination of whole-network and community-level comparisons, the explicit availability of raw data on Zenodo, and a clearly exploratory framing. If the observed associations survive controls for degree and density, the findings would offer credible evidence that journal gatekeeping, authorship, and intellectual organization are structurally coupled. The main limitation, the absence of degree-preserving null models, currently tempers the strength of the central claim.
major comments (3)
- [Section 3, Table 1] The permutation test used for the generalized distance correlation rejects independence, but it does not rule out the alternative that the dissimilarity matrices are correlated because they share a common degree or density gradient. Jaccard dissimilarity is strongly influenced by set sizes, and Table 2 shows that the IA networks are an order of magnitude denser than the IE networks (e.g., statistics IA density 0.91 versus IE density 0.12). A degree-preserving null model, such as randomizing the underlying bipartite graphs while fixing journal degrees before projection, or rewiring the one-mode networks with fixed degree sequences, is needed to benchmark the observed sqrt(R_d) values. Without such a control, the central claim that intellectual proximity is also author and editor proximity is overstated.
- [Section 4, Table 2 and footnote 1] The resolution parameter 0.8 for the ILS IA network was selected post hoc because it produced better E-I indices. This selection is not accompanied by a stability analysis or a pre-specified criterion, so the community-based association results for that field carry an optimistic bias. The acknowledged instability of community detection in economics similarly weakens the community-level conclusions; a sensitivity analysis over a range of resolution parameters would be needed to establish that the reported associations are not artifacts of the chosen partitions.
- [Section 5 and Abstract] The claim that CC-IA is the strongest association is based on point estimates only. For statistics, the CC-IA sqrt(R_d) of 0.6431 is close to the IE-IA value of 0.5985, and no uncertainty intervals or tests of equality of dependent correlations are provided. The ordering of correlation strengths should be treated as descriptive unless accompanied by bootstrap intervals or a formal comparison of dependent correlation coefficients.
minor comments (5)
- [Section 3, Eq. (1)] The sets A and B are defined explicitly only for the IE network; please specify what the sets represent for the CC and IA networks, namely citing-article sets and author sets, respectively.
- [Table 1] P-values are reported with inconsistent numbers of decimal places (e.g., 0.00001 versus 0.00058); please use a uniform formatting convention.
- [Section 4] The asymmetric Rajski coherence variants ('right' and 'left') are described only verbally; providing the formulas would make the direction of prediction unambiguous and improve reproducibility.
- [Abstract and Section 5] There is a grammatical error in 'The strongest correlations is' which should read 'The strongest correlations are'.
- [Section 2, Figures 1-9] The figure captions are minimal; since the visual comparison of networks is part of the exploratory analysis, please indicate the layout algorithm and how edge weights are represented in each figure.
Circularity Check
No significant circularity: the network comparisons are built from independent empirical data sources with no fitted-parameter prediction loop.
full rationale
The paper's central claim is that co-citation (CC), interlocking-authorship (IA), and interlocking-editorship (IE) journal networks are associated within each of three fields. The three network families are constructed from separate empirical inputs: IE networks come from manually collected editorial-board lists (cited to prior papers by the same authors, but these are data-collection citations, not theoretical assumptions), while IA and CC networks come from Web of Science author and cited-reference data. No quantity is fitted to a subset of the data and then presented as a prediction of a closely related quantity; the reported generalized distance correlations are descriptive indices of association between independently built dissimilarity matrices, and the permutation test is a standard independence test rather than a derivation from the paper's own conclusions. The community-detection comparison likewise uses Louvain partitions and external association indices, with no self-defined equivalence linking the input and output. The skeptical concern about degree-preserving null models is a robustness or confounding critique, not an instance of circular reasoning, because the observed correlations would not equal the inputs by construction even if they were confounded by network density. Self-citations are used mainly as sources of already-collected editorial-board data, and the raw data are deposited in an external repository; thus the citation chain does not carry the paper's substantive claim. The paper is exploratory and its inference is limited by methodological choices, but its reasoning is not circular.
Assumptions & free parameters
free parameters (1)
- Louvain resolution parameter for ILS IA network =
0.8
assumptions (4)
- domain assumption Jaccard dissimilarity on sets of editors, authors, or citing articles is a valid proxy for journal (dis)similarity.
- standard math Generalized distance correlation is an appropriate measure for comparing dissimilarity matrices.
- domain assumption The Louvain algorithm with modularity optimization identifies meaningful journal communities.
- domain assumption Editorial board data from a prior year and WoS data from the following five years are comparable and stable.
Cite this review
Pith. "Pith review of Intellectual and social similarity among scholarly journals: an exploratory comparison of the networks of editors, authors and co-citations." pith.science (2026). https://pith.science/paper/RZ5BESFV
@misc{pith2026190809120,
author = {Pith},
title = {Pith review of: Intellectual and social similarity among scholarly journals: an exploratory comparison of the networks of editors, authors and co-citations},
year = {2026},
howpublished = {\url{https://pith.science/paper/RZ5BESFV}},
note = {Machine review of arXiv:1908.09120}
}
read the original abstract
This paper explores, by using suitable quantitative techniques, to what extent the intellectual proximity among scholarly journals is also a proximity in terms of social communities gathered around the journals. Three fields are considered: statistics, economics and information and library sciences. Co-citation networks (CC) represent the intellectual proximity among journals. The academic communities around the journals are represented by considering the networks of journals generated by authors writing in more than one journal (interlocking authorship: IA), and the networks generated by scholars sitting in the editorial board of more than one journal (interlocking editorship: IE). For comparing the whole structure of the networks, the dissimilarity matrices are considered. The CC, IE and IA networks appear to be correlated for the three fields. The strongest correlations is between CC and IA for the three fields. Lower and similar correlations are obtained for CC and IE, and for IE and IA. The CC, IE and IA networks are then partitioned in communities. Information and library sciences is the field where communities are more easily detectable, while the most difficult field is economics. The degrees of association among the detected communities show that they are not independent. For all the fields, the strongest association is between CC and IA networks; the minimum level of association is between IE and CC. Overall, these results indicate that the intellectual proximity is also a proximity among authors and among editors of the journals. Thus, the three maps of editorial power, intellectual proximity and authors communities tell similar stories.
Figures
Figures from the paper (6 more)
Reference graph
Works this paper leans on
-
[1]
Baccini, A. (2009). Italian economic journals. a network-based ranking and an exploratory analysis of their influence on setting international professional standards. Rivista Italiana degli Economisti\/ 14\/ (3), 491--511
work page 2009
-
[2]
Baccini, A. and L. Barabesi (2010). Interlocking editorship. a network analysis of the links between economic journals. Scientometrics\/ 82\/ (2), 365--389
work page 2010
-
[3]
Baccini, A. and L. Barabesi (2011). Seats at the table: The network of the editorial boards in information and library science. Journal of Informetrics\/ 5\/ (3), 382--391
work page 2011
-
[4]
Baccini, A. and L. Barabesi (2014). Gatekeepers of economics: the network of editorial boards in economic journals. In A. Lanteri and J. Vromen (Eds.), The Economics of Economists , pp.\ 104--150. Cambridge: Cambridge University Press
work page 2014
-
[5]
Baccini, A., L. Barabesi, and M. Marcheselli (2009). How are statistical journal linked? a network analysis. Chance\/ 22\/ (3), 34--43
work page 2009
- [6]
-
[7]
Bretz, F., T. Hothorn, and P. Westfall (2011). Multiple Comparisons Using R . Boca Raton: CRC Press
work page 2011
-
[8]
Brogaard, J., J. Engelberg, and C. A. Parsons (2014). Networks and productivity: Causal evidence from editor rotations. Journal of Financial Economics\/ 111\/ (1), 251--270
work page 2014
Show all 35 references
-
[9]
Cramér, H. (1946). Mathematical Methods of Statistics . Princeton: Princeton University Press
1946
-
[10]
Crane, D. (1967). The gatekeepers of science: Some factors affecting the selection of articles of scientific journals. American Sociologist\/ 32\/ (2), 195--201
1967
-
[11]
Csiszar, A. (2018). The Scientific Journal. Authorship and Politicis of Knwoledge in the Nineteenth Century . Chicago: Chicago University Press
2018
-
[12]
Mrvar, and V
de Nooy, W., A. Mrvar, and V. Batagelj (2018). Exploratory Social Network Analysis with Pajek. Revised and expanded edition for updated software . Cambridge: Cambridge University Press
2018
-
[13]
Dutilleul, P., J. D. Stockwell, D. Frigon, and P. Legendre (2000). The mantel test versus pearson's correlation analysis: Assessment of the differences for biological and environmental studies. Journal of Agricultural, Biological, and Environmental Statistics\/ 5\/ (2), 131--150
2000
-
[14]
Erfanmanesh, M. and M. Morovati (2017). Interlocking editorships in scientific journals. Science and Engineering Ethics\/ 2017
2017
-
[15]
Heckman, J. J. and S. Moktan (2018). Publishing and promotion in economics: The tyranny of the top five. National Bureau of Economic Research Working Paper Series\/ No. 25093
2018
-
[16]
Hoenig, B. (2015). Gatekeepers in social science. In J. D. Wright (Ed.), International Encyclopedia of the Social and Behavioral Sciences (Second Edition) , pp.\ 618--622. Oxford: Elsevier
2015
-
[17]
Hubert, L. and P. Arabie (1985). Comparing partitions. Journal of Classification\/ 2\/ (1), 193--218
1985
-
[18]
Klein, D. B. and T. DiCola (2004). Institutional ties of journal of development economics authors and editors. Econ Journal Watch\/ 1\/ (2), 319--330
2004
-
[19]
Kulczycki, E., T. C. E. Engels, J. Pölönen, K. Bruun, M. Dušková, R. Guns, R. Nowotniak, M. Petr, G. Sivertsen, A. Istenič Starčič, and A. Zuccala (2018). Publication patterns in the social sciences and humanities: evidence from eight european countries. Scientometrics\/ 116\/...
2018
-
[20]
Laband, D. N. and M. J. Piette (1994). Favoritism versus search for good papers: Empirical evidence regarding the behavior of journal editors. The Journal of Political Economy\/ 102\/ (1), 194--203
1994
-
[21]
Legendre, P. and L. Legendre (1998). Numerical Ecology . Amsterdam: Elsevier Science
1998
-
[22]
Lehmann, E. and J. Romano (2005). Testing Statistical Hypotheses . New York, Springer
2005
-
[23]
Levandowsky, M. and D. Winter (1971). Distance between sets. Nature\/ 234\/ (5323), 34--35
1971
-
[24]
Newman, M. E. J. (2004, Jun). Fast algorithm for detecting community structure in networks. Phys. Rev. E\/ 69 , 066133
2004
-
[25]
Newman, M. E. J. and M. Girvan (2004, Feb). Finding and evaluating community structure in networks. Phys. Rev. E\/ 69 , 026113
2004
-
[26]
Ni, C., C. R. Sugimoto, and B. Cronin (2013). Visualizing and comparing four facets of scholarly communication: producers, artifacts, concepts, and gatekeepers. Scientometrics\/ 94\/ (3), 1161--1173
2013
-
[27]
Ni, C., C. R. Sugimoto, and J. Jiang (2013). Venue-author-coupling: A measure for identifying disciplines through author communities. Journal of the American Society for Information Science and Technology\/ 64\/ (2), 265--279
2013
-
[28]
Omelka, M. and S. Hudecova (2013). A comparison of the mantel test with a generalised distance covariance test. Environmetrics\/ 24\/ (7), 449--460
2013
-
[29]
Hartley, L
Potts, J., J. Hartley, L. Montgomery, C. Neylon, and E. Rennie (2017). A journal is a club: a new economic model for scholarly publishing. Prometheus\/ 35\/ (1), 75--92
2017
-
[30]
R: A Language and Environment for Statistical Computing
R Core Team (2018). R: A Language and Environment for Statistical Computing . Vienna, Austria
2018
-
[31]
Rizzo, M. and G. Székely (2018). E-statistics: Multivariate inference via the energy of data. R package 'energy'\/ version 1.7-5
2018
-
[32]
Small, H. (1973). Co-citation in the scientific literature: A new measure of the relationship between two documents. Journal of the American Society for Information Science\/ 24 , 265--269
1973
-
[33]
Székely, G. J., M. L. Rizzo, and N. K. Bakirov (2007). Measuring and testing dependence by correlation of distances. The Annals of Statistics\/ 35\/ (6), 2769--2794
2007
-
[34]
Todeschini, R. and A. Baccini (2016). Handbook of Bibliometric Indicators. Quantitative Tools for Studying and Evaluating Research . Weinheim (Germany): Wiley-VCH
2016
-
[35]
Wasserman, S. and K. Faust (1994). Social Network Analysis: Method and Application . Cambridge: Cambridge University Press
1994
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.