{"id":"202e02cf-8a0d-44af-ab6e-0678d556e16a","arxiv_id":"1908.04653","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A belief propagation algorithm for multilayer modularity is derived, benchmarked against GenLouvain, and shown to provide soft community assignments and a convergence-based detectability signal.","lead":"The authors extend a message-passing method for community detection, modularity belief propagation, to multilayer networks with tunable resolution and layer coupling. The method matches a standard tool on benchmarks and adds convergence-based signals for when community structure is genuinely detectable.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Main-text update Eq. (2.4) has a '+' on the null-model field term, whereas the supplement's derivation (S.6–S.7) requires '−'; if the published sign is implemented, multimodbp does not optimize objective (1.1).","rationale":"Good-faith reading: the paper's contribution is an algorithm and an empirical claim. The algorithm is defined by Eq. (2.4); the empirical claim depends on that algorithm being modularity BP. The sign of the field term is not a cosmetic detail. Modularity's null model subtracts the expected edge weight, so the corresponding BP field must suppress, not promote, assignment of high-degree nodes to the same community. The supplement's derivation (S.1) is explicit about this, giving a minus sign in S.7; the main text's plus sign in Eqs. (2.1) and (2.4) is an internal contradiction. Because the released code is not in the manuscript, a reader cannot know which variant was benchmarked. If the code follows the main text, the GenLouvain comparison is invalid as a modularity comparison and the detectability interpretation is not about modularity at all. If the code follows the supplement, the paper needs a correction. Either way the central claim's foundation is insecure. The reader's chosen weakest assumption (convergence ⇔ detectability) is important but the paper explicitly acknowledges that caveat (Section 2.4, Fig. S.24); the sign inconsistency is unacknowledged and more fundamental. Agreement with reader: disagree, since the reader identified a different, already-qualified concern. The verdict should remain conditional, but the condition is now on resolving the equation/source discrepancy.","tokens_in":33347,"tokens_out":13175,"duration_ms":128908,"concrete_test":"Inspect the released multimodbp source (github.com/bwalker1/ModularityBP_Cpp) and verify the sign of the field term in the message update: it should be exp(−γβ d_i θ_{l_i}^t/(2m_{l_i})) as in S.7. Then rerun the DSBM experiment of Fig. 5 with the published '+' sign. If the results change materially, the main-text equations are incorrect and the claim as stated fails; if the code already uses the minus sign, the paper must correct Eqs. (2.1) and (2.4) and the central claim survives.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim asserts that Eq. (2.4) is the belief-propagation algorithm for the multilayer modularity objective (1.1). This requires Boltzmann factors exp(+β Ã_ij δ) for edges and exp(−β γ d_i d_j/(2m_l) δ) for the null model. The supplement derives the field term as exp(−γβ d_i θ_{l_i}^t/(2m_{l_i})) in Eqs. S.6–S.7. The main-text Eq. (2.4) instead has a plus sign: exp(+γβ d_i θ_{l_i}^t/(2m_{l_i})). The same sign appears in the single-layer equation (2.1). With a plus sign the null model becomes an attractive interaction, pulling high-degree nodes into the same community, which is the opposite of the expected-edge penalty in Eq. (1.1). Thus a program implementing the published update equations would be solving a different objective, and the benchmark comparisons against GenLouvain (Figs. 5, 7) would not compare two modularity optimizers. The paper never notes this discrepancy; the released code is the only place that could resolve it.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper develops multimodbp, a belief-propagation algorithm for maximizing multilayer modularity Q(γ,ω)=Σ_ij(A_ij−γP_ij+ωC_ij)δ(c_i,c_j). It extends the Zhang–Moore single-layer modularity belief-propagation approach by adding a resolution parameter γ, layer-specific null-model fields, and interlayer coupling in the update equations, and it derives a Bethe free energy and a β* stability criterion for selecting the inverse temperature. The authors evaluate the method on stochastic block models, LFR benchmarks, and multilayer generative benchmarks against GenLouvain, and they use convergence behavior and node-level marginals on U.S. Senate voting and Lazega lawyer networks to argue that the algorithm can signal the detectability of community structure.","tokens_in":33623,"tokens_out":12599,"duration_ms":123449,"significance":"If the technical issues are resolved, this is a useful contribution: it gives practitioners a multilayer community-detection heuristic with soft output (marginals), a convergence-based detection signal, and an accompanying Python package. The supplement contains detailed derivations of the update equations, the free energy, and β*; the synthetic benchmark comparisons are broad (DSBM, temporal, uniform and block multiplex); and the real-world examples illustrate the claimed extra information. That said, the manuscript currently contains a sign inconsistency in the printed update equations, and the empirical comparisons lack uncertainty quantification, so the results should be treated as provisional.","major_comments":[{"comment":"In Eq. (2.4) the null-model field enters with a plus sign: the update is proportional to exp(γβd_iθ_{l_i}^t/(2m_{l_i}) + ...), where θ_{l_i}^t = Σ_{j∈V_{l_i}} d_jψ_j^t. In the derivation in Supplement S.1 the same field is derived with a minus sign (Eqs. S.6–S.7 and S.10): exp(−βd_iθ_{l_i}^t/(2m_{l_i})). The plus sign makes the null model attractive for high-degree nodes, which is the opposite of the penalty in Eq. (1.1), so a program implementing Eq. (2.4) would not be solving the stated multilayer modularity objective. The Bethe free energy in Eq. (2.5), whose final term has a plus sign, is consistent with the minus-sign update, as is the S.1 derivation; the same plus sign appears in Eq. (S.23). The authors must reconcile these equations and, ideally, confirm from the released code that the implemented update uses the minus sign.","section":"Eq. (2.4), Eq. (2.5), Supplement S.1 (S.6–S.7, S.10) and S.23"},{"comment":"Figures 5 and 7 report point averages over 50 and 100 realizations, respectively, but no error bars, confidence intervals, or significance tests are given. Several claims of comparable or superior performance (e.g., the statement in §3.2.2 that GenLouvain has \"a slight edge overall\" in the uniform multiplex case, Figure 7B) depend on differences whose magnitude relative to run-to-run variability cannot be assessed. The authors should add error bars or otherwise quantify uncertainty.","section":"Figs. 5 and 7"},{"comment":"The paper's title and contributions center on using convergence to non-trivial fixed points as a signal of detectable community structure, but this premise is explicitly heuristic: §2.4 notes that the β* heuristic \"provides no guarantees,\" and Figure S.24 shows that some non-converged runs still align with ground truth. To support the detectability claim, the authors should quantify the behavior of the convergence signal on structureless null networks (e.g., Erdős–Rényi or configuration-model graphs with matched degree sequences) and report the overlap of the converged and non-converged AMI distributions over the parameter grid; Figure 8 alone covers one topology and one value of p.","section":"§2.4, Fig. 8, Fig. S.24"}],"minor_comments":[{"comment":"In Section 3.2.2 the temporal benchmark is described as having \"150 node-layers in each layer with 100 layers,\" while the caption of Figure 7A says \"100 node-layers in each layer with 150 layers\"; both give 15,000 node-layers, but the text should agree.","section":"§3.2.2 vs Fig. 7 caption"},{"comment":"Equation (S.10) contains a garbled symbol in the definition of θ_{l_i}^t (\"V↕⟩\"), which should be V_{l_i}.","section":"Eq. (S.10)"},{"comment":"The reference for the released software [63] points to a GitHub repository, while the text states that the tool is available on PyPI; the reference should be updated or clarified.","section":"Reference [63]"},{"comment":"In Section 2.4, \"Schülkeet al.\" should read \"Schülke et al.\"","section":"Section 2.4"},{"comment":"The Figure 1 caption phrase \"From left to right, the plots show the retrieval modularity, number of iterations to convergence, and the AMI ... and the effective number of communities\" describes four panels but is awkwardly worded; please revise for clarity.","section":"Fig. 1 caption"}],"recommendation":"major_revision","confidential_remarks":"The sign inconsistency in Eq. (2.4) is the key stumbling block. I would ask the authors to confirm the implemented update direction from the released package and correct the printed equations accordingly; if the implementation matches the minus-sign derivation, the paper is likely acceptable after a thorough revision. The contribution is incremental but useful, and the real-world demonstrations are illustrative rather than conclusive. I would also encourage the editor to verify that the companion code is accessible at the time of publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuine contribution here is the multilayer extension of modularity belief propagation: layer-specific null model, resolution and coupling parameters, a Bethe free energy, and the beta* stability heuristic. The supplement has careful derivations, the synthetic benchmarks against GenLouvain show the method is competitive, and the authors released a Python package. That is a real step beyond single-layer modbp and worth building on.\n\nBut there is a load-bearing problem: the main-text update equation (2.4) has a plus sign on the null-model field term, while the supplement's derivation (Eqs. S.6–S.7 and S.10) consistently gives a minus sign. Same for the single-layer equation (2.1). With the plus sign, the null model becomes an attractive interaction instead of a penalty, so the algorithm would not be optimizing the multilayer modularity objective (1.1). The published equations and the supplement cannot both be right. The released code is the only way to tell which version was actually run. If the code has the plus sign, the benchmark comparisons against GenLouvain are not comparisons of two modularity optimizers, and the paper's central claim collapses. If the code has the minus sign, the main text has a typo that is trivial to fix. This has to be checked before anything else.\n\nSmaller issues: Figures 5 and 7 report averages over many realizations without error bars, which makes it hard to judge whether the reported differences are meaningful. The convergence-based detectability check is explicitly heuristic and the paper says so, but the claim that non-convergence means undetectable structure is stronger than the evidence, especially since Supplement Figure S.24 shows some non-converged runs align with ground truth. And in the real-world examples, the parameters are selected using knowledge of the ground-truth labels; that is fine for an illustrative analysis, but it should be stated as such.\n\nThe derivations themselves look plausible and the empirical work is honest. If the sign issue resolves in favor of the supplement, this is a solid methods paper that deserves publication. If it resolves the other way, the paper needs major revision or rejection. A serious referee should see the code and verify the sign, and the authors should add error bars to the main comparisons.","headline":"A real multilayer extension of modularity belief propagation with a serious sign error in the published update equation that needs to be resolved before the results can be trusted.","tokens_in":34146,"tokens_out":3355,"would_cite":false,"duration_ms":37651,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68Q25","68R10","68U05"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper extends modularity-based belief propagation to multilayer networks and argues that convergence to a non-trivial fixed point is a reliable signal that communities are detectable, while matching a leading modularity optimizer in…","keywords":["multilayer networks","community detection","modularity","belief propagation","message passing","resolution parameter","detectability","soft partitioning"],"falsifier":"Generate multilayer networks with no planted communities but with degree sequences, edge weights, and layer coupling matched to benchmarks where multimodbp converges; count how often the algorithm converges to a non-trivial fixed point and how high the adjusted mutual information of that partition is against arbitrary labels. If a substantial fraction of such null networks produce converged non-trivial partitions, the convergence signal is not specific to real structure. A complementary test is to measure, on the temporal benchmark, whether non-converged runs' partitions align with ground truth often enough to undermine convergence as a necessary condition.","tokens_in":33168,"feed_emoji":"🕸️","tokens_out":10120,"duration_ms":95707,"temperature":0.7,"pith_summary":"The paper extends message-passing (belief propagation) community detection from single-layer to multilayer networks, starting from the multilayer modularity objective and adding a resolution parameter plus an interlayer coupling parameter. It tries to establish that convergence of the algorithm to a non-trivial fixed point is a trustworthy sign that the network actually has detectable community structure, and that the retrieved partition, measured by adjusted mutual information (AMI), is competitive with a widely used modularity optimizer on synthetic benchmarks. A second claim is that the node-level marginals give a soft, uncertainty-weighted assignment that practitioners can use to choose parameters and interpret real-world structure. If these claims hold, the method gives network analysts a way to ask whether a modularity-optimizing partition reflects real mesoscale structure or just noise fitting.","feed_headline":"Belief propagation flags only detectable multilayer communities","feed_subtitle":"Converging message passing finds real multilayer structure; its marginals also give per-node confidence in the partition.","key_machinery":"The central object is the multilayer message update $\\psi_{i\\to k}^{t} \\propto \\exp\\left[\\gamma \\beta \\frac{d_i}{2m_{l_i}}\\theta_{l_i}^{t} + \\sum_{j\\in\\partial i\\setminus k}\\log\\left(1+\\psi_{j\\to i}^{t}(e^{\\tilde{A}_{ij}\\beta}-1)\\right)\\right]$, with $\\tilde{A}_{ij}=A_{ij}\\delta(l_i,l_j)+\\omega C_{ij}(1-\\delta(l_i,l_j))$. The first term is the layer-specific field from the modularity null model, generalized to the multilayer objective; the sum over neighbors carries both intralayer edge weights and interlayer coupling. The companion stability formula, $\\langle ((e^{\\beta^*\\tilde{A}_{ij}}-1)/(e^{\\beta^*\\tilde{A}_{ij}}+q-1))^2 \\rangle_{ij}\\hat{c}=1$, selects the inverse temperature at which the symmetric fixed point destabilizes, which is what makes convergence to a non-trivial fixed point a structured signal rather than a generic property of the iteration.","core_discovery":"At the paper's core is a belief-propagation update for multilayer modularity in which each node-layer's belief about belonging to community t is proportional to an exponential of a layer-local null-model field, a sum over intralayer neighbors, and a sum over interlayer neighbors weighted by the coupling parameter. The updates are derived from the multilayer modularity Hamiltonian, and their fixed points are stationary points of the associated free energy. Linearizing around the symmetric fixed point gives a stability condition that selects the inverse temperature $\\beta^*$ for each assumed number of communities, and the paper reports that running the algorithm at these $\\beta^*$ values lands in the retrieval phase for the synthetic and real networks tested. The central empirical discovery is that the algorithm's convergence itself indicates detectability: when the planted structure becomes too weak, multimodbp stops converging to a non-trivial partition, whereas a modularity optimizer continues to return high-modularity partitions. On top of that, the marginals at the fixed point are used as a soft partition whose per-node entropy measures confidence in the community assignment.","pith_inferences":["Editorial inference: the convergence-versus-non-convergence dichotomy could be calibrated against a formal null model, such as degree-preserved random multilayer networks, to yield a model-free test of community significance; the paper demonstrates the signal but does not quantify its false-positive rate.","Editorial inference: the marginals could be thresholded or resampled to produce overlapping or ensemble partitions, since a node with near-uniform beliefs is naturally compatible with multiple communities; the paper only reads them as confidence values.","Editorial inference: the strong dependence of multiplex performance on spectral initialization suggests the fixed-point landscape has multiple basins; mapping which initializations converge to which partitions would tell users when a converged result is stable versus initialization-driven.","Editorial inference: the same stability analysis could be applied to directed, signed, or edge-heterogeneous multilayer null models, giving analogous inverse-temperature formulas for those settings; the paper does not attempt those cases."],"forward_implications":["On the dynamic stochastic block model benchmark, setting the resolution parameter to $\\gamma=0.5$ widens the retrieval phase and matches single-layer detectability limits for low intralayer noise, while larger $\\gamma$ and larger coupling $\\omega$ pool signal across layers and push detection closer to the layer-aggregation bound.","On temporal, uniform-multiplex, and block-multiplex benchmarks, the retrieved partitions have AMI comparable to or better than a widely used modularity optimizer, with the largest gains in temporal networks with strong interlayer persistence.","Because non-convergence tracks undetectable structure, the method can be used as a check on greedy modularity optimization: a high-modularity partition on a network where multimodbp never converges should be regarded as potentially spurious.","Node-level marginals provide a confidence measure per node-layer that can be averaged over layers; in the voting-network example this reproduces known polarization history and labels bipartisan legislators, and in the law-firm example it maps which parameter regions align with metadata."],"supporting_citations":[{"why":"Supplies the original single-layer modularity belief propagation framework and the retrieval-phase convergence idea this paper extends.","marker":"[66]"},{"why":"Defines the multilayer modularity objective and null model that the update equations are built on.","marker":"[38]"},{"why":"Provides the weighted-edge generalization and the stability-based inverse-temperature selection formula adopted here.","marker":"[54]"},{"why":"Is the widely used multilayer modularity optimizer used as the accuracy baseline in the benchmark comparisons.","marker":"[24]"},{"why":"Supplies the generative benchmark models for temporal, uniform-multiplex, and block-multiplex networks used in the comparisons.","marker":"[3]"},{"why":"Gives the marginal-condensing approach the paper uses to let the number of communities float up to an upper limit.","marker":"[52]"},{"why":"Establishes the single-layer stochastic block model detectability threshold that anchors the performance limits the method approaches.","marker":"[10]"},{"why":"Provides the dynamic stochastic block model and its detectability analysis, the first multilayer test case.","marker":"[15]"},{"why":"Supplies the voting network used for the real-world demonstration of convergence, entropy, and parameter selection.","marker":"[60]"},{"why":"Provides the earlier multislice voting-network analysis that the voting example builds on.","marker":"[37]"}],"fun_headline_variants":["Multilayer BP: convergence signals detectable structure","Belief propagation flags only detectable multilayer states","No convergence, no community: BP for multilayer nets","Marginals from BP show per-node confidence in multilayers","Multilayer modularity BP: stopping means weak structure"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's practical value rests on the unproven premise that when the iterative message-passing updates settle into a non-trivial partition, the network genuinely has community structure rather than the iterations simply getting trapped; the paper says its heuristic provides no guarantees and acknowledges that some non-converged runs still match the planted structure.","fun_headline_variants_meta":{"raw":{"variants":["Multilayer BP: convergence signals detectable structure","Belief propagation flags only detectable multilayer states","No convergence, no community: BP for multilayer nets","Marginals from BP show per-node confidence in multilayers","Multilayer modularity BP: stopping means weak structure"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000309,"raw_usage":{"total_tokens":1770,"prompt_tokens":953,"completion_tokens":817,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":569,"completion_tokens_details":{"reasoning_tokens":742}},"tokens_in":569,"tokens_out":817,"duration_ms":8122,"temperature":1.0,"reasoning_tokens":742,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:36:11.263931+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate multilayer networks with no planted communities but with degree sequences, edge weights, and layer coupling matched to benchmarks where multimodbp converges; count how often the algorithm converges to a non-trivial fixed point and how high the adjusted mutual information of that partition is against arbitrary labels. If a substantial fraction of such null networks produce converged non-trivial partitions, the convergence signal is not specific to real structure. A complementary test is to measure, on the temporal benchmark, whether non-converged runs' partitions align with ground truth often enough to undermine convergence as a necessary condition.","supporting_citations":[{"cited_title":"Zhang and C","cited_arxiv_id":null,"evidence_quote":"Supplies the original single-layer modularity belief propagation framework and the retrieval-phase convergence idea this paper extends."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the multilayer modularity objective and null model that the update equations are built on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the weighted-edge generalization and the stability-based inverse-temperature selection formula adopted here."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Is the widely used multilayer modularity optimizer used as the accuracy baseline in the benchmark comparisons."},{"cited_title":"Bazzi, L","cited_arxiv_id":null,"evidence_quote":"Supplies the generative benchmark models for temporal, uniform-multiplex, and block-multiplex networks used in the comparisons."},{"cited_title":"Sch¨ulke and F","cited_arxiv_id":null,"evidence_quote":"Gives the marginal-condensing approach the paper uses to let the number of communities float up to an upper limit."},{"cited_title":"Decelle, F","cited_arxiv_id":null,"evidence_quote":"Establishes the single-layer stochastic block model detectability threshold that anchors the performance limits the method approaches."},{"cited_title":"Ghasemian, P","cited_arxiv_id":null,"evidence_quote":"Provides the dynamic stochastic block model and its detectability analysis, the first multilayer test case."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the voting network used for the real-world demonstration of convergence, entropy, and parameter selection."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the earlier multislice voting-network analysis that the voting example builds on."}],"review_version":1}