{"id":"a18c8958-7961-4c23-abda-ec9c44ea76b5","arxiv_id":"2507.10005","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Community-structured sparse relational graphs improve 5-layer CIFAR-10 accuracy over fully connected baselines, but the advantage reverses at 8 layers due to over-smoothing.","lead":"This paper tests whether neural networks built from graphs with community structure and scale-free degree distributions learn CIFAR-10 image classification better than fully connected networks. It finds a depth-dependent effect: community structure helps at 5 layers, but at 8 layers all structured networks fall below the baseline.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Parameter-count confound threatens the central claim: community-structured graphs have fewer trainable weights than the complete baseline, so the reported 5-layer advantage may be capacity regularization rather than topology.","rationale":"I read the paper in good faith: the authors provide a systematic sweep, share code, and report some 10-seed verification runs. The qualitative depth-reversal finding is interesting and plausible in light of over-smoothing. However, the primary headline claim - that community structure and inter-community density improve learning - is not identifiable under the parameter-count confound. The paper itself acknowledges that sparsity helps via capacity reduction (Sec. IV.A), so the burden is on the authors to show that modularity adds something beyond sparsity. The reader's weakest_assumption identified this same issue, and I agree with that assessment. My proposed control experiment would settle whether the modularity-specific claim survives when edge count is matched. Because the current evidence is compatible with a simpler parameter-count explanation, conditional acceptance is appropriate; the verdict need not change from the reader's CONDITIONAL assessment.","tokens_in":14086,"tokens_out":5952,"duration_ms":71338,"concrete_test":"Run a matched-parameter control for the optimal ER-with-communities configuration in Fig. 3(b) (c=2 at the reported optimal p and mu). Construct two networks with the same number of nodes and exactly the same number of trainable weights: (1) the modular graph, and (2) an ER graph with no planted community structure but the same edge count (sample p to match |E|). Train both with identical width, depth, optimizer, epochs (200), and the same 10 seeds used for the cross-verification runs. If the modular graph's top-1 error is not better than the equal-edge ER graph by more than the seed-noise scale (about 0.1-0.2%), the 'community advantage' reduces to parameter-count regularization. Repeat for c=4, c=8, and for the static scale-free networks with m=3 to test generality.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central 5-layer result is that community-structured sparse relational graphs outperform the complete-graph MLP by up to 5.3% top-1 error (Sec. IV.B). However, the experimental design does not match the number of trainable parameters across configurations. In the relational-graph construction (Sec. II, Eq. 1), each round r has its own weight matrix W^(r), whose sparsity pattern is fixed by the graph adjacency and whose values are learned independently. Thus the trainable-parameter count per layer is exactly the number of graph edges |E|, plus biases; the complete-graph baseline has N^2 weights per layer, while every sparse configuration in Figs. 2-4 has fewer. Section IV.A explicitly attributes the sparse-network advantage to reduced model capacity ('reducing model capacity relative to a fixed training set size improves generalization'), which is a parameter-count explanation rather than a topology explanation. Yet the later modularity claims (Secs. IV.B, IV.E and the Conclusion) treat community organization as the driver, with inter-community density mu reported as the dominant factor (Spearman rho down to -0.91). Since both mu and the number of communities change the edge count, the observed monotonic dependence on mu is indistinguishable from a monotonic parameter-count effect. The data therefore do not support the strong conclusion that 'coherent, densely interconnected communities demonstrate enhanced learning capabilities' as a claim about topology.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript studies how the topology of a fixed-width MLP's hidden layers, represented as a relational graph, affects CIFAR-10 classification performance. The authors generate Erdős–Rényi and static scale-free networks with and without community structure (using a simplified LFR benchmark), translate them into 5-layer and 8-layer message-passing MLPs, and measure top-1 error against a fully connected baseline. They report that community-structured sparse graphs improve over the complete baseline by up to 5.3% at five layers, that inter-community density mu correlates strongly with lower error, that the advantage reverses at eight layers, and that a C. elegans neural network outperforms frontal and ER baselines. They interpret the results as evidence for a modular inductive bias and structural regularization.","tokens_in":14286,"tokens_out":7610,"duration_ms":85368,"significance":"If the conclusions were established, the paper would provide a valuable systematic map of how mesoscale community structure affects learning in a controlled relational-graph setting, with potential design guidance for architecture search and biologically inspired networks. The study is empirically extensive for a 128-node setting, includes multiple random seeds with a cross-check at ten seeds, and ships code and an interactive visualization. The main weaknesses are that the parameter-count confound prevents the 5-layer advantage from being attributed to topology, and the 8-layer reversal is demonstrated only for a restricted subset of the parameter space. These issues are fixable with additional matched controls, so the work is a worthwhile candidate for major revision.","major_comments":[{"comment":"The experimental design does not match the number of trainable parameters across configurations. Because each message-passing round r has an independent weight matrix W^(r) whose sparsity pattern is fixed by the graph adjacency, the hidden-layer parameter count is proportional to the number of edges |E| in the relational graph. The complete-graph baseline has N^2 weights per round, while every sparse configuration in Figs. 2–4 has fewer. The 5-layer claim that community-structured graphs outperform the complete baseline by up to 5.3% top-1 error (Sec. IV.B) is therefore indistinguishable from a capacity-regularization effect, and the manuscript itself invokes capacity reduction in Sec. IV.A ('reducing model capacity relative to a fixed training set size improves generalization'). The later modularity claims (Secs. IV.B and IV.E) would require either edge-matched non-community baselines or an analysis that holds the parameter count fixed while varying community organization; otherwise the monotonic dependence on inter-community density mu in Fig. 7 could be an edge-count effect rather than a community-topology effect.","section":"Sec. II, Eq. (1); Secs. IV.A–IV.B"},{"comment":"The categorical depth-reversal claim is not supported by the presented evidence. The 8-layer results in Fig. 6(a) are for static scale-free networks with communities at c=3 and a single fixed inter-community density mu=0.667; Fig. 6(b) is also restricted to c=3. No 8-layer results are shown for ER networks, for other community counts, or for the no-community baselines. The abstract and Sec. V state that extending the architecture to eight layers 'reverses the effect entirely,' but such a categorical conclusion requires 8-layer sweeps over the same parameter space used for the 5-layer phase diagrams. The over-smoothing interpretation is plausible, but the paper does not measure representation collapse or compare against complete-graph over-smoothing; it only cites Refs. [43,44].","section":"Sec. IV.D, Fig. 6"},{"comment":"The Spearman correlations in Fig. 7 are computed after aggregating across all intra-community parameters p (or gamma), and the p range reported in the Fig. 7 caption (p∈[0.167,1.0]) differs from the range stated in the text for Fig. 3 (p∈[0.1,1.0]). Because mu also changes the expected edge count, the correlation does not isolate community topology from network density. Moreover, a Spearman rho of -0.91 does not mean that 'nearly 91% of the variance' is explained (Sec. IV.E); rho is a rank correlation, not an R^2. The authors should report partial correlations controlling for edge count or hold p/gamma fixed, and should rephrase the variance claim.","section":"Sec. IV.E, Fig. 7"}],"minor_comments":[{"comment":"The captions of Fig. 4 and Fig. 7 are essentially identical, although the figures are referred to in different sections; one of the captions likely describes the wrong figure.","section":"Fig. 4 and Fig. 7 captions"},{"comment":"There is a typo in the phrase 'thesmall world” property' — the opening quotation mark and a space are missing.","section":"Sec. II"},{"comment":"The sentence 'the implementation of any relational graph generally enhances the MLP architecture’s predictive performance by 2.8%' is ambiguous: it should specify whether 'any' means every tested graph, and whether 2.8% is a mean improvement over the baseline.","section":"Sec. IV.B"},{"comment":"The footnote states that only the largest connected component of each community is retained, so the actual node count and edge count vary across configurations; this should be reported explicitly in the methods because it affects both the parameter-count analysis and the comparison across community numbers.","section":"Footnote 31, Sec. III.C"},{"comment":"The C. elegans comparison in Fig. 5 reports small differences (about 0.3–0.6 percentage points) with error bars; the text should state which pairwise differences are statistically significant and should clarify that the 'Random' sample is not matched to the frontal network in degree sequence or edge density.","section":"Sec. IV.C, Fig. 5"}],"recommendation":"major_revision","confidential_remarks":"To the editor: The paper has a fresh empirical contribution and the open code is a strength. The parameter-count confound is the central issue; I would ask for edge-matched baselines or a partial-correlation analysis before publication. The 8-layer claim also needs broader evidence. I do not see grounds for rejection, since the observations are reproducible and the interpretation can be corrected within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know: this is a solid, reproducible empirical sweep that extends graph2nn from Watts-Strogatz graphs to ER and scale-free graphs with LFR-style community structure. The headline result — community structure and moderate sparsity help at 5 layers, and the advantage categorically reverses at 8 layers — is plausible, consistent with prior over-smoothing results, and worth taking seriously. The paper is honest about its scope and ships code and data, which earns real credit.\n\nWhat is genuinely new: the controlled sweep over community count c, inter-community density mu, and degree exponent gamma, plus the depth comparison. The C. elegans comparison is a nice addition, and the 8-layer reversal is independently documented, even if it partly extends the failure case already noted in Ref. [13]. The SFN experiments with fixed average degree m are a useful design feature: here total edge count is roughly constant as mu varies, so the strong negative Spearman correlations (down to -0.87 for SFN) are not simply a parameter-count effect. The stress-test note overstates the confound for that part.\n\nThe soft spots are real but not fatal. First, the comparison against the complete-graph baseline is confounded: sparse graphs have fewer trainable weights, and the paper itself says the advantage comes from reduced capacity. That makes the claim that community structure specifically — beyond sparsity — is the driver less clean. The ER experiments also vary total edge count with mu, so the ER correlations could partly reflect capacity. A matched-control experiment, e.g. random graphs with the same edge count as the modular graphs, would settle this. Second, the phase diagrams and heatmaps lack error bars; aggregate boxplots help, but the heatmaps are the visual centerpiece. Third, the 8-layer claim is shown only for a slice (c=3, m=3), though the text implies a broader sweep. Fourth, the conclusion that sparse structured replacements are 'universally beneficial' overreaches from 128-node CIFAR-10 experiments.\n\nWho gets value: network-science-minded ML readers who want a map of how mesoscale structure affects small relational-graph classifiers. The paper is a serious empirical contribution, not a breakthrough. It deserves a careful referee and a request for matched-control experiments and explicit error bars. I would engage with it in review, and I might cite the depth reversal after the confound is addressed.","headline":"Competent empirical extension of graph2nn with a genuine depth-dependent reversal, but the modularity claim needs parameter-matched controls before it is fully clean.","tokens_in":14898,"tokens_out":3237,"would_cite":false,"duration_ms":39014,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that the community structure of a neural network's hidden-layer graph improves learning at five layers—by up to 5.3% top-1 error on CIFAR-10—and that this advantage fully reverses at eight layers.","keywords":["relational graphs","community structure","network modularity","neural network depth","CIFAR-10","scale-free networks","over-smoothing","image classification"],"falsifier":"Train the same family of relational graphs with the number of trainable weights matched across configurations—for example, by widening the sparse layers or adding untrained connections so every graph has the same parameter count—and check whether the five-layer community advantage and the eight-layer reversal persist; if the advantage disappears, it was driven by parameter reduction, not community structure.","tokens_in":13785,"feed_emoji":"🧩","tokens_out":9008,"duration_ms":94588,"temperature":0.7,"pith_summary":"The authors test whether the wiring pattern of a neural network's hidden layers—not just how many connections exist, but how they are arranged into communities—changes how well the network learns. They do this with a relational-graph representation in which each hidden layer is one round of message exchange along a fixed sparse graph, and they build such graphs as random or scale-free networks split into a tunable number of communities, trained on the CIFAR-10 image classification task. They find that at five layers, every sparse graph tested beats the fully connected baseline, and graphs with densely interconnected communities do best, up to 5.3% lower top-1 error. They also find that increasing the density of connections between communities steadily improves performance, with rank correlation reaching −0.91 at eight communities. The pattern is depth-bounded: at eight layers all tested configurations perform worse than the fully connected baseline, which the authors attribute to over-smoothing, the collapse of node representations toward uniformity after many rounds of message passing.","feed_headline":"Community wiring beats dense nets at 5 layers, flops at 8","feed_subtitle":"Modular graph structure cuts CIFAR-10 error by up to 5.3%, but depth flips the advantage—a warning for deep sparse nets.","key_machinery":"The central object is the relational graph: a fixed sparse graph whose nodes pair one input and one output channel of a hidden layer, and whose edges define which pairs of neurons exchange messages. Each hidden layer corresponds to one round of message passing over the same sparsity pattern, with independently learned weights per round. The paper's generative machinery is a simplified community-benchmark construction: the network is partitioned into a variable number of communities, each generated as a random or scale-free subgraph, and connected to one another by a tunable inter-community density parameter µ. This design lets the authors sweep modularity while holding layer width and depth fixed, and then map top-1 error over the (intra-community density, µ, number-of-communities) phase space against a fully connected baseline, and correlate error with µ using a non-parametric rank correlation.","core_discovery":"The paper's central claim is that modularity of the relational graph is a first-order, depth-bounded determinant of learning performance. On the CIFAR-10 task with a fixed-width five-layer MLP, a sparse relational graph of either random or scale-free type consistently improves top-1 error relative to the complete-graph baseline, and imposing community structure on that graph improves it further, with the best configuration outperforming the complete graph by 5.3%. The paper identifies inter-community density as the dominant structural lever: across configurations with 2 to 8 communities, higher inter-community density is monotonically associated with lower error, with rank correlation reaching −0.91 at 8 communities. The same experiment at eight layers produces a categorical reversal: every tested configuration, including community-structured ones, falls below the complete-graph baseline, and the variance across topologies grows. The authors explain the reversal as over-smoothing, where repeated rounds of message passing drive node representations toward a uniform, information-poor state, and they conclude that five layers is a principled optimum for this architecture and dataset.","pith_inferences":["Beyond the paper: if parameter counts were matched across configurations, the community advantage at five layers might vanish entirely, because edge count alone—not modularity—could explain the gains; a matched-parameter experiment would settle this.","Beyond the paper: the strong monotonic link between inter-community density and lower error implies a practical, untested recipe—raising inter-community integration at moderate depth is a cheap performance knob even when the overall edge count is fixed.","Beyond the paper: the roundworm brain network outperforming its random subsample suggests that global community organization, not just local wiring, carries the benefit; a further untested extension would ablate communities by targeted edge rewiring to localize where the performance loss concentrates."],"forward_implications":["If modular sparse wiring helps at five layers, network architecture search should treat community structure as a tunable design axis on top of width, depth, and edge count.","If the eight-layer reversal is real, depth and topology cannot be optimized independently: a graph that is beneficial at moderate depth becomes harmful at greater depth, so depth sweeps are required before any wiring claim is adopted.","If inter-community density is the dominant lever, then for a fixed edge budget, wiring that distributes edges across communities—rather than only within them—should be prioritized for fixed-width classifiers.","A direct corollary is that full connectivity is not the best use of parameters at this scale, so sparse structured layers can be adopted as a default in small fixed-width MLPs."],"supporting_citations":[{"why":"Introduces the relational-graph representation that maps MLP layers to rounds of message passing on a fixed graph, the foundation this paper extends.","marker":"[13]"},{"why":"Provides the CIFAR-10 dataset and task on which all performance comparisons in the paper are measured.","marker":"[17]"},{"why":"Defines the random graph model used as the basic sparse topology for the hidden-layer relational graphs.","marker":"[22]"},{"why":"Defines the static scale-free network model used to introduce heterogeneous degree distributions into the hidden-layer graphs.","marker":"[23]"},{"why":"Supplies the community-benchmark idea and the mixing-parameter framework used to generate modular relational graphs.","marker":"[27]"},{"why":"Documents the over-smoothing phenomenon in graph convolutional networks, invoked to explain the eight-layer performance collapse.","marker":"[43]"},{"why":"Formalizes exponential loss of expressive power in deep graph message passing, cited as the theoretical basis for the depth reversal.","marker":"[44]"}],"fun_headline_variants":["Modular graphs boost accuracy at 5 layers, reverse at 8","Community wiring aids learning at depth 5, backfires at depth 8","Graph modularity helps CIFAR-10 at 5 layers, hurts at 8","Sparse modular nets beat dense ones at 5 layers, flop at 8","Depth flips modularity gain: 5-layer win, 8-layer loss"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the performance differences between graph configurations reflect the topology itself, even though the compared networks contain different numbers of trainable weights—the sparse community graphs have far fewer parameters, so part of their advantage could be an ordinary regularization effect.","fun_headline_variants_meta":{"raw":{"variants":["Modular graphs boost accuracy at 5 layers, reverse at 8","Community wiring aids learning at depth 5, backfires at depth 8","Graph modularity helps CIFAR-10 at 5 layers, hurts at 8","Sparse modular nets beat dense ones at 5 layers, flop at 8","Depth flips modularity gain: 5-layer win, 8-layer loss"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000316,"raw_usage":{"total_tokens":1821,"prompt_tokens":1011,"completion_tokens":810,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":627,"completion_tokens_details":{"reasoning_tokens":704}},"tokens_in":627,"tokens_out":810,"duration_ms":8267,"temperature":1.0,"reasoning_tokens":704,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T17:41:00.969387+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same family of relational graphs with the number of trainable weights matched across configurations—for example, by widening the sparse layers or adding untrained connections so every graph has the same parameter count—and check whether the five-layer community advantage and the eight-layer reversal persist; if the advantage disappears, it was driven by parameter reduction, not community structure.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the relational-graph representation that maps MLP layers to rounds of message passing on a fixed graph, the foundation this paper extends."},{"cited_title":"Krizhevsky,Learning Multiple Layers of Features from Tiny Images, Tech","cited_arxiv_id":null,"evidence_quote":"Provides the CIFAR-10 dataset and task on which all performance comparisons in the paper are measured."},{"cited_title":"Erd ˝os and A","cited_arxiv_id":null,"evidence_quote":"Defines the random graph model used as the basic sparse topology for the hidden-layer relational graphs."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the static scale-free network model used to introduce heterogeneous degree distributions into the hidden-layer graphs."},{"cited_title":"Lancichinetti and S","cited_arxiv_id":null,"evidence_quote":"Supplies the community-benchmark idea and the mixing-parameter framework used to generate modular relational graphs."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Documents the over-smoothing phenomenon in graph convolutional networks, invoked to explain the eight-layer performance collapse."}],"review_version":1}