{"id":"6bafc9b5-3ec7-4648-b18c-b848a27b1b23","arxiv_id":"2411.14159","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Graph neural networks classify rigidity and connectivity percolation of 2D particle networks, exceeding 95 percent accuracy on diluted lattices but falling below 90 percent on correlated off-lattice gels.","lead":"This paper trains graph neural networks to classify whether simulated 2D particle networks are rigid or percolate across periodic boundaries, reaching over 95 percent accuracy on lattice networks but lower accuracy on dynamically generated gels. A generalist should read it to see how far machine learning can replace expensive physics simulations for structure-property screening of soft materials, and where it currently fails.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"High lattice accuracy may be a local-density artifact: K=4 receptive fields cannot resolve system-spanning rigidity, and no non-GNN baseline is reported to rule out mean-degree classification.","rationale":"I am not objecting to the internal consistency; the paper is honest and its negative results are valuable. The concern is about what the positive lattice result demonstrates. The architecture's locality plus the lack of a baseline makes the >95% accuracy ambiguous: it may be a sophisticated implementation of a mean-coordination threshold. This is not a circularity or fraud concern; it is a missing control. A baseline is cheap and would settle it. If the baseline matches, the abstract's general claims about GNNs predicting rigidity of particulate networks would need to be narrowed to local-density proxies working on uncorrelated lattices, and the practical value for off-lattice gels is already limited by Fig. 7. I retain the reader's CONDITIONAL verdict; the condition should be an explicit baseline comparison and, ideally, validation of the labels against a pebble-game rigidity criterion on a subset.","tokens_in":12083,"tokens_out":11332,"duration_ms":120723,"concrete_test":"Take the same L=50 and L=100 diluted-lattice datasets and the same 4:1:1 train/validation/test folds. Train a logistic-regression (or random-forest) baseline on a handful of easily computed graph features - number of edges, mean degree, degree variance, largest-connected-component fraction, cyclomatic number, and triangle count - and evaluate on the same test folds with the same accuracy metric. If the simple baseline's accuracy is within one standard error of the reported GNN accuracy (e.g., at least 95% for L=100), the positive claim is a local-density artifact; if the baseline is substantially below the GNN, the GNN has learned nontrivial rigidity information.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central positive result is the 93-97% rigidity-classification accuracy on diluted triangular lattices (Sec. III A). But the ChebConv model uses a polynomial filter of order K=4 (Sec. II B), so each node's effective receptive field is only about four hops. For L=50 or L=100 lattices this is a strictly local view, while rigidity percolation is a system-spanning, nonlocal property. On randomly diluted triangular lattices generated at p=0.60-0.70 around pc ~ 0.66, rigidity is almost perfectly correlated with mean coordination z=6p; a threshold on edge count or mean degree may reproduce most of the labels. The paper reports no non-GNN baseline, so the >95% accuracy could be explained by the GNN learning a local density statistic rather than rigidity percolation. Consistent with this reading, the same model drops to about 80% on correlated lattices and visibly fails the backbone task in Fig. 7. The absence of a simple baseline is therefore the most load-bearing gap for the paper's main claim: without it, the high accuracy does not yet establish that the GNN is learning the long-range rigidity transition.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper trains graph neural networks (ChebConv models with K=4 filters) to classify rigidity and connectivity percolation in two classes of 2D disordered networks: bond-diluted triangular lattices and off-lattice Brownian dynamics gels with permanent crosslinks. On lattice rigidity classification, the reported accuracies are 93--97%, increasing with system size; off-lattice rigidity accuracies are below 90%; connectivity percolation is treated as a ternary classification problem with class imbalance and reaches overall accuracies near 90% but with poor minority-class f1 scores. The paper also tests transfer across system sizes, variations of model inputs (edge lengths, node positions, largest-cluster filtering), open boundary conditions, and the effect of spatial correlations, including a backbone-identification failure for models trained on uncorrelated lattices. Trained models and code are made available.","tokens_in":12338,"tokens_out":5517,"duration_ms":54609,"significance":"The paper is a carefully executed proof-of-concept: the evaluation protocol is sound (held-out test sets, k-fold cross-validation, confusion matrices, and out-of-sample system-size tests), the authors report negative results honestly, and they provide open-source models. If the central claim survives closer scrutiny, it would establish that GNN surrogates can classify rigidity of uncorrelated 2D spring lattices with high accuracy and at lower computational cost than the pebble game. However, the significance is currently limited by two unresolved issues: the absence of a simple non-GNN baseline for the lattice rigidity task, and the dependence of off-lattice labels on an ad hoc shear-modulus threshold. The paper also demonstrates, through its own Fig. 7, that the high accuracy does not transfer to correlated networks, which substantially narrows the scope of the claimed applicability.","major_comments":[{"comment":"The central claim of >95% rigidity accuracy on diluted triangular lattices is not yet established because no non-GNN baseline is reported. The ChebConv model uses K=4 (Sec. II B), giving each node an effective receptive field of only about four hops on L=20 to L=100 lattices, while rigidity percolation is a system-spanning property. For randomly diluted lattices with p in [0.60, 0.70] centered near p_c≈0.66, the global bond density (or mean coordination number) is an almost sufficient statistic for the label, so a simple classifier such as logistic regression on mean degree or edge count may achieve comparable accuracy. Without such baselines evaluated on the same train/test splits, the high accuracy does not demonstrate that the GNN has learned rigidity percolation rather than a local density statistic. Please add these baselines and report their confusion matrices and accuracies.","section":"Sec. III A and Sec. II B"},{"comment":"The off-lattice rigidity labels are defined by comparing the simulated shear modulus to G'_thresh, whose values (0.15, 0.10, 0.05 for N=10^2, 10^3, 10^4) were chosen from preliminary runs. The reported accuracies therefore quantify agreement with this specific threshold criterion, not with a physically validated definition of rigidity. This matters because the threshold values vary with system size and are not justified by a physical argument. Please provide a sensitivity analysis of the trained-model accuracy with respect to G'_thresh (and, if feasible, t_max and omega), and validate the labels for small systems against an independent rigidity criterion such as the pebble game. Without this, the off-lattice accuracy numbers in Fig. 1(b) and Fig. 6 cannot be interpreted as physical prediction accuracy.","section":"Sec. II A and Table I"},{"comment":"The backbone-removal application shows that a model trained on uncorrelated lattices produces a clearly non-rigid, disconnected structure when asked to identify a load-bearing backbone. This is an important negative result, but it also delimits the central claim: the >95% accuracy applies only to graphs drawn from the same uncorrelated bond-dilution distribution as the training data, while off-lattice and correlated networks are predicted with substantially lower accuracy (below 90% off-lattice, and about 80% at the highest correlation strength in Fig. 7(a)). The abstract and introduction should state this scope restriction explicitly, since the phrase 'suitability for large, macromolecular assemblies' could be read as implying broader applicability than the data support.","section":"Sec. IV and Fig. 7"}],"minor_comments":[{"comment":"The statement that connectivity percolation scores are only 'slightly lower' is not supported by the f1 scores in Fig. 3: the minority class dconn=1 has f1 near or below 0.5 for all oversampling factors. The abstract should qualify that overall accuracy is near 90% but intermediate percolation states are poorly classified.","section":"Abstract and Sec. III B"},{"comment":"Reference [40] is a placeholder ('URL for SI will be added'); the supplementary tables and figures (S1--S6) are essential for reproducing the hyperparameter search and should be accessible before publication.","section":"Reference [40]"},{"comment":"The comparison with non-spectral GNNs is only qualitative ('no clear evidence that any meaningful training was achieved'); please report the numerical accuracies and loss curves so readers can assess this claim quantitatively.","section":"Sec. III C"},{"comment":"The phrase 'colinear springs' should be 'collinear springs', and the sentence describing the threshold selection from preliminary runs would benefit from a brief statement of how the authors judged a threshold to be 'suitable'.","section":"Sec. II A"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is honest, well-structured, and the evaluation protocol is a strength. The main risk is that the headline accuracy on lattice rigidity may be reproducible with a trivial mean-degree classifier; the authors should be asked to provide that baseline. The off-lattice label threshold is a secondary but real concern. The placeholder for the supplementary information should also be resolved before the paper can be fully evaluated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe thing to know: this is the first GNN treatment of rigidity percolation classification, and it is honestly done — but the headline 95%+ lattice accuracy is probably a local-density artifact, and the paper lacks the simple baseline that would settle it. The negative results (oversampling doesn't fix class imbalance; correlated networks drop to ~80%) are the more useful part.\n\nWhat's new and good: the evaluation is proper — held-out test sets, cross-validation, confusion matrices, out-of-system-size tests. The paper ships trained models (GitLab) and compares timings: 2.5 ms to classify rigidity vs 0.81 s with the pebble game for L=100. The authors are transparent about failure modes, especially the backbone-removal test (Fig. 7b) where the model trained on uncorrelated lattices produces clearly non-rigid fragments.\n\nSoft spots: the central claim rests on the 93–97% rigidity accuracy for bond-diluted triangular lattices. The ChebConv model uses K=4, so each node sees about four hops—strictly local. On random dilution near p_c ≈ 0.66, rigidity is almost perfectly correlated with mean coordination z = 6p, so a threshold on mean degree would likely match most labels. No non-GNN baseline (e.g., logistic regression on edge count) is reported, so we cannot know whether the GNN is learning rigidity percolation or a local density statistic. The off-lattice labels are defined by shear-modulus thresholds G'_thresh chosen from preliminary runs (Table I); those thresholds are disclosed but ad hoc, so the <90% accuracy there measures agreement with a self-defined criterion. Also, the abstract says connectivity performance is \"slightly lower\" when the minority f1 score is around 50% and overall accuracy barely exceeds 90% — that's an overstatement, though the body is frank. No raw data or SI is available (SI URL is a placeholder).\n\nThe stress-test concern holds up: without a baseline, the main claim does not establish that the GNN is learning the long-range transition.\n\nWho should read it: people working on ML surrogates for network properties and the gel modelling community. It deserves a serious referee, but the referee should ask for a simple baseline comparison and some analysis of what the model is actually using (e.g., ablation of node degree features). I'd send it to review with major revision, not desk-reject.\n\nBest,\n[Your name]","headline":"First GNN rigidity-percolation classifier, honestly evaluated but likely learning a local density proxy on lattices; the missing non-GNN baseline is the load-bearing gap.","tokens_in":12884,"tokens_out":3418,"would_cite":true,"duration_ms":33755,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Graph neural networks can classify lattice rigidity percolation with over 95% accuracy, the paper shows.","keywords":["graph neural networks","rigidity percolation","connectivity percolation","disordered spring networks","gel modelling","Chebyshev convolution","class imbalance","machine learning classification"],"falsifier":"Relabel a fixed set of off-lattice configurations with the rigidity threshold $G'_{\\text{thresh}}$ shifted by 20% in either direction from the values in Table I, retrain the best model, and check whether reported accuracy changes by more than training noise; a substantial change would mean the model is learning the threshold rule, not a physical rigidity transition. Independently, measure accuracy on correlated lattices at the strongest correlation strength shown in Fig. 7 and verify it lands near the reported ~80% value.","tokens_in":11853,"feed_emoji":"🕸️","tokens_out":6842,"duration_ms":64147,"temperature":0.7,"pith_summary":"This paper tries to establish that graph neural networks can act as fast, data-driven surrogates for rigidity and connectivity percolation in disordered particulate networks, using only the graph of particles and bonds. For bond-diluted triangular lattices, the trained models classify rigidity with accuracies around 93% for small systems up to 97% for larger systems. For off-lattice networks generated by Brownian dynamics, accuracy stays below 90%, and the paper argues the gap comes from spatial correlations that are absent in uncorrelated lattices. If the central claim holds, it offers a practical way to screen gel-like networks for rigidity far faster than running shear-modulus simulations.","feed_headline":"Neural network predicts rigid spring networks with up to 97% accuracy","feed_subtitle":"A trained graph model classifies rigidity in milliseconds, but correlated gel networks still defeat it.","key_machinery":"The central object is the graph neural network with Chebyshev spectral convolution layers (ChebConv), which uses the graph Laplacian approximated by a polynomial filter of order $K-1$ to pass messages across the particle-bond graph and read out a graph-level classification. The network starts from uniform scalar node features, optionally augmented with edge lengths or node positions, and is trained with cross-entropy loss with class weighting. The classification labels it learns against come from a periodic-boundary-aware connectivity percolation algorithm and from a thresholded simulated shear modulus for rigidity, with the threshold defined from preliminary runs.","core_discovery":"The central claim is that a graph neural network whose input is just the connectivity of particles and bonds can learn to classify rigidity percolation on diluted triangular lattices with accuracies that rise from about 93% for $L=20$ to 97% for $L=100$, and can classify connectivity percolation into dimensions 0, 1, or 2 with lower accuracy, limited by the rarity of the intermediate class. The paper also claims that off-lattice gel networks are harder, with best accuracies below 90%, and demonstrates that the likely cause is correlation: models trained on uncorrelated lattices lose accuracy when applied to correlated lattices, dropping from about 95% to about 80% as correlation strength increases. A further claim is that the trained rigidity classifier is much faster than the reference calculation, taking about 2.5 ms versus 0.81 s for $L=100$ lattices.","pith_inferences":["Beyond the paper, the natural next step is regression to the full shear modulus instead of binary rigidity; the paper notes this would demand more data and that current graph neural network layers cannot directly use multifidelity training inputs.","The backbone-extraction failure suggests the model learns statistical graph features rather than mechanical load paths, so extrapolation to geometries unlike the training data should be checked against a physics calculation before being trusted.","The correlation result implies a domain-shift problem: reported off-lattice accuracies likely depend on how well the training distribution matches the correlated structures being classified, so cheap correlated-lattice tests could serve as a diagnostic before expensive simulations are run.","If extended to regression, such a tool could support inverse-design workflows that search for microstructures with a target material response, using the fast model to filter candidates and simulations only for final verification."],"forward_implications":["Rigidity screening for bond-diluted two-dimensional spring networks can be done in milliseconds by a trained graph neural network, making it practical to scan many configurations or system sizes.","Any deployment on dynamically generated gels must include correlated training data; without it, accuracy is expected to sit near 80% rather than 95%.","Connectivity percolation classification into three classes is not fixed by oversampling, so applications needing the intermediate dimension should use binary formulations or enhanced sampling near the transition.","The speed advantage is currently specific to rigidity classification, since the connectivity percolation algorithm is already faster than the graph neural network in the paper's timing test.","The released trained models let researchers feed in their own simulated networks and test whether the reported accuracies transfer to other particulate systems."],"supporting_citations":[{"why":"Supplies the graph-based percolation algorithm that correctly handles periodic boundaries and generates the connectivity percolation labels.","marker":"[10]"},{"why":"Provides the pebble-game rigidity algorithm used as the reference method in the timing comparison.","marker":"[11]"},{"why":"Provide the viscoelastic-response code that generates the lattice networks and their simulated shear-modulus rigidity labels.","marker":"[31, 32]"},{"why":"Provides the Brownian dynamics protocol used to generate the off-lattice, thermally crosslinked gel networks.","marker":"[33]"},{"why":"Introduces the Chebyshev spectral convolution scheme that forms the core of the graph neural network architecture.","marker":"[37]"},{"why":"Supplies the correlated-lattice model used to test whether spatial correlations explain the lower off-lattice accuracies.","marker":"[42]"},{"why":"Provides the earlier convolutional neural network approach to connectivity percolation that this work extends to rigidity and graph-based inputs.","marker":"[26]"}],"fun_headline_variants":["97% accurate rigidity classification by graph nets","Graph neural net predicts rigidity with 97% accuracy","Rigidity percolation classified by graph net, but correlated gels stump it","Graph neural net classifies rigidity in 2.5 ms, gels harder"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a network counts as rigid exactly when its simulated shear modulus exceeds a threshold chosen from preliminary runs, and that the uncorrelated-lattice training distribution is representative of the correlated systems the model will face; if either fails, the high accuracies measure agreement with a self-defined rule or a mismatched training set rather than physical rigidity.","fun_headline_variants_meta":{"raw":{"variants":["97% accurate rigidity classification by graph nets","Graph neural net predicts rigidity with 97% accuracy","Rigidity percolation classified by graph net, but correlated gels stump it","Graph neural net classifies rigidity in 2.5 ms, gels harder"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000792,"raw_usage":{"total_tokens":3454,"prompt_tokens":878,"completion_tokens":2576,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":494,"completion_tokens_details":{"reasoning_tokens":2504}},"tokens_in":494,"tokens_out":2576,"duration_ms":18663,"temperature":1.0,"reasoning_tokens":2504,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T15:28:19.462290+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Relabel a fixed set of off-lattice configurations with the rigidity threshold $G'_{\\text{thresh}}$ shifted by 20% in either direction from the values in Table I, retrain the best model, and check whether reported accuracy changes by more than training noise; a substantial change would mean the model is learning the threshold rule, not a physical rigidity transition. Independently, measure accuracy on correlated lattices at the strongest correlation strength shown in Fig. 7 and verify it lands near the reported ~80% value.","supporting_citations":[{"cited_title":"Livraghi, K","cited_arxiv_id":null,"evidence_quote":"Supplies the graph-based percolation algorithm that correctly handles periodic boundaries and generates the connectivity percolation labels."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Brownian dynamics protocol used to generate the off-lattice, thermally crosslinked gel networks."},{"cited_title":"Defferrard, X","cited_arxiv_id":null,"evidence_quote":"Introduces the Chebyshev spectral convolution scheme that forms the core of the graph neural network architecture."},{"cited_title":"Zhang, L","cited_arxiv_id":null,"evidence_quote":"Supplies the correlated-lattice model used to test whether spatial correlations explain the lower off-lattice accuracies."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the earlier convolutional neural network approach to connectivity percolation that this work extends to rigidity and graph-based inputs."}],"review_version":1}