{"id":"3031bf24-a7f8-4eec-ba53-10c683a9bef7","arxiv_id":"2412.12034","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A thermodynamics-constrained graph neural network predicts liver deformation, velocity, and stress in milliseconds on unseen geometries, but its test evaluation is contaminated by model selection on the test set.","lead":"The paper trains a graph neural network with a thermodynamic structure to simulate liver tissue deformation in milliseconds, reporting relative position errors below 0.15% and forward passes as fast as 1.65 ms. It matters because it aims to make patient-specific, real-time digital twins for surgical planning and haptics plausible, though the evaluation has serious selection issues.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Evaluation leakage: early stopping on the full test set and a test set that includes validation simulations invalidate the reported generalization accuracy; a disjoint validation/test split is the decisive check.","rationale":"The reader's verdict is REJECT and I agree with the rejection, but for a more proximal reason than the stated weakest assumption. The paper's headline numbers are generated under a protocol in which the test set is used for early-stopping model selection (Section 2.4), and the 'test' set deliberately includes the validation simulations. This is a textbook information leak: the model is selected to minimize error on the exact simulations later reported as 'unseen.' Hence Tables 3–4 and the rollout figures cannot support the abstract's sub-0.15% position and sub-7% stress/velocity claims. The synthetic-data realism point raised by the reader is real but is an external-validity concern; if the evaluation is contaminated, the numbers are uninterpretable even for the synthetic population. I note also the thermodynamic 'ensured' claim is unsupported because only a soft degeneracy loss is described and the hard-constraint construction is not specified, but I do not need that to reject. A retraining experiment with a disjoint validation/test split is the decisive check; without it, the central claim of accurate generalization to unseen anatomies is unverified.","tokens_in":16735,"tokens_out":4275,"duration_ms":39986,"concrete_test":"Retrain the hybrid model using the same data and hyperparameters, but split the 190 L5 simulations into disjoint sets: for example, 38 validation simulations for early stopping and 152 test simulations never used for any model selection. Select the checkpoint with lowest validation RRMSE, then compute Tables 3–4 and rollout RRMSE on the untouched 152. If q RRMSE remains near 1.27e-3 and v/σ RRMSE remain below 5.5e-2/3.0e-2, the selection-leak concern is not supported; if q RRMSE rises to ~2e-3 or more, or the hybrid-vanilla gap shrinks, the paper's generalization claims are overstated and the current REJECT stands.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claims—position RRMSE ~1.27e-3, velocity/stress RRMSE ~5.2e-2/2.7e-2—rest on test-set metrics. Section 2.4 states that validation uses 20% of simulations from held-out anatomy L5, and that the primary testing dataset is based on the 190 total simulations of L5, including those also used for validation to avoid biases. Section 2.4 further states that weight selection for test inference is done through early stopping based on the mean relative rollout error across the complete test dataset. This means the test set is not held out: checkpoints are chosen to minimize error on the very simulations later reported as unseen. Tables 3–4 and Figures 7–10 therefore conflate model selection with evaluation and give optimistically biased generalization estimates. This is an internal protocol flaw, not a matter of consensus; it directly undercuts the previously unseen anatomy claim even if the synthetic constitutive model were accepted. The thermodynamic guarantee is a separate unsupported assertion (soft degeneracy loss plus unspecified hard constraints), but the evaluation contamination is the most load-bearing because it invalidates the headline accuracy claims. A clean re-evaluation with a disjoint validation/test split would settle whether the reported numbers hold.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a graph neural network surrogate for real-time simulation of human liver deformations. The architecture combines a central mesh graph with actuator subgraphs and augments a vanilla message-passing encoder-processor-decoder with a \"metriplectic\" decoder that outputs energy/entropy gradients and per-edge operators intended to realize a port-metriplectic (GENERIC) structure. Training uses synthetic finite element data from a homogeneous Ogden-Prony constitutive model, with noise-based data augmentation and both a vanilla and a hybrid (physics-informed) variant. On a test anatomy L5, the hybrid model reports lower RMSE/RRMSE than the vanilla baseline and inference times of about 1.65–7.3 ms in a forward pass. The abstract and conclusions claim generalization to previously unseen anatomies and thermodynamic consistency by construction.","tokens_in":16831,"tokens_out":5637,"duration_ms":51691,"significance":"If the accuracy and thermodynamic-compliance claims were rigorously established, this would be a valuable step toward real-time, patient-specific surgical simulation. The multi-graph scheme with actuator subgraphs and the explicit comparison against a parameter-matched vanilla baseline are well-targeted design choices, and reporting per-seed standard deviations in Tables 3–4 is good practice. However, the evaluation protocol uses the test set for early stopping and includes validation simulations in the test set, which invalidates the headline generalization numbers; the thermodynamic guarantee is neither formalized in the text nor empirically verified on test data; and the external validity is limited by the synthetic, homogeneous constitutive model. These issues are fixable in a revision, but they are central to the paper's claims.","major_comments":[{"comment":"The evaluation protocol does not use a truly held-out test set. The text states that \"the primary testing dataset is based on the 190 total simulations of L5, including those also used for validation to avoid biases\" and that \"weight selection for test inference is done through early stopping based on the mean relative rollout error across the complete test dataset.\" Because the test set is used for model selection, the RMSE/RRMSE values in Tables 3–4 and the rollout plots in Figures 7–10 are optimistically biased estimates of generalization to an unseen anatomy. Please re-run the evaluation with a disjoint validation split (for example, a subset of L5 simulations reserved for validation only) and report test metrics from the checkpoint selected on that validation split.","section":"2.4"},{"comment":"The claim that the metriplectic structure \"ensures that the network satisfies thermodynamic principles during inference\" is not supported by any formal guarantee or empirical verification. Equation (29) adds a soft loss Ldeg with lambda_d = 5, which encourages but does not enforce the degeneracy conditions (20)–(21), and the hard constraints mentioned in Figure 4 are not defined mathematically anywhere in the text: no explicit parameterization of L and M that guarantees skew-symmetry and positive semi-definiteness, nor any construction satisfying the degeneracy conditions, is provided. Please either specify the hard-constraint construction in detail or report test-time residuals of dE/dt, dS/dt, and the degeneracy conditions on the held-out anatomy; without this, the thermodynamic guarantee is not supported.","section":"2.2.1 and 2.3.4"},{"comment":"The RRMSE definition is ambiguous and potentially misleading. The denominator ||zref||^2_infinity is not defined; if it is the squared infinity norm of the entire reference state over all nodes and snapshots, then the metric is a global relative error and is not directly interpretable as a per-node relative error, which is how the abstract's \"relative position errors below 0.15%\" is phrased. Please clarify the normalization and, if appropriate, report per-snapshot or per-node relative errors as well.","section":"3.1, Eq. (34)"}],"minor_comments":[{"comment":"The phrase \"soft and hard constrains\" should be \"soft and hard constraints.\"","section":"Abstract"},{"comment":"There is a typo: \"Terefore\" should be \"Therefore.\"","section":"Appendix A"},{"comment":"The label \"Train\" for rollout error on training data is not a generalization measure; consider renaming it (e.g., \"Memorized (Train)\") or removing it from the generalization comparison to avoid confusing memorization with extrapolation.","section":"3.3, Figure 10"},{"comment":"Table 5 reports no standard deviations, unlike Tables 3–4; please add per-seed variation or state clearly that these are single-run results.","section":"Appendix E, Table 5"}],"recommendation":"major_revision","confidential_remarks":"The paper leans heavily on a lineage of prior works from the same group (refs. 23–29), and the incremental novelty over Tierz et al. (2024) and Hernandez et al. (2023) should be articulated more clearly: what exactly is new beyond applying the already-published port-metriplectic GNN architecture to liver data? The evaluation-leakage issue is the most urgent technical problem; a clean re-evaluation with a disjoint validation/test split could change the headline numbers, so the editors should request that re-analysis before considering publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth reading, but the headline accuracy claims don't survive contact with the evaluation protocol. What is actually new here: an application of thermodynamics-informed graph neural networks to liver deformation, with a five-anatomy benchmark, a systematic comparison against a vanilla GNN baseline, and real-time inference measurements. The multi-graph actuator scheme is adapted from Pfaff et al., and the metriplectic machinery from the authors' own prior work, so the novelty is incremental—but the liver dataset, the ablations on message-passing steps, and the timing measurements are genuine engineering work.\n\nThe soft spot is load-bearing, and it is right there in Section 2.4: the test set includes the validation simulations, and model weights are selected by early stopping on the mean rollout error across the complete test dataset. That means the reported RRMSE on L5 is not an independent test of generalization to an unseen anatomy; the 'unseen' numbers are optimistically biased, and the abstract's position error below 0.15% and stress under 7% are not trustworthy as stated. The hybrid-vs-vanilla comparison may still be informative under the same flawed protocol, but the absolute claims need a disjoint validation split before I'd cite them.\n\nThe thermodynamic guarantee is a separate unsupported assertion. The degeneracy conditions are added as a soft loss; the hard constraints are mentioned but never specified, and there is no inference-time verification of energy conservation or entropy production. Saying the network 'ensures' thermodynamic principles during inference is stronger than the evidence. The synthetic material model is also an acknowledged simplification—homogeneous, omitting vascularization—which limits the clinical 'digital twin' framing, though that is a reasonable simplification for a methods paper. No code or data are released, so the numbers are hard to verify independently.\n\nMy recommendation: if this came to me as an editor, I'd send it to review. The application and benchmark are valuable, the paper is clearly written, and the evaluation flaw is fixable. The authors should re-run with a proper validation split, report only test-set metrics after model selection, and either specify the hard constraints or soften the thermodynamic guarantee. With those changes this could become a solid reference for real-time biomechanical surrogates.","headline":"Solid application paper with a serious evaluation leakage: early stopping on the test set undermines the headline accuracy claims, but the benchmark and architecture work may still be worth a careful revision.","tokens_in":17489,"tokens_out":3181,"would_cite":false,"duration_ms":28254,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A graph neural network constrained by the GENERIC metriplectic structure predicts the response of a previously unseen liver anatomy in about 1.65 milliseconds.","keywords":["graph neural networks","GENERIC","metriplectic structure","soft tissue simulation","digital twin","real-time inference","liver biomechanics","thermodynamics-informed machine learning"],"falsifier":"Run the trained model on an ex vivo or in vivo liver under the same traction and compression protocols and compare measured surface displacement and stress with predictions; if positional or stress errors on the real organ exceed the reported 0.15% and 7% thresholds, then the accuracy claim is an artifact of the synthetic ground truth. A cheaper intermediate check is to rerun the finite element data with a heterogeneous, vascularized constitutive model and see whether the network's error on unseen anatomies stays below the reported bounds.","tokens_in":16414,"feed_emoji":"⚡","tokens_out":8819,"duration_ms":69939,"temperature":0.7,"pith_summary":"This paper tries to show that a graph neural network can act as a real-time digital twin of a human liver, predicting how the organ deforms, moves, and stresses under surgical traction or compression loads. The key claim is that embedding the GENERIC thermodynamic structure into the network, by making each mesh node an open system that exchanges energy with its neighbours, keeps predictions physically consistent even on liver anatomies never seen in training. If correct, the work closes the speed gap between offline finite element simulation and interactive haptic surgery, with forward-pass inference in the single-digit milliseconds. The paper reports relative position errors below 0.15% and stress and velocity errors under 7%, with the physics-informed hybrid model consistently beating a vanilla graph network of equal size.","feed_headline":"Physics-aware graph network predicts liver response in 1.65 ms","feed_subtitle":"Thermodynamic constraints hold position error below 0.15% on anatomies never used in training.","key_machinery":"The machinery is a multi-graph message-passing network combined with a nodal port-metriplectic implementation of GENERIC. GENERIC writes the time evolution of a dissipative system as $\\dot z = L\\,\\partial E/\\partial z + M\\,\\partial S/\\partial z$, where $L$ is a skew-symmetric Poisson operator and $M$ a symmetric positive-semidefinite friction matrix; the paper reformulates this per node so each mesh node is an open subsystem exchanging energy with neighbours through graph edges. Instead of predicting $\\dot z$ directly, the hybrid decoder outputs the local energy and entropy gradients and the flattened edge operators, then a reparametrization layer aggregates boundary contributions and assembles the update, with forward Euler integration rolling the state forward. This object carries the argument because thermodynamic consistency is built into the architecture itself rather than relying on the data alone.","core_discovery":"The central discovery claimed is that a graph neural network whose decoder is forced to output the local building blocks of the metriplectic GENERIC evolution equation, per-node energy and entropy gradients plus per-edge friction and Poisson operators, rather than raw state derivatives, learns a dissipative soft-tissue simulator that is both faster and more stable than a data-only network. The imposed structure is enforced twice: softly through a degeneracy loss term in training, and strongly through the reparametrization that assembles the nodal update from the decoded blocks. On a test set of 190 simulations on an unseen fifth liver geometry, the hybrid model lowers rollout RMSE and RRMSE for position, velocity, and Cauchy stress, suppresses error spikes near loaded nodes and stress concentrators, and runs at up to 600 Hz for one-pass configurations. The authors interpret this as evidence that thermodynamic inductive biases improve extrapolation to unseen anatomy and load states, making patient-specific real-time simulation feasible.","pith_inferences":["Because the synthetic ground truth is homogeneous and vascularization-free, the clinically relevant claim is untested; if real tissue response differs, the method still works as a simulator but not as a patient twin.","The same architecture could be applied to other viscoelastic soft tissues, since its tissue-specific content is confined to the training data and constitutive parameters, not the network design.","The observed quadratic scaling of inference time with message-passing steps suggests an adaptive scheme that varies pass count by region or required accuracy could push interactive rates on larger meshes.","The noise-injection training strategy contributes to rollout stability independently of the physics bias; ablating it separately would clarify how much of the stability comes from thermodynamics versus data augmentation."],"forward_implications":["Surgical simulators could get haptic feedback at more than 600 Hz, since one-pass models infer the full liver state in 1.65 ms.","A twin trained on four anatomies transfers to a fifth unseen anatomy with lower rollout errors than a vanilla graph network, so patient-specific deployment may not require per-patient retraining.","Imposing the metriplectic structure improves rollout stability, preventing stress-induced error spikes from corrupting position predictions over 20-time-step simulations.","The port-metriplectic locality removes the need to assemble global matrices $L$ and $M$, so the method can scale to finer meshes as message passing and hardware allow."],"supporting_citations":[{"why":"Supplies the thermodynamics-informed graph network framework and the GENERIC-based physical bias that this paper extends to open systems.","marker":"[23]"},{"why":"Supplies the mesh-based graph network message-passing processor and the multi-graph training scheme used for the simulator.","marker":"[14]"},{"why":"Provides the locally informed graph neural network approach that treats each node as an open thermodynamic system, avoiding global matrix assembly.","marker":"[27]"},{"why":"Establishes the port-metriplectic formulation for open systems that the nodal GENERIC implementation builds on.","marker":"[28]"},{"why":"Provides the experimentally calibrated liver material parameters used for the Ogden and Prony constitutive model in the synthetic ground truth.","marker":"[7]"},{"why":"Supplies the five liver anatomies used to build training and test geometries.","marker":"[38]"},{"why":"Defines the GENERIC equation that the metriplectic structure is based on.","marker":"[33]"},{"why":"Defines the Ogden hyperelastic strain energy density used in the finite element simulations.","marker":"[39]"},{"why":"Defines the Prony series used to model the viscous time-dependent behavior of the tissue.","marker":"[40]"}],"fun_headline_variants":["Liver sim in 1.65 ms with physics-aware GNN","Thermo-constrained GNN predicts liver response in 1.65 ms","1.65 ms liver prediction via physics-aware graph net","Physics-aware GNN: liver response in 1.65 ms, 0.15% error"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the synthetic finite element data, produced with a homogeneous Ogden-Prony constitutive model whose parameters come from literature, captures the true mechanical response of a real human liver; the paper explicitly omits vascularization and other anatomical features.","fun_headline_variants_meta":{"raw":{"variants":["Liver sim in 1.65 ms with physics-aware GNN","Thermo-constrained GNN predicts liver response in 1.65 ms","1.65 ms liver prediction via physics-aware graph net","Physics-aware GNN: liver response in 1.65 ms, 0.15% error"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000387,"raw_usage":{"total_tokens":2061,"prompt_tokens":981,"completion_tokens":1080,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":597,"completion_tokens_details":{"reasoning_tokens":998}},"tokens_in":597,"tokens_out":1080,"duration_ms":8727,"temperature":1.0,"reasoning_tokens":998,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:20:24.527156+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the trained model on an ex vivo or in vivo liver under the same traction and compression protocols and compare measured surface displacement and stress with predictions; if positional or stress errors on the real organ exceed the reported 0.15% and 7% thresholds, then the accuracy claim is an artifact of the synthetic ground truth. A cheaper intermediate check is to rerun the finite element data with a heterogeneous, vascularized constitutive model and see whether the network's error on unseen anatomies stays below the reported bounds.","supporting_citations":[{"cited_title":"Port-metriplectic neural net- works: thermodynamics-informed machine learning of complex physical systems","cited_arxiv_id":null,"evidence_quote":"Establishes the port-metriplectic formulation for open systems that the nodal GENERIC implementation builds on."},{"cited_title":"Martinez Martinez","cited_arxiv_id":null,"evidence_quote":"Provides the experimentally calibrated liver material parameters used for the Ogden and Prony constitutive model in the synthetic ground truth."},{"cited_title":"Dynamics and thermodynamics of complex fluids","cited_arxiv_id":null,"evidence_quote":"Defines the GENERIC equation that the metriplectic structure is based on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Ogden hyperelastic strain energy density used in the finite element simulations."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Prony series used to model the viscous time-dependent behavior of the tissue."}],"review_version":1}