{"id":"c6c62756-8990-42a8-876b-2d6109800019","arxiv_id":"2508.15015","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"SEAL makes GNN predictions inherently interpretable by decomposing molecules into fragments and defining the prediction as a sum of per-fragment contributions, enforced with regularized fragment-local message passing.","lead":"This paper introduces SEAL, a graph neural network that predicts molecular properties by splitting molecules into chemical fragments and defining the prediction as the sum of each fragment's contribution. The benefit is a model whose answer automatically comes with a chemically meaningful explanation, which matters when domain experts must trust predictions in drug discovery and materials design.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Fragment contributions are exact by construction, but the leap from algebraic additivity to causal, fragment-local explanations is untested: with nonzero inter-fragment weights, c_i can depend on the whole molecule.","rationale":"The reader's weakest_assumption is the fragment-additivity hypothesis, and this stress-test converges on the same point: the paper must show that inter-fragment information flow is sufficiently suppressed that each c_i reflects only fragment-local signal. The reader's conditional verdict already flags this, along with the fidelity protocol asymmetry and the user-study limitations. My analysis agrees that these are fixable but real concerns. The most load-bearing of them is the causal/fragment-local interpretation of c_i, because it is the central intellectual contribution of the paper: an exact additive decomposition is trivially true by construction, but interpreting the additive terms as chemically meaningful causal contributions requires an empirical check that the paper does not perform. The synthetic results on boron and rings-max provide direct evidence that when the fragmentation and the true substructure do not align, the explanation quality collapses, which is exactly what the fragment-local hypothesis predicts. This does not warrant rejection: the architecture is transparent, the algebra is clean, and the synthetic benchmark offers some genuine support where the hypothesis holds. It does warrant conditioning the acceptance on an explicit counterfactual validation of the contribution scores and on either a fair fidelity protocol or a clear statement that the real-data comparison is not apples-to-apples. Since the reader already reached CONDITIONAL, my verdict is UNCHANGED.","tokens_in":30380,"tokens_out":4834,"duration_ms":65114,"concrete_test":"Take a trained SEAL model on hERG (λ=10^-4) and on PAINS (λ=0). For 100 test molecules, compute c_i for each fragment, then compute the counterfactual difference Δ_i = ŷ(G) - ŷ(G'), where G' is the graph with fragment i's atoms removed and the model re-run on the remaining graph. Compute the rank correlation between c_i and Δ_i across fragments and molecules. If the correlation is weak (e.g., Spearman ρ < 0.7) on either task, the claim that c_i is a fragment-local causal contribution fails. As a secondary check, rerun Table 4 for SEAL using input-level node-feature masking instead of zeroing c_i; if SEAL's positive-fidelity advantage shrinks to the baseline range, the real-data 'outperforms' claim is an artifact of the asymmetric protocol.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that the fragment contributions c_i in Eq. (2) are faithful and 'causal influence' estimates. What has to be true for that claim is that suppressing inter-fragment message passing via the L1 penalty in Eq. (4) actually makes each fragment's representation depend only on that fragment, so zeroing c_i is equivalent to removing the fragment's causal effect. This is never verified. The L1 penalty on W_inter does not zero the inter-fragment weights unless λ is extremely large, and the paper itself selects λ=0 for PAINS and small λ for hERG and AqSol (Tables 1, 3). With λ=0 there is unrestricted information flow between fragments, so c_i is a function of the whole molecule, not of fragment i alone. The synthetic benchmark exposes exactly this failure mode: SEAL's SE drops to 0.34 on rings-max and 0.88 on boron, where the target substructure is not aligned with the BRICS fragmentation. Therefore the algebraic identity y-hat = sum c_i + b guarantees only an additive accounting of the model's output, not that c_i measures the fragment's counterfactual causal contribution. The real-data fidelity comparisons additionally mask contributions rather than input features for SEAL while masking input features for baselines, so the headline 'outperforms' on hERG and CYP2C9 may partly reflect this protocol asymmetry. This is a load-bearing gap because the paper's scientific contribution is precisely the claim that SEAL's explanations are chemically meaningful and causal, not merely that the architecture is additive.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces SEAL, a graph neural network for molecular property prediction that decomposes molecules into BRICS fragments, applies a modified GCN layer with separate intra-fragment and inter-fragment weights, and produces predictions as the sum of per-fragment contributions (Eqs. 1–2). An L1 penalty on inter-fragment weights (Eq. 4) is intended to reduce cross-fragment information leakage and make the fragment contributions more local and interpretable. The method is evaluated on the B-XAIC synthetic benchmark, three TDC real-world datasets (hERG, CYP2C9, AqSolDB), and a user study with chemistry experts. The authors report competitive predictive accuracy and claim state-of-the-art explanation fidelity and human-aligned interpretability.","tokens_in":30693,"tokens_out":6559,"duration_ms":74315,"significance":"The idea of making a GNN's prediction an additive sum of fragment-level contributions is attractive: if the contributions are genuinely local and chemically meaningful, SEAL would provide built-in, directly usable fragment explanations without post-hoc explainers. The synthetic benchmark with ground-truth substructures is an appropriate test bed, and the paper is commendably honest about the boron and rings-max failures. The code is released, and the user study, despite its reporting gaps, addresses an important dimension rarely evaluated. However, the paper's central interpretability claim is currently not established: additive decomposition alone does not make the contributions causal or fragment-local, and the headline real-world fidelity comparison uses different masking protocols for SEAL than for the baselines.","major_comments":[{"comment":"The fidelity comparison is protocol-asymmetric. The Metrics text states 'For all models, we mask node features at the input level, ensuring a fair comparison,' yet the SEAL rows in Table 4 exactly match the 'SEAL-mask' rows in Appendix C (e.g., hERG λ=10⁻⁴: 0.63/0.09/0.71/0.07/0.78/0.05 in both Table 4 and Table 14), meaning SEAL is evaluated by zeroing its fragment contributions c_i, not by masking input features. Since the prediction is defined as ŷ = Σ c_i + b (Eq. 2), zeroing c_i is a built-in linear sensitivity operation, which inflates positive fidelity and deflates negative fidelity by construction. The claim that SEAL 'outperforms all other methods in both positive and negative fidelity' on hERG and CYP2C9 is therefore not supported by the current evaluation. The authors should either re-run all methods under the same input-masking protocol or justify contribution masking with ap","section":"Real-world datasets, Metrics; Table 4; Appendix C"},{"comment":"The central claim that c_i is a 'causal influence' or fragment-local explanation is not established. Eq. (2) is an algebraic decomposition, not a causal statement. c_i is computed from a fragment representation that is the sum-pooled output of multiple GNN layers, and with nonzero inter-fragment weights W_inter, h_i depends on the entire molecule. The L1 penalty in Eq. (4) does not zero W_inter unless λ is infinite, and the selected λ is small or zero for several tasks (λ=0 for PAINS, λ=10⁻⁴ for hERG and AqSol, λ=10⁻³ for rings-count). The synthetic results show exactly the failure mode: when the ground-truth substructure does not align with the BRICS fragmentation, SEAL's SE drops to 0.34 on rings-max and 0.88 on boron (Table 2). To support the locality/causal reading, the authors should report the actual magnitudes or sparsity of W_inter after training, compare zeroing c_i with removin","section":"Fragment Contributions/SEAL-GCN, Eqs. (2)–(4); Tables 1–2, 6–7"},{"comment":"The user study is reported without essential experimental detail. The text does not state the number of participants, the total number of votes, or any statistical test comparing SEAL to the other methods. The sentence 'Deconvolution and IntegratedGradients were chosen 5 times, and InputXGradients was selected 3 times' is ambiguous and does not permit the reader to assess whether SEAL's apparent preference is significant or due to chance. For a central claim about 'human-aligned interpretability,' the authors should report participant count, recruitment procedure, per-method vote totals, and a significance test (e.g., per-question or per-participant mixed-effects model).","section":"User study"},{"comment":"The abstract's claim that SEAL 'outperforms other explainability methods in both quantitative attribution metrics' is too broad given the paper's own results. On the rings-max task, SEAL's SE is 0.34, while GuidedBackprop reaches 0.67 and Saliency 0.55; on boron, SEAL (0.88) is also below several gradient baselines (1.00). The text in the results section acknowledges the boron and rings-max difficulties, but the abstract, introduction, and discussion generalize the outperformance claim. The authors should qualify the claim and report whether the overall advantage is statistically significant across tasks, or restrict the claim to the tasks where SEAL is indeed strong (PAINS, rings-count, indole, X, P).","section":"Abstract and Discussion; Table 2"}],"minor_comments":[{"comment":"The fragment representation is described as sum pooling followed by LayerNorm, but LayerNorm is not shown in Eq. (1); please align the equation with the text.","section":"Methods, Eq. (1)"},{"comment":"The baseline list says 'Deconvolution (Mahendran and Vedaldi 2016), (Shrikumar et al. 2016), and Guided Backpropagation' — the Shrikumar method name is missing (likely DeepLIFT).","section":"Related Work / Baselines"},{"comment":"The naming convention for masking strategies ('mask-abs', 'mask', 'abs', 'zero') is confusing and appears to mix 'zeroing features' with 'masking contributions.' Please define each term explicitly and use consistent terminology in the main text.","section":"Appendix C"},{"comment":"For the regression task (AqSolDB), the fidelity metric is defined only for classification ('proportion of times the predicted class changes'). Table 13 reports numerical values for regression without defining the metric; please provide the regression-specific definition.","section":"Metrics, real-world datasets"},{"comment":"The 'Baseline' row is not named per task; since different tasks have different best-performing baselines, please identify which method is shown for each column.","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The architecture is simple and potentially useful, and the synthetic evaluation is honestly reported, including failures. The main issues are the asymmetric fidelity protocol (which undermines the headline comparison) and the unverified leap from additive decomposition to causal, fragment-local explanation. Both are addressable with additional experiments and re-analysis, so I recommend major revision rather than rejection. The user study also needs more rigorous reporting before the human-alignment claim can be assessed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid, useful paper with one load-bearing assumption that needs to be surfaced. The architecture is genuinely new relative to fragment GNNs that rely on external explainers: separate intra/inter-fragment weights, L1 regularization on inter-fragment weights, and a sum-of-contributions readout. Eq. 2 makes the attributions exact by construction for the model's own output. The synthetic benchmark is a real strength: SEAL nails indole, rings-count, and PAINS explanation, and the paper honestly shows failure on boron and rings-max, which are exactly the tasks where the fragment-additivity hypothesis breaks. That honesty earns credit.\n\nThe soft spots are real but fixable. The real-data fidelity comparison is not apples-to-apples. The headline numbers come from zeroing SEAL's own contribution terms (SEAL-mask rows), while baselines mask input node features. Appendix C shows SEAL-mask-abs, a closer analog to input masking, lands much closer to baseline values. So the claim that SEAL 'outperforms with a significant margin' is partly a protocol artifact. Second, the causal language. With λ=0 for PAINS and small λ for hERG and AqSol, the L1 penalty does not zero the inter-fragment weights, so c_i can depend on the whole molecule. Algebraic additivity does not imply causal locality. The paper should either test the locality condition or drop 'causal influence' from the abstract. Third, the user study has no participant count, no significance test, and no per-question statistics beyond '14 of 19.' Suggestive, not conclusive. Fourth, λ is selected per task on the validation set; that's defensible but should be flagged when reporting explanation scores, since the explanation quality is conditional on a tuned hyperparameter.\n\nWho this is for: people working on GNN interpretability and molecular property prediction. The architecture is simple enough to reproduce, and the fragment-level output is genuinely aligned with how chemists talk. I'd send it to peer review. The fixes are major but not rethink-the-approach: matched fidelity protocols, tempered language, user-study numbers, and a discussion of when c_i is actually local. My own verdict is conditional, not reject.","headline":"SEAL is a clean, useful fragment-additive GNN with a real claim to novelty and an honest synthetic benchmark; the main gap is that 'causal fragment influence' is asserted rather than tested, and the real-data fidelity protocol flatters SEAL.","tokens_in":31222,"tokens_out":2452,"would_cite":true,"duration_ms":29376,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper introduces SEAL, a graph neural network whose prediction for a molecule is, by construction, the sum of separate contributions from chemically meaningful fragments, and argues these contributions are faithful explanations because","keywords":["graph neural networks","molecular property prediction","interpretability","fragment contributions","attribution","explainable AI","substructure explanation","message-passing regularization"],"falsifier":"Build a synthetic task whose ground truth is a known nonlinear interaction between two distant fragments, for example a positive label only when both fragments appear together. If SEAL's explanation quality on positive examples stays near chance (SE close to 0.5) or its prediction accuracy falls far below GIN, then the fragment-additivity premise fails exactly where inter-fragment interactions carry the signal.","tokens_in":30232,"feed_emoji":"🧪","tokens_out":6086,"duration_ms":63697,"temperature":0.7,"pith_summary":"SEAL is a graph neural network for molecular property prediction that rewrites the prediction head so the output is, by construction, the sum of per-fragment contributions plus a bias. Each fragment contribution is produced by an MLP over the sum-pooled representations of the fragment's atoms, so no separate explainer is needed: the score the model uses is the explanation. To keep those contributions local and chemically meaningful, SEAL-GCN layers use separate weights for edges inside a fragment and edges between fragments, with an L1 penalty that shrinks the inter-fragment weights. The paper argues that this design yields explanations that are faithful by construction and competitive in accuracy, and supports this with synthetic benchmarks, fidelity tests on real-world datasets, and a user study in which chemists preferred SEAL's explanations in 14 of 19 compounds.","feed_headline":"SEAL makes molecular predictions a sum of fragment scores","feed_subtitle":"Chemists picked SEAL's fragment explanations 14 of 19 times over seven standard baselines.","key_machinery":"The load-bearing object is the fragment-contribution decomposition: molecule, BRICS-style fragments, sum-pooled fragment representations, fragment scores c_i, and prediction as an exact sum. The new layer SEAL-GCN carries the argument: it applies separate weight matrices to same-fragment neighbors (W_intra) and cross-fragment neighbors (W_inter), and an L1 penalty on W_inter forces the model to encode as much signal as possible locally inside fragments. The hyperparameter λ controls the strength of this locality pressure; larger λ means less inter-fragment information flow and, the paper finds, cleaner explanations, at some cost on tasks that genuinely need long-range information.","core_discovery":"The central claim is that a GNN can be built so that its prediction is literally a sum of fragment-level numbers, and those numbers are the explanation. After message passing, each fragment's atoms are sum-pooled and passed through an MLP to give c_i; the prediction is y-hat = sum of c_i plus a bias. Because there is no graph-level readout that mixes all atoms before the final layer, attributing the prediction to fragments is not a post-hoc guess but a restatement of the model's arithmetic. The architecture, SEAL-GCN, enforces locality by giving intrafragment and interfragment edges separate weights and adding an L1 penalty on interfragment weights; the authors hypothesize that for many chem","pith_inferences":["The additivity premise is itself testable and marks the paper's implicit boundary: one could construct a benchmark where the true label is a nonlinear function of two distant fragments, and SEAL's explanation quality should collapse; the paper's own boron and largest-ring results point in that direction.","Because fragment definitions determine what a 'contribution' means, the same architecture with finer or coarser fragmentation would produce different explanations; a chemical-discovery tool built on this idea should probably expose fragmentation as a user choice rather than a fixed preprocessing step.","The exact-sum head makes SEAL a natural base for counterfactual reasoning, since replacing or removing a fragment changes the prediction by exactly the change in that fragment's contribution; the paper does not pursue this, but it could support a generative optimization loop.","When the optimal λ is near zero, the task is dominated by long-range interactions and fragment-level reasoning may be inappropriate; SEAL's own regularization search could therefore double as a diagnostic for whether a dataset is fragment-local."],"forward_implications":["For fragment-local properties, explanations are exact by construction: the highlighted fragment scores are the arithmetic components of the predicted value, so no separate explainer is required and there is no approximation step to argue about.","The model can trade a small amount of accuracy for interpretability: SEAL reaches AUROC 0.85 on hERG and 0.81 on CYP2C9 versus 0.86 for GIN, with substantially better fidelity scores.","Fragment contributions give chemists a direct handle for hypothesis generation: a predicted property can be decomposed into which fragment adds or subtracts, and those numbers can be ranked, compared across molecules, or used to propose modifications.","The λ selection protocol, which picks the highest regularizer not significantly worse than the best model by a Wilcoxon signed-rank test, provides a principled way to choose how much locality to enforce on a given task.","On tasks requiring cross-fragment interactions, the same mechanism predicts where SEAL will fail: the boron and largest-ring synthetic tasks degrade precisely because the ground-truth substructure is spread over larger motifs that the chosen fragmentation cannot cleanly isolate."],"supporting_citations":[{"why":"Supplies the BRICS fragmentation scheme that defines the molecular fragments whose contributions are summed.","marker":"(Degen et al. 2008)"},{"why":"Provides the B-XAIC synthetic benchmark with ground-truth substructure explanations and the SE/NE evaluation metrics.","marker":"(Proszewska, Danel, and Rymarczyk 2025)"},{"why":"Defines GNNExplainer, the main perturbation-based baseline SEAL is compared against.","marker":"(Ying et al. 2019)"},{"why":"Defines Integrated Gradients, a strong gradient-based baseline in the fidelity and user-study comparisons.","marker":"(Sundararajan, Taly, and Yan 2017)"},{"why":"Supplies the TDC platform from which the real-world hERG, CYP2C9, and solubility datasets are drawn.","marker":"(Huang et al. 2021)"},{"why":"Supplies the hERG inhibition dataset used for the cardiac-safety classification task.","marker":"(Karim et al. 2021)"},{"why":"Supplies the CYP2C9 inhibition dataset used for the metabolism task.","marker":"(Veith et al. 2009)"},{"why":"Supplies the AqSolDB aqueous solubility regression dataset.","marker":"(Sorkun, Khetan, and Er 2019)"},{"why":"Supplies LayerNorm used in forming fragment representations before the contribution MLP.","marker":"(Ba, Kiros, and Hinton 2016)"},{"why":"Supports the paper's premise that toxicity-related molecular behavior is often associated with specific fragment patterns.","marker":"(von Korff and Sander 2006)"}],"fun_headline_variants":["SEAL's predictions are literally sums of fragment scores","Chemists preferred SEAL's fragment explanations 14 out of 19 times","No post-hoc guesses: SEAL explains by summing fragment numbers","Fragment scores are the prediction, not an afterthought—SEAL"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The load-bearing premise is that molecular properties, and the networks that predict them, are dominated by fragment-local signals, so suppressing inter-fragment message passing does not destroy prediction accuracy or explanation truth; the boron and largest-ring failures in the paper show places where this premise breaks.","fun_headline_variants_meta":{"raw":{"variants":["SEAL's predictions are literally sums of fragment scores","Chemists preferred SEAL's fragment explanations 14 out of 19 times","No post-hoc guesses: SEAL explains by summing fragment numbers","Fragment scores are the prediction, not an afterthought—SEAL"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000302,"raw_usage":{"total_tokens":1570,"prompt_tokens":731,"completion_tokens":839,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":475,"completion_tokens_details":{"reasoning_tokens":766}},"tokens_in":475,"tokens_out":839,"duration_ms":8728,"temperature":1.0,"reasoning_tokens":766,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T18:11:04.640058+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build a synthetic task whose ground truth is a known nonlinear interaction between two distant fragments, for example a positive label only when both fragments appear together. If SEAL's explanation quality on positive examples stays near chance (SE close to 0.5) or its prediction accuracy falls far below GIN, then the fragment-additivity premise fails exactly where inter-fragment interactions carry the signal.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the BRICS fragmentation scheme that defines the molecular fragments whose contributions are summed."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines GNNExplainer, the main perturbation-based baseline SEAL is compared against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines Integrated Gradients, a strong gradient-based baseline in the fidelity and user-study comparisons."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the hERG inhibition dataset used for the cardiac-safety classification task."},{"cited_title":"P.; Lloyd, D","cited_arxiv_id":null,"evidence_quote":"Supplies the CYP2C9 inhibition dataset used for the metabolism task."},{"cited_title":"C.; Khetan, A.; and Er, S","cited_arxiv_id":null,"evidence_quote":"Supplies the AqSolDB aqueous solubility regression dataset."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supports the paper's premise that toxicity-related molecular behavior is often associated with specific fragment patterns."}],"review_version":1}