{"id":"e9fe7575-9225-4bf1-9cd8-cad16b4834b8","arxiv_id":"2501.12523","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Federated training of the DiGress molecular diffusion model produces molecules with validity and uniqueness close to a centralized model on the QM9 dataset.","lead":"This paper shows that a molecule-generating AI model (DiGress) can be trained collaboratively across separate data sites without pooling the data, using Intel's OpenFL framework. The federated model performs nearly as well as a model trained on the combined dataset, suggesting that privacy-preserving collaboration in drug discovery is feasible at a small scale.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The comparability claim in Table 1 is based on a single CL and a single FL run with no error bars or seed repetitions; the headline validity and uniqueness gaps are within the sampling error of 1,000 generated molecules, so the central claim is not yet statistically established.","rationale":"The reader identified the IID split as the weakest assumption, which is a genuine external-validity limitation, but the more load-bearing issue for the paper's central claim is the absence of any uncertainty quantification on the single CL/FL comparison. The abstract's claim is about this experiment's outcome; if the reported numbers are one draw from a high-variance process, the claim is not established even for the favorable IID setting. The IID limitation is acknowledged in the future-work section, whereas the lack of error bars is not, and it affects the interpretation of every metric in Table 1. I agree with the reader's overall CONDITIONAL verdict because the core feasibility result is plausible and the integration of DiGress with OpenFL is a useful proof-of-concept, but the requested revisions should prioritize multi-seed evaluation with confidence intervals. The paper has positive aspects: it builds on a standard public model (DiGress) and an established FL framework (OpenFL), and the described workflow integration is credible. However, without repeated runs or code release, the central comparability claim remains underdetermined, so the CONDITIONAL verdict should stand with this additional experimental requirement.","tokens_in":6317,"tokens_out":5588,"duration_ms":56211,"concrete_test":"Rerun the CL and FL protocols exactly as in Section 5 with at least 5 independent seeds (varying model initialization, data shuffling, and diffusion sampling). Report mean ± standard deviation for validity, uniqueness, NLL, and MAE, and construct a 95% confidence interval for the CL minus FL difference in validity and uniqueness. If the intervals include zero and the validity difference stays within a pre-specified equivalence margin (e.g., ±0.01), the comparability claim is supported; if the intervals exclude zero or the FL model is systematically worse, the claim should be weakened to 'within statistical noise' or 'worse.'","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that federated training is comparable to centralized training rests entirely on the small differences between single-point estimates in Table 1. Section 5 reports one CL run (100 epochs) and one FL run (100 rounds, 1 epoch per collaborator) and evaluates 1,000 generated molecules, but no random seeds, repeated runs, or confidence intervals are reported. The observed gaps are of the same order as the sampling noise of the evaluation protocol: with n = 1,000, a validity difference of 0.004 (0.9560 vs 0.9600) is well within one standard error (~0.006), and the uniqueness difference of 0.003 is about 1.3 standard errors. Because model initialization, data shuffling, and molecular sampling are stochastic, a different seed could plausibly reverse or enlarge these gaps. The paper's qualitative conclusion in Section 6 that FL 'can match the performance of centralized learning' therefore goes beyond what the evidence supports; the current numbers are consistent with comparability but do not demonstrate it. This concern is load-bearing because if the single-run comparison is unrepresentative, the abstract's strongest claim fails even before considering the acknowledged limitation that the QM9 split is IID rather than a realistic heterogeneous silo.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a federated training framework for DiGress, a discrete denoising diffusion model for molecular graph generation, implemented with the OpenFL workflow and task-runner APIs. The authors simulate a two-client federation by randomly splitting the QM9 dataset, train DiGress in both federated and centralized modes, and compare NLL, regressor MAE, validity, and uniqueness on 1,000 generated molecules. They report that the federated model achieves performance comparable to the centralized model, with small absolute percent differences across all four metrics. The manuscript also documents the OpenFL integration details and includes training/validation curves and sampled molecules. The central claim is that federated training can preserve molecular generation quality while enabling privacy-preserving collaborative learning.","tokens_in":6537,"tokens_out":2619,"duration_ms":28756,"significance":"If the comparability claim holds, the paper provides a useful feasibility demonstration: a widely used molecular diffusion model can be trained in a federated fashion with negligible degradation, addressing a real barrier to multi-institutional drug discovery. The contribution is primarily empirical and systems-oriented rather than algorithmic, but it fills a relevant gap by showing OpenFL's workflow API can accommodate a complex generative model with a separate regressor. The paper is clear about the architecture and provides a concrete integration recipe that other practitioners could follow. However, the strength of the empirical claim is currently limited by the absence of statistical uncertainty quantification and by the use of a randomized (near-IID) data split, which the authors themselves acknowledge does not mimic real heterogeneous silos.","major_comments":[{"comment":"The central claim of comparability rests on single point estimates from one centralized run and one federated run, with no random seeds, repeated trials, or confidence intervals. For the evaluation protocol using 1,000 generated molecules, the validity difference of 0.004 (0.9560 vs 0.9600) is well within the standard error of the estimate (~0.006), and the uniqueness difference of 0.003 is about 1.3 standard errors. The paper should report mean and variance over multiple seeds (or at least confirm the observed differences are reproducible across initializations) before claiming that federated learning 'can match the performance of centralized learning' in Section 6. Without such information, the results are consistent with comparability but do not statistically establish it.","section":"Section 5, Table 1"},{"comment":"The experiment randomly splits QM9 across two collaborators, producing near-IID partitions that are not representative of real pharmaceutical data silos, which are typically heterogeneous and non-IID. While Section 6 lists non-IID and scaffold splits as future work, the abstract's claim that the federated model 'achieves comparable performance with a model trained on centralized data' is not qualified to the IID setting and could be misread as evidence for real-world federations. The authors should either add experiments with a non-IID split (e.g., scaffold-based) or explicitly restrict the key claim to the IID simulation setting.","section":"Section 5, data split; Section 6, limitations"}],"minor_comments":[{"comment":"The reported absolute percent difference for NLL is inconsistent: Table 1 states 3.06%, while the text reports 3.11%. The value computed from Eq. (2) using the table's numbers is 3.06%, so the text should be corrected.","section":"Section 5, text vs Table 1"},{"comment":"The uniqueness metric is described as 'assessed over 1,000 generated molecules', but it is not stated whether duplicates are removed across the 1,000 samples or within each diffused chain; defining the exact protocol would improve reproducibility.","section":"Section 5, evaluation metrics"},{"comment":"The paper does not include a reproducibility statement listing software versions, hardware, or random seeds. Providing these details, along with a link to any released code for the DiGress-OpenFL integration, would strengthen the experimental contribution.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable feasibility study for a systems-oriented venue, but the empirical evidence needs strengthening before publication. The self-citation of OpenFL is understandable given the authors are also OpenFL developers, but the paper should more clearly position any potential conflict of interest. The use of a random split and single-run comparisons are the main technical weaknesses; if the authors can add seed repetitions and non-IID experiments, the claim would be substantially more convincing."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a clean, honest feasibility study that says “we wired DiGress into OpenFL and on a random two-client split of QM9 it behaves about like the centralized baseline.” That is what it demonstrates, and the paper mostly does not overclaim except in the conclusion. The contribution is integration plus a head-to-head table, not a new method, and the table is thinner than it looks because there are no error bars.\n\nWhat is new and what it does well: first integration of DiGress with OpenFL using the Task Runner API; the workflow description is clear and practical. The evaluation uses standard external metrics (RDKit validity, uniqueness on 1000 samples, NLL, MAE). Training budgets are matched (100 rounds vs 100 epochs, 1 epoch per client). The loss and sampling curves are helpful. The authors explicitly acknowledge in Section 6 that scaffold splits, larger datasets like GuacaMol, and heterogeneous data remain to be tested. There is no circularity: evaluation metrics are independent benchmarks.\n\nSoft spots, in proportion: the central comparability claim rests on single runs. With 1,000 sampled molecules, the validity gap (0.9560 vs 0.9600) is within one standard error (~0.006), and the uniqueness gap is about 1.3 standard errors. So the numbers are consistent with comparability, not proof of it. The conclusion sentence that FL “can match” centralized performance goes beyond the evidence. Second, the random split produces near-IID clients, which is the easy case; real pharmaceutical silos are heterogeneous, and the paper acknowledges this only as future work. This does not invalidate the feasibility result but does limit what it means. Minor: the NLL percent difference is 3.06 in the table but 3.11 in the text (and MAE 1.35 vs 1.36). No code is released for the DiGress–OpenFL integration, so reproducibility is limited to the OpenFL framework itself.\n\nWho this is for: researchers working on federated molecular generation or using OpenFL, as a proof-of-concept data point. It would benefit from a revision adding multiple seeds or confidence intervals, a non-IID split, and code release. As a proof of concept it is reasonable and the authors are transparent about limitations.\n\nRecommendation: I would send it to peer review, not desk reject. It is a legitimate integration result, and the main concern is one the authors can address with a modest amount of additional computation.","headline":"A straightforward integration of DiGress into OpenFL with an honest but statistically under-powered comparability claim; worth reviewing, not worth citing.","tokens_in":7076,"tokens_out":1784,"would_cite":false,"duration_ms":18384,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper shows that a discrete denoising diffusion model for molecules, trained across separate data sites with federated learning, matches a model trained on a single centralized dataset.","keywords":["federated learning","molecular generation","discrete denoising diffusion","DiGress","OpenFL","drug discovery","QM9 dataset","graph generation"],"falsifier":"Reproduce the same two-site federation with a scaffold split of QM9, where chemically related molecules are kept on the same site, then compare validity and uniqueness to a centralized model; a drop materially larger than the reported 0.42% and 0.31% differences would show the comparability claim depends on the random split.","tokens_in":6093,"feed_emoji":"🧪","tokens_out":7512,"duration_ms":71411,"temperature":0.7,"pith_summary":"This paper tries to establish that collaborative, privacy-preserving training can replace centralized training for AI-driven molecular generation without a meaningful performance penalty. It does this by adapting DiGress, a graph-based discrete denoising diffusion model, to the OpenFL federated learning framework instead of a single data server. A two-client federation trained on the QM9 molecule dataset produced nearly the same validity and uniqueness of generated molecules as the centralized model, with slightly higher negative log-likelihood and regressor error. If the comparison holds, drug-discovery teams could pool proprietary data across institutions without moving or exposing that data.","feed_headline":"Federated diffusion model matches centralized molecule training","feed_subtitle":"Training on siloed drug data without sharing it yields nearly identical validity and uniqueness.","key_machinery":"The central object is DiGress, a discrete denoising diffusion model that generates molecules as graphs, with atoms as nodes and bonds as edges. The argument is carried by a federated training loop defined in OpenFL's workflow API: the aggregator sends the global denoiser and regressor weights to collaborators, each trains locally on private data, and the aggregator combines updates by weighted federated averaging, $w_{t+1}=\\sum_{k=1}^{K}\\alpha_k w_k^t$. The denoising model minimizes cross-entropy between its predicted node and edge types and the original graph, while the regressor predicts the highest occupied molecular orbital and dipole moment by mean squared error; both models are federated together. OpenFL's Task Runner API is then able to run the same training in a production-ready setting with encrypted communication and trusted execution environments.","core_discovery":"The paper reports that a federated DiGress model, trained over 100 rounds with one epoch per collaborator at two sites, matches the quality of a DiGress model trained for 100 epochs on the whole QM9 dataset. On validity (RDKit-sanitized molecules) the federated model scored 0.9560 versus 0.9600 for centralized learning; on uniqueness over 1,000 generated molecules it scored 0.9989 versus 0.9958. Negative log-likelihood and regressor mean absolute error were also close, within roughly 3% and 1.4% respectively. The authors interpret these results as evidence that federated learning can support collaborative drug-design model training while keeping data localized.","pith_inferences":["The random split used in the experiment is a favorable case where the two sites' data are statistically similar; a federation of companies with different chemical libraries would likely show a larger federated-versus-centralized gap, so the comparability claim is best read as a lower bound on degradation.","Because QM9 molecules are small and simple, validity and uniqueness are near ceiling; tests on larger, more complex datasets such as GuacaMol would reveal whether the federated model degrades more sharply outside this regime.","The same federated wrapper could be applied to other graph generative models with minor code changes, making the OpenFL workflow API a general way to privacy-preserve generative chemistry, not just DiGress.","The paper evaluates aggregate metrics, not whether the generated molecules are novel relative to each collaborator's private data; a useful next check is measuring scaffold novelty and overlap with each site's training set."],"forward_implications":["A pharmaceutical federation can train a molecular generator on data that stays at each institution, with no raw molecules leaving the site.","The small performance gap (validity 0.9560 versus 0.9600, uniqueness 0.9989 versus 0.9958) means a federated model is usable in practice for early drug-design screening.","Because only the model weights cross the network, the same architecture can be paired with stronger security layers without changing the molecular generation algorithm.","The approach extends beyond the two-client experiment: any number of collaborators can join the same round-based averaging loop, subject to communication and data-size constraints.","Conditional property guidance survives federation, since the regressor's MAE moved only from 0.6932 to 0.7026."],"supporting_citations":[{"why":"This reference supplies the DiGress model architecture and training algorithm, including the discrete denoising diffusion loss and the property regressor.","marker":"[12]"},{"why":"This reference supplies the OpenFL framework whose workflow and task runner APIs execute the federated training.","marker":"[3]"},{"why":"This reference supplies the weighted federated averaging rule used to aggregate collaborator model updates.","marker":"[6]"},{"why":"This reference supplies the QM9 dataset of more than 130k molecules used for both the centralized and federated training runs.","marker":"[8]"},{"why":"Together with [8], this reference provides the source of the QM9 molecular structures and quantum-chemical properties.","marker":"[10]"},{"why":"This reference provides the denoising diffusion probabilistic model background that the discrete graph diffusion approach builds on.","marker":"[4]"}],"fun_headline_variants":["Federated diffusion matches centralized molecule generation","Privacy-first training yields same drug candidate quality","OpenFL-driven model rivals central on molecular validity","Federated learning for drug design: equal validity, uniqueness","Siloed data can train molecule generator as well as pooled"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole comparison rests on the assumption that randomly splitting QM9 into two sites resembles a real collaboration between data owners; if the sites' molecule distributions are sharply different, federated averaging may not converge to centralized quality.","fun_headline_variants_meta":{"raw":{"variants":["Federated diffusion matches centralized molecule generation","Privacy-first training yields same drug candidate quality","OpenFL-driven model rivals central on molecular validity","Federated learning for drug design: equal validity, uniqueness","Siloed data can train molecule generator as well as pooled"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000364,"raw_usage":{"total_tokens":1903,"prompt_tokens":827,"completion_tokens":1076,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":443,"completion_tokens_details":{"reasoning_tokens":1015}},"tokens_in":443,"tokens_out":1076,"duration_ms":8622,"temperature":1.0,"reasoning_tokens":1015,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T17:05:37.693637+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Reproduce the same two-site federation with a scaffold split of QM9, where chemically related molecules are kept on the same site, then compare validity and uniqueness to a centralized model; a drop materially larger than the reported 0.42% and 0.31% differences would show the comparability claim depends on the random split.","supporting_citations":[{"cited_title":"In: The Eleventh International Conference on Learning Representations (2023),https://openreview.net/forum? id=UaAD-Nu86WX","cited_arxiv_id":null,"evidence_quote":"This reference supplies the DiGress model architecture and training algorithm, including the discrete denoising diffusion loss and the property regressor."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"This reference supplies the OpenFL framework whose workflow and task runner APIs execute the federated training."},{"cited_title":"Scientific Data1 (2014)","cited_arxiv_id":null,"evidence_quote":"This reference supplies the QM9 dataset of more than 130k molecules used for both the centralized and federated training runs."}],"review_version":1}