{"id":"ba4d1244-a01e-4e69-b500-0da27f580268","arxiv_id":"2502.19391","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Igformer co-designs antibody CDR sequences and structures by adding personalized propagation, global attention, and dual equivariant message passing to dyMEAN, with modest benchmark improvements.","lead":"Igformer is a neural network that predicts both the amino acid sequence and 3D structure of antibody binding loops from a target antigen, building on the prior model dyMEAN. On standard benchmarks it reports small but consistent gains, such as higher amino-acid recovery and lower docking RMSD, though the comparison omits error bars and shared training conditions.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section 2.1's data-leakage claim is unsupported because RAbD and IgFold test complexes are PDB-derived and very likely present in the Nov-2022 SAbDab training set; without a reported CDR-H3 identity overlap check, the headline performance improvements cannot be trusted.","rationale":"The reader's weakest assumption is exactly the concern I would prioritize: evaluation-set leakage. This is the most load-bearing because every reported comparison on RAbD and IgFold depends on the test complexes being disjoint from the training snapshot. Additional issues are real but secondary: baseline numbers are taken from the dyMEAN paper rather than retrained on an identical split, no error bars or significance tests are provided, and the dimension-wise coordinate normalization in Appendix D.3.3 appears inconsistent with the claimed E(3)-equivariance in Theorem C.3. However, even if those issues were fixed, a training/test overlap would invalidate the central claim. I therefore agree with the reader's CONDITIONAL verdict and would make the overlap analysis the primary condition for acceptance. If the proposed CDR-H3 identity check finds any test antibody within the 40% threshold of a training antibody, the verdict should move to REJECT; if it finds no overlap, the concern is resolved and the remaining issues reduce to reproducibility and statistical reporting.","tokens_in":26130,"tokens_out":5504,"duration_ms":51244,"concrete_test":"For each of the 60 RAbD and 51 IgFold test complexes, extract the CDR-H3 sequence using the same IMGT numbering and MMseqs2 clustering procedure described in Section 2.1 and Appendix E.1, and run MMseqs2 with the same 40% identity threshold over the union of the 3,246 training antibodies and all test sequences. Report (i) the number of test complexes whose exact PDB ID appears in the training set, and (ii) the maximum CDR-H3 identity between any test complex and any training antibody, together with a histogram of identities. If any test complex shares more than 40% CDR-H3 identity with a training antibody, or appears by exact PDB ID, then Tables 1 and 4 are compromised; if all test complexes are below the threshold and absent from the training set, the leakage concern is resolved. This check should be run with the released training-split metadata and preprocessing code.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.1 states that the SAbDab data are clustered at a 40% CDR-H3 sequence-identity threshold and split into 3,246 training and 365 validation antibodies, then asserts that evaluating on RAbD (60 complexes) and IgFold (51 complexes) 'prevents data leakage.' No overlap analysis is reported between the training set and either test set. Because RAbD and IgFold complexes are antibody-antigen structures curated from the PDB and SAbDab is a PDB-derived database with a November 2022 snapshot, the RAbD complexes are in the same database and are very likely present in the training set unless explicitly removed. If an exact test antibody, or one sharing more than 40% CDR-H3 identity with a training antibody, is evaluated, then the reported AAR 43.50%, RMSD 7.15 Å, and DockQ 0.450 on RAbD (Table 1) and DockQ 0.522 on IgFold (Table 4) could reflect memorization of the masked CDR-H3 sequence and structure rather than generalization. Appendix E.1 even describes the curated SAbDab database as serving as the foundation for both benchmark sets, making an explicit de-duplication check essential. This is the most load-bearing concern because every downstream comparison in Tasks 1-4 depends on the integrity of the evaluation split.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes Igformer, an end-to-end equivariant graph network for full-atom antibody CDR co-design. It represents antibody-antigen complexes as intra- and inter-graphs, uses Approximate Personalized Propagation and a simplified graph transformer to refine the paratope-epitope interface, and applies triangle multiplicative and axial attention modules before dual equivariant message passing for coordinate and sequence prediction. Evaluation on the RAbD and IgFold benchmarks reports improved AAR, RMSD, and DockQ over dyMEAN and earlier baselines across CDR-H3 design, multiple CDR design, full antibody design, and complex structure prediction.","tokens_in":26429,"tokens_out":8333,"duration_ms":77489,"significance":"If the empirical results survive a leakage-free evaluation, Igformer would represent a meaningful advance in antibody co-design, improving both sequence recovery and docking quality over the previous end-to-end state of the art. The proposed interface-refinement architecture is plausible, and the ablation studies consistently show contributions from APP, SGFormer, triangle modules, and dual EMP. However, the headline comparisons currently rest on an asserted but unverified separation of training and test sets, on baselines whose training conditions are not shown to be matched, on missing uncertainty estimates, and on an equivariance theorem that is proved only for the EMP submodule. These gaps must be closed before the central claims can be accepted.","major_comments":[{"comment":"Section 2.1 states that 'This test set selection prevents data leakage during the evaluation phase,' but no overlap analysis is reported between the 3,246 training antibodies and the 60 RAbD or 51 IgFold test complexes. Appendix E.1 explicitly says that the curated SAbDab dataset 'serves as the foundation for both the RAbD and the IgFold benchmark dataset,' so the test complexes are drawn from the same source snapshot used for training. Because the training set is clustered at 40% CDR-H3 identity, the decisive check is an identity and structural overlap report after that clustering; without it, the reported gains in AAR, RMSD, and DockQ in Tables 1-4 could reflect memorization rather than generalization. Please report overlap statistics and remove any training antibodies within the clustering threshold of a test antibody, or justify why this is unnecessary.","section":"Section 2.1 and Appendix E.1"},{"comment":"Tables 1-4 report single-point performance values without error bars, and Appendix E.3 says that baselines are run with 'hyperparameters and training procedures from their official releases' rather than retrained on the same SAbDab train/validation split used for Igformer. The caption of Table 1 further says that results with * are 'collected from the dyMEAN paper,' which is not the same as reproducing those baselines under identical conditions. To support the headline comparisons, including AAR 43.50 versus 42.64, RMSD 7.15 versus 8.42, and DockQ 0.450 versus 0.408 in Table 1, the authors should either retrain all baselines on the identical split with multiple seeds or provide explicit evidence that the published numbers were obtained on the same snapshot and split, and they should report means and standard deviations over seeds or checkpoints.","section":"Appendix E.3 and Tables 1-4"},{"comment":"Theorem C.3 claims E(3)-equivariance of the full Igformer pipeline, but the proof in Appendix D.5.2-D.5.4 only shows that the EMP coordinate update is equivariant under rotations and translations of a fixed set of input coordinates. It does not cover the template-based coordinate initialization and Kabsch alignment of Appendix D.3.2, the chain-wise centering and dimension-specific normalization of Appendix D.3.3, or the iterative replacement of epitope and paratope coordinates in Algorithm 1. The normalization step Xi = (X'_i - mu_d)/sigma_d is not an E(3) transformation in general, so Theorem C.3 is not established as stated. Either extend the proof to the full pipeline or state the equivariance claim only for the EMP module.","section":"Theorem C.3 and Appendix D.5"},{"comment":"Appendix F.1 (Table 11) selects the paratope definition per task, using H3 for Tasks 1, 3, and 4 and H3+L3 for Task 2, and Appendix F.2 (Figure 5) chooses w = 0.2 as the value with peak DockQ across all tasks. If these choices were made after observing test benchmark results, the reported performance is optimistic and the comparisons to dyMEAN are not on equal footing. Please state whether these hyperparameters were fixed before evaluating on RAbD and IgFold, and describe the validation-set selection procedure.","section":"Appendix F.1 and F.2"}],"minor_comments":[{"comment":"The abstract reports an 11.84% reduction in RMSD, while Section 2.5 reports a 12.44% relative improvement in RMSD; these numbers should be reconciled.","section":"Abstract and Section 2.5"},{"comment":"The text says Igformer achieves a 2.2% relative improvement in AAR over dyMEAN, while the abstract and Table 1 imply a relative improvement of 2.02% (43.50/42.64 - 1); please make the description consistent.","section":"Section 2.2 and Abstract"},{"comment":"The caption states that results with * are collected from the dyMEAN paper, but the table itself contains no star symbols; please insert the markers or remove the note.","section":"Table 1 caption"},{"comment":"The sentence referring to the RAbD benchmark cites Dunbar et al. (2013), but the RAbD benchmark is defined by Adolf-Bryfogle et al. (2018); the citation should be corrected.","section":"Appendix E.1"},{"comment":"The final sentence says that 'we update the global coordinates again based on Equations 12 and 13,' which is too vague to determine whether normalization is reapplied; please specify the exact coordinate transformation used at this stage.","section":"Appendix D.3.3"},{"comment":"The caption contains a typo, 'inter-grpah refinement module,' which should read 'inter-graph refinement module.'","section":"Table 8 caption"}],"recommendation":"major_revision","confidential_remarks":"The core technical work appears worth pursuing, but the data-leakage question is the decisive risk. If an overlap analysis shows that test complexes share more than 40% CDR-H3 identity with training antibodies, the headline performance claims would be invalid, so the authors should be asked to report that analysis prominently. Clarifying which baselines were newly trained against which split is equally important for the fairness of the comparison. The manuscript is best handled as a major revision rather than a rejection because the identified problems are addressable within the scope of the paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Xingyi,\n\nQuick take: Igformer is a plausible incremental extension of dyMEAN, and its ablations suggest the new modules do something real. But the headline numbers are not verifiable as reported, and the E(3)-equivariance claim looks wrong. It deserves peer review, but the referee should demand a de-duplication check, retrained baselines, and a corrected equivariance statement.\n\nWhat's actually new: the paper adds an inter-graph refinement stage to dyMEAN, combining approximate personalized propagation (APP) with a simplified graph transformer (SGFormer), plus triangle and axial attention and a dual EMP module. That combination for the paratope-epitope interface is not in dyMEAN, and the ablations in Table 5 show each piece contributes to the reported gains. The consistency of improvements across the four tasks is evidence that the engineering is sound. This is a legitimate contribution to an active subfield.\n\nThe soft spots, in proportion:\n\n1. The data-leakage concern is real. Section 2.1 asserts that RAbD and IgFold evaluation 'prevents data leakage' but gives no overlap analysis against the 3,246 training antibodies. Both benchmarks are PDB-derived, and the training snapshot is SAbDab from November 2022. Without a CDR-H3 sequence identity check, the reported AAR and DockQ gains could be partly memorization. This is the load-bearing issue.\n\n2. Baselines are not retrained on the same split; the dyMEAN numbers are taken from the original paper. That makes the comparison weaker than the text implies.\n\n3. There are no error bars or significance tests, so we cannot tell whether the 1-5% relative improvements are stable. The abstract's 11.84% RMSD reduction versus 12.44% in the body is a minor but telling inconsistency.\n\n4. Theorem C.3, the E(3)-equivariance claim, does not hold for the pipeline as described. The chain-specific centering and per-dimension normalization in D.3.3 are not global rigid transformations, so the proof covers a cleaner version of the model than the one actually run. This is fixable, but the claim should be weakened or the normalization changed.\n\nNone of these kill the architecture's potential, but they do mean 'state of the art' is not currently established.\n\nWho this is for: readers working on antibody co-design, equivariant GNNs, or protein interface models. It deserves a serious referee with the expectation of major revisions, and I'd want code and the overlap analysis before trusting the numbers.","headline":"Plausible incremental model for antibody co-design, but the headline performance claim rests on an unverified train/test split and the equivariance theorem appears to break on the actual pipeline.","tokens_in":27008,"tokens_out":3484,"would_cite":false,"duration_ms":30164,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Igformer, an end-to-end full-atom antibody co-design model, claims new best results on CDR-H3 sequence recovery and antibody-antigen docking by refining the binding-interface graph with personalized propagation plus global attention.","keywords":["antibody co-design","complementarity-determining region","CDR-H3","equivariant graph neural network","full-atom structure prediction","antigen-binding interface","amino acid recovery","DockQ"],"falsifier":"Compare every one of the 3,246 training antibodies with each of the 60 RAbD and 51 IgFold test complexes by CDR-H3 sequence identity at the 40% threshold used for clustering; any test complex that lands in a training cluster would refute the leakage-free evaluation claim.","tokens_in":25773,"feed_emoji":"🧬","tokens_out":9815,"duration_ms":84795,"temperature":0.7,"pith_summary":"This paper tries to establish that antibody co-design—jointly predicting the amino acid sequence and the 3D structure of the antibody loops that bind an antigen—can be made substantially more accurate by modeling the antibody-antigen binding interface as a graph refined at two scales. It introduces Igformer, an end-to-end full-atom model whose interface update first propagates local chemical and geometric information with personalized propagation, then captures global dependencies with transformer-style attention, and uses the refined interface to drive sequence and structure prediction. The reported payoff is concrete: on the RAbD CDR-H3 task Igformer reaches 43.50% amino acid recovery, 7.15 Å RMSD, and DockQ 0.450, compared with dyMEAN's 42.64%, 8.42 Å, and 0.408, and on the IgFold complex-structure benchmark it reaches DockQ 0.522 versus 0.452. If these numbers hold, multi-scale interface refinement of this kind would give computational antibody design a better starting point for therapeutic candidates, improving both the sequence and the docking quality without a separate docking stage.","feed_headline":"Igformer beats dyMEAN on antibody CDR sequence and docking","feed_subtitle":"End-to-end model reports 43.5% amino-acid recovery and DockQ 0.450 on CDR-H3, topping prior antibody designers.","key_machinery":"The load-bearing mechanism is the two-scale inter-graph refinement between the antibody paratope and the antigen epitope. An Equivariant Message Passing (EMP) module, which updates residue embeddings and atomic coordinates while preserving $E(3)$-equivariance, supplies initial residue representations. The inter-graph is then refined by Approximate Personalized Propagation (APP), which balances local neighborhood context against initial residue features, and by a Simplified Graph Transformer (SGFormer), which applies global attention across the whole binding interface. The refined inter-graph is passed through a Triangle Multiplicative module and Axial Attention module, and finally through separate EMP modules for intra- and inter-graph processing that generate the predicted coordinates and amino acid sequences. The paper proves that coordinates are $E(3)$-equivariant and embeddings are $E(3)$-invariant, and its ablations identify the inter-graph refinement and dual-scale message passing as the main sources of the reported DockQ improvement.","core_discovery":"On its own terms, the paper's central discovery is that a carefully refined antibody-antigen inter-graph converts interface modeling into a two-scale problem: local chemical and geometric constraints are propagated by Approximate Personalized Propagation, long-range dependencies are captured by a Simplified Graph Transformer, and the refined edge structure then drives separate intra- and inter-graph equivariant message passing for full-atom structure and sequence prediction. The paper claims this yields state-of-the-art epitope-binding CDR-H3 co-design on the RAbD benchmark, with amino acid recovery 43.50% and DockQ 0.450 versus dyMEAN's 42.64% and 0.408, and stronger complex structure prediction on the IgFold benchmark, with DockQ 0.522 versus 0.452. Ablation studies in the paper attribute the gain to each component, with the personalized propagation and the dual EMP design showing the largest drops when removed.","pith_inferences":["Editorial inference: if the leakage-free split holds, the model should generalize to epitopes deposited after the November 2022 SAbDab snapshot; testing on a time-split of newer complexes would separate learned binding physics from training-set familiarity.","Editorial inference: the design principle of local propagation plus global attention over an inter-graph is not antibody-specific and could plausibly transfer to nanobodies or T-cell receptor interfaces, though the paper does not test this.","Editorial inference: even at 43.5% CDR-H3 recovery, most residues in the most variable loop are not recovered, so practical therapeutic use would likely pair Igformer with affinity maturation or experimental screening, a step the paper leaves implicit.","Editorial inference: because the reported gains are measured on docking metrics against the full antigen, the interface loss's 6.6 Å contact threshold may understate long-range or conformational binding contributions; ablating that threshold would clarify whether the DockQ gain comes from local contact geometry or global paratope placement."],"forward_implications":["CDR-H3 co-design improves on both sequence and docking fronts: 43.50% amino acid recovery, 7.15 Å RMSD, and DockQ 0.450 on RAbD, each ahead of dyMEAN.","Designing all six CDRs also improves (63.55% versus 60.05% AAR, and DockQ 0.4817 versus 0.3973), while full-antibody design improves less but consistently (73.69% AAR and DockQ 0.4600).","On complex structure prediction from complete antibody sequences, Igformer reaches DockQ 0.522 versus dyMEAN's 0.452 on the IgFold benchmark, and the paper reports it also outperforms a ground-truth-guided HERN upper bound.","Ablations show that every proposed component contributes, with removing APP causing the largest DockQ drop on Task 1 (0.416 versus 0.450) and removing the dual EMP design being close behind.","Because the coordinates are $E(3)$-equivariant and the embeddings are $E(3)$-invariant, predictions are consistent under rigid rotations and translations, a physical requirement for generated antibody structures."],"supporting_citations":[{"why":"dyMEAN, the direct end-to-end competitor whose reported numbers are the baseline for Igformer's improvements and for the '*' entries in Table 1.","marker":"Kong et al. (2023a)"},{"why":"RAbD benchmark and RosettaAb baseline; defines the 60-complex test set for Tasks 1-3.","marker":"Adolf-Bryfogle et al. (2018)"},{"why":"SAbDab, the source database from which the training and validation antibodies are drawn.","marker":"Dunbar et al. (2013)"},{"why":"SAbDab in the biotherapeutics era; the updated database snapshot used for training.","marker":"Schneider et al. (2021)"},{"why":"IgFold benchmark and IgFold structure-prediction baseline used in Task 4.","marker":"Ruffolo et al. (2023a)"},{"why":"Personalized PageRank propagation, the basis of the APP module's local-global information balance.","marker":"Klicpera et al. (2019)"},{"why":"Triangle attention and related pair-update mechanisms that the iterative update module adapts.","marker":"Jumper et al. (2021)"},{"why":"MEAN, the earlier equivariant baseline that Igformer compares against in Task 1.","marker":"Kong et al. (2023b)"}],"fun_headline_variants":["Igformer: 43.5% CDR-H3 recovery, DockQ 0.450, beats dyMEAN","Igformer beats dyMEAN with 43.5% CDR-H3 recovery","Igformer improves full-atom antibody co-design over dyMEAN"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the 60 RAbD and 51 IgFold test complexes are genuinely absent from the 3,246-antibody SAbDab training set; the paper asserts this prevents leakage but reports no overlap check, so a hidden match at the 40% CDR-H3 clustering threshold would turn the reported gains into a memorization artefact.","fun_headline_variants_meta":{"raw":{"variants":["Igformer: 43.5% CDR-H3 recovery, DockQ 0.450, beats dyMEAN","Igformer beats dyMEAN with 43.5% CDR-H3 recovery","Igformer improves full-atom antibody co-design over dyMEAN"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001647,"raw_usage":{"total_tokens":6523,"prompt_tokens":902,"completion_tokens":5621,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":518,"completion_tokens_details":{"reasoning_tokens":5546}},"tokens_in":518,"tokens_out":5621,"duration_ms":33473,"temperature":1.0,"reasoning_tokens":5546,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T12:22:36.129808+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare every one of the 3,246 training antibodies with each of the 60 RAbD and 51 IgFold test complexes by CDR-H3 sequence identity at the 40% threshold used for clustering; any test complex that lands in a training cluster would refute the leakage-free evaluation claim.","supporting_citations":[],"review_version":1}