{"id":"59409a09-e65a-4ead-bbf8-81a7e8440061","arxiv_id":"2502.19395","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"IgSeek retrieves structurally similar CDRs from a database of solved antibody structures and uses their sequences to predict the query CDR sequence, outperforming inverse folding baselines in amino-acid recovery.","lead":"This paper presents IgSeek, a method that predicts antibody CDR sequences by retrieving structurally similar CDRs from a database of known antibody structures and combining their sequences. It reports higher sequence recovery and much faster inference than current inverse folding models, offering a retrieval-based alternative for antibody design.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Test-set masking for duplicate/near-duplicate CDRs: the 2.9% AAR lead may be inflated because IgSeek retrieves the query's own near neighbors more easily than baselines can reconstruct them.","rationale":"The reader flagged a related but distinct concern: whether each query has a structurally similar CDR in the database. My concern strengthens and refines that into a concrete, testable threat: the SAbDab split removes only identical sequences, and CDR sequence space is highly redundant, so near-duplicate sequences (e.g., 70-100% identity) can still drive retrieval-based AAR. The paper's reported similarity distribution (Fig. 7) only gives means around 0.3-0.5, which is not enough to rule out a long tail of high-similarity pairs. The effect is specifically favorable to IgSeek because its prediction is literally a consensus of retrieved sequences: when a retrieved neighbor is nearly sequence-identical, that position is almost perfectly recovered, whereas baselines must generate sequences from structure. The baseline comparison is also not apples-to-apples: AntiFold and AbMPNN receive only the CDR structure (or CDR plus flanking residues), while IgSeek effectively has access to the entire database of natural sequences at inference time. The Appendix G experiments show IgSeek's advantage grows when flanking residues are added, which is consistent with retrieval improving with more contextual structure. Thus the main load-bearing question is not whether retrieval works in principle, but whether the reported accuracy advantage survives after controlling for database redundancy. The paper reports no such control, so the result is plausible but not yet established. A stratified analysis by nearest-neighbor sequence identity is a simple, decisive check that would either confirm the method's practical value or reduce the claim to a retrieval-redundancy artifact. This does not change the reader's CONDITIONAL verdict; it makes the condition concrete and falsifiable.","tokens_in":19075,"tokens_out":1805,"duration_ms":15345,"concrete_test":"For every query CDR in SAbDab-2024, compute (1) the sequence identity of the closest same-length CDR in SAbDab-before-2024 and (2) the RMSD of that nearest neighbor. Report AAR for the subset of queries with nearest-neighbor sequence identity below a threshold (e.g., <0.5) and across deciles of the nearest-neighbor distribution. If the IgSeek lead over ProteinMPNN/AbMPNN/AntiFold persists on the low-similarity tail, the claim survives; if the lead is concentrated in high-similarity queries, the headline claim is inflated by database redundancy.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that IgSeek outperforms state-of-the-art inverse folding on sequence recovery for antibody CDRs in SAbDab-2024 (Fig. 2a). IgSeek's inference mechanism is template-based: it retrieves K=10 CDRs of equal length from SAbDab-before-2024 and samples per-position amino acids by consensus. Its accuracy therefore depends almost entirely on how often the database contains a near-identical sequence whose structure is also near-identical to the query. The paper only removes CDR sequences that are identical to those in the dataset (Appendix D) and reports mean sequence similarity of 0.3-0.5 (Fig. 7). A mean of 0.4 with a long tail allows a substantial fraction of test queries to have a near-duplicate or highly similar CDR sequence in the training/database set. For such queries, consensus retrieval can achieve high AAR trivially, while inverse folding baselines must infer the sequence from structure alone and are not given the benefit of near-duplicate memorization. Without reporting the exact distribution of maximum sequence identity between each query and its retrieved neighbors, or the AAR stratified by distance to the nearest database neighbor, the claimed >2.9% improvement on light-chain CDRs is not robust evidence of generalizable sequence-design ability; it may reflect database coverage and redundancy rather than method quality. This is the load-bearing issue because the method's core novelty is retrieval, and the headline result is precisely the retrieval advantage.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes IgSeek, a retrieval-based method for antibody and T-cell receptor CDR sequence design. A multi-channel equivariant graph neural network (MEGNN) is pre-trained on SAbDab CDR pairs to predict pairwise RMSD (Eq. 14), then used to embed all CDRs in a pre-2024 SAbDab database. For a query CDR backbone, IgSeek retrieves K equal-length nearest neighbors in embedding space and samples each position from the empirical amino-acid histogram of the retrieved sequences. The authors evaluate amino-acid recovery (AAR) on SAbDab-2024 and STCRDab against ProteinMPNN, ESM-IF1, AbMPNN, and AntiFold, report order-of-magnitude speedups, and include a variant (IgSeek+Kabsch) that re-ranks retrieved candidates by true RMSD. The central claim is that retrieval-based inference outperforms inverse-folding baselines on CDR sequence recovery, especially for light-chain CDRs and T-cell receptors.","tokens_in":19529,"tokens_out":6492,"duration_ms":60689,"significance":"If the central comparison were robust, IgSeek would offer a useful and fast alternative to inverse folding for CDR design, and the temporal split is a commendable design choice. The paper also contains a formal equivariance proof and a clear algorithmic description. However, the current evidence is not sufficient to establish the headline advantage: the main benchmark gives baselines only CDR backbones, the STCRDab generalization test uses a random split that permits near-duplicate retrieval, and the database-coverage assumption is not quantified. These are correctable with additional experiments, so the work has potential but needs major revision.","major_comments":[{"comment":"The central comparison is not apples-to-apples. The authors state in §2.3 that only CDR backbone structures are given to all methods, and the Remark confirms that AntiFold and AbMPNN degrade because they rely on full antibody backbone and framework sequence information. IgSeek is specifically designed for CDR-only input, so part of its advantage may reflect task adaptation rather than design quality. The extended-context experiments in Appendix G (Fig. 8) add only up to three flanking residues, still not the full variable domain required by AntiFold. Moreover, the Abstract's claim that IgSeek 'outperforms state-of-the-art approaches' is stronger than what Fig. 2a shows for the primary variant: on heavy-chain CDRs IgSeek is comparable to baselines, and only IgSeek+Kabsch, which uses true RMSD for re-ranking, exceeds all baselines. Please re-run AntiFold and AbMPNN under their native input requirements, or clearly define the task as CDR-only and adapt the baselines accordingly, and make the abstract reflect the results of the primary variant.","section":"§2.3 (Fig. 2a), Remark"},{"comment":"The temporal split does not control for near-duplicate CDRs between the retrieval database and the test set. Appendix D says identical sequences are removed, but Fig. 7 reports only average sequence similarity of 0.3–0.5; an average does not rule out a long tail of near-duplicates, and for a retrieval method the tail is what matters. IgSeek's prediction is literally a consensus of the retrieved sequences, so a query with a close database homolog can be recovered trivially, whereas the inverse-folding baselines must infer the sequence from structure alone. The paper does not report the distribution of maximum sequence identity or minimum structural RMSD between each query and database entries, nor AAR stratified by these distances. Please add these stratifications and include a non-learned retrieval baseline (e.g., nearest neighbor by TM-align RMSD) to separate database-coverage effects from MEGNN embedding quality.","section":"§2.3, Appendix D (Fig. 7)"},{"comment":"The STCRDab generalization experiment uses a random 80/20 split of the same database to form templates and queries. Under a random split, many query CDRs will have close structural and sequence neighbors among the templates, so the reported ≥30% AAR lead over inverse-folding baselines in Fig. 2b is expected for a retrieval method and does not demonstrate generalization to genuinely novel receptors. Please use a temporal or cluster-based split, remove near-identical sequences between template and query sets, and report AAR after such filtering. Without this, the STCRDab result does not support the generalization claim made in the text.","section":"§2.3 Generalization Performance, Appendix D (STCRDab)"},{"comment":"No error bars, confidence intervals, or significance tests are reported for any AAR comparison. The headline difference on light-chain CDRs is 2.9 percentage points, and generation is stochastic (n_sample=2 in Table 3), so run-to-run variability and model-seed variability could change the ranking. The authors also report 'best results' for baselines but do not describe how many samples were used for each method. Please provide mean ± standard deviation over at least five runs and a paired significance test (e.g., Wilcoxon signed-rank) across CDR types to establish that the observed differences are not noise.","section":"Fig. 2, Table 3, Table 5"}],"minor_comments":[{"comment":"The procedure is described as 'Bernoulli sampling,' but sampling an amino acid from a 20-category probability distribution is categorical/multinomial sampling, not Bernoulli sampling. Please rename.","section":"§C.2, Algorithm 3"},{"comment":"The statement of Theorem 1 is malformed: the notation 'hi, TY(g)X_i^(L) = MEGNN(...)' mixes an equivariance equation with a tuple in an unclear way. Please rewrite the theorem statement with clear domain and codomain notation.","section":"§C.3, Theorem 1"},{"comment":"The sentence 'we remove CDR sequences that are identical to those in the dataset to eliminate redundancy' is ambiguous: it is unclear whether removal is from the training set, the retrieval database, or the test set. Please clarify the exact filtering step and report how many CDRs were removed.","section":"Appendix D"},{"comment":"The inference-time comparison on OAS-H3 should specify what exactly is timed for each method: IgSeek timing appears to be retrieval-plus-sampling only, while baselines include neural network inference; please define the measurement protocol and hardware.","section":"Fig. 2c"},{"comment":"There is a typo in the Conclusion: 'investigatino' should be 'investigation.'","section":"§3"}],"recommendation":"major_revision","confidential_remarks":"The manuscript would be considerably strengthened by releasing code and database construction scripts, as no code availability statement is given. The authors should also revisit the abstract's unqualified 'outperforms state-of-the-art' claim, since the primary variant does not clearly beat baselines on heavy-chain CDRs."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about IgSeek (arXiv:2502.19395). First, it's a genuinely new angle on antibody CDR sequence design: instead of inverse folding with a learned generative model, it retrieves similar CDR structures from SAbDab and assembles the sequence as a consensus of the retrieved templates. That's clean, interpretable, and fast, and it sidesteps hallucination by construction. The MEGNN embedding that maps CDR backbones to vectors, trained to predict pairwise RMSD, is sensible, and the retrieval comparison against FoldSeek is a real, positive result.\n\nSecond, the headline claim—that IgSeek beats state-of-the-art inverse folding on AAR—is not yet supported. The main benchmark gives the baselines only the isolated CDR backbone, while AntiFold and AbMPNN expect the full variable domain. The authors acknowledge this in a Remark, but they don't fix it in the main results. The appendix adds flanking residues, but even then the baselines are not given their native input format.\n\nThe bigger issue is the one the stress-test note flags: IgSeek's accuracy depends almost entirely on whether the database contains a near-structural homolog of the query. The paper removes exact duplicates and uses a temporal split, which is good, but it only reports mean sequence similarity of 0.3–0.5 between train and test. That average hides the tail. If a substantial fraction of test CDRs have a near-duplicate in the retrieval database, the consensus method trivially recovers those sequences, while the inverse folding baselines get no such shortcut. Without stratifying AAR by distance to the nearest database neighbor or reporting the max-sequence-identity distribution for each query, the 2.9% lead on light chains could reflect database coverage rather than generalizable design ability. That's a load-bearing omission.\n\nThere are smaller soft spots: no error bars, no code or data, and no non-parametric retrieval baseline (e.g., TM-align or FoldSeek) used directly for sequence assembly in the AAR comparison. The FoldSeek comparison is only for retrieval quality, not for end-to-end AAR.\n\nIf the authors add the stratification and a fair baseline setting, this could be a solid contribution. As it stands, it's a promising idea with an evaluation that overclaims. I'd still send it to peer review—the idea is novel enough and the retrieval result interesting enough to warrant referee time to push on exactly these questions.","headline":"A clever retrieval-based CDR design idea whose headline numbers are undercut by unfair baselines and unquantified database redundancy.","tokens_in":19927,"tokens_out":3276,"would_cite":false,"duration_ms":29004,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"IgSeek establishes that antibody CDR sequences can be designed by retrieving structurally similar natural loops from a vector database, outperforming inverse-folding models on sequence recovery while reducing inference cost by at least 20x.","keywords":["antibody design","CDR sequence design","structure retrieval","inverse folding","equivariant graph neural network","sequence recovery","T-cell receptor","hallucination reduction"],"falsifier":"Run IgSeek on a test set of CDR backbones that are deliberately shifted so that no database loop is within 2 Å RMSD; if the average amino-acid recovery does not drop sharply toward random, the method's gains are not actually coming from retrieved structural templates.","tokens_in":18901,"feed_emoji":"🧬","tokens_out":9841,"duration_ms":83558,"temperature":0.7,"pith_summary":"Antibodies bind their targets through hypervariable loops called CDRs, and the standard deep-learning route designs them by inverse folding: predicting a sequence from a generated backbone. This paper argues that a simpler route works better for CDRs—retrieve the most structurally similar loops from a database of natural antibodies and assemble a sequence from their conserved motifs. The proposed framework, IgSeek, embeds 24,479 solved antibody CDR structures into a vector space, then for any query loop pulls the ten nearest equal-length neighbors and samples each residue from the aligned sequences of those neighbors. On antibodies released in 2024, IgSeek recovers at least 2.9% more native amino acids than state-of-the-art inverse-folding models on light-chain CDRs, transfers to T-cell receptor CDRs without retraining, and is at least 20x faster at inference. If right, this reframes antibody sequence design as a database coverage problem rather than a generative modeling problem, and it directly reduces the hallucination risk that plagues generative inverse folding.","feed_headline":"Antibody CDR sequences recovered by retrieval, not generation","feed_subtitle":"IgSeek beats four inverse-folding baselines on light-chain CDR recovery and runs 20x faster.","key_machinery":"The load-bearing object is MEGNN (Multi-channel Equivariant Graph Neural Network), an E(3)-equivariant encoder that represents each residue by its four backbone atoms as separate channels and reads out a fixed-length embedding per CDR. It is trained self-supervised to predict the RMSD of CDR pairs whose TM-score is over 0.6, so embeddings live in a space where distance means structural similarity. At inference, the query embedding is the key for a K-NN search over 24,479 solved CDR loops from SAbDab-before-2024; the top-10 nearest equal-length CDRs contribute their aligned sequences as an empirical amino-acid frequency at each loop position, from which the final sequence is sampled. An optional variant, IgSeek+Kabsch, re-ranks the retrieved candidates by exact RMSD to simulate perfect retrieval and shows the headroom in the recovery numbers.","core_discovery":"IgSeek's central discovery is that CDR sequence inference can be posed as isomorphic structure retrieval: an E(3)-equivariant multi-channel graph neural network (MEGNN) encodes each CDR backbone into a fixed-length vector, trained self-supervised so that embedding distance predicts pairwise backbone RMSD; then the query embedding retrieves top-10 equal-length nearest neighbors from a vector database of 24,479 solved CDR loops from before 2024. The predicted sequence is generated position by position by counting amino acids at aligned positions of the retrieved sequences and sampling from that empirical distribution. The recovery numbers on the 4,449 SAbDab-2024 CDRs are the claim: at least 2.9% higher average amino-acid recovery than ProteinMPNN, ESM-IF1, AbMPNN, and AntiFold on light-chain CDRs, comparable recovery on heavy-chain CDRs, consistent wins over all baselines on all six CDR types when the retrieved candidates are re-ranked by exact RMSD (IgSeek+Kabsch), at least 30% higher recovery on T-cell receptor CDRs without retraining, and at least 20x faster inference than the generative baselines.","pith_inferences":["A direct test of the hallucination-reduction claim that the paper does not report: fold IgSeek-generated sequences with an independent structure predictor and compare CDR backbone RMSD to the query structure; retrieval should visibly suffer less distribution shift than inverse-folding baselines.","The dependence on database coverage suggests an experiment the authors did not run—construct a retrieval database augmented with high-confidence predicted antibody structures and measure whether long, rare CDR-H3 loops, where solved examples are scarce, gain the most.","Position-wise sampling discards correlations between residue choices within a loop; modeling the joint distribution over aligned retrieved motifs could improve both recovery and naturalness while keeping the retrieval mechanism."],"forward_implications":["Larger retrieval databases should translate directly into higher sequence recovery, because every added natural loop expands the set of structural templates IgSeek can draw from.","IgSeek can replace the inverse-folding step in existing antibody structure-generation pipelines; in the paper's 8R1C case study, RFdiffusion-plus-IgSeek recovered 0.67 of native residues at 0.38 Å backbone RMSD, versus 0.33 for dyMEAN alone.","Because IgSeek needs only the CDR backbone and no framework sequence, it can design CDR sequences on any fixed antibody framework, which is the standard developability-friendly setup.","The 20x inference speed-up makes high-throughput design over large repertoires (the paper tests 5,000 predicted CDR-H3 loops) practical without retraining."],"supporting_citations":[{"why":"Establishes the canonical-structure observation that limited CDR conformations are scaffolded by conserved residues, the empirical motivation for structure-based retrieval.","marker":"Chothia et al. (1989)"},{"why":"SAbDab supplies the 24,479 pre-2024 CDRs for the vector database and training pairs, plus the 4,449 post-2024 test CDRs.","marker":"Dunbar et al. (2013); Schneider et al. (2021)"},{"why":"TM-align computes the TM-scores used to sample structurally similar training pairs and the RMSD used in retrieval evaluation.","marker":"Zhang & Skolnick (2005)"},{"why":"ProteinMPNN is the primary generative inverse-folding baseline and the archetype of the hallucination problem IgSeek targets.","marker":"Dauparas et al. (2022)"},{"why":"AbMPNN is the antibody-fine-tuned ProteinMPNN baseline compared on amino-acid recovery.","marker":"Dreyer et al. (2023)"},{"why":"AntiFold is the antibody-specific inverse-folding baseline that IgSeek must outperform on SAbDab-2024 and STCRDab.","marker":"Høie et al. (2024)"},{"why":"FoldSeek is the state-of-the-art structure-search baseline against which IgSeek's retrieval AUROC and 2.6x speed-up are measured.","marker":"Van Kempen et al. (2024)"},{"why":"STCRDab provides the held-out T-cell receptor CDR dataset used to demonstrate generalization without retraining.","marker":"Leem et al. (2018)"}],"fun_headline_variants":["Retrieval beats generation for antibody CDR design","IgSeek: structure retrieval outperforms inverse folding for CDRs","20x faster and higher recovery: IgSeek retrieves CDR sequences","For CDRs, retrieval beats generation: IgSeek sets new state of the art","IgSeek retrieves CDR sequences, dodging diffusion hallucinations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes the database holds, for every query CDR of a given length, a few loops whose backbone shapes are close enough (roughly within 1 Å RMSD) that their aligned sequence motifs can be recombined to recover the native sequence; if a query has no such close structural neighbor, retrieval has no information to contribute.","fun_headline_variants_meta":{"raw":{"variants":["Retrieval beats generation for antibody CDR design","IgSeek: structure retrieval outperforms inverse folding for CDRs","20x faster and higher recovery: IgSeek retrieves CDR sequences","For CDRs, retrieval beats generation: IgSeek sets new state of the art","IgSeek retrieves CDR sequences, dodging diffusion hallucinations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000698,"raw_usage":{"total_tokens":3163,"prompt_tokens":966,"completion_tokens":2197,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":582,"completion_tokens_details":{"reasoning_tokens":2103}},"tokens_in":582,"tokens_out":2197,"duration_ms":13369,"temperature":1.0,"reasoning_tokens":2103,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T12:22:28.388000+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run IgSeek on a test set of CDR backbones that are deliberately shifted so that no database loop is within 2 Å RMSD; if the average amino-acid recovery does not drop sharply toward random, the method's gains are not actually coming from retrieved structural templates.","supporting_citations":[],"review_version":1}