Pith. sign in

REVIEW 4 major objections 5 minor 54 references

Evolution-Aware MSA Reasoning for Subsampling via Factor Graphs

T0 review · 4 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read AP-REASONER recasts MSA subsampling as a factor-graph optimization with two control knobs and outperforms heuristic subsamplers on long-range contact prediction and protein conformational ensemble recovery.

desk verdict AP-REASONER is a genuinely new factor-graph formulation of MSA subsampling with a solid central result, but the paper overclaims consistency and needs variance estimates before the superiority claim lands. read the letter →

arxiv 2607.22314 v1 pith:DHOEDGBF submitted 2026-07-24 cs.LG

Zhangzhi Xiong , Minzhang Li , Haotian Yu , Sixian Shen , Kexin Zhang , Mingrui Li , Jie Zheng , Kewei Tu
show 1 more author
Jingyi Yu
This is my paper
classification cs.LG
keywords MSAsubsamplingaffinitypropagationfactorgraphsproteinlanguagemodelslong-rangecontactpredictionconformationalensemblessequencediversityqueryidentity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's thesis is that subsampling a multiple sequence alignment — choosing which few hundred sequences a protein model actually sees — should be treated as an explicit optimization problem rather than a filtering heuristic. Its method, AP-REASONER, turns the choice into a factor-graph inference over which rows should represent which other rows, with two knobs that directly control sequence diversity and query identity. If the paper is right, this reframing matters because the subsampling step becomes a tunable lever on downstream biology: models pretrained on AP-chosen subsets reach higher long-range contact precision than models trained on random, identity-filtered, or diversity-greedy subsets (Top-L precision 35.1% versus 31.5% for random and 22.3% for HHfilter on CASP15), and a single knob can steer structure prediction toward alternative protein conformations that are otherwise missed. The value, then, is not just a better sampler but a demonstration that subsampling is a composable component of the modeling pipeline, not a cheap preprocessing necessity.

What carries the argument

The central object is AP-REASONER, an affinity-propagation factor graph adapted to fixed-budget MSA subsampling. The unary factor for row i choosing exemplar k is exp(S_ik), with off-diagonal similarities S_ik = -(d_gap(i,k)/L)^α built from normalized gap-aware Hamming distance (α is the diversity control knob) and diagonal self-preferences S_kk = p + β d_gap(k,query)/L (β is the query-identity control knob, p a global preference offset). Exemplar-consistency factors forbid a row from being chosen as an exemplar by others unless it chooses itself. The subset is obtained by log-domain max-product message passing (responsibility and availability updates), followed by a fixed-cardinality wrappe

What would settle it

Replace the similarity matrix in the unary factors with a co-evolution-aware similarity (e.g., a direct-coupling analysis or mutual-information score), retrain the same checkpoints, and compare CASP15 Top-L precision; if the advantage over random and diversity-greedy subsamplers disappears, the Hamming geometry is the active mechanism. Alternatively, evaluate on a set of protein families engineered so that contacts arise predominantly from third-order covariation rather than pairwise sequence disagreement; the reported gains should vanish if the proxy is insufficient.

Watch

Extended reading notes

Core claim

AP-REASONER formulates MSA subsampling as a maximum-a-posteriori problem on an affinity-propagation factor graph. Each alignment row i has a variable indicating which row represents it; unary factors score how well candidate k represents i using a normalized gap-aware Hamming distance — off-diagonal similarity S_ik = -(d/L)^α, where the exponent α controls how strongly local mutations are tolerated and hence the global diversity of chosen exemplars — while diagonal self-preferences S_kk = p + β d(k,query)/L bias exemplar choice toward or away from the query sequence. Exemplar-consistency factors require any row chosen by others to choose itself. Log-domain max-product message passing coordin

Load-bearing premise

The argument collapses if normalized gap-aware Hamming distance between aligned rows is not a faithful proxy for the evolutionary signal that matters to downstream structure: if the relevant information lives in higher-order co-evolutionary statistics, AP-REASONER optimizes the wrong objective and the contact/conformation gains need not survive outside this geometry.

Editorial extensions

If this is right

  • Subsampling strategy is a train-time determinant of structure-aware representations: the AP-REASONER checkpoint outperforms all baselines on CASP15 Top-L contact precision, widening on the 36-domain informative subset (43.3% vs 38.8% random, 27.3% HHfilter).
  • The diversity knob α works as designed and is usable at inference time: average Top-L/5 contact precision across checkpoints rises monotonically from 45.7% at α=0.4 to 47.3% at α=1.6.
  • The query-identity knob β can steer a frozen structure predictor between states: in KaiB, raising β from -11 to +10 reconfigures the fold-switch region from Ground-state to Fold-switch topology, with analogous transitions for MAD2 and RfaH.
  • Targeted sampling is dramatically cheaper than brute force: 56 AP-REASONER samples match the KaiB fold-switch recovery that AF-Cluster only reached with 1008 samples, and the best RfaH autoinhibited-state RMSD improves from 4.54 Å (random) to 2.15 Å.
  • Contact precision versus MSA diversity and query identity is hump-shaped, indicating a joint optimum that single-axis heuristic samplers cannot reach; AP-REASONER's factor-graph optimization is the mechanism proposed to land there.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if the Hamming proxy is the operative ingredient, replacing it with a co-evolution-aware similarity (e.g., a direct-coupling or mutual-information kernel) is the natural next experiment; the gains should either shrink or grow in a way that isolates what the factor graph is actually optimizing.
  • Editorial inference: the appendix's identity-coverage decomposition (slope ≈ 1.20 × 3.04, R²=0.989) suggests diversity and query identity are coupled through alignment coverage; practitioners tuning α and β independently may in practice move one effective knob, so joint calibration rather than grid search over the two scalars is indicated.
  • Editorial inference: since AP-REASONER runs offline and deterministically on the full MSA, the same factor-graph subsetter can be dropped into any MSA-conditioned protein model without altering training objectives, so the reported contact-gain pattern is worth testing across other MSA-based architectures and larger pretraining corpora.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper recasts MSA subsampling as an explicit optimization problem solved with affinity propagation on a factor graph. AP-REASONER defines off-diagonal similarities via a gap-aware Hamming distance controlled by an exponent α and self-preferences with a query-distance prior controlled by β, then wraps AP in a fixed-cardinality bisection procedure. The authors pretrain MSA Transformer with six subsampling strategies on a 260K-MSA OpenProteinSet split and evaluate on masked-LM, CASP15 long-range contact prediction, and AF2-based conformational ensemble prediction on KaiB, RfaH, and MAD2. The headline results are that the AP-trained checkpoint improves CASP15 Top-L precision over random and HHfilter (Table 2), the α knob shows a monotone trend under the Top-L/5 budget (Table 3), and the β knob can steer predicted ensembles between known deposited states (Figures 5–6).

Significance. If the empirical claims hold, this is a useful contribution: it converts a heuristic preprocessing step into a principled, deterministic optimization with interpretable controls, and it opens a training-time dimension for MSA-based PLM subsampling. The factor-graph formulation is a clean adaptation of standard affinity propagation, with MSA-specific unary factors and an exact-budget wrapper. The paper is also honest about the MLM result (Table 1), where random sampling is competitive, and it isolates downstream evaluation through a frozen-backbone LRCP head. The main weaknesses are the lack of repeated-seed statistics, the post hoc definition of the 'informative subset', and the fact that test-time divMax still outperforms all AP α variants in Table 3; these limit the strength of the 'consistently outperforms' claim rather than invalidate the method.

major comments (4)
  1. [§4.2, Table 2; Appendix B.7] The headline LRCP comparison rests on a single pretraining run per sampler. AP selection is deterministic (B.7), but MSA Transformer training and the random/divMax baseline protocols are stochastic, so the 3.6-point Top-L gap on full CASP15 and 4.5-point gap on the informative subset could be training noise. Please report mean±std over at least three seeds or paired significance tests. Without variance, the abstract's 'consistently outperforms' is not supported by the reported evidence.
  2. [§4.2, Table 2; Appendix D.5] The post hoc exclusion of 9 of 45 CASP15 domains as 'universal-failure' domains inflates the AP margin from 3.6 to 4.5 points. D.5 describes these targets qualitatively (low Neff, orphan folds) but gives no quantitative threshold or pre-registered criterion. If the exclusion is made after inspecting results, the informative-subset numbers are cherry-picked. In addition, Table 3 shows that the divMax inference-time sampler has a higher column mean (50.2) than any AP α variant (max 47.3), so the paper's claim that AP-REASONER 'outperforms baseline subsamplers' holds only for training-time sampling and should be scoped accordingly in the abstract and Section 4.2.
  3. [§4.3, §4.4, Table 4, Figure 5] The control-knob validation is partly circular and not fully quantified. The α knob is validated on the same Top-L/5 LRCP metric that motivated its design, and the β knob is demonstrated by selecting β per protein to hit already-deposited states (KaiB GS/FS, MAD2 Closed/Open, RfaH Auto/Active), not by blind prediction on unseen proteins. Table 4 also contains counterexamples to the 'consistently populates these regions' claim: for Mad2 Open, AP has avg RMSD 8.54 vs AF-Cluster's 7.42 and best RMSD 6.67 vs 5.32; for RfaH Active, Random has best RMSD 3.20 vs AP's 3.53. Please report cluster-level success rates across replicates and temper the qualitative claims, or test the β knob on proteins whose alternative states are not used in the validation.
  4. [§3.2, Eq. (1)–(2)] The entire AP-REASONER objective uses normalized gap-aware Hamming distance as the sole evolutionary similarity. The paper asserts that this proxy 'sufficiently preserves the coarse structural neighborhood' but gives no direct evidence that per-column residue disagreement retains the higher-order co-evolutionary statistics that LRCP and conformational prediction depend on. As a correctness-risk check, I suggest comparing AP-REASONER with a variant whose distance is replaced by a co-evolution-aware metric (e.g., normalized mutual information, phylogenetic distance, or an MSA-transformer embedding distance) and reporting whether the Table 2 gains persist. If they do not, the optimization may be tuned to the wrong geometry.
minor comments (5)
  1. [Abstract, §4.2] The phrase 'consistently outperforms baseline subsamplers' is contradicted by Table 3, where divMax has the highest column mean. Please qualify this as 'training-time sampling' or 'on the LRCP Top-L benchmark under divMax test-time subsampling'.
  2. [Appendix D.5] The 'informative subset' definition should include the actual Neff thresholds, sequence-identity cutoffs, or the list of the 9 excluded domains. The current wording ('physically groundless') is not an operational criterion.
  3. [Figure 14 caption] Typo: 'RafH' should be 'RfaH'. Similar minor typos appear in 'controled'.
  4. [Appendix B.6] The fixed-cardinality wrapper performs bisection on p and then truncates or pads by diagonal belief. This post-processing can break the exemplar-consistency constraint implied by Eq. (4). Please justify that the final subset still corresponds to a valid AP assignment or state how much the truncation alters the AP solution.
  5. [Table 4, Appendix D.6] The table reports n as 'number of confident top-1 predictions inside the cluster box', while the text calls these 'targeted samples' and compares 56 AP samples against 1008 AF-Cluster samples. The column header and the comparison basis (number of AF2 forward passes vs number of retained clusters) should be clarified. Also, '96 seeds' for Random is ambiguous; specify whether these are random seeds, sampled MSAs, or structure predictions.

Circularity Check

1 steps flagged · score 2.0 of 10

Main held-out CASP15 superiority claims are not circular; the only definitional reduction is the Appendix C.2 'hump' identity, which is algebraic given a gap-free query yet is presented as an empirically validated relation.

  1. self definitional [Appendix C.2, Eq. (20) and surrounding text]
    "In the OpenProteinSet a3m corpus the query row is gap-free by construction, so for any non-query row i we have pidi = pidungap_i · cov_i, since every gap column counts as a mismatch against the gap-free query. Averaging over rows yields μ_col_PID ≈ μ_ungap_PID · cov. (20) On our nine samplers Eq. 20 holds with R2 = 0.989 and a maximum relative error of 5.06%."

    With a gap-free query, Eq. (18) defines PID_i = (1/L) Σ 1(s_ij = q_j); every gap in row i is a mismatch, so the matches are exactly the matched non-gap positions. Since cov_i is the non-gap fraction and un-gapped PID is matches divided by the non-gap count, PID_i = udngap_PID_i · cov_i follows immediately from the definitions for each row. Eq. (20) is therefore an algebraic identity, not an empirical discovery. Reporting R^2 = 0.989 and attributing cross-sampler variance in column-wise query identity to coverage is thus a tautology built into the metric definitions. This step supports an interpretative 'hump' narrative in the appendix and is not load-bearing for the main held-out CASP15 contact or conformation results.

full rationale

The paper's central derivation chain — treating MSA subsampling as an AP factor-graph optimization with Hamming-distance-based unary factors, exemplar-consistency factors, and α/β control knobs — is self-contained and does not reduce to its inputs. The main evidence for the superiority claim is evaluation on held-out CASP15 domains (Table 2 and appendix tables) and structure-based conformational recovery, which are external to the subsampler's construction. No parameter of AP-REASONER is fitted to the CASP15 targets; α = 1 and β = 0 are fixed operating points, and the α-trend and β-steering experiments report genuine downstream measurements, even if single-run. I found no load-bearing self-citation: the cited algorithmic components (Frey-Dueck AP, MSA Transformer, HHfilter, divMax, AF-Cluster) are external prior work, and no uniqueness theorem is imported from the current authors. The only concrete circular reduction is the Appendix C.2 'hump' identity, where column-wise PID equals un-gapped PID times coverage by definition for a gap-free query; presenting R^2 = 0.989 as empirical validation is a definitional tautology, not evidence of a discovered coupling. This does not affect the core held-out benchmark conclusions. The lack of repeated-seed variance is a statistical robustness concern, not a circularity concern.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on AP's message-passing as a suitable optimizer, on gap-aware Hamming distance as an evolutionary proxy, on the fixed-cardinality wrapper, and on the post-hoc exclusion of 9 CASP15 domains. No new physical entities are invented; alpha, beta, p, and pq are hand-set or per-MSA tuned parameters that the experiments depend on.

free parameters (4)
  • alpha (α) = 1 (training); swept 0.4–1.6 in evaluation
    Controls curvature of the Hamming-distance to similarity mapping. The value 1 is a manual choice; the monotone LRCP trend in Table 3 is used to validate the knob, not to derive it.
  • beta (β) = 0 (training); −10…+10 per target for conformation steering
    Controls query-identity preference in the diagonal self-preferences. Chosen by hand per protein to hit known deposited states, so the 'controllable recovery' demonstration is not a blind prediction.
  • global preference offset p = set by bisection to meet N_B(L)
    AP self-preference tuned per MSA to hit the fixed row budget. The fixed-cardinality wrapper is load-bearing for exact-budget comparison and adds a per-MSA tuned parameter.
  • query anchor preference pq = chosen sufficiently large
    Hard query anchoring: S00 = pq. No criterion is given for 'sufficiently large', and it can affect the subset composition while being held fixed during bisection.
assumptions (5)
  • domain assumption Affinity propagation max-product message passing converges to a stable exemplar set for protein MSAs
    The method relies on the practical sklearn AP implementation (Appendix B.4, B.5) and assumes the message-passing schedule with damping λ=0.5 yields a valid subsampling solution.
  • domain assumption Gap-aware Hamming distance is an adequate proxy for evolutionary diversity and query identity relevant to structure prediction
    Section 3.2 states it 'sufficiently preserves the coarse structural neighborhood' while 'bypassing complex evolutionary discrepancies'; this is load-bearing for the entire factor design.
  • domain assumption Exemplar count is empirically monotone non-decreasing in the global preference p
    Appendix B.6 relies on this monotonicity for bisection over p to hit the exact row budget N_B(L).
  • domain assumption OpenProteinSet query rows are gap-free so Eq. 20 (pid_i = pid_ungap_i * cov_i) holds
    Appendix C.2 uses this to derive the 'hump' decomposition; if queries contain gaps, the identity fails and the variance-decomposition claim breaks.
  • ad hoc to paper The 9 excluded CASP15 domains are 'universal-failure' domains where MSA-based inference is physically groundless
    Appendix D.5 defines the 36-domain informative subset by excluding these 9; the exclusion is post hoc and affects the headline numbers in Table 2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Evolution-Aware MSA Reasoning for Subsampling via Factor Graphs." pith.science (2026). https://pith.science/paper/DHOEDGBF

@misc{pith2026260722314,
  author       = {Pith},
  title        = {Pith review of: Evolution-Aware MSA Reasoning for Subsampling via Factor Graphs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DHOEDGBF}},
  note         = {Machine review of arXiv:2607.22314}
}
read the original abstract

Multiple Sequence Alignments (MSAs) provide protein language models with explicit evolutionary context, but their large depth makes subsampling unavoidable under limited token budgets. Existing strategies, including random selection, identity-based filtering, and diversity-driven sampling, are effective heuristics, yet provide limited control over the evolutionary signals retained in the subset. In this work, we recast MSA subsampling as an explicit optimization problem, where key evolutionary measures, including query identity and diversity, are treated as controllable objectives. Building on this view, we introduce AP-REASONER, an Affinity-Propagation-based factor-graph approach. With evolution-aware unary factors, exemplar-consistency factors, and two control knobs, AP-REASONER performs factor-graph reasoning through message passing to infer a fixed-budget MSA subset. Experiments on long-range contact prediction and conformational ensemble prediction show that AP-REASONER outperforms baseline subsamplers on structure-sensitive downstream tasks and enables controllable recovery of alternative protein conformations. These results highlight the value of modeling MSA subsampling as a controllable optimization problem, where factor-graph reasoning offers an effective alternative to heuristic selection.

Figures

Figures reproduced from arXiv: 2607.22314 by the authors.

Figure 1
Figure 1. The left part illustrates two main MSA indicators for evolution: query identity and diversity. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. AP factor graph and control knob. We adopt normalized gap-aware Hamming distance as an alignment-native proxy for sequence redundancy and divergence, in line with the residue-disagreement measures widely used in evolutionary-coupling analy￾sis [31]. It measures direct residue disagree￾ment at homologous positions and avoids auxiliary model assumptions. While bypass￾ing complex evolutionary discrepancies, it sufficie… view at source ↗
Figure 3
Figure 3. Contact prediction under different MSA subsampling strategies on CASP15 targets. We [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Comprehensive display for the r.m.s.d. of protein structure predictions (conformations [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Structural transition of KaiB controlled by the [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: r.m.s.d. of protein structure predictions by [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: MLM evaluation metrics as a function of sampled-MSA diversity (top row) and mean query [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: Early training loss and perplexity under different MSA subsampling strategies. Sampler [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: Contact prediction under different MSA subsampling strategies on CASP15 targets. We [PITH_FULL_IMAGE:figures/full_fig_p025_9.png]
Figure 10
Figure 10. Figure 10: Training-MSA Diversity / Query Identity v.s. Informative CASP15 Top-L Contact [PITH_FULL_IMAGE:figures/full_fig_p025_10.png]
Figure 11
Figure 11. Figure 11: Training-MSA Diversity / Query Identity v.s. full CASP15 Top-L Contact Precision [PITH_FULL_IMAGE:figures/full_fig_p026_11.png]
Figure 12
Figure 12. Figure 12: Column-mean precision v.s. AP α knob under all settings [PITH_FULL_IMAGE:figures/full_fig_p026_12.png]
Figure 13
Figure 13. Figure 13: Structural transition of KaiB controled by the [PITH_FULL_IMAGE:figures/full_fig_p026_13.png]
Figure 14
Figure 14. Figure 14: Structural transition of RafH controled by the [PITH_FULL_IMAGE:figures/full_fig_p028_14.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

54 extracted references · 5 canonical work pages

  1. [1]

    Accurate structure prediction of biomolecular interactions with alphafold 3.Nature, 630(8016):493–500, 2024

    Josh Abramson, Jonas Adler, Jack Dunger, Richard Evans, Tim Green, Alexander Pritzel, Olaf Ronneberger, Lindsay Willmore, Andrew J Ballard, Joshua Bambrick, et al. Accurate structure prediction of biomolecular interactions with alphafold 3.Nature, 630(8016):493–500, 2024

    2024

  2. [2]

    Watkins, Stephen Ra, Richard Bonneau, and Mohammed AlQuraishi

    Gustaf Ahdritz, Nazim Bouatta, Sachin Kadyan, Lukas Jarosch, Daniel Berenberg, Ian Fisk, Andrew M. Watkins, Stephen Ra, Richard Bonneau, and Mohammed AlQuraishi. Openpro- teinset: Training data for structural biology at scale, 2023. URL https://arxiv.org/abs/ 2308.05326

    arXiv 2023

  3. [3]

    Principles that govern the folding of protein chains.Science, 181(4096): 223–230, 1973

    Christian B Anfinsen. Principles that govern the folding of protein chains.Science, 181(4096): 223–230, 1973

    1973

  4. [4]

    Accurate prediction of protein structures and interactions using a three-track neural network.Science, 373(6557):871–876, 2021

    Minkyung Baek, Frank DiMaio, Ivan Anishchenko, Justas Dauparas, Sergey Ovchinnikov, Gyu Rie Lee, Jue Wang, Qian Cong, Lisa N Kinch, R Dustin Schaeffer, et al. Accurate prediction of protein structures and interactions using a three-track neural network.Science, 373(6557):871–876, 2021

    2021

  5. [5]

    Cyrus Chothia and Arthur M. Lesk. The relation between the divergence of sequence and structure in proteins.The EMBO Journal, 5(4):823–826, 1986. doi: 10.1002/j.1460-2075.1986. tb04288.x

    arXiv 1986

  6. [6]

    Sampling alternative conformational states of transporters and receptors with alphafold2.elife, 11:e75751, 2022

    Diego Del Alamo, Davide Sala, Hassane S Mchaourab, and Jens Meiler. Sampling alternative conformational states of transporters and receptors with alphafold2.elife, 11:e75751, 2022

    2022

  7. [7]

    BERT: pre-training of deep bidirectional transformers for language understanding.CoRR, abs/1810.04805, 2018

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding.CoRR, abs/1810.04805, 2018. URL http://arxiv.org/abs/1810.04805

    arXiv 2018

  8. [8]

    The protein-folding problem, 50 years on.science, 338 (6110):1042–1046, 2012

    Ken A Dill and Justin L MacCallum. The protein-folding problem, 50 years on.science, 338 (6110):1042–1046, 2012

    2012

Show all 54 references
  1. [9]

    Prottrans: toward understanding the language of life through self-supervised learning.IEEE transactions on pattern analysis and machine intelligence, 44(10):7112–7127, 2021

    Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rehawi, Yu Wang, Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steinegger, et al. Prottrans: toward understanding the language of life through self-supervised learning.IEEE transactions on pattern a...

    2021

  2. [10]

    Protein complex prediction with alphafold-multimer.biorxiv, pages 2021–10, 2021

    Richard Evans, Michael O’neill, Alexander Pritzel, Natasha Antropova, Andrew Senior, Tim Green, Augustin Žídek, Russ Bates, Sam Blackwell, Jason Yim, et al. Protein complex prediction with alphafold-multimer.biorxiv, pages 2021–10, 2021

    2021

  3. [11]

    Disease variant prediction with deep generative models of evolutionary data.Nature, 599(7883):91–95, 2021

    Jonathan Frazer, Pascal Notin, Mafalda Dias, Aidan Gomez, Joseph K Min, Kelly Brock, Yarin Gal, and Debora S Marks. Disease variant prediction with deep generative models of evolutionary data.Nature, 599(7883):91–95, 2021

    2021

  4. [12]

    Mixture modeling by affinity propagation

    Brendan J Frey and Delbert Dueck. Mixture modeling by affinity propagation. In Y . Weiss, B. Schölkopf, and J. Platt, editors,Advances in Neural Information Processing Systems, volume 18. MIT Press, 2005. URL https://proceedings.neurips.cc/paper_files/ paper/2005/file/327708dd...

    2005

  5. [13]

    Frey and Delbert Dueck

    Brendan J. Frey and Delbert Dueck. Clustering by passing messages between data points. Science, 315(5814):972–976, 2007. doi: 10.1126/science.1136800. URL https://www. science.org/doi/abs/10.1126/science.1136800

    2007 doi

  6. [14]

    Modeling aspects of the language of life through transfer-learning protein sequences.BMC bioinformatics, 20(1):723, 2019

    Michael Heinzinger, Ahmed Elnaggar, Yu Wang, Christian Dallago, Dmitrii Nechaev, Florian Matthes, and Burkhard Rost. Modeling aspects of the language of life through transfer-learning protein sequences.BMC bioinformatics, 20(1):723, 2019

    2019

  7. [15]

    Hofmann and A

    A. Hofmann and A. Wlodawer. Pcsb—a program collection for structural biology and bio- physical chemistry.Bioinformatics, 18(1):209–210, 01 2002. ISSN 1367-4803. doi: 10.1093/ bioinformatics/18.1.209. URLhttps://doi.org/10.1093/bioinformatics/18.1.209

    2002 doi

  8. [16]

    A-prot: protein structure modeling using msa trans- former.BMC bioinformatics, 23(1):93, 2022

    Yiyu Hong, Juyong Lee, and Junsu Ko. A-prot: protein structure modeling using msa trans- former.BMC bioinformatics, 23(1):93, 2022

    2022

  9. [17]

    Mutation effects predicted from sequence co-variation

    Thomas A Hopf, John B Ingraham, Frank J Poelwijk, Charlotta PI Schärfe, Michael Springer, Chris Sander, and Debora S Marks. Mutation effects predicted from sequence co-variation. Nature biotechnology, 35(2):128–135, 2017

    2017

  10. [18]

    Bidirectional lstm-crf models for sequence tagging.arXiv preprint arXiv:1508.01991, 2015

    Zhiheng Huang, Wei Xu, and Kai Yu. Bidirectional lstm-crf models for sequence tagging.arXiv preprint arXiv:1508.01991, 2015

    2015 arXiv

  11. [19]

    John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ron- neberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, Alex Bridgland, Clemens Meyer, Simon A. A. Kohl, Andrew J. Ballard, Andrew Cowie, Bernardino Romera-Paredes, Sta...

    2021

  12. [20]

    Efficient inference in fully connected crfs with gaussian edge potentials.Advances in neural information processing systems, 24, 2011

    Philipp Krähenbühl and Vladlen Koltun. Efficient inference in fully connected crfs with gaussian edge potentials.Advances in neural information processing systems, 24, 2011

    2011

  13. [21]

    Andriy Kryshtafovych, Maciej Antczak, Marta Szachniuk, Tomasz Zok, Rachael C. Kretsch, Ramya Rangan, Phillip Pham, Rhiju Das, Xavier Robin, Gabriel Studer, Janani Durairaj, Jerome Eberhardt, Aaron Sweeney, Maya Topf, Torsten Schwede, Krzysztof Fidelis, and John Moult. New pred...

    2023 doi

  14. [22]

    Factor graphs and the sum-product algorithm.IEEE Transactions on information theory, 47(2):498–519, 2001

    Frank R Kschischang, Brendan J Frey, and H-A Loeliger. Factor graphs and the sum-product algorithm.IEEE Transactions on information theory, 47(2):498–519, 2001

    2001

  15. [23]

    Conditional random fields: Probabilistic models for segmenting and labeling sequence data

    John Lafferty, Andrew McCallum, and Fernando CN Pereira. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. 2001

    2001

  16. [24]

    Schafer, Jeshuwin Prabakaran, Devlina Chakravarty, Madeleine F

    Myeongsang Lee, Joseph W. Schafer, Jeshuwin Prabakaran, Devlina Chakravarty, Madeleine F. Clore, and Lauren L. Porter. Large-scale predictions of alternative protein conformations by AlphaFold2-based sequence association.Nature Communications, 16(1):5622, 2025. ISSN 2041-1723....

    2025 doi

  17. [25]

    Clustering by soft-constraint affinity propaga- tion: applications to gene-expression data.Bioinformatics, 23(20):2708–2715, 2007

    Michele Leone, Sumedha, and Martin Weigt. Clustering by soft-constraint affinity propaga- tion: applications to gene-expression data.Bioinformatics, 23(20):2708–2715, 2007. ISSN 1367-4803. doi: 10.1093/bioinformatics/btm414. URL http://dx.doi.org/10.1093/ bioinformatics/btm414

    2007 doi

  18. [26]

    Evolutionary-scale prediction of atomic-level protein structure with a language model.Science, 379(6637):1123–1130, 2023

    Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, et al. Evolutionary-scale prediction of atomic-level protein structure with a language model.Science, 379(6637):1123–1130, 2023. 11

    2023

  19. [27]

    Large language models generate functional protein sequences across diverse families.Nature biotechnology, 41 (8):1099–1106, 2023

    Ali Madani, Ben Krause, Eric R Greene, Subu Subramanian, Benjamin P Mohr, James M Holton, Jose Luis Olmos Jr, Caiming Xiong, Zachary Z Sun, Richard Socher, et al. Large language models generate functional protein sequences across diverse families.Nature biotechnology, 41 (8):1...

    2023

  20. [28]

    Protein 3d structure computed from evolutionary sequence variation

    Debora S Marks, Lucy J Colwell, Robert Sheridan, Thomas A Hopf, Andrea Pagnani, Riccardo Zecchina, and Chris Sander. Protein 3d structure computed from evolutionary sequence variation. PloS one, 6(12):e28766, 2011

    2011

  21. [29]

    Colabfold: making protein folding accessible to all.Nature methods, 19(6): 679–682, 2022

    Milot Mirdita, Konstantin Schütze, Yoshitaka Moriwaki, Lim Heo, Sergey Ovchinnikov, and Martin Steinegger. Colabfold: making protein folding accessible to all.Nature methods, 19(6): 679–682, 2022

    2022

  22. [30]

    Cui, David C

    Gabriel Monteiro da Silva, Jennifer Y . Cui, David C. Dalgarno, George P. Lisi, and Brenda M. Rubenstein. High-throughput prediction of protein conformational distributions with sub- sampled AlphaFold2.Nature Communications, 15(1):2464, 2024. ISSN 2041-1723. doi: 10.1038/s4146...

    2024 doi

  23. [31]

    Direct-coupling analysis of residue coevolution captures native contacts across many protein families.Proceedings of the National Academy of Sciences, 108(49):E1293–E1301, 2011

    Faruck Morcos, Andrea Pagnani, Bryan Lunt, Arianna Bertolino, Debora S Marks, Chris Sander, Riccardo Zecchina, José N Onuchic, Terence Hwa, and Martin Weigt. Direct-coupling analysis of residue coevolution captures native contacts across many protein families.Proceedings of th...

    2011

  24. [32]

    Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval

    Pascal Notin, Mafalda Dias, Jonathan Frazer, Javier Marchena-Hurtado, Aidan N Gomez, Debora Marks, and Yarin Gal. Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval. InInternational Conference on Machine Learning, pages 16990–...

    2022

  25. [33]

    Elsevier, 2014

    Judea Pearl.Probabilistic reasoning in intelligent systems: networks of plausible inference. Elsevier, 2014

    2014

  26. [34]

    Scikit- learn: Machine learning in python.the Journal of machine Learning research, 12:2825–2830, 2011

    Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit- learn: Machine learning in python.the Journal of machine Learning research, 12:2825–2830, 2011

    2011

  27. [35]

    End-to-end learning of multiple sequence alignments with differentiable smith–waterman.Bioinformatics, 39(1):btac724, 2023

    Samantha Petti, Nicholas Bhattacharya, Roshan Rao, Justas Dauparas, Neil Thomas, Juannan Zhou, Alexander M Rush, Peter Koo, and Sergey Ovchinnikov. End-to-end learning of multiple sequence alignments with differentiable smith–waterman.Bioinformatics, 39(1):btac724, 2023

    2023

  28. [36]

    Collaborative summarization: When collaborative filtering meets document summarization

    Yang Qu and Qunxiu Chen. Collaborative summarization: When collaborative filtering meets document summarization. In Olivia Kwong, editor,Proceedings of the 23rd Pacific Asia Conference on Language, Information and Computation, Volume 2, pages 474–483, Hong Kong, December 2009....

    2009

  29. [37]

    Canny, Pieter Abbeel, and Yun S

    Roshan Rao, Nicholas Bhattacharya, Neil Thomas, Yan Duan, Xi Chen, John F. Canny, Pieter Abbeel, and Yun S. Song. Evaluating protein transfer learning with TAPE.CoRR, abs/1906.08230, 2019. URLhttp://arxiv.org/abs/1906.08230

    1906 arXiv

  30. [38]

    Transformer protein language models are unsupervised structure learners

    Roshan Rao, Joshua Meier, Tom Sercu, Sergey Ovchinnikov, and Alexander Rives. Transformer protein language models are unsupervised structure learners. In9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net,

    2021

  31. [39]

    Msa transformer

    Roshan M Rao, Jason Liu, Robert Verkuil, Joshua Meier, John Canny, Pieter Abbeel, Tom Sercu, and Alexander Rives. Msa transformer. InInternational conference on machine learning, pages 8844–8856. PMLR, 2021

    2021

  32. [40]

    Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C Lawrence Zitnick, Jerry Ma, et al. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences.Proceedings of the national a...

    2021

  33. [41]

    Improved protein structure prediction using potentials from deep learning.Nature, 577(7792):706–710, 2020

    Andrew W Senior, Richard Evans, John Jumper, James Kirkpatrick, Laurent Sifre, Tim Green, Chongli Qin, Augustin Žídek, Alexander WR Nelson, Alex Bridgland, et al. Improved protein structure prediction using potentials from deep learning.Nature, 577(7792):706–710, 2020

    2020

  34. [42]

    Stein and Hassane S

    Richard A. Stein and Hassane S. Mchaourab. Speach_af: Sampling protein ensembles and conformational heterogeneity with alphafold2.PLOS Computational Biology, 18(8):1–16, 08 2022. doi: 10.1371/journal.pcbi.1010483. URL https://doi.org/10.1371/journal. pcbi.1010483

    2022 doi

  35. [43]

    Speach_af: Sampling protein ensembles and conformational heterogeneity with alphafold2.PLoS computational biology, 18(8):e1010483, 2022

    Richard A Stein and Hassane S Mchaourab. Speach_af: Sampling protein ensembles and conformational heterogeneity with alphafold2.PLoS computational biology, 18(8):e1010483, 2022

    2022

  36. [44]

    MMseqs2 enables sensitive protein sequence searching for the analysis of massive data sets.Nature Biotechnology, 35(11):1026–1028, 2017

    Martin Steinegger and Johannes Söding. MMseqs2 enables sensitive protein sequence searching for the analysis of massive data sets.Nature Biotechnology, 35(11):1026–1028, 2017. doi: 10.1038/nbt.3988. URLhttps://doi.org/10.1038/nbt.3988

    2017 doi

  37. [45]

    Hh-suite3 for fast remote homology detection and deep protein annotation

    Martin Steinegger, Markus Meier, Milot Mirdita, Harald Vöhringer, Stephan J Haunsberger, and Johannes Söding. Hh-suite3 for fast remote homology detection and deep protein annotation. BMC bioinformatics, 20(1):473, 2019

    2019

  38. [46]

    Bertology meets biology: Interpreting attention in protein language models.arXiv preprint arXiv:2006.15222, 2020

    Jesse Vig, Ali Madani, Lav R Varshney, Caiming Xiong, Richard Socher, and Nazneen Fatema Rajani. Bertology meets biology: Interpreting attention in protein language models.arXiv preprint arXiv:2006.15222, 2020

    2006 arXiv

  39. [47]

    Graphical models, exponential families, and variational inference.Foundations and Trends® in Machine Learning, 1(1-2):1–305, 2008

    Martin J Wainwright and Michael I Jordan. Graphical models, exponential families, and variational inference.Foundations and Trends® in Machine Learning, 1(1-2):1–305, 2008

    2008

  40. [48]

    Accurate de novo prediction of protein contact map by ultra-deep learning model.PLoS computational biology, 13(1): e1005324, 2017

    Sheng Wang, Siqi Sun, Zhen Li, Renyu Zhang, and Jinbo Xu. Accurate de novo prediction of protein contact map by ultra-deep learning model.PLoS computational biology, 13(1): e1005324, 2017

    2017

  41. [49]

    Predicting multiple conformations via sequence clustering and alphafold2.Nature, 625(7996):832–839, 2024

    Hannah K Wayment-Steele, Adedolapo Ojoawo, Renee Otten, Julia M Apitz, Warintra Pit- sawong, Marc Hömberger, Sergey Ovchinnikov, Lucy Colwell, and Dorothee Kern. Predicting multiple conformations via sequence clustering and alphafold2.Nature, 625(7996):832–839, 2024

    2024

  42. [50]

    Improved protein structure prediction using predicted interresidue orientations

    Jianyi Yang, Ivan Anishchenko, Hahnbeom Park, Zhenling Peng, Sergey Ovchinnikov, and David Baker. Improved protein structure prediction using predicted interresidue orientations. Proceedings of the National Academy of Sciences, 117(3):1496–1503, 2020. 13 Contents of the Append...

    2020

  43. [52]

    MSA-tailored factor formulation: We construct the factors specifically for the MSA reasoning scenario as elaborated in Section 3.2

  44. [53]

    Hard query anchoring: We force the primary query sequence to act as its own exemplar by assigning it an overwhelmingly large preference (S00 =p q)

  45. [54]

    orphan folds

    Exact-budget search wrapper: Because protein language model training generally favors a fixed sequence cardinality to maximize the utilization of the limited token budget, we wrap the AP reasoning in an outer-loop bisection. This mechanism iteratively adjusts the global baseli...

    2022

  46. [2021]

    URLhttps://openreview.net/forum?id=fylclEqgvgd

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.