actifpTM: a refined confidence metric of AlphaFold2 predictions involving flexible regions

Julia K. Varga; Ora Schueler-Furman; Sergey Ovchinnikov

REVIEW

actifpTM: a refined confidence metric of AlphaFold2 predictions involving flexible regions

Julia K. Varga, Sergey Ovchinnikov, Ora Schueler-Furman

Quantitative Methods · Submitted 20 Dec 2024 · 1 reference

Pith's verdict: Useful practical refinement of ipTM with working code and encouraging case studies, but the length-invariance claim is not established because d0 is still computed from the full complex length.

T0 · AI review

How this review was made: An AI referee (deepseek-v4-flash) read the paper on August 11, 2026 and answered a fixed list of questions about it. T0 is the starting level every paper gets. The levels describe how deeply Pith has checked the paper, never the authors.

T0 AI review An AI read the paper and answered a fixed list of questions.
T1 Audit Automatic checks that the references and files the paper relies on exist and match.
T2 Compute The paper's code or calculations re-run with the same results.
T3 Formal statement The claims written out in a formal math language a computer can read.
T4 Proof Those claims proved, and the proof checked by computer.

How Pith checks papers →

· 3 major objections · 4 minor · Ask about this paper · The compiled record · Read the full review →

Challenge this review Comment (0) Read on arXiv ↗

Cite ▾

arXiv ID2412.15970 (version 1)

Pith IDX2RH5LNZ, Pith's permanent record of this paper and its review

CitationJulia K. Varga, Ora Schueler-Furman, Sergey Ovchinnikov. "actifpTM: a refined confidence metric of AlphaFold2 predictions involving flexible regions." arXiv:2412.15970 (2024). https://pith.science/paper/X2RH5LNZ

Cite this reviewBibTeX for Pith's review of this paper

Claim paper

Why Pith says this

Highlight any text to comment on it.

Editor's note

read the letter

The core idea is simple and genuinely useful: reweight interchain residue pairs by AF2-predicted contact probabilities instead of treating all pairs equally. That directly attacks a known failure mode where flexible flanks drag ipTM down. The implementation is concrete—it is in ColabFold, reproducible, and the paper includes a notebook. The three case studies are chosen but they do show the expected effect: ipTM collapses with longer flanks while actifpTM stays close to the bound-motif score. The method is a legitimate refinement, and the authors are honest that it cannot make an unconfident prediction confident.

That said, the central theoretical claim overreaches. The paper says actifpTM “does not depend anymore on the length of the additional regions,” but the TM-score length scale d0 is still calculated from the full concatenated input length, exactly as the paper itself describes earlier. The modification only changes the mask/weights; it does not touch d0. Since d0 grows with total length, every pairwise TM contribution for the interface also grows, so the score is still length-dependent through d0. The “de facto shortening Lcommon” explanation addresses the sum over pairs, not the normalization scale. The near-constant values in Figure 1E-G could partly reflect d0 inflation masking residual interface effects. This is not fatal—the empirical improvement is credible—but it means the length-invariance claim needs either a corrected derivation or a benchmark that explicitly separates the d0 effect.

The broader validation is thin. Three selected families are not a systematic test. The contact-probability map is checked against one solved structure in Supplementary Figure 2. The paper would be stronger with a larger retrospective analysis, even on a few dozen complexes, and with a comparison against iPAE-based masking. As is, it reads as a methods note with encouraging examples rather than a fully tested metric.

Who is this for? Anyone using ColabFold to score peptide-protein or motif-mediated interactions will want this flag. It deserves a serious referee—the code and the practical impact justify it—but the authors should be asked to temper the abstract, address the d0 issue, and ideally add a broader validation set.

Referee report

Major objections (3)

[Implementation of the modified score (Figure 1C-D)] The central claim that actifpTM "does not depend anymore on the length of the additional regions" is not established by the calculation as described. In the Introduction the authors state that the TM-score length scale d0 is calculated from the length of the concatenated input sequences. The actifpTM modification only changes the residue-pair weights; it does not recount d0 or the normalization length L_target. Since d0 = 1.24*(L-15)^(1/3)-1.8 increases with total L, the same interface distances produce different 1/(1+(d_i/d0)^2) terms when flanks are added, and the 1/L_target normalization also changes. The phrase "de facto, shortening Lcommon" addresses only which residue pairs enter the sum, not the length scale or denominator. Please either recompute d0 and the normalization from the effective interface length, or demonstrate numerically that the residual length dependence is negligible over a realistic range of flank lengths.
[Results (case studies) and Supplementary Figure 2] The robustness claim is supported by only three hand-selected examples (SIAH1-CACYBP, MAPK-SH3BP5, KEAP1-NF2L2), with no systematic benchmark or statistical summary. The validation that the predicted contact probability map agrees with experimental contacts is shown for a single peptide (Supplementary Figure 2). Because the central contribution is precisely that actifpTM is a more robust, comparable confidence measure across different flanking lengths, a larger and more diverse evaluation set is needed—at minimum, a table listing ipTM and actifpTM for all test cases in the Lee et al. set, with the number of cases in each category and a quantitative comparison of the variance across extensions.
[Implementation of the modified score (contact probability weighting)] The weights are derived from AF2's own predicted distance distributions, and the one validation example (Supplementary Figure 2) may not reflect performance in flexible flanking regions, which are exactly the regions where AF2's distance predictions are least reliable. If the pseudo-contact map misassigns low contact probability to true interface residues or high probability to non-interface residues, actifpTM could inherit or even amplify the bias it aims to correct. I ask for a targeted check: for a set of complexes with known flexible flanks, compare the contact probability map against the experimental interface, and report sensitivity/specificity or a similar quantification.

Minor objections (4)

[Introduction] The sentence "This is theoretically equivalent to the model and the native structure being superimposed onto each other on every possible residue" is an oversimplification of the pTM calculation; please rephrase to avoid implying an actual exhaustive superimposition.
[Results] The paper says "The implemented code was run on selected domain-motif instances of the dataset from Lee et al."; please specify the total number of instances selected and the criteria for selection.
[Implementation of the modified score] The phrase "contact probability map output by AF2" should be defined explicitly as the sum of distogram bin probabilities with predicted Cβ-Cβ distance below 8 Å (Cα for glycine), since this is not a direct output of AlphaFold2.
[Figure 1D] The label "actifpTM normalized residue weights" is ambiguous; a small equation or a reference to the original ipTM equations would clarify the exact normalization.

How Pith checks papers → · Full audit of this paper →

The paper

The paper claims

A modified AlphaFold2 confidence score, actifpTM, removes the penalty that flexible flanking regions impose on the standard ipTM score.

The reading

This paper argues that AlphaFold2's interchain confidence score, ipTM, is systematically biased by flexible flanking regions because it weights every interchain residue pair equally. The authors propose actifpTM (actual interface pTM), a reweighted version that uses AlphaFold2's predicted contact probabilities to focus the score on residues that are actually predicted to touch. The consequence is a confidence metric that stays comparable across predictions of the same interaction even when the input sequences differ in how much disordered context they include, which matters for peptide–protein and motif-mediated interactions where the structured interface is short relative to the flanking regions. The method is implemented in a widely used AlphaFold2 pipeline and is demonstrated on several domain–peptide examples.

What carries the argument

The central object is the predicted contact probability map that AlphaFold2 produces from its distance-distribution output: for each residue pair, the probability of a Cβ–Cβ distance below 8 Å (Cα for glycines). actifpTM feeds these probabilities into the TM-score machinery as residue-pair weights, replacing the uniform interchain weights of ipTM. After row-wise normalization, each weighted pair contributes its theoretical maximum TM-score per predicted-error bin, the rows are summed, and the per-residue maximum is selected as the final score. This weighting effectively shortens the alignment length by skipping residue pairs that are not predicted to be in contact, which removes the penalty from flexible flanking regions.

Extended reading notes

Core claim

The paper claims that actifpTM, a modified ipTM score, gives a more robust measure of interaction confidence than the original ipTM when models contain flexible flanking regions. The key change is that interchain residue pairs are weighted by the predicted contact probability map — the probability that two residues' Cβ atoms (Cα for glycines) are within 8 Å — instead of being given uniform weight. After row-wise normalization, the usual TM-score summation over predicted-error bins is applied, and the maximum per-residue value is the final actifpTM. In the presented examples, actifpTM remains nearly constant when flexible extensions are added to an identically modeled interface, while ipTM drops sharply; it also decreases appropriately when the modeled interface conformation becomes less confident.

Load-bearing premise

The method assumes that AlphaFold2's predicted contact probabilities (Cβ–Cβ within 8 Å) correctly identify the true interface residues, so that weighting by them removes the flanking-region bias rather than adding a new one.

Editorial extensions

If this is right

actifpTM values remain comparable for the same structured interface regardless of how much disordered flanking sequence is included, whereas ipTM values drop with longer flanks.
actifpTM retains sensitivity to genuine changes in interface confidence, as shown by its decrease when a peptide's binding conformation becomes less reliable.
The score enables fair comparison of models built from sequences with different context lengths, which is especially relevant for peptide–protein and motif-mediated interactions.
The method is available in a widely used AlphaFold2 pipeline through a simple flag, making it usable without custom post-processing.
Because it reports pairwise values for every chain pair, actifpTM can flag which interface in a multi-chain complex is confidently modeled.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same contact-probability weighting could be applied to monomer pTM so that flexible tails within a single chain do not drag down the confidence of a folded domain.
A testable extension is to run a large peptide-binder screen and check whether actifpTM improves ranking stability when the same peptide is tested with different construct lengths; the paper itself does not benchmark this.
If AlphaFold2's contact probabilities are well calibrated beyond structured interfaces, actifpTM could also serve as a rough estimate of the fraction of a binding region that is structured, linking the score to disorder predictions.

Original abstract

One of the main advantages of deep learning models of protein structure, such as Alphafold2, is their ability to accurately estimate the confidence of a generated structural model, which allows us to focus on highly confident predictions.The ipTM score provides a confidence estimate of interchain contacts in protein-protein interactions. However, interactions, in particular motif-mediated interactions, often also contain regions that remain flexible upon binding. These non-interacting flanking regions are assigned low confidence values and will affect iPTM, as it considers all interchain residue pairs, and two models of the same motif-domain interaction, but differing in the length of their flanking regions, would be assigned very different values. Here we propose actifpTM (actual interface pTM), a modified ipTM measure, that focuses on the confident region of an interaction, resulting in a more robust measure of interaction confidence, even when not the full interaction is structured. actifpTM has been incorporated into ColabFold.

The compiled record

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 3,832 input tokens.

What the abstract itself states, as probabilities: claims first 0.07 · states limitation 0.15 · human participants 0.02 · controlled comparison 0.22 · randomized assignment 0.01 · states quantitative result 0.04 · compares with prior results 0.69 · mentions code or data availability 0.54.

The abstract, sentence by sentence

arXiv listing abstract v1. Results and the main claim are underlined; show every role.

One of the main advantages of deep learning models of protein structure, such as Alphafold2, is their ability to accurately estimate the confidence of a generated structural model, which allows us to focus on highly confident predictions.The ipTM score provides a confidence estimate of interchain contacts in protein-protein interactions. However, interactions, in particular motif-mediated interactions, often also contain regions that remain flexible upon binding. These non-interacting flanking regions are assigned low confidence values and will affect iPTM, as it considers all interchain residue pairs, and two models of the same motif-domain interaction, but differing in the length of their flanking regions, would be assigned very different values. Here we propose actifpTM (actual interface pTM), a modified ipTM measure, that focuses on the confident region of an interaction, resulting in a more robust measure of interaction confidence, even when not the full interaction is structured. actifpTM has been incorporated into ColabFold.

background contribution method result interpretation main claim

What the classifier reports for each sentence. The number is the probability it gave the label.
#	sentence	role	the authors' own work	states the main conclusion	strength
s0	One of the main advantages of deep learning models of protein structure, such as Alphafold2, is their ability to accurately estimate the confidence of a generated structural model, which allows us to focus on highly confident predictions.The ipTM score provides a confidence estimate of interchain contacts in protein-protein interactions.	background 1.00	0.06	0.05	asserted 1.00
s1	However, interactions, in particular motif-mediated interactions, often also contain regions that remain flexible upon binding.	background 0.84	0.07	0.03	asserted 0.94
s2	These non-interacting flanking regions are assigned low confidence values and will affect iPTM, as it considers all interchain residue pairs, and two models of the same motif-domain interaction, but differing in the length of their flanking regions, would be assigned very different values.	background 0.83	0.29	0.07	asserted 1.00
s3	Here we propose actifpTM (actual interface pTM), a modified ipTM measure, that focuses on the confident region of an interaction, resulting in a more robust measure of interaction confidence, even when not the full interaction is structured. actifpTM has been incorporated into ColabFold.	contribution 0.99	0.94	0.82	asserted 1.00

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from argument 0.63.

What was examined, and what was not

sources read

arXiv listing abstract v1

not examined

full text
figures and tables
references and citing sentences
relations to other papers

levels run

P abstract

not run

S sections
T body sentences
C citations
R cross-paper relations

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. Every number in the measurements table is the authors', checked by code. Record as JSON

References

Discussion

Disagree with the review, or know something it missed? Leave a comment, highlight any passage above to comment on it, or challenge this review formally.

[1] [1]

Ahdritz,G. et al. (2024) OpenFold: retrainingAlphaFold2yieldsnewinsightsintoitslearningmechanismsandcapacityfor generalization. Nat. Methods, 21, 1514–1524.Baek,M. et al. (2023) Efficient andaccuratepredictionof proteinstructureusingRoseTTAFold2.bioRxiv.Bret,H. et al. (2024) Frominteractionnetworkstointerfaces, scanningintrinsicallydisorderedregionsusingA...

View on Pith 2024