{"id":"a776f72e-af48-46f0-b841-7dfadd645cdf","arxiv_id":"2412.13722","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Contrastive metric learning on TCR co-specificity data yields a distance metric that generalizes to unseen pMHCs and highlights steric over hydrophobic amino-acid properties.","lead":"This paper learns rules for when two T cell receptors recognize the same viral peptide, directly from thousands of immune-cell sequences. The learned rules transfer to peptides the model was never trained on, and they indicate that amino-acid shape matters more than hydrophobic character.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The generalization claim to peptides ≥6 amino acids away rests on an unquantified, possibly small subset of test pMHCs; group-level confidence intervals are needed.","rationale":"The reader's weakest_assumption is the factorization in Eq. 14–15, which is indeed a genuine modeling concern: if P_S and P_A are not independent, the learned site and amino-acid weights may be biased, and the biophysical interpretations (steric vs. hydrophobic importance, non-contact positions) could be artifacts. However, the central claim as framed in the reader's strongest_claim is about predictive generalization to dissimilar pMHCs and outperformance over TCRdist. That claim could survive even if the factorization is imperfect, because the metric is empirically tested on hold-out pMHCs; if d_AS predicts well on those, the factorization is adequate for prediction. The more load-bearing risk is therefore whether the dissimilar-peptide test is statistically reliable: it is the specific quantitative claim in the abstract, yet the paper does not report the size of that subset or group-level confidence intervals. The concrete test I propose would settle whether this risk materializes. The reader's factorization concern is valid but secondary for the headline empirical claim, hence 'partial' agreement. The reader's CONDITIONAL verdict remains appropriate: the paper's main result is likely true, but the missing robustness details on the dissimilar-peptide subset should be addressed before full acceptance. I recommend no change to the verdict, as the concern does not by itself invalidate the work; it identifies a gap that supports the existing conditional status.","tokens_in":17532,"tokens_out":11473,"duration_ms":107491,"concrete_test":"Using the released Zenodo code and the MIRA dataset, enumerate all hold-out pMHC groups whose minimum Levenshtein distance to the 26 training peptides is ≥6. Count the number of groups and the number of co-specific and cross-specific TCR pairs in this subset, then recompute the d_AS AUROC with 95% confidence intervals obtained by bootstrapping over pMHC groups (e.g., 10,000 resamples). Report the group count and the CI; if the lower bound of the CI is at or below 0.5, or the number of groups is fewer than 5, the generalization claim as stated is not supported by the MIRA test set alone.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that the learned co-specificity rules generalize to pMHCs differing by at least 6 amino acids from the nearest training peptide (Section IV D, Fig. S5C). The paper does not report how many of the 24 hold-out pMHC groups meet this threshold, nor the number of co-specific and cross-specific TCR pairs in this subset. The error bars shown are over 15 resampled training/test batches, which resample from the same groups and therefore do not capture group-level variability. If only a handful of pMHC groups satisfy the ≥6 amino-acid condition, the reported AUROC could be driven by a few groups that are easy to discriminate, and the 'striking breadth' conclusion is not established. The external VDJdb validation supports transfer to other pathogens, but it does not quantify the specific ≥6-amino-acid distance to the training peptides, so it does not directly back the quantitative claim as stated. This is the load-bearing risk because it targets the headline generalization assertion itself, not a modeling premise that might be violated without affecting predictive performance.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a supervised contrastive learning framework, derived from pseudo-likelihood maximization, to learn interpretable sequence distance metrics for TCR co-specificity. Using single-edit CDR3beta pairs from the MIRA SARS-CoV-2 dataset with matched V genes and restricted lengths, the authors learn an amino-acid substitution matrix A, a site-weight matrix S, and a joint factorized metric d_AS. They report that d_AS outperforms TCRdist on held-out MIRA pMHC groups and on external VDJdb data, including for TCR pairs with multiple edit steps. They also claim the learned rules generalize to pMHCs that differ by at least six amino acids from any training peptide. In addition, they find that steric properties of amino acids correlate more strongly with the learned substitution penalties than hydrophobic properties, and that non-contact CDR3beta positions contribute substantially to co-specificity.","tokens_in":17740,"tokens_out":4945,"duration_ms":48720,"significance":"If fully supported, this is a valuable contribution to TCR specificity prediction. The pseudo-likelihood derivation connects contrastive metric learning to established statistical physics inference, and the learned metrics are simple, interpretable, and available as drop-in replacements for TCRdist. The external validation on VDJdb and the transfer to TCRalpha chains strengthen the claim that local co-specificity rules are shared across ligands and chains. The authors provide public code and a clear data-availability statement, and they explicitly discuss several limitations, including the factorization assumption and the positive-unlabeled nature of the evaluation. The paper's central empirical assertion, however, requires stronger statistical support for the specific claim of generalization to pMHCs at least six amino acids away from the training set, which is currently not quantified at the group level.","major_comments":[{"comment":"The headline empirical claim—that the learned rules generalize to pMHCs differing by at least six amino acids from the nearest training peptide—is not supported with sufficient quantitative detail. Fig. S5A shows a distribution of minimum peptide Levenshtein distances, but the paper does not report how many of the 24 held-out MIRA pMHC groups fall in the >=6 amino-acid category, nor how many TCR pairs contribute to that subset. The error bars in Fig. S5C are computed over resampled training/test batches, which resample within the same pMHC groups and therefore do not capture group-level variability. If only a few groups satisfy the >=6 condition, the reported AUROC could be driven by easy-to-discriminate groups, and the conclusion that the rules generalize broadly to dissimilar peptides would not be established. I request that the authors report the number of pMHC groups and TCR pairs in the dissimilar-peptide subset, and provide either per-group AUROC values or a bootstrap over pMHC groups (e.g., cluster bootstrap) to quantify group-level uncertainty.","section":"Section IV D, Fig. S5C"},{"comment":"The evaluation treats all pairs from different specificity groups as negative, which is a positive-unlabeled (PU) setup because cross-reactive pairs are labeled as negative. The authors acknowledge this and note that about one in ten single edits maintain co-specificity, bounding AUROC below one. However, this labeling issue can also affect the comparison between metrics if the rate of cross-reactivity differs across edit distances or between the MIRA and VDJdb test sets. For instance, at edit distance 1 the mislabeling rate may be higher than at distance 5, and if the composition of true positives differs between datasets, the reported AUROC gap between d_AS and TCRdist could be partly an artifact of label noise rather than metric quality. I recommend a sensitivity analysis that either restricts to high-confidence negatives (e.g., pairs with large TCR sequence dissimilarity or different HLA restrictions) or estimates and corrects for mislabeling rates, to confirm that the relative improvements are robust.","section":"Section III, Section VI"},{"comment":"The joint metric d_AS assumes that site contributions and amino-acid substitution effects factorize, i.e., that P_S and P_A are statistically independent. The authors mention this assumption in the Discussion, but the empirical section does not test it. Because the metric's extrapolation to multi-edit pairs and to dissimilar pMHCs relies on this factorization, I ask the authors to report how the model's performance compares with a non-factorized alternative (e.g., a full site-by-amino-acid matrix trained on the same data) or to provide diagnostic evidence that the independence assumption is not the main cause of the observed performance decline at edit distances beyond 3-4. This would strengthen the interpretation of the learned weights as biophysical quantities rather than as phenomenological fit parameters.","section":"Section IV D, Eq. 14-15"}],"minor_comments":[{"comment":"In the sentence 'P(σ) represents the highly highly non-uniform', 'highly' is repeated; this typo should be corrected.","section":"Section I"},{"comment":"The captions state 'resampling training data with replacement fiveteen times'; 'fiveteen' should be 'fifteen'.","section":"Figure S3 and S4 captions"},{"comment":"The word 'biopysical' in 'biopysical interpretation' should be 'biophysical'.","section":"Section V"},{"comment":"In the Steric Property rows, 'STERMIMOL' should be 'STERIMOL' (three occurrences).","section":"Table S1"},{"comment":"The notation switches between A(Δ_i) in Eq. 12 and P_A(Δ_i) in Eq. 15; please define the relationship explicitly to avoid confusion when interpreting the learned matrix A.","section":"Section IV D, Eq. 14"},{"comment":"Fig. S5C reports error bars 'across 10 test batches', while other figures use 15 resampled batches; please clarify why the number differs and ensure the caption describes the resampling procedure consistently.","section":"Figure S5"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is well-organized, the framework is clearly presented, and the code availability is a strength. The central empirical claim about generalization to pMHCs at least six amino acids away from training is plausible but currently lacks the quantitative group-level analysis needed to support it firmly. The missing analyses—counts of dissimilar-peptide groups/pairs and group-level uncertainty—are straightforward to add and do not require new data. I see no evidence of circularity or misconduct; the concerns are about statistical support and robustness, not about the validity of the overall approach."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The real news here is a jointly learned site- and identity-dependent TCR distance metric that beats TCRdist on held-out MIRA data and on external VDJdb data, including for the alpha chain. The pseudolikelihood derivation is clean and does real work: it explains why the contrastive loss behaves like a log-odds ratio, with an empirical null instead of BLOSUM's factorized frequencies. The analytical solution for the simple case is correct and ties the method to established substitution-matrix thinking without overselling it. Code is available, the authors are explicit about the positive-unlabeled label problem and the single-edit training restriction, and they cite tcrBLOSUM as independent confirmation rather than hiding it. The novelty is not the substitution matrix idea per se; it is the joint optimization of position and identity weights and the out-of-sample generalization claim.\n\nThe soft spots are real but mostly fixable. The one that matters is the claim that the rules generalize to peptides differing by at least 6 amino acids from the nearest training peptide. Figure S5C apparently shows this, but the text never reports how many of the 24 hold-out pMHC groups satisfy that threshold or how many co-specific and cross-specific TCR pairs sit in that subset. The error bars come from resampling the same groups, so they do not capture group-level variability. If the >=6-aa subset is a handful of easy groups, the \"striking breadth\" conclusion is not established. This needs either a group-level bootstrap or explicit counts, or the claim should be softened. Relatedly, the factorization in Eq. 14-15 is a modeling premise, and the paper's own Fig. 6 shows performance degrading beyond edit distance 3-4, which is consistent with independence being only a first-order approximation. That does not invalidate the metric, but it should be discussed as a known limit rather than a minor detail. The biophysical property comparison in Fig. 5 has no confidence intervals; the steric-versus-hydrophobic claim would be stronger with some measure of spread.\n\nThe PU-label issue is acknowledged and bounded by the authors' earlier estimate that about one in ten single edits preserves co-specificity, so it depresses absolute AUROC but does not obviously bias the ranking of metrics. The lack of a direct benchmark against tcrBLOSUM is a missed opportunity, especially since the authors cite it as confirmation.\n\nThis deserves a serious referee. The paper is not circular; the generalization claim is genuinely out-of-sample. The path is revision, not rejection: quantify the dissimilar-peptide subset, add group-level resampling, add intervals to the property correlations, and benchmark against tcrBLOSUM. I would engage with it and would bring it to a reading group to argue about the right way to sample groups.","headline":"A genuinely useful TCR distance metric with a real generalization claim, but the headline extrapolation rests on a subset whose size the paper never reports.","tokens_in":714,"tokens_out":1976,"would_cite":true,"duration_ms":31393,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that simple, interpretable distance metrics learned from T cell receptor sequence data predict whether two receptors share a peptide target, and that these rules transfer to peptides highly dissimilar from any seen in…","keywords":["T cell receptor","TCR co-specificity","contrastive learning","distance metric learning","amino acid substitution matrix","SARS-CoV-2","two-point statistics"],"falsifier":"Measure co-specificity rates for TCR pairs that differ at two positions where each single substitution is known to preserve co-specificity: the factorized model (Eq. 14) predicts the double-substitution co-specificity should be the product of the two single-site probabilities; if the observed double-mutant co-specificity is substantially lower, the independence assumption fails and the metric's weights are mis-calibrated.","tokens_in":17294,"feed_emoji":"🧬","tokens_out":7455,"duration_ms":63719,"temperature":0.7,"pith_summary":"The paper aims to show that the chance two T cell receptors bind the same peptide can be captured by a simple, interpretable distance metric learned from receptor sequence data, and that the resulting rules transfer to peptides very different from any seen during training. It derives the metric from a contrastive loss that it connects to pseudo-likelihood maximization, then fits position- and amino-acid-specific weights on a large set of SARS-CoV-2-specific TCRs. The learned joint metric outperforms the standard TCRdist metric on held-out and external data, and remains predictive for peptide targets differing by at least six amino acids from the nearest training peptide. The fitted weights indicate that matching the steric (shape) properties of substituted amino acids matters more for co-specificity than hydrophobic properties, and that residues not in direct peptide contact still contribute substantially. These findings support a view of TCR specificity as governed by local, transferable sequence rules.","feed_headline":"TCR distance metric predicts shared specificity on unseen targets","feed_subtitle":"Shape-matching, not hydrophobicity, decides which amino-acid changes destroy shared peptide binding.","key_machinery":"The machinery is a factorized, interpretable distance metric defined over edit steps between TCR CDR3 sequences. For each edit step $i$, the metric combines a site weight $S(k_i, \\ell)$ and an amino-acid substitution weight $A(\\Delta_i)$ through $d_{AS}(\\sigma,\\sigma') = -\\sum_i \\log\\left[1 - \\left(1 - e^{-S(k_i,\\ell)}\\right)\\left(1 - e^{-A(\\Delta_i)}\\right)\\right]$, which corresponds to a co-selection factor $Q_{AS}$ proportional to the product over sites of $[1 - P_S(k_i,\\ell)(1 - P_A(\\Delta_i))]$, with $P_S$ a per-site contribution probability and $P_A$ a per-substitution co-specificity retention probability. Parameters are fitted by minimizing a contrastive loss that the paper derives as a negative log-pseudo-likelihood, whose optimum sets each feature weight to the log-ratio of co-specific pair frequency to unlabeled pair frequency — a direct data-driven analog of log-odds substitution matrices. This two-point-statistics view is what lets the learned rules generalize across ligand landscapes rather than fitting one peptide at a time.","core_discovery":"On the paper's own terms, the central discovery is that TCR co-specificity — whether two receptors share a ligand — is governed by local and largely ligand-independent sequence rules that can be learned from data. Fitting an exponential model of co-selection factors, the authors optimize a factorized distance metric $d_{AS}$ that assigns separate weights to the position of an amino-acid substitution and the identity of the substitution, with the two combined as independent probabilities per edit step. This metric outperforms BLOSUM-based TCRdist on both held-out SARS-CoV-2 specificity groups and an external dataset covering other pathogens, and it continues to separate co-specific from cross-specific pairs when the peptide differs by at least six amino acids from any training peptide. The learned substitution matrix is only moderately correlated with BLOSUM62 and tracks steric properties more strongly than hydrophobicity, while the learned site weights show that positions not contacting the peptide matter for specificity. The paper concludes that simple learned distance metrics can substitute for heuristic ones and that contrastive learning provides a principled route to two-point statistics of receptor–ligand maps.","pith_inferences":["A consequence not explored in the paper: the same two-point, pseudo-likelihood machinery should transfer to antibody–antigen co-specificity data, where the learned substitution matrix could be compared directly.","Because the factorized model degrades past 3–4 edits, an explicit next step is to add pairwise, context-dependent terms to Eq. 14; if that restores performance at larger edit distances, the independence assumption is the bottleneck.","The paper's property analysis suggests a broader rule: flexible binding loops (TCR, antibody) are governed by steric matching, while rigid pockets (protease) are governed by hydrophobicity; this can be tested on additional protein–protein interaction classes.","The successful transfer from one virus's peptides to unrelated epitopes implies co-specificity rules could serve as a zero-shot prior for newly discovered epitopes with no known TCRs, something the paper mentions only as an eventual aim."],"forward_implications":["Learned TCR distance metrics can replace heuristic ones like TCRdist as drop-in similarity scores for clustering and annotation of TCR repertoires.","The same contrastive framework can be retrained on larger paired-chain data to yield a machine-optimized metric with more parameters and, presumably, further reach.","Because the learned substitution matrix emphasizes steric shape over hydrophobicity, sequence-similarity tools for T cells should be recalibrated accordingly rather than relying on BLOSUM.","The finding that non-contact CDR3 positions influence co-specificity argues against trimming loop edges when scoring TCR similarity.","Some co-specificity rules are shared between the $\\alpha$ and $\\beta$ chains, since metrics trained on CDR3$\\beta$ transfer to CDR3$\\alpha$ pairs in external data."],"supporting_citations":[{"why":"supplies the large SARS-CoV-2 TCR-pMHC training set with experimentally determined specificity.","marker":"[11]"},{"why":"established that co-selection factors decay with Levenshtein distance, motivating the exponential ansatz and two-point statistics.","marker":"[26]"},{"why":"defines the TCRdist baseline metric that the learned metrics must outperform.","marker":"[3]"},{"why":"defines the BLOSUM substitution matrix the paper compares its learned matrix against.","marker":"[24]"},{"why":"provides the TCR-pMHC structures used to compute contact probabilities for validating site weights.","marker":"[46]"},{"why":"supplies thermodynamic mutation data used to test whether steric properties dominate other protein-protein interfaces.","marker":"[48]"},{"why":"documents that sequence-based models fail to generalize to unseen epitopes, setting the benchmark the paper's extrapolation claim beats.","marker":"[21]"},{"why":"provides the external TCR dataset from other pathogens used to validate generalization.","marker":"[41]"}],"fun_headline_variants":["Shape matching, not hydrophobicity, governs TCR co-specificity","Learned TCR distance metric generalizes to unseen ligands","Data-driven rules predict shared TCR specificity on new peptides","Steric properties outweigh hydrophobicity in TCR specificity"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything rests on the assumption that the impact of an amino-acid change depends only on the site and the amino-acid identity, with these two factors multiplying independently per edit step; if a substitution's effect changes with its sequence context, the learned weights are biased and the demonstrated extrapolation to multi-edit pairs is not assured.","fun_headline_variants_meta":{"raw":{"variants":["Shape matching, not hydrophobicity, governs TCR co-specificity","Learned TCR distance metric generalizes to unseen ligands","Data-driven rules predict shared TCR specificity on new peptides","Steric properties outweigh hydrophobicity in TCR specificity"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000177,"raw_usage":{"total_tokens":1291,"prompt_tokens":940,"completion_tokens":351,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":556,"completion_tokens_details":{"reasoning_tokens":294}},"tokens_in":556,"tokens_out":351,"duration_ms":3883,"temperature":1.0,"reasoning_tokens":294,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T12:51:29.727513+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure co-specificity rates for TCR pairs that differ at two positions where each single substitution is known to preserve co-specificity: the factorized model (Eq. 14) predicts the double-substitution co-specificity should be the product of the two single-site probabilities; if the observed double-mutant co-specificity is substantially lower, the independence assumption fails and the metric's weights are mis-calibrated.","supporting_citations":[{"cited_title":"Nolan, M","cited_arxiv_id":null,"evidence_quote":"supplies the large SARS-CoV-2 TCR-pMHC training set with experimentally determined specificity."},{"cited_title":"Mayer and C","cited_arxiv_id":null,"evidence_quote":"established that co-selection factors decay with Levenshtein distance, motivating the exponential ansatz and two-point statistics."},{"cited_title":"Henikoff and J","cited_arxiv_id":null,"evidence_quote":"defines the BLOSUM substitution matrix the paper compares its learned matrix against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"provides the TCR-pMHC structures used to compute contact probabilities for validating site weights."},{"cited_title":"Jankauskait˙ e, B","cited_arxiv_id":null,"evidence_quote":"supplies thermodynamic mutation data used to test whether steric properties dominate other protein-protein interfaces."},{"cited_title":"Nielsen, A","cited_arxiv_id":null,"evidence_quote":"documents that sequence-based models fail to generalize to unseen epitopes, setting the benchmark the paper's extrapolation claim beats."},{"cited_title":"Goncharov, D","cited_arxiv_id":null,"evidence_quote":"provides the external TCR dataset from other pathogens used to validate generalization."}],"review_version":1}