REVIEW 3 major objections 4 minor 44 references
Antibody Ranking Learns From Its Own Test Data, With Caveats
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
AbICL adapts antibody affinity ranking at test time by conditioning on labeled pairwise comparisons through a transformer context head trained with episodic meta-learning, achieving improved AUROC on the AbRank benchmark.
T0 review reviewed 2026-07-08 challenge →
load-bearing objection Real methodological novelty in applying ICL to antibody affinity ranking, but headline claims rest on a test-context protocol that is at least partially transductive; honest training-context results are modest and mixed. the 3 major comments →
AbICL: In-Context Learning for Antigen-Specific Antibody Affinity Ranking
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
Core claim
The paper's central claim is that antibody affinity ranking is inherently antigen-specific: a single global ranking function learned during training cannot capture the binding landscape of every target antigen, but a model trained to condition on a handful of labeled affinity comparisons for the target antigen can adapt its predictions at inference time without parameter updates. The authors demonstrate this by showing that episodic meta-training is the load-bearing component: an architecturally identical model trained without episodic support sets derives almost no benefit from contextual demonstrations, while the episodically trained model uses them effectively. The authors also find that,
What carries the argument
Context Ranking Head: a Pre-LayerNorm Transformer encoder (4 layers, 768 hidden dim, 8 heads) that takes as input a concatenated sequence of support pair tokens (each augmented with a learnable label embedding) and query pair tokens (without labels), and produces predictions only from the query outputs. The self-attention mechanism allows each query to attend to all labeled support pairs, enabling antigen-specific adaptation. No positional encoding is used, since the support set is unordered. The structural encoder (two GCNs over antibody and antigen molecular graphs, initialized from a pretrained affinity ranking model) is frozen during Context Ranking Head training.
Load-bearing premise
The Test-context protocol draws labeled demonstrations from the test set itself, meaning the model sees pairs from the same distribution (and for the Local Perturbation benchmark, the same antigen mutant set) it is being evaluated on. For Local Perturbation, where all pairs are near-neighbor mutants of one antigen, even a single labeled pair from the same set reveals the local affinity landscape. This is transductive evaluation, not standard in-context generalization from a独立
What would settle it
If the Training-context protocol (using only historical training data as demonstrations) showed no improvement over the No-context baseline, or if the episodic and non-episodic variants performed identically when given the same context, the claim that in-context adaptation (rather than transductive leakage or added model capacity) drives the gains would be falsified. The Training-context results do show improvement in several settings but underperform the baseline in others, which partially tempers the claim.
If this is right
- If in-context adaptation genuinely helps antibody ranking, the practical workflow changes: instead of running a fixed predictor on all candidates, one would first experimentally characterize a small number of affinity comparisons for the target antigen, then feed those as context to rank the remaining candidates.
- The finding that demonstrations from biologically related (but not identical) antigens still improve performance suggests that context need not come from the exact target, which matters for antigens where no prior data exists.
- The episodic meta-training recipe (random support size, antigen-grouped episodes, frozen encoder) could be transferred to other biomolecular ranking tasks where a small number of labeled comparisons is available at test time, such as T-cell receptor binding or small-molecule potency ranking.
- The observation that context helps most under distribution shift and fine-grained discrimination implies that in-context learning is complementary to, not redundant with, a strong global ranking function: the global function handles easy cases, and context rescues hard ones.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes AbICL, an in-context learning (ICL) framework for antigen-specific antibody affinity ranking. The method combines a pretrained graph-based structural encoder with a Transformer-based Context Ranking Head trained via episodic meta-learning. The model is evaluated on the AbRank benchmark under three inference protocols: No-context, Training-context (demonstrations from the training set), and Test-context (demonstrations from the test set itself). The authors claim that AbICL consistently outperforms existing baselines and that the gains arise from effective in-context adaptation rather than increased model capacity.
Significance. Applying in-context learning to antibody-antigen affinity ranking is a novel and well-motivated direction. The episodic meta-training ablation (Figure 3, Table III) provides a fair, parameter-matched comparison demonstrating that the ICL mechanism, rather than mere capacity increase, drives the observed improvements under the Training-context protocol. The analysis of context source relevance (Figure 4, Table IV) is a valuable contribution, showing that biologically related context remains informative. However, the significance of the headline results is substantially undermined by the evaluation methodology used for the Test-context protocol.
major comments (3)
- Section IV-A3 (Test-context protocol) and Table II: The Test-context protocol draws labeled demonstrations from the test set itself. For the Local Perturbation benchmark, where all pairs are near-neighbor mutants of the same antigen, providing labeled test pairs as context constitutes transductive leakage. The model gains access to the local affinity landscape of the specific test distribution. The headline claim of 'consistent outperformance' (Abstract, Contributions) relies heavily on Test-context results (e.g., 0.872 AUROC on Local Perturbation Balanced, a 0.204 jump over Training-context). This is not standard in-context generalization. The claim must be re-scaled to match the Training-context results, which show only modest improvements and actually underperform WALLE-Affinity on Hard Ag (0.581 vs 0.637 on Local Perturbation). The abstract and contributions should be revised to明确区分由
- Abstract and Section IV-B: The claim that AbICL 'consistently outperforms existing ranking and regression baselines across almost all data splits' is only supported when the transductive Test-context protocol is included. Under the more standard Training-context protocol, AbICL underperforms WALLE-Affinity on several settings (e.g., Local Perturbation Balanced: 0.668 vs 0.671; Local Perturbation Hard Ag: 0.581 vs 0.637; Unrelated Complex Hard Ag: 0.711 vs 0.746). The framing of the results in the abstract and main text conflates these two protocols, overstating the method's generalization capabilities.
- Section IV-B and Table II: No error bars, confidence intervals, or statistical significance tests are reported for any of the results. Given that the differences between AbICL (Training-context) and WALLE-Affinity are small in several splits (e.g., 0.880 vs 0.866 on Unrelated Complex Balanced), it is impossible to assess whether these improvements are statistically meaningful without variance estimates.
minor comments (4)
- Section I: The phrase 'can predictions benefit from using by using antigen-specific evidence available?' contains a duplicated 'using by using'.
- References: Reference [1] is cited twice in the same bracket in Section I: '[1, 1, 2, 3, 4]'. Please correct the citation numbering.
- Figure 2: The y-axis scale for the Local Perturation benchmark starts at 0.5, while the Unrelated Complex benchmark starts at 0.5 as well but the visual layout makes the Local Perturbation bars appear to have larger gaps. Ensure visual representations are not misleading regarding the magnitude of differences.
- Section IV-B1: The text states 'our reproduced structural encoder obtained in Stage 1 is weaker than the original model.' While the authors transparently acknowledge this, it complicates the baseline comparison. It would be helpful to clarify whether the WALLE-Affinity numbers reported in Table II are from the original paper or from a re-run of the reproduced encoder.
Simulated Author's Rebuttal
We thank the referee for a careful and substantive review. The core methodological concern about the Test-context protocol and the framing of headline claims is well-taken. We will revise the manuscript to clearly distinguish the two inference protocols and re-scale claims accordingly, and we will add statistical significance reporting. We provide point-by-point responses below.
read point-by-point responses
-
Referee: Test-context protocol constitutes transductive leakage on Local Perturbation; headline claims rely on Test-context results and must be re-scaled to Training-context.
Authors: We agree with the referee's core observation. On the Local Perturbation benchmark, where all pairs are near-neighbor mutants of the same antigen, the Test-context protocol does provide the model with access to the local affinity landscape of the test distribution. This is a form of transductive inference, and it is not equivalent to standard in-context generalization from training-distribution demonstrations. We acknowledge that the current framing in the abstract and contributions conflates the two protocols and overstates the method's generalization capabilities by leaning on Test-context results. We will revise the abstract, contributions, and Section IV-B to clearly separate the two protocols and to present Training-context as the primary evaluation for generalization claims. Under Training-context, AbICL shows modest but real improvements on several settings (e.g., Unrelated Complex Balanced: 0.880 vs 0.866; Unrelated Complex Hard Ab: 0.838 vs 0.763), while underperforming WALLE-Affinity on others (e.g., Local Perturbation Hard Ag: 0.581 vs 0.637). We will state these comparisons transparently. The Test-context results will be retained as a separate analysis motivated by the practical scenario where a small number of experimental measurements are available at inference time (following the CAMP protocol), but they will be explicitly labeled as transductive and excluded from the headline generalization claims. revision: yes
-
Referee: Abstract and Section IV-B claim 'consistent outperformance' but this is only supported with Test-context; under Training-context, AbICL underperforms WALLE-Affinity on several splits.
Authors: The referee is correct on the specific numbers. Under Training-context, AbICL underperforms WALLE-Affinity on Local Perturbation Balanced (0.668 vs 0.671), Local Perturbation Hard Ag (0.581 vs 0.637), and Unrelated Complex Hard Ag (0.711 vs 0.746). The current abstract claim that AbICL 'consistently outperforms existing ranking and regression baselines across almost all data splits' is not supported by the Training-context results alone and will be revised. We will replace the blanket claim with a more precise statement: AbICL under Training-context achieves improvements on the Unrelated Complex benchmark (particularly on Balanced and Hard Ab splits) and shows competitive but not uniformly superior performance on the Local Perturbation benchmark. We will also note explicitly where WALLE-Affinity remains stronger. The contributions bullet claiming 'state-of-the-art performance on the AbRank benchmark' will be revised to specify that this holds only under the Test-context (transductive) protocol, and that under the standard Training-context protocol the gains are more modest and setting-dependent. revision: yes
-
Referee: No error bars, confidence intervals, or statistical significance tests are reported.
Authors: This is a valid and important criticism. We did not report variance estimates or significance tests in the original submission. We will address this by running multiple seeds (at least 5) for both Stage 1 and Stage 2 training and reporting mean AUROC with standard deviations for all results in Table II. Where the differences between AbICL (Training-context) and WALLE-Affinity are small (e.g., 0.880 vs 0.866 on Unrelated Complex Balanced, or 0.668 vs 0.671 on Local Perturbation Balanced), we will report paired bootstrap confidence intervals or DeLong test p-values to assess whether the differences are statistically meaningful. We acknowledge that some of the smaller differences may not reach statistical significance, and we will state this transparently in the revised manuscript. revision: yes
Circularity Check
No significant circularity; Test-context protocol raises evaluation-validity concerns but predictions are not equivalent to inputs by construction
full rationale
The paper's derivation chain is self-contained. The model architecture (Eqs. 2-7) defines a standard encoder + Transformer attention head trained via episodic meta-training; no equation reduces a claimed prediction to its inputs by definition. The Test-context protocol (Section IV-A3, Appendix C2) draws labeled demonstration pairs from the test set and evaluates on non-overlapping query pairs from the same antigen. While this raises legitimate transductive-leakage and evaluation-validity concerns—especially on Local Perturbation where all pairs are near-neighbor mutants of one antigen—the model's predictions on query pairs are not equivalent to the demonstration labels by construction. The model processes demonstrations through self-attention and produces independent predictions on different pairs; there is no definitional identity between input demonstrations and output predictions. The Training-context protocol, which is more methodologically independent, shows mixed results (underperforming WALLE-Affinity on several settings), but this is a correctness/evaluation concern rather than circularity. No self-citation chain is load-bearing: the paper cites external works (AbRank [10], CAMP [35]) by different author groups. The headline claim of 'consistent outperformance' is overstated when restricted to Training-context results, but overclaiming is not circularity. Score 1 reflects the minor concern that the abstract does not distinguish between Test-context and Training-context protocols when claiming consistent outperformance, but the paper itself is transparent about the protocol definitions.
Axiom & Free-Parameter Ledger
free parameters (6)
- K_max =
8
- Context Ranking Head depth =
4 layers
- d_model =
768
- Learning rate =
1e-4
- Episode batch size =
8
- Context demonstration ratio r =
not specified explicitly
axioms (3)
- domain assumption The WALLE-Affinity GCN encoder architecture produces meaningful structural representations for antibody-antigen complexes
- domain assumption Pairwise binary ranking labels from AbRank are reliable ground truth
- ad hoc to paper Self-attention over support-query pairs is a sufficient mechanism for in-context adaptation
Cite this review
Pith. "Pith review of AbICL: In-Context Learning for Antigen-Specific Antibody Affinity Ranking." pith.science (2026). https://pith.science/paper/OCIRP7XP
@misc{pith2026260705846,
author = {Pith},
title = {Pith review of: AbICL: In-Context Learning for Antigen-Specific Antibody Affinity Ranking},
year = {2026},
howpublished = {\url{https://pith.science/paper/OCIRP7XP}},
note = {Machine review of arXiv:2607.05846}
}
read the original abstract
Accurate ranking of antibody candidates according to their binding affinity is essential for therapeutic antibody discovery. However, existing methods treat affinity comparisons independently and ignore the contextual information encoded in other labeled comparisons, limiting their ability to capture antigen-specific binding landscapes. For many target antigens, a small number of experimentally characterized affinity comparisons are often available. An important question is whether the model can exploit these existing comparisons to infer antigen-specific ranking patterns that facilitate subsequent affinity ranking. This form of learning from labeled demonstrations closely resembles the paradigm of In-Context Learning, motivating us to revisit antibody affinity ranking from an ICL perspective. To this end, we propose AbICL, an ICL framework for antigen-specific antibody affinity ranking. AbICL combines a pretrained structural encoder with a context ranking head and is trained with an episodic meta-training strategy that enables the model to leverage support demonstrations for test-time adaptation without gradient updates. Experiments on the AbRank benchmark demonstrate that AbICL consistently outperforms existing ranking baselines across almost all data splits and evaluation benchmarks. Further analysis shows that the value of contextual demonstrations depends on how well they match the target inference task, and becomes increasingly pronounced under distribution shift and fine-grained affinity discrimination. These findings highlight the potential of ICL as an effective paradigm for antigen-specific antibody affinity ranking, particularly in challenging settings where a single global ranking function is insufficient.
Figures
Reference graph
Works this paper leans on
-
[1]
Ziyang Wang, Yu Zhang, Youli Zhang, Jianwei Huang, Xiaoli Lu, Xiaoping Min, Shengxiang Ge, Jun Zhang, and Ningshao Xia. Predicting antibody–antigen affinity with a dual-level representation model.Bioinformatics, 42(4):btag109, 04 2026
work page 2026
-
[2]
Learning the language of protein-protein interactions.Nature Communications, 2026
Varun Ullanat, Bowen Jing, Samuel Sledzieski, and Bon- nie Berger. Learning the language of protein-protein interactions.Nature Communications, 2026
work page 2026
-
[3]
Minghui Li, Yao Shi, Shengqing Hu, Shengshan Hu, Peijin Guo, Wei Wan, Leo Yu Zhang, Shirui Pan, Jizhou Li, Lichao Sun, and Xiaoli Lan. Mvsf-ab: accurate anti- body–antigen binding affinity prediction via multi-view sequence feature learning.Bioinformatics, 41(5):btae579, 05 2025
work page 2025
-
[4]
Ye Yuan, Qushuo Chen, Jun Mao, Guipeng Li, and Xiaoyong Pan. Dg-affinity: predicting antigen–antibody affinity with language models from sequences.BMC bioinformatics, 24(1):430, 2023
work page 2023
-
[5]
Nuwan Bandara, Dasun Premathilaka, Sachini Chan- danayake, Sahan Hettiarachchi, Vithurshan Varenthirara- jah, Aravinda Munasinghe, Kaushalya Madhawa, and Subodha Charles. Deep geometric framework to predict antibody-antigen binding affinity.Journal of Structural Biology, page 108257, 2025
work page 2025
-
[6]
Alissa M Hummer, Constantin Schneider, Lewis Chinery, and Charlotte M Deane. Investigating the volume and diversity of data needed for generalizable antibody– antigenδδg prediction.Nature Computational Science, 5(8):635–647, 2025
work page 2025
-
[7]
Anushriya Subedy, Siddharth Bhadra-Lobo, and Guil- laume Lamoureux. Se3bind: Se (3)-equivariant model for antibody-antigen binding affinity prediction.bioRxiv, pages 2026–01, 2026
work page 2026
-
[8]
Zhengshan Chen, Song He, Xiangyang Chi, and Xi- aochen Bo. Predicting antibody affinity changes upon mutation based on unbound protein structures.Interna- tional Journal of Molecular Sciences, 26(3):1343, 2025
work page 2025
-
[9]
Huiyu Cai, Zuobai Zhang, Mingkai Wang, Bozitao Zhong, Quanxiao Li, Yuxuan Zhong, Yanling Wu, Tianlei Ying, and Jian Tang. Pretrainable geometric graph neural network for antibody affinity maturation.Nature communications, 15(1):7785, 2024
work page 2024
-
[10]
AbRank: A Benchmark Dataset and Metric-Learning Framework for Antibody-Antigen Affinity Ranking
Chunan Liu, Aurelien Pelissier, Yanjun Shao, Lilian Denzler, Andrew CR Martin, Brooks Paige, and Mar ´ıa Rodr ´ıguez Mart ´ınez. Abrank: A benchmark dataset and metric-learning framework for antibody-antigen affinity ranking.arXiv preprint arXiv:2506.17857, 2025
work page internal anchor Pith review Pith/arXiv arXiv 2025
-
[11]
Fan Xu, Zhi-an Huang, Haohuai He, Yidong Song, Wei Liu, Dongxu Zhang, Yao Hu, and Kay Chen Tan. Ablwr: A context-aware listwise ranking framework for antibody-antigen binding affinity prediction via positive- unlabeled learning.arXiv preprint arXiv:2604.11272, 2026
work page internal anchor Pith review Pith/arXiv arXiv 2026
-
[12]
Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, et al. Evolutionary- scale prediction of atomic-level protein structure with a language model.Science, 379(6637):1123–1130, 2023
work page 2023
-
[13]
Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C Lawrence Zitnick, Jerry Ma, et al. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences.Proceedings of the na- tional academy of sciences, 118(15):e2016239118, 2021
work page 2021
-
[14]
Language modeling materializes a world model of protein biology.bioRxiv, pages 2026–06, 2026
Salvatore Candido, Thomas Hayes, Alexander Derry, Roshan Rao, Zeming Lin, Robert Verkuil, Bryan Z Wu, Jin Sub Lee, Elise S Bruguera, Jehan A Keval, et al. Language modeling materializes a world model of protein biology.bioRxiv, pages 2026–06, 2026
work page 2026
-
[15]
Josh Abramson, Jonas Adler, Jack Dunger, Richard Evans, Tim Green, Alexander Pritzel, Olaf Ronneberger, Lindsay Willmore, Andrew J Ballard, Joshua Bambrick, et al. Accurate structure prediction of biomolecular interactions with alphafold 3.Nature, 630(8016):493– 500, 2024
work page 2024
-
[16]
Rohith Krishna, Jue Wang, Woody Ahern, Pascal Sturm- fels, Preetham Venkatesh, Indrek Kalvet, Gyu Rie Lee, Felix S Morey-Burrows, Ivan Anishchenko, Ian R Humphreys, et al. Generalized biomolecular model- ing and design with rosettafold all-atom.Science, 384(6693):eadl2528, 2024
work page 2024
-
[17]
Boltz-1 de- mocratizing biomolecular interaction modeling.BioRxiv, pages 2024–11, 2025
Jeremy Wohlwend, Gabriele Corso, Saro Passaro, Noah Getz, Mateo Reveiz, Ken Leidal, Wojtek Swiderski, Liam Atkinson, Tally Portnoi, Itamar Chinn, et al. Boltz-1 de- mocratizing biomolecular interaction modeling.BioRxiv, pages 2024–11, 2025
work page 2024
-
[18]
Boltz-2: Towards accurate and efficient binding affinity prediction.BioRxiv, 2025
Saro Passaro, Gabriele Corso, Jeremy Wohlwend, Mateo Reveiz, Stephan Thaler, Vignesh Ram Somnath, Noah Getz, Tally Portnoi, Julien Roy, Hannes Stark, et al. Boltz-2: Towards accurate and efficient binding affinity prediction.BioRxiv, 2025
work page 2025
-
[19]
Chai-1: Decoding the molecular interactions of life.BioRxiv, pages 2024–10, 2024
Chai Discovery team, Jacques Boitreaud, Jack Dent, Matthew McPartlon, Joshua Meier, Vinicius Reis, Alex Rogozhonikov, and Kevin Wu. Chai-1: Decoding the molecular interactions of life.BioRxiv, pages 2024–10, 2024
work page 2024
-
[20]
Zero-shot antibody design in a 24-well plate.bioRxiv, pages 2025–07, 2025
Chai Discovery Team, Jacques Boitreaud, Jack Dent, Danny Geisz, Matthew McPartlon, Joshua Meier, Zhuo- ran Qiao, Alex Rogozhnikov, Nathan Rollins, Paul Wol- lenhaupt, et al. Zero-shot antibody design in a 24-well plate.bioRxiv, pages 2025–07, 2025
work page 2025
-
[21]
ByteDance AML AI4Science Team, Xinshi Chen, Yux- uan Zhang, Chan Lu, Wenzhi Ma, Jiaqi Guan, Chengyue Gong, Jincai Yang, Hanyu Zhang, Ke Zhang, et al. Protenix-advancing structure prediction through a com- prehensive alphafold3 reproduction.BioRxiv, pages 2025–01, 2025
work page 2025
-
[22]
Xiaotong Xu, Ilaria Coratella, Victor Reys, and Alexan- dre MJJ Bonvin. Deeprank-ab: a scoring function for antibody-antigen complexes based on geometric deep learning.Communications Biology, 2026
work page 2026
-
[23]
Jeffrey A Ruffolo, Lee-Shin Chu, Sai Pooja Mahajan, and Jeffrey J Gray. Fast, accurate antibody structure prediction from deep learning on massive set of natural antibodies.Nature communications, 14(1):2389, 2023
work page 2023
-
[24]
Ruofan Jin, Qing Ye, Jike Wang, Zheng Cao, Dejun Jiang, Tianyue Wang, Yu Kang, Wanting Xu, Chang-Yu Hsieh, and Tingjun Hou. Attabseq: an attention-based deep learning prediction method for antigen–antibody binding affinity changes based on protein sequences. Briefings in Bioinformatics, 25(4):bbae304, 07 2024
work page 2024
-
[25]
Chen Liu, Mingchen Li, Yang Tan, Wenrui Gou, Guisheng Fan, and Bingxin Zhou. Sequence-only pre- diction of binding affinity changes: a robust and inter- pretable model for antibody engineering.Bioinformatics, 41(8):btaf446, 2025
work page 2025
-
[26]
Matching networks for one shot learning
Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. Advances in neural information processing systems, 29, 2016
work page 2016
-
[27]
Jake Snell, Kevin Swersky, and Richard Zemel. Prototyp- ical networks for few-shot learning.Advances in neural information processing systems, 30, 2017
work page 2017
-
[28]
Model- agnostic meta-learning for fast adaptation of deep net- works
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model- agnostic meta-learning for fast adaptation of deep net- works. InInternational conference on machine learning, pages 1126–1135. PMLR, 2017
work page 2017
-
[29]
On First-Order Meta-Learning Algorithms
Alex Nichol, Joshua Achiam, and John Schulman. On first-order meta-learning algorithms.arXiv preprint arXiv:1803.02999, 2018
work page internal anchor Pith review Pith/arXiv arXiv 2018
-
[30]
A Simple Neural Attentive Meta-Learner
Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, and Pieter Abbeel. A simple neural attentive meta-learner.arXiv preprint arXiv:1707.03141, 2017
work page internal anchor Pith review Pith/arXiv arXiv 2017
-
[31]
Meta-learning with memory-augmented neural networks
Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. Meta-learning with memory-augmented neural networks. InInterna- tional conference on machine learning, pages 1842–
-
[32]
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott G...
work page 1901
-
[33]
Transformers learn in-context by gradient descent
Johannes V on Oswald, Eyvind Niklasson, Ettore Ran- dazzo, Jo˜ao Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, and Max Vladymyrov. Transformers learn in-context by gradient descent. InInternational Confer- ence on Machine Learning, pages 35151–35174. PMLR, 2023
work page 2023
-
[34]
Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Shuming Ma, Zhifang Sui, and Furu Wei. Why can gpt learn in- context? language models secretly perform gradient de- scent as meta-optimizers. InFindings of the Association for Computational Linguistics: ACL 2023, pages 4005– 4019, 2023
work page 2023
-
[35]
In-Context Learning for Few-Shot Molecular Property Prediction
Christopher Fifty, Jure Leskovec, and Sebastian Thrun. In-context learning for few-shot molecular property pre- diction.arXiv preprint arXiv:2310.08863, 2023
work page internal anchor Pith review Pith/arXiv arXiv 2023
-
[36]
Pin-tuning: parameter-efficient in- context tuning for few-shot molecular property predic- tion
Liang Wang, Qiang Liu, Shaozhen Liu, Xin Sun, Shu Wu, and Liang Wang. Pin-tuning: parameter-efficient in- context tuning for few-shot molecular property predic- tion. InProceedings of the 38th International Conference on Neural Information Processing Systems, NIPS ’24, Red Hook, NY , USA, 2024. Curran Associates Inc
work page 2024
-
[37]
Metalic: Meta-learning in-context with pro- tein language models
Jacob Beck, Shikha Surana, Manus McAuliffe, Oliver Bent, Thomas Barrett, Juan Jose Garau-Luis, and Paul Duckworth. Metalic: Meta-learning in-context with pro- tein language models. InInternational Conference on Learning Representations, volume 2025, pages 48716– 48732, 2025
work page 2025
-
[38]
Kristof Sch ¨utt, Pieter-Jan Kindermans, Huziel Enoc Sauceda Felix, Stefan Chmiela, Alexandre Tkatchenko, and Klaus-Robert M ¨uller. Schnet: A continuous-filter convolutional neural network for modeling quantum in- teractions.Advances in neural information processing systems, 30, 2017
work page 2017
-
[39]
Attention is all you need.Advances in neural information processing systems, 30, 2017
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017
work page 2017
-
[40]
Set transformer: A framework for attention-based permutation-invariant neural networks
Juho Lee, Yoonho Lee, Jungtaek Kim, Adam Kosiorek, Seungjin Choi, and Yee Whye Teh. Set transformer: A framework for attention-based permutation-invariant neural networks. InInternational conference on machine learning, pages 3744–3753. PMLR, 2019
work page 2019
-
[41]
Jeffrey A Ruffolo, Jeffrey J Gray, and Jeremias Sulam. Deciphering antibody affinity maturation with language models and weakly supervised learning.Machine Learn- ing for Structural Biology Workshop, NeurIPS, 2021. APPENDIX A. Details of Model Structures The encoder follows the W ALLE-Affinity architecture . For each antibody-antigen complex, residue-leve...
work page 2021
-
[42]
Training-context:Context demonstrations are drawn from training-set pairs of the same antigen as the test antigen, with up toK max pairs selected. For the Local Perturbation benchmark, where test antigens are point mutants not seen during training, a prefix fallback strategy is applied: if no exact antigen match exists in the training set, the antigen nam...
-
[43]
Test-context:Context demonstration and query pairs are both drawn from the test set for the same antigen and are non-overlapping. Specifically,n context = min(Kmax, ,⌊N×r⌋) pairs are used as context demonstration, whereris the context demonstration ratio, and the remaining pairs serve as queries
-
[44]
No-context:No context demonstrations are provided; predictions rely solely on the encoder representations. D. Details of Experiments Table II presents the main performance comparison on the AbRank benchmark, while Table III and Table IV provide an ablation study on episodic meta-training and a detailed anal- ysis of various training and inference strategi...
This paper was first reviewed by glm-5.2 on July 8, 2026.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.