REVIEW 4 major objections 7 minor 44 references
Preference-based Antibody Expression Ranking: Scaling with Large-scale Weak Supervision
T0 review · 4 major / 7 minor · reviewed 2026-08-02 · deepseek-v4-flash
Pith's one-line read Antibody expression ranking can be learned from scarce yield labels plus millions of weakly labeled immunization-derived sequences; the recipe beats supervised baselines and improves as weak data grows.
desk verdict A practical, well-ablated approach to antibody expression ranking, but the weak-supervision signal likely teaches 'camelid-likeness' rather than 'expressibility', and all results rest on an unreleased private dataset. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The preference pair is the load-bearing object. Strong pairs compare yields only past a 30 mg/L margin; weak pairs declare each of 4.2 million immunization-derived camelid VHHs preferred over a zero-yield design, converting the assumption that such sequences are mostly expressible into training signal. DPO becomes tractable for masked protein language models via IMGT alignment to a shared 128-position grid and a union-masked pseudo-log-likelihood, scoring only differing positions in one forward pass as identical positions cancel. Around this: Stage I masked-language pretraining on the 4.2M corpus, Stage II DPO with a KL penalty keeping the model near its Stage I reference.
What would settle it
Measure recombinant expression for a few hundred randomly sampled sequences from Camel-4M under the same SDS-PAGE assay used for Exp-1K: if the detected fraction is well below the assumed ~90%, the weak preference pairs carry corrupted supervision and the Figure 4 scaling curve is an artifact of a clean weak pool rather than of preference learning itself. A complementary check is to run the exact two-stage pipeline on an external VHH expression dataset; if the ranking gains over supervised baselines do not transfer beyond the internal Exp-1K test set, the claim is distribution-specific.
Extended reading notes
Core claim
The paper's central claim: antibody expressibility becomes learnable when supervision is expressed as relative preferences rather than pointwise labels. Measured yields form pairs only past a 30 mg/L margin; each of 4.2 million immunization-derived camelid VHHs is preferred over a zero-yield design, assuming such sequences are mostly expressible. IMGT alignment to 128 positions and a union-masked pseudo-log-likelihood score only differing positions in one pass, making DPO tractable for masked language models. On Exp-1K, the two-stage pipeline posts the best composite score on all three backbones (AntiBERTa2 45.0 vs 33.1/29.0); without the weak corpus DPO underperforms regression (Table 5).
Load-bearing premise
The framework rests on the assumption that the 4.2 million immunization-derived camelid VHH sequences in Camel-4M are predominantly expressible — supported only by 16 of 17 internally measured sequences (Appendix C.3) and an evolutionary argument — because if a substantial share of those sequences are actually secretion-incompetent or misfolded under the authors' assay, the weak preference pairs encode a corrupted signal and the scaling gains of Figure 4 would not transfer to
Editorial extensions
If this is right
- Discovery campaigns that routinely generate immunization-derived sequence libraries gain a new use for them: as weak expressibility supervision, candidate prioritization improves without additional wet-lab yield measurements.
- The union-masked score approximation removes the main computational obstacle to preference optimization for masked protein language models, so the same recipe transfers to other protein properties that come with relative rather than absolute labels, such as stability, aggregation, or affinity.
- Because weak preference pairs narrow the validation-to-test generalization gap and smooth the optimization trajectory, small labeled datasets trained this way should generalize more reliably than regression heads trained on the same labels alone.
- The scaling result implies that accumulating more immunization data — a byproduct of normal discovery work — directly translates into better ranking, so unmeasured industrial sequence archives become economically valuable assets.
- The paper's own Appendix B shows a supervised binary classifier augmented with the same weak positives still wins on pure expressibility classification; the preference-based model's demonstrated advantage is ranking, so its natural deployment is as a prioritizer inside screening pipelines rather than as a hard express/no-express filter.
Reading between the lines
- The weak-positive premise is the hinge: support for '>90% of camelid VHHs are expressible' is 16 of 17 internal measurements plus an evolutionary argument. A direct assay of a few hundred randomly sampled Camel-4M sequences would reveal whether the Figure 4 scaling curve is a property of the learning method or of an unusually clean weak pool; if the true positive rate is lower, the framework exten
- The union-masked approximation weights every IMGT position equally, so hypervariable loops and gap positions carry as much preference signal as conserved framework residues; weighting positions by conservation, or masking gaps, is a concrete testable extension.
- Because the fine-tuned model scores sequences by average pseudo-log-likelihood, it can serve as a ranking layer inside generative antibody design loops, reranking model-sampled candidates rather than only screening pre-existing libraries.
- Pair construction discards most quantitative information: intermediate yields matter only when they clear the 30 mg/L margin. Graded preferences across multiple yield bins would let the scarce labeled data contribute more, and the paper's own sensitivity analysis shows the result is sensitive to how such signals are mixed.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a two-stage framework for ranking antibody sequences by expression yield. Stage I performs continual masked-language-model pretraining on Camel-4M, a proprietary corpus of 4.2M camelid VHH sequences. Stage II applies Direct Preference Optimization (DPO) to protein language models, using a union-masked pseudo-log-likelihood approximation and IMGT alignment to handle variable-length sequences. Preference pairs come from two sources: strong pairs constructed from Exp-1K yield measurements with a margin of 30 mg/L, and weak pairs that pair Camel-4M sequences (assumed expressible) against zero-yield Exp-1K sequences. On the internal Exp-1K test set (n=126), the full pipeline outperforms supervised regression/classification baselines on most metric-by-backbone cells of Table 2, and Figure 4 shows increasing performance with weak-data scale. The paper also includes ablations on weak-data removal, sampling ratio, and similarity filtering.
Significance. If the central claims hold, the framework would be a practical way to exploit large industrial immunization datasets to improve expression ranking without large labeled sets. The paper's strengths include evaluation across three backbones, a sensible ablation structure (Tables 3-6, Appendix B), a clear statement of the binary-supervision comparison in Appendix B, and an analysis of the generalization gap (Table 4). However, two load-bearing issues need to be resolved before the claims are fully established: the validity of the union-masked PLL approximation and the source/expression confound in the weak preference signal. The evaluation is also based on a single split of a proprietary dataset, with no error bars or external validation.
major comments (4)
- [Section 3.3.2] Union-masked approximation: The claim that 'positions outside M need not be evaluated, as their contributions cancel in the score difference' is not correct under the PLL definition in Section 3.2. For a pair (yw, yl), the exact difference PLL(yw)-PLL(yl) includes, for positions i not in M, terms where the token is identical but the conditioning contexts differ because the M positions are different in yw and yl; these terms do not cancel. In addition, the proposed single-forward-pass evaluation of positions i in M uses a context in which all other M positions are masked, whereas exact PLL masks only position i. The approximation therefore has an unquantified bias. Because the efficiency contribution of the paper rests on this approximation, please provide a theoretical justification or an empirical comparison with exact PLL on a subset of pairs, showing that the relative ordering of pref
- [Section 3.3.3 / Section 4.1 / Appendix C.3] Weak supervision signal: The weak pairs (yk in Camel-4M, y0 in Exp-1K with zero yield) differ in two correlated attributes: expression status and origin (immunized camelid PBMC vs synthetic design). DPO only requires the preferred sequence to receive a higher score, so the model can satisfy the weak-pair signal by learning a 'camelid-likeness' boundary rather than an 'expressible vs non-expressible' ranking. The n=17 expressibility measurement in Appendix C.3 does not rule this out, and its 95% CI is wide (16/17, approximately 71-100%). Because the Exp-1K test set is entirely synthetic, the scaling gains in Figure 4 could reflect source discrimination that transfers only if expressible synthetic designs are systematically closer to natural camelid VHHs. Please add a control that separates origin from expression status, e.g., weak pairs whose positives are expressible synthetic sequences,
- [Section 4.2 / Table 2] Statistical evaluation: The main conclusion rests on a single 80/10/10 split with n=126 test sequences and no variance estimates, repeated seeds, or significance tests. Some comparisons are close or go in the opposite direction (e.g., IgBERT AUC 74.8 vs 78.6 for the best supervised baseline), so the 'most metrics' claim is hard to interpret without uncertainty quantification. Please report bootstrapped confidence intervals or multiple-seed means for the central results, and consider evaluating on one of the public benchmarks in Table 1 to support external validity.
- [Section 4.7 / Table 5] Inconsistent claim: The text states that 'without the inclusion of large-scale weak supervision (Camel-4M), the pure DPO approach yields slightly inferior results compared to direct regression.' The table shows the opposite for the paper's own composite metric: DPO without Camel-4M has Avg = sqrt(32.8 * 35.1) ~ 33.9, while supervised regression has Avg = sqrt(29.1 * 37.6) ~ 33.1, and MCC is higher (32.8 vs 29.1). Please correct the claim or specify which metric 'inferior' refers to, since this paragraph is used to argue that weak supervision is necessary.
minor comments (7)
- [Table 2] IgBERT row: '78.67.5' appears to be a formatting error; should read '78.6 7.5' for AUC and MCC.
- [Table 1] Header 'NYield Range (mg/L)' is ambiguous; clarify which columns are N, yield range, mean, and standard deviation.
- [Figure 4] Report numerical values or confidence intervals for the scaling curves; the current figure cannot be quantitatively assessed.
- [Appendix A.4] Typo: 'n the preference optimization phase' should be 'In the preference optimization phase'.
- [Appendix C.3] The 94.1% expressibility estimate from n=17 should be reported with a confidence interval, and the random sampling procedure should be specified.
- [Section 4.2] Clarify how 'Recall at top 20%' is computed: ranking by predicted score, and which score (PLL or DPO score)?
- [Appendix A.5] Unify notation Nsup/Nweak (Section 4.5) with N_S/N_L in Appendix A.5.
Circularity Check
No significant circularity: the empirical comparisons rest on measured labels and held-out data; the weak-supervision premise is an assumption with empirical support, not a definitional reduction.
full rationale
The paper's central claims are empirical: Stage I CPt and Stage II DPO are evaluated against measured yields in a held-out Exp-1K test split (§4.3), and strong preference pairs are constructed directly from quantitative yield differences (§3.3.3). No equation in the method defines the target ranking in terms of the fitted parameters or vice versa; the DPO objective is a standard learning objective, and the PLL/union-mask approximation is cited to external work (Ferragu et al., 2025; Zhao et al., 2024), not to a self-citation chain. The weak-supervision premise that camelid-derived sequences are mostly expressible is an input assumption supported by internal n=17 measurements (Appendix C.3) and external biology; this is a data-quality or confound concern (e.g., weak pairs may encode source identity rather than expressibility), not a circular reduction of the claimed result to its inputs. The paper even reports that removing weak data or using pure DPO without it does not yield the same gains (Table 5), which confirms the result is data-driven rather than forced by construction. Therefore no specific circular step can be exhibited, and the appropriate finding is no significant circularity.
Assumptions & free parameters
free parameters (5)
- yield margin δ =
30 mg/L
- DPO KL regularization β =
0.1
- supervised oversampling rate α =
1000 (p_weak ≈ 74%)
- binary decision percentile threshold =
≈73rd percentile of PLL scores (train positive fraction)
- non-expressible cutoff =
<1 mg/L
assumptions (6)
- domain assumption Immunization-derived camelid VHH sequences are predominantly expressible (>90%)
- domain assumption Pseudo-log-likelihood (PLL) is a serviceable surrogate for sequence log-likelihood inside the DPO objective
- domain assumption The union-masked single-forward-pass estimate preserves the DPO preference gradients
- domain assumption IMGT numbering provides valid positional homology across variable-length VHHs
- domain assumption Semi-quantitative SDS-PAGE yield estimates support pair construction at margin δ=30
- standard math The Bradley-Terry preference model underlying DPO maps yields and weak positives to valid pairwise preferences
Cite this review
Pith. "Pith review of Preference-based Antibody Expression Ranking: Scaling with Large-scale Weak Supervision." pith.science (2026). https://pith.science/paper/DWPP7VBK
@misc{pith2026260716263,
author = {Pith},
title = {Pith review of: Preference-based Antibody Expression Ranking: Scaling with Large-scale Weak Supervision},
year = {2026},
howpublished = {\url{https://pith.science/paper/DWPP7VBK}},
note = {Machine review of arXiv:2607.16263}
}
read the original abstract
Antibody expression ranking is a critical task in antibody design, yet its modelling is severely hindered by the scarcity of labeled experimental data. To address this, we propose a unified preference-based learning framework that integrates scarce quantitative expression data with large-scale weak positive supervision from immunization data. We adapt Direct Preference Optimization (DPO) to protein language models by introducing a union-masked log-likelihood approximation and IMGT-based alignment, enabling efficient training on variable-length sequences. Evaluating on a diverse internal dataset of 1254 labeled sequences and 4 million unlabeled camelid-derived antibodies, we show that our method consistently outperforms baselines on most metrics. Our results demonstrate that preference learning can effectively learn from weak supervision, providing a scalable solution for antibody expressibility optimization in data-constrained settings. Project page: https://kisoji-biotechnology-inc.github.io/Preference-Expression-Ranking/.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
BioRxiv , pages=
FLAb: Benchmarking deep learning methods for antibody fitness prediction , author=. BioRxiv , pages=. 2024 , publisher=
2024
-
[2]
Proceedings of the National Academy of Sciences , volume=
Biophysical properties of the clinical-stage antibody landscape , author=. Proceedings of the National Academy of Sciences , volume=. 2017 , publisher=
2017
-
[3]
and Borhani, David , title =
Arsiwala, Ammar and Bhatt, Rebecca and Yang, Yaoyu and Quintero Cadena, Porfi and Anderson, KC and Ao, Xiang and van Niekerk, Lood and Rosenbaum, Adam and Bhatt, Aanal and Smith, Alex and Grippo, Lucia and Cao, Xing and Cohen, Rich and Patel, Jay and Allen, Olga and Faraj, Ali and Nandy, Anisha and Hocking, Jason and Tural, Berk and Salvador, Sara and Jac...
2025
-
[4]
Tushar Jain and Tingwan Sun and Stéphanie Durand and Amy Hall and Nga Rewa Houston and Juergen H. Nett and Beth Sharkey and Beata Bobrowicz and Isabelle Caffry and Yao Yu and Yuan Cao and Heather Lynaugh and Michael Brown and Hemanta Baruah and Laura T. Gray and Eric M. Krauland and Yingda Xu and Maximiliano Vásquez and K. Dane Wittrup , title =. Proceedi...
-
[5]
Szkodny, Alana C. and Lee, Kelvin H. , title =. Biotechnology Progress , volume =. doi:https://doi.org/10.1002/btpr.3466 , url =. https://aiche.onlinelibrary.wiley.com/doi/pdf/10.1002/btpr.3466 , abstract =
-
[6]
Advances in Neural Information Processing Systems , volume=
A SARS-CoV-2 interaction dataset and VHH sequence corpus for antibody language models , author=. Advances in Neural Information Processing Systems , volume=
-
[7]
arXiv preprint arXiv:2510.19474 , year=
g-DPO: Scalable Preference Optimization for Protein Language Models , author=. arXiv preprint arXiv:2510.19474 , year=
-
[8]
Advances in neural information processing systems , volume=
Direct preference optimization: Your language model is secretly a reward model , author=. Advances in neural information processing systems , volume=
Show all 44 references
-
[9]
Advances in neural information processing systems , volume=
Language models enable zero-shot prediction of the effects of mutations on protein function , author=. Advances in neural information processing systems , volume=
-
[10]
arXiv preprint arXiv:2004.03497 , year=
Progen: Language modeling for protein generation , author=. arXiv preprint arXiv:2004.03497 , year=
2004 arXiv
-
[11]
Proceedings of the 58th annual meeting of the association for computational linguistics , pages=
Masked language model scoring , author=. Proceedings of the 58th annual meeting of the association for computational linguistics , pages=
-
[12]
Nature methods , volume=
Unified rational protein engineering with sequence-based deep representation learning , author=. Nature methods , volume=. 2019 , publisher=
2019
-
[13]
Nature methods , volume=
Low-N protein engineering with data-efficient deep learning , author=. Nature methods , volume=. 2021 , publisher=
2021
-
[14]
Nature communications , volume=
An alpaca nanobody neutralizes SARS-CoV-2 by blocking receptor interaction , author=. Nature communications , volume=. 2020 , publisher=
2020
-
[15]
International Conference on Research in Computational Molecular Biology , pages=
Contrastive fitness learning: Reprogramming protein language models for low-n learning of protein fitness landscape , author=. International Conference on Research in Computational Molecular Biology , pages=. 2024 , organization=
2024
-
[16]
Likelihood-based fine-tuning of protein language models for few-shot fitness prediction and design , author=
-
[17]
Nature biotechnology , volume=
Learning protein fitness models from evolutionary and assay-labeled data , author=. Nature biotechnology , volume=. 2022 , publisher=
2022
-
[18]
Protein Language Model Fitness is a Matter of Preference , author=
-
[19]
, author=
Lora: Low-rank adaptation of large language models. , author=. ICLR , volume=
-
[20]
The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=
Protein Inverse Folding From Structure Feedback , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=
-
[21]
arXiv preprint arXiv:2506.00297 , year=
Improving Protein Sequence Design through Designability Preference Optimization , author=. arXiv preprint arXiv:2506.00297 , year=
-
[22]
Advances in Neural Information Processing Systems , volume=
Antigen-specific antibody design via direct energy-based preference optimization , author=. Advances in Neural Information Processing Systems , volume=
-
[23]
arXiv preprint arXiv:2412.20984 , year=
Alignab: Pareto-optimal energy alignment for designing nature-like antibodies , author=. arXiv preprint arXiv:2412.20984 , year=
-
[24]
arXiv preprint arXiv:2512.17815 , year=
Structure-Aware Antibody Design with Affinity-Optimized Inverse Folding , author=. arXiv preprint arXiv:2512.17815 , year=
-
[25]
arXiv preprint arXiv:2510.21161 , year=
RiboPO: Preference Optimization for Structure-and Stability-Aware RNA Design , author=. arXiv preprint arXiv:2510.21161 , year=
-
[26]
Bioinformatics , volume=
Protein--Sol: a web tool for predicting protein solubility from sequence , author=. Bioinformatics , volume=. 2017 , publisher=
2017
-
[27]
Bioinformatics Advances , volume=
G-RANK: an equivariant graph neural network for the scoring of protein--protein docking models , author=. Bioinformatics Advances , volume=. 2023 , publisher=
2023
-
[28]
Khade and Michael Maser and Vladimir Gligorijevic and Andrew Watkins , title =
Pranav M. Khade and Michael Maser and Vladimir Gligorijevic and Andrew Watkins , title =. 2023 , doi =. https://www.biorxiv.org/content/early/2023/06/28/2023.06.26.546331.full.pdf , journal =
2023
-
[29]
BioRxiv , pages=
Enhancing antibody language models with structural information , author=. BioRxiv , pages=. 2024 , publisher=
2024
-
[30]
PLOS Computational Biology , volume=
Large scale paired antibody language models , author=. PLOS Computational Biology , volume=. 2024 , publisher=
2024
-
[31]
BioRxiv , volume=
Language models of protein sequences at the scale of evolution enable accurate structure prediction , author=. BioRxiv , volume=
-
[32]
Bioinformatics , volume=
ProteinBERT: a universal deep-learning model of protein sequence and function , author=. Bioinformatics , volume=. 2022 , publisher=
2022
-
[33]
International Journal of Molecular Sciences , volume=
DeepSTABp: a deep learning approach for the prediction of thermal protein stability , author=. International Journal of Molecular Sciences , volume=. 2023 , publisher=
2023
-
[34]
Synthetic and Systems Biotechnology , volume=
Transfer learning with pre-trained language models for protein expression level prediction in Escherichia coli , author=. Synthetic and Systems Biotechnology , volume=. 2026 , publisher=
2026
-
[35]
Bioinformatics , volume=
NetSolP: predicting protein solubility in Escherichia coli using language models , author=. Bioinformatics , volume=. 2022 , publisher=
2022
-
[36]
Advances in Neural Information Processing Systems , volume=
Proteingym: Large-scale benchmarks for protein fitness prediction and design , author=. Advances in Neural Information Processing Systems , volume=
-
[37]
Developmental & Comparative Immunology , volume=
IMGT unique numbering for immunoglobulin and T cell receptor variable domains and Ig superfamily V-like domains , author=. Developmental & Comparative Immunology , volume=. 2003 , publisher=
2003
-
[38]
Bioinformatics , volume=
ANARCI: antigen receptor numbering and receptor classification , author=. Bioinformatics , volume=. 2016 , publisher=
2016
-
[39]
2009 , publisher=
The ELISA guidebook , author=. 2009 , publisher=
2009
-
[40]
Clinical Chemistry , volume=
The ELISA, enzyme-linked immunosorbent assay , author=. Clinical Chemistry , volume=. 2010 , publisher=
2010
-
[41]
Bioinformatics , pages=
RP3Net: a deep learning model for predicting recombinant protein production in Escherichia coli , author=. Bioinformatics , pages=. 2026 , publisher=
2026
-
[42]
PLoS biology , volume=
A case of convergence: why did a simple alternative to canonical antibodies arise in sharks and camels? , author=. PLoS biology , volume=. 2011 , publisher=
2011
-
[43]
Frontiers of Agricultural Science and Engineering , volume=
A comprehensive analysis of immunoglobulin heavy chain genes in the Bactrian camel (Camelus bactrianus) , author=. Frontiers of Agricultural Science and Engineering , volume=
-
[44]
Frontiers in Immunology , volume=
Genetic removal of the CH1 exon enables the production of heavy chain-only IgG in mice , author=. Frontiers in Immunology , volume=. 2018 , publisher=
2018
Reviewed August 2, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.