Pith. sign in

REVIEW 4 major objections 6 minor 75 references

A Cross Modal Knowledge Distillation & Data Augmentation Recipe for Improving Transcriptomics Representations through Morphological Features

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Freezing a pretrained microscopy encoder and training only a small gene-expression adapter — Semi-Clipped — distills morphology into transcriptomics under weak pairing, and PEA augmentation lifts recall of known biology on unseen datasets.

desk verdict Solid empirical recipe for distilling morphology into transcriptomics, but the load-bearing weak-pairing assumption is only indirectly tested; a permutation-pairing control would settle it. read the letter →

arxiv 2505.21317 v1 pith:4SOTNR5A submitted 2025-05-27 cs.LG cs.AI

Ihab Bendidi , Yassir El Mesbahi , Alisandra K. Denton , Karush Suri , Kian Kenyon-Dean , Auguste Genovesio , Emmanuel Noutahi This is my paper
classification cs.LGcs.AI
keywords cross-modalknowledgedistillationcontrastivelearningtranscriptomicsmicroscopyimagingweaklypaireddataaugmentationbatchcorrectionbiologicalrelationshiprecall
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that scarce, weakly paired data — gene-expression profiles and microscopy images that share a cell line and a drug treatment but come from different biological replicates — is enough to transfer morphological knowledge into transcriptomics. Its method, Semi-Clipped, freezes a large pretrained microscopy encoder and trains only a small adapter on the gene-expression side, using the CLIP contrastive objective to pull expression representations toward image representations. A companion augmentation, PEA, turns standard batch-correction steps (centering, scaling, and typical-variation normalization) into a stochastic source of training variation for gene-expression embeddings. If the claims hold, gene-expression representations become better at recalling known biological relationships on new cell types, new measurement platforms, and even single-cell data, without giving up the gene-level interpretability that makes transcriptomics useful.

What carries the argument

The load-bearing object is the Semi-Clipped objective: the CLIP contrastive loss computed between frozen teacher embeddings $z_T$ (microscopy) and adapted student embeddings $h_S = f_S(z_S)$, where the adapter $f_S$ — a three-layer MLP — is the only trainable component. Training nothing on the image side enforces one-way knowledge transfer and prevents mutual drift between modalities that share little information. The second mechanism is PEA (Perturbation Embedding Augmentation), a stochastic function $A$ drawn from a set of batch-correction transformations — centering, center-scaling, and Typical Variation Normalization (TVN) — applied to student embeddings during training, with a random subset of correction steps dropped and a random subset of control samples used for the correction. This repurposes a standard post-processing cleanup step as a source of controlled training variation, while a fixed TVN correction is applied to the teacher embeddings and to all embeddings at evaluation time.

What would settle it

Replace the random within-group pairing with replicate-matched pairing (image and expression from the same biological replicate) and retrain: if known-relationship recall does not rise, the weak pairing was not what carried the signal. A second check is to compare adapter outputs with the mean embedding of each (cell line, perturbation) batch, since a collapse toward batch-level averages would show that PEA's batch corrections, not biological alignment, explain the gains.

Watch

Extended reading notes

Core claim

The central claim is that a label-free distillation recipe, Semi-Clipped, outperforms both label-supervised distillation methods and other multimodal alignment baselines at enriching transcriptomics representations with morphology, and that the gains survive large distribution shifts. The teacher is the frozen Phenom-1 microscopy encoder; the student is a frozen transcriptomics encoder (scVI, an scVI-like MLP, or scGPT) whose embeddings pass through a three-layer adapter trained with the CLIP loss against the teacher space. Pairing is deliberately weak: each epoch, one transcriptomics sample and one microscopy image are matched at random solely because they share cell line and perturbation. On retrieval of known biological relationships, Semi-Clipped leads on HUVEC-KO and SC-RPE1, matches the best unsupervised methods on LINCS, and stays close to the unimodal baseline on transcriptomic interpretability. PEA alone improves relationship recall over no augmentation by 17% on HUVEC-KO, 55% on LINCS, and 20% on SC-RPE1; stacking it on top of all compared augmentations raises those gains to 25%, 69%, and 26%. The paper further claims that Semi-Clipped, uniquely among the compared methods, retrieves relationships absent from both unimodal modalities, with enriched cell-cycle and post-translational-modification pathways.

Load-bearing premise

The load-bearing premise is that randomly pairing one gene-expression sample with one microscopy image per epoch — matched only by shared cell line and perturbation, not by biological replicate — gives the training enough signal to transfer real morphological knowledge into gene expression; the paper itself concedes that such random pairing may dilute representation quality when subtle differences exist within a treatment group.

Editorial extensions

If this is right

  • Representations produced by the recipe improve known-biological-relationship recall on out-of-distribution data: unseen CRISPR knockouts in HUVEC-KO, the L1000 assay in LINCS, and single-cell transcriptomics in SC-RPE1.
  • Label-free distillation beats label-supervised distillation (KD, SHAKE, C2KD) in this setting, which suggests that weak biological labels are too coarse to serve as the training signal for cross-modal transfer.
  • Because only a small adapter is trained, the recipe is inexpensive: 1.3 million weakly paired samples distill in about 19 hours on a single GPU, so the method can scale as weakly paired multimodal datasets grow.
  • PEA is a drop-in augmentation: it improves KD, SHAKE, and VICReg as well as Semi-Clipped, with statistically significant gains across all three out-of-distribution datasets.
  • The distilled representations recover relationships that neither unimodal modality retrieves on its own, enriched for cell-cycle and post-translational-modification pathways, indicating synergy rather than purely additive transfer.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Extension: the same frozen-teacher, trainable-adapter recipe should apply to other modality pairs, such as proteomics or metabolomics as students of microscopy, whenever weakly paired batches share a biological state.
  • Extension: random pairing sets the method's ceiling, so replacing it with concentration- or time-matched pairing is the most direct untested route to higher recall, and the paper's own Discussion points to within-group dilution as the residual error source.
  • Extension: the emergent cell-cycle enrichment invites a dissection the paper does not perform — ablating only the random control-subset sampling inside PEA, with all other components fixed, would test whether that specific stochasticity exposes the cross-modal synergy.
  • Extension: the benchmarks measure recall of already-annotated relationships, so a prospective test of whether enriched embeddings improve a real drug-discovery decision, such as hit ranking in a screening campaign, remains open.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper proposes and evaluates a recipe for improving transcriptomics representations by distilling knowledge from microscopy images under weakly paired, data-scarce conditions. The method, Semi-Clipped, freezes a pretrained microscopy encoder (Phenom-1) and a transcriptomics encoder (scVI or scGPT) and trains only an adapter with a CLIP-style contrastive loss, using one randomly chosen microscopy image per transcriptomics sample per epoch as a positive pair. The paper also introduces PEA, which stochastically applies batch-correction transformations (centering, scaling, TVN) as data augmentation during adapter training. Evaluations on three OOD transcriptomics datasets (HUVEC-KO, LINCS, SC-RPE1) report improved Known Biological Relationship Recall while largely preserving Transcriptomic Interpretability, with 15-seed averages and Wilcoxon tests in Table 1 and a monotonic PEA-component ablation in Table 2. The authors claim that the approach achieves state-of-the-art cross-modal distillation and that PEA synergizes with existing augmentations.

Significance. If the claims hold, the paper offers a computationally efficient and practical recipe: it transfers morphological information into a unimodal transcriptomics representation, works under data scarcity, and improves OOD relationship recall. The empirical backbone is credible: 15-seed averages with standard deviations, Wilcoxon signed-rank tests, and a monotonic PEA ablation. The main weakness is that the central mechanism—random weak pairing as a carrier of morphology-to-transcriptomics signal—is not directly tested, and some interpretability claims are contradicted by Table 1. With a pairing control and corrected claims, this would be a solid contribution.

major comments (4)
  1. [Section 4, Algorithm 1, Section 6] The core claim that Semi-Clipped transfers morphological knowledge depends on positive pairs in Algorithm 1 carrying information beyond the shared condition label. Since pairs are formed by randomly sampling one microscopy image per transcriptomics sample per epoch based only on cell line, perturbation, and concentration, the method could reduce to aligning treatment/concentration categories. The Discussion (Section 6) concedes that random pairing may dilute representation quality when subtle intra-group differences exist. The null experiments in Figures 6 and 7 permute perturbation labels of an already trained model and therefore do not test whether training-time pair assignments matter. I recommend a control experiment that breaks pairing at training time—e.g., permuting image embeddings across perturbations while keeping all other hyperparameters fixed—and reporting whether relationship recall drops. If it does not drop, the reported gains should be attributed to regularization/distribution alignment rather than morphological distillation.
  2. [Section 5.2, Table 1] The statement that 'PEA consistently enhances performance across all three OOD datasets for every distillation approach' and that it 'significantly improves Transcriptomic Interpretability' is contradicted by Table 1: for SC-RPE1, SHAKE+PEA decreases Tx Preservation from 36.95±0.46 to 36.15±0.51 and VICReg+PEA decreases it from 36.75±0.17 to 36.33±0.22. Please restrict the claim to Known Relationship Recall or define an aggregate metric with a significance test; as written, the interpretability-preservation claim is overbroad.
  3. [Reproducibility] The manuscript does not provide code, trained adapters, or public access to the HUVEC-CMPD training data. Because the method is an empirical recipe with no formal guarantees, the central results cannot be independently reproduced or stress-tested by reviewers. At minimum, release the training and evaluation code, and provide a data-sharing or re-creation plan for the training set.
  4. [Section 5.3, Table 3] The emergent-synergy claim is based on GSEA p-values without multiple-testing correction; Table 3 lists dozens of pathways, several with p-values around 0.03–0.05, and the claim that KD, SHAKE, and VICReg 'fail to significantly enrich any biological pathway' may reflect threshold effects. Report corrected q-values (e.g., BH-FDR) and the number of gene sets tested, or temper the synergy claim.
minor comments (6)
  1. [Figure 2] Add error bars or confidence intervals to the 5-seed results; as shown, the visual comparisons do not convey variance.
  2. [Algorithm 1] The line 'Apply TVN correction to teacher embeddings: zb S = B(zT)' uses the student subscript for a teacher quantity; rename to avoid confusion.
  3. [Section 4] Clarify whether the one randomly selected pair per epoch is sampled with or without replacement within a batch, and whether the 20,000 microscopy images are reused across epochs; this affects interpretation of the weak-pairing signal.
  4. [Figure 4 caption] The word 'Detailled' should be 'Detailed'.
  5. [Related Works] The XKD reference (Sarkar & Etemad) lacks a year and venue; complete the citation.
  6. [Table 1 caption] State that the reported p-values apply only to improvements; the two decreases in Tx Preservation for SC-RPE1 are not covered by the significance claim.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the distillation and augmentation results are not pre-wired to the evaluation metric, and the self-cited benchmark is externally validated.

full rationale

I find no step in the derivation chain that reduces to its own inputs. Semi-Clipped optimizes a standard CLIP contrastive loss (Algorithm 1) between a trainable student adapter output h_S and frozen teacher embeddings z_T; the teacher encoder is fixed before training, and the evaluation metric (Known Biological Relationship Recall) is never used as a training objective, so the reported gains are not fitted by construction. PEA is an input-space stochastic batch-correction augmentation rather than a fit to the benchmark; its components are standard centering, scaling, and TVN. The evaluation framework is taken from the authors' prior work (Bendidi et al., 2024b), but that self-citation is not load-bearing: the benchmark is validated against external databases (CORUM, HuMAP, StringDB, Reactome, SIGNOR) and is applied uniformly to all baselines and to the proposed method, so it does not smuggle in the conclusion. The Discussion's caveat that random pairing within treatment groups may dilute representation quality is a genuine validation risk and would be best addressed by a permutation-pairing control, but it is not a circularity: random weak pairing is an input assumption, not a quantity derived from the evaluation metric. Minor self-citations exist, but none carries the argument by itself; the central claim has independent empirical content.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on empirical evaluation rather than derivation. The method assumes that weak pairing via shared metadata is a valid biological correspondence signal, that frozen pretrained encoders provide suitable embedding spaces, that batch correction preserves perturbation signal, and that the chosen retrieval benchmark reflects biological utility. No new physical or biological entities are introduced. Hyperparameters are tuned by grid search, but the main results are measured on out-of-distribution datasets.

free parameters (6)
  • CLIP temperature tau = 0.1
    Tuned via grid search on a validation split; controls the sharpness of the contrastive distribution in Semi-Clipped (Section 4).
  • Adapter learning rate = 0.001
    Selected by grid search; fixed for Semi-Clipped training (Section 4).
  • Batch size = 1024
    Selected by grid search; determines the number of in-batch contrastive negatives (Section 4).
  • Training epochs = 150
    Selected by grid search; Figure 3 shows more epochs improves recall (Section 4).
  • Adapter architecture dimensions = 256 input, 512 and 1024 hidden, 768 output
    Chosen architecture for the transcriptomics adapter fS (Appendix A.2).
  • PEA step-drop and control-sampling randomization = not specified numerically
    The probability of dropping batch-correction steps and the control-sample count distribution are described qualitatively in Section 3, leaving a replication gap.
assumptions (4)
  • domain assumption Samples sharing cell line, perturbation, and concentration are weakly paired and represent comparable biological states.
    Stated in Sections 1 and 3; this pairing signal defines the positive pairs in the CLIP loss.
  • domain assumption Frozen pretrained encoders provide embedding spaces that are compatible enough for a small MLP adapter to align transcriptomics to microscopy.
    Section 3 and Appendix A.1 assume Phenom-1 and scVI or scGPT latents carry transferable biological structure.
  • domain assumption Batch correction using control samples removes technical variation while preserving perturbation signal.
    PEA is built on this premise; Appendix C defines centering, standardization, and TVN.
  • domain assumption Known relationship recall on top and bottom 5% cosine similarities of aggregated gene embeddings is a valid proxy for biological utility.
    Appendix B.1; this benchmark protocol from Celik et al. and Bendidi et al. shapes all reported conclusions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Cross Modal Knowledge Distillation & Data Augmentation Recipe for Improving Transcriptomics Representations through Morphological Features." pith.science (2026). https://pith.science/paper/4SOTNR5A

@misc{pith2026250521317,
  author       = {Pith},
  title        = {Pith review of: A Cross Modal Knowledge Distillation & Data Augmentation Recipe for Improving Transcriptomics Representations through Morphological Features},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4SOTNR5A}},
  note         = {Machine review of arXiv:2505.21317}
}
read the original abstract

Understanding cellular responses to stimuli is crucial for biological discovery and drug development. Transcriptomics provides interpretable, gene-level insights, while microscopy imaging offers rich predictive features but is harder to interpret. Weakly paired datasets, where samples share biological states, enable multimodal learning but are scarce, limiting their utility for training and multimodal inference. We propose a framework to enhance transcriptomics by distilling knowledge from microscopy images. Using weakly paired data, our method aligns and binds modalities, enriching gene expression representations with morphological information. To address data scarcity, we introduce (1) Semi-Clipped, an adaptation of CLIP for cross-modal distillation using pretrained foundation models, achieving state-of-the-art results, and (2) PEA (Perturbation Embedding Augmentation), a novel augmentation technique that enhances transcriptomics data while preserving inherent biological information. These strategies improve the predictive power and retain the interpretability of transcriptomics, enabling rich unimodal representations for complex biological tasks.

Figures

Figures reproduced from arXiv: 2505.21317 by the authors.

Figure 1
Figure 1. [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Performance comparison of the distillation and augmentation components of our approach compared to existing distillation methods (a) and biological data augmentation techniques (b) across five training seeds. Higher is better for all metrics. Semi-Clipped and PEA maintain interpretability and achieve the highest performance on all OOD datasets. (a) Z-scores of evaluation metrics (relationship recall and Tx preservab… view at source ↗
Figure 3
Figure 3. Ablation study on the known relationship recall score of hyperparameters choices (Tx Adapter learning rate, CLIP loss temperature, batch size, and training epochs) for training Semi-Clipped on the HUVEC-KO dataset, including the selected optimal configuration (dotted vertical line). For each studied parameter, we set all other hyperparameters at their best performing value. While performance varies with parameter ch… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Venn diagrams of retrieved biological relationships for KD, SHAKE, VICReg, and Semi-Clipped (all trained with PEA) on the HUVEC-KO OOD dataset. Semi-Clipped shows the high￾est overlap with transcriptomics while integrating morphological insights, whereas KD and SHAKE e…
Figure 5
Figure 5. Figure 5: Comparison of relationship gains and losses across cross-modal distillation methods shown in [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: Literature-known biological relationships retrieved through the LINCS dataset by the transcriptomics and microscopy imaging unimodal encoders, alongside our proposed Semi-Clipped approach, without data augmentations, and a null distribution through randomization of the…
Figure 7
Figure 7. Figure 7: Literature-known biological relationships retrieved by the transcriptomics and microscopy imaging unimodal encoders, alongside our proposed Semi-Clipped approach (first row), without data augmentations, across different retrieval thresholds (columns) on the HUVEC-KO da…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

75 extracted references · 62 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Rotation-agnostic image representation learning for digital pathology

    Alfasly, S., Shafique, A., Nejat, P., Khan, J., Alsaafin, A., Alabtah, G., and Tizhoosh, H. Rotation-agnostic image representation learning for digital pathology. In CVPR, 2024

    work page 2024

  3. [3]

    M., McLean, C

    Ando, D. M., McLean, C. Y., and Berndl, M. Improving phenotypic measurements in high-content imaging screens. bioRxiv, July 2017

    work page 2017

  4. [4]

    Vicreg: Variance-invariance-covariance regularization for self-supervised learning

    Bardes, A., Ponce, J., and LeCun, Y. Vicreg: Variance-invariance-covariance regularization for self-supervised learning. In ICLR, 2022

    work page 2022

  5. [5]

    No free lunch in self supervised representation learning, 2023

    Bendidi, I., Bardes, A., Cohen, E., Lamiable, A., Bollot, G., and Genovesio, A. No free lunch in self supervised representation learning, 2023

    work page 2023

  6. [6]

    Exploring self-supervised learning biases for microscopy image representation

    Bendidi, I., Bardes, A., Cohen, E., Lamiable, A., Bollot, G., and Genovesio, A. Exploring self-supervised learning biases for microscopy image representation. Biological Imaging, 4, 2024 a . doi:10.1017/S2633903X2400014X

    doi:10.1017/s2633903x2400014x 2024

  7. [7]

    B., Mesbahi, Y

    Bendidi, I., Whitfield, S., Kenyon-Dean, K., Yedder, H. B., Mesbahi, Y. E., Noutahi, E., and Denton, A. K. Benchmarking transcriptomics foundation models for perturbation analysis : one pca still rules them all. In NeurIPS AIDrugX Workshop, 2024 b

    work page 2024

  8. [8]

    ChAda-ViT : Channel Adaptive Attention for Joint Representation Learning of Heterogeneous Microscopy Image

    Bourriez, N., Bendidi, I., Cohen, E., Watkinson, G., Sanchez, M., Bollot, G., and Genovesio, A. ChAda-ViT : Channel Adaptive Attention for Joint Representation Learning of Heterogeneous Microscopy Image . In CVPR, 2024

    work page 2024

Show all 75 references
  1. [9]

    Integrating single-cell transcriptomics with cellular phenotypes: cell morphology, ca2+ imaging and electrophysiology

    Camunas-Soler, J. Integrating single-cell transcriptomics with cellular phenotypes: cell morphology, ca2+ imaging and electrophysiology. Biophys. Rev., 16 0 (1): 0 89--107, February 2024

    2024

  2. [10]

    E., Jones, T

    Carpenter, A. E., Jones, T. R., Lamprecht, M. R., Clarke, C., Kang, I. H., Friman, O., Guertin, D. A., Chang, J. H., Lindquist, R. A., Moffat, J., Golland, P., and Sabatini, D. M. CellProfiler : image analysis software for identifying and quantifying cell phenotypes. Genome Bi...

    2006

  3. [11]

    M., Lazar, N

    Celik, S., Hütter, J.-C., Carlos, S. M., Lazar, N. H., Mohan, R., Tillinghast, C., Biancalani, T., Fay, M. M., Earnshaw, B. A., and Haque, I. S. Building, benchmarking, and exploring perturbative maps of transcriptional and morphological data. PLOS Computational Biology, 20 0 ...

    2024 doi

  4. [12]

    N., Ackerman, J., Alix, E., Ando, D

    Chandrasekaran, S. N., Ackerman, J., Alix, E., Ando, D. M., Arevalo, J., Bennion, M., Boisseau, N., Borowa, A., Boyd, J. D., Brino, L., Byrne, P. J., Ceulemans, H., Ch'ng, C., Cimini, B. A., Clevert, D.-A., Deflaux, N., Doench, J. G., Dorval, T., Doyonnas, R., Dragone, V., Eng...

    2023

  5. [13]

    J., Ding, T., Lu, M

    Chen, R. J., Ding, T., Lu, M. Y., Williamson, D. F. K., Jaume, G., Song, A. H., Chen, B., Zhang, A., Shao, D., Shaban, M., Williams, M., Oldenburg, L., Weishaupt, L. L., Wang, J. J., Vaidya, A., Le, L. P., Gerber, G., Sahai, S., Williams, W., and Mahmood, F. Towards a general-...

    2024

  6. [14]

    W., Gaffney, D

    Conesa, A., Madrigal, P., Tarazona, S., Gomez-Cabrero, D., Cervera, A., McPherson, A., Szcze \'s niak, M. W., Gaffney, D. J., Elo, L. L., Zhang, X., and Mortazavi, A. A survey of best practices for RNA-seq data analysis. Genome Biol., 17 0 (1), December 2016

    2016

  7. [15]

    Diffdock: Diffusion steps, twists, and turns for molecular docking

    Corso, G., Stärk, H., Jing, B., Barzilay, R., and Jaakkola, T. Diffdock: Diffusion steps, twists, and turns for molecular docking. In ICLR, 2023

    2023

  8. [16]

    scGPT : toward building a foundation model for single-cell multi-omics using generative AI

    Cui, H., Wang, C., Maan, H., Pang, K., Luo, F., Duan, N., and Wang, B. scGPT : toward building a foundation model for single-cell multi-omics using generative AI . Nat. Methods, 21 0 (8): 0 1470--1480, August 2024

    2024

  9. [17]

    and Taylor, G

    DeVries, T. and Taylor, G. W. Dataset augmentation in feature space, 2017

    2017

  10. [18]

    Protein complex prediction with AlphaFold-Multimer

    Evans, R., O'Neill, M., Pritzel, A., Antropova, N., Senior, A., Green, T., Z \' dek, A., Bates, R., Blackwell, S., Yim, J., Ronneberger, O., Bodenstein, S., Zielinski, M., Bridgland, A., Potapenko, A., Cowie, A., Tunyasuvunakool, K., Jain, R., Clancy, E., Kohli, P., Jumper, J....

    2021

  11. [19]

    Compressing visual-linguistic model via knowledge distillation

    Fang, Z., Wang, J., Hu, X., Wang, L., Yang, Y., and Liu, Z. Compressing visual-linguistic model via knowledge distillation. In ICCV, 2021

    2021

  12. [20]

    M., Kraus, O., Victors, M., Arumugam, L., Vuggumudi, K., Urbanik, J., Hansen, K., Celik, S., Cernek, N., Jagannathan, G., Christensen, J., Earnshaw, B

    Fay, M. M., Kraus, O., Victors, M., Arumugam, L., Vuggumudi, K., Urbanik, J., Hansen, K., Celik, S., Cernek, N., Jagannathan, G., Christensen, J., Earnshaw, B. A., Haque, I. S., and Mabey, B. Rxrx3: Phenomics map of biology. bioRxiv, 2023. doi:10.1101/2023.02.07.527350

    2023 doi

  13. [21]

    How molecules impact cells: Unlocking contrastive phenomolecular retrieval

    Fradkin, P., Azadi, P., Suri, K., Wenkel, F., Bashashati, A., Sypetkowski, M., and Beaini, D. How molecules impact cells: Unlocking contrastive phenomolecular retrieval. In NeurIPS, 2024

    2024

  14. [22]

    Cross modal distillation for supervision transfer

    Gupta, S., Hoffman, J., and Malik, J. Cross modal distillation for supervision transfer. In CVPR, 2016

    2016

  15. [23]

    J., and Rueckert, D

    Hager, P., Menten, M. J., and Rueckert, D. Best of Both Worlds: Multimodal Contrastive Learning with Tabular and Imaging Data . In CVPR, pp.\ 23924--23935, Los Alamitos, CA, USA, June 2023. IEEE Computer Society. doi:10.1109/CVPR52729.2023.02291. URL https://doi.ieeecomputerso...

    2023

  16. [24]

    P., and Topcu, U

    han Li, P., Chinchali, S. P., and Topcu, U. Csa: Data-efficient mapping of unimodal features to multimodal features, 2024

    2024

  17. [25]

    Distilling the knowledge in a neural network

    Hinton, G., Vinyals, O., and Dean, J. Distilling the knowledge in a neural network. In NeurIPS Deep Learning Workshop, 2015

    2015

  18. [26]

    Knowledge distillation from a stronger teacher

    Huang, T., You, S., Wang, F., Qian, C., and Xu, C. Knowledge distillation from a stronger teacher. In NeurIPS, 2022

    2022

  19. [27]

    C2kd: Bridging the modality gap for cross-modal knowledge distillation

    Huo, F., Xu, W., Guo, J., Wang, H., and Guo, S. C2kd: Bridging the modality gap for cross-modal knowledge distillation. In CVPR, 2024

    2024

  20. [28]

    Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Z \'i dek, A., Potapenko, A., Bridgland, A., Meyer, C., Kohl, S. A. A., Ballard, A. J., Cowie, A., Romera-Paredes, B., Nikolov, S., Jain, R., Adler, J., Back, T., Pete...

    2021

  21. [29]

    J., Urbanik, J., Donhauser, K., Hartford, J., Saberian, S., Sahin, N., Bendidi, I., Celik, S., Fay, M., Vera, J

    Kenyon-Dean, K., Wang, Z. J., Urbanik, J., Donhauser, K., Hartford, J., Saberian, S., Sahin, N., Bendidi, I., Celik, S., Fay, M., Vera, J. S. R., Haque, I. S., and Kraus, O. Vitally consistent: Scaling biological representation learning for cell microscopy. In ICML, 2025

    2025

  22. [30]

    Kharchenko, P. V. The triumphs and limitations of computational methods for scRNA-seq . Nat. Methods, 18 0 (7): 0 723--732, July 2021

    2021

  23. [31]

    Augmentation of transcriptomic data for improved classification of patients with respiratory diseases of viral origin

    Kircher, M., Chludzinski, E., Krepel, J., Saremi, B., Beineke, A., and Jung, K. Augmentation of transcriptomic data for improved classification of patients with respiratory diseases of viral origin. Int. J. Mol. Sci., 23 0 (5): 0 2481, February 2022

    2022

  24. [32]

    V., Morse, K., Makes, M., Mabey, B., and Earnshaw, B

    Kraus, O., Kenyon-Dean, K., Saberian, S., Fallah, M., McLean, P., Leung, J., Sharma, V., Khan, A., Balakrishnan, J., Celik, S., Beaini, D., Sypetkowski, M., Cheng, C. V., Morse, K., Makes, M., Mabey, B., and Earnshaw, B. Masked autoencoders for microscopy are scalable learners...

    2024

  25. [33]

    Lafarge, M. W. and Koelzer, V. H. Rotation invariance and extensive data augmentation: A strategy for the mitosis domain generalization (midog) challenge. In Biomedical Image Registration, Domain Generalisation and Out-of-Distribution Analysis, pp.\ 62--67, Cham, 2022. Springe...

    2022

  26. [34]

    Multimodal emotion recognition using deep generalized canonical correlation analysis with an attention mechanism

    Lan, Y.-T., Liu, W., and Lu, B.-L. Multimodal emotion recognition using deep generalized canonical correlation analysis with an attention mechanism. In IJCNN, 2020. doi:10.1109/IJCNN48605.2020.9207625

    2020

  27. [35]

    Decomposed cross-modal distillation for rgb-based temporal action detection

    Lee, P., Kim, T., Shim, M., Wee, D., and Byun, H. Decomposed cross-modal distillation for rgb-based temporal action detection. In CVPR, 2023

    2023

  28. [36]

    Exploring representation-level augmentation for code search, 2022

    Li, H., Miao, C., Leung, C., Huang, Y., Huang, Y., Zhang, H., and Wang, Y. Exploring representation-level augmentation for code search, 2022

    2022

  29. [37]

    and Zhe, J

    Li, L. and Zhe, J. Shadow knowledge distillation: Bridging offline and online knowledge transfer. In NeurIPS, 2022

    2022

  30. [38]

    Mdwgan-gp: data augmentation for gene expression data based on multiple discriminator WGAN-GP

    Li, R., Wu, J., Li, G., Liu, J., Xuan, J., and Zhu, Q. Mdwgan-gp: data augmentation for gene expression data based on multiple discriminator WGAN-GP . BMC Bioinformatics, 24 0 (1): 0 427, November 2023

    2023

  31. [39]

    Evaluating the utilities of foundation models in single-cell data analysis

    Liu, T., Li, K., Wang, Y., Li, H., and Zhao, H. Evaluating the utilities of foundation models in single-cell data analysis. bioRxiv, September 2023

    2023

  32. [40]

    B., Jordan, M

    Lopez, R., Regier, J., Cole, M. B., Jordan, M. I., and Yosef, N. Deep generative modeling for single-cell transcriptomics. Nature Methods, 15 0 (12): 0 1053--1058, 2018

    2018

  33. [41]

    Integrative analysis methods for spatial transcriptomics

    Lu, S., F \"u rth, D., and Gillis, J. Integrative analysis methods for spatial transcriptomics. Nat. Methods, 18 0 (11): 0 1282--1283, November 2021

    2021

  34. [42]

    D., McMahon, A

    Miao, Z., Humphreys, B. D., McMahon, A. P., and Kim, J. Multi-omics integration in the age of million single-cell data. Nat. Rev. Nephrol., 17 0 (11): 0 710--724, November 2021

    2021

  35. [43]

    You don t need domain-specific data augmentations when scaling self-supervised learning

    Moutakanni, T., Oquab, M., Szafraniec, M., Vakalopoulou, M., and Bojanowski, P. You don t need domain-specific data augmentations when scaling self-supervised learning. In NeurIPS, 2024

    2024

  36. [44]

    Single-cell RNA-seq data augmentation using generative fourier transformer

    Nouri, N. Single-cell RNA-seq data augmentation using generative fourier transformer. Commun. Biol., 8 0 (1): 0 113, January 2025

    2025

  37. [45]

    Relational knowledge distillation

    Park, W., Kim, D., Lu, Y., and Cho, M. Relational knowledge distillation. In CVPR, June 2019

    2019

  38. [46]

    and Plummer, B

    Pham, C. and Plummer, B. A. Enhancing feature diversity boosts channel-adaptive vision transformers. In NeurIPS, 2024

    2024

  39. [47]

    W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I

    Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I. Learning transferable visual models from natural language supervision. In NeurIPS, 2021

    2021

  40. [48]

    M., Saunders, R

    Replogle, J. M., Saunders, R. A., Pogson, A. N., Hussmann, J. A., Lenail, A., Guna, A., Mascibroda, L., Wagner, E. J., Adelman, K., Lithwick-Yanai, G., Iremadze, N., Oberstrass, F., Lipson, D., Bonnar, J. L., Jost, M., Norman, T. M., and Weissman, J. S. Mapping information-ric...

    2022

  41. [49]

    The chemical space project

    Reymond, J.-L. The chemical space project. Acc. Chem. Res., 48 0 (3): 0 722--730, March 2015

    2015

  42. [50]

    S., Krim, H., and Dai, L

    Roheda, S., Riggan, B. S., Krim, H., and Dai, L. Cross-modality distillation: A case for conditional generative adversarial networks. In ICASSP, 2018. doi:10.1109/ICASSP.2018.8462082

    2018

  43. [51]

    R., and Leskovec, J

    Rosen, Y., Roohani, Y., Agarwal, A., Samotor c an, L., Tabula Sapiens Consortium , Quake, S. R., and Leskovec, J. Universal cell embeddings: A foundation model for cell biology. bioRxiv, November 2023

    2023

  44. [52]

    Optimizing Transformations for Contrastive Learning in a Differentiable Framework, pp.\ 96–105

    Ruppli, C., Gori, P., Ardon, R., and Bloch, I. Optimizing Transformations for Contrastive Learning in a Differentiable Framework, pp.\ 96–105. Springer Nature Switzerland, 2022. ISBN 9783031167607. doi:10.1007/978-3-031-16760-7_10. URL http://dx.doi.org/10.1007/978-3-031-16760-7_10

    2022 doi

  45. [53]

    H-optimus-0, 2024

    Saillard, C., Jenatton, R., Llinares-López, F., Mariet, Z., Cahané, D., Durand, E., and Vert, J.-P. H-optimus-0, 2024. URL https://github.com/bioptimus/releases/tree/main/models/h-optimus/v0

    2024

  46. [54]

    Large scale cell painting guided compound selection reveals activity cliffs and functional relationships

    Sanchez, M., Bourriez, N., Bendidi, I., Cohen, E., Svatko, I., Del Nery, E., Tajmouati, H., Bollot, G., Calzone, L., and Genovesio, A. Large scale cell painting guided compound selection reveals activity cliffs and functional relationships. May 2025

    2025

  47. [55]

    CLOOME : contrastive learning unlocks bioimaging databases for queries with chemical structures

    Sanchez-Fernandez, A., Rumetshofer, E., Hochreiter, S., and Klambauer, G. CLOOME : contrastive learning unlocks bioimaging databases for queries with chemical structures. Nat. Commun., 14 0 (1): 0 7339, November 2023

    2023

  48. [56]

    and Etemad, A

    Sarkar, P. and Etemad, A. Xkd: Cross-modal knowledge distillation with domain alignment for video representation learning. Proceedings of the AAAI Conference on Artificial Intelligence, 38. doi:10.1609/aaai.v38i13.29407

    doi

  49. [57]

    K., Mukherjee, S., Ebert, B

    Subramanian, A., Tamayo, P., Mootha, V. K., Mukherjee, S., Ebert, B. L., Gillette, M. A., Paulovich, A., Pomeroy, S. L., Golub, T. R., Lander, E. S., and Mesirov, J. P. Gene set enrichment analysis: A knowledge-based approach for interpreting genome-wide expression profiles. P...

    2005

  50. [58]

    M., Peck, D

    Subramanian, A., Narayan, R., Corsello, S. M., Peck, D. D., Natoli, T. E., Lu, X., Gould, J., Davis, J. F., Tubelli, A. A., Asiedu, J. K., Lahr, D. L., Hirschman, J. E., Liu, Z., Donahue, M., Julian, B., Khan, M., Wadden, D., Smith, I. C., Lam, D., Liberzon, A., Toder, C., et ...

    2017

  51. [59]

    V., Xiao, L., Chopra, A., Chaffin, M

    Theodoris, C. V., Xiao, L., Chopra, A., Chaffin, M. D., Al Sayed, Z. R., Hill, M. C., Mantineo, H., Brydon, E. M., Zeng, Z., Liu, X. S., and Ellinor, P. T. Transfer learning enables predictions in network biology. Nature, 618 0 (7965): 0 616--624, June 2023

    2023

  52. [60]

    H., Wu, Y., Salakhutdinov, R., and Morency, L.-P

    Tsai, Y.-H. H., Wu, Y., Salakhutdinov, R., and Morency, L.-P. Self-supervised learning from a multi-view perspective, 2021. URL https://arxiv.org/abs/2006.05576

    2021 arXiv

  53. [61]

    K., Kunz, J

    Vorontsov, E., Bozkurt, A., Casson, A., Shaikovski, G., Zelechowski, M., Liu, S., Severson, K., Zimmermann, E., Hall, J., Tenenholtz, N., Fusi, N., Mathieu, P., van Eck, A., Lee, D., Viret, J., Robert, E., Wang, Y. K., Kunz, J. D., Lee, M. C. H., Bernhard, J., Godrich, R. A., ...

    2024 arXiv

  54. [62]

    Multimodal adaptive distillation for leveraging unimodal encoders for vision-language tasks, 2022

    Wang, Z., Codella, N., Chen, Y.-C., Zhou, L., Dai, X., Xiao, B., Yang, J., You, H., Chang, K.-W., fu Chang, S., and Yuan, L. Multimodal adaptive distillation for leveraging unimodal encoders for vision-language tasks, 2022. URL https://arxiv.org/abs/2204.10496

    2022 arXiv

  55. [63]

    Weakly supervised cross-modal learning in high-content screening

    Watkinson, G., Cohen, E., Bourriez, N., Bendidi, I., Bollot, G., and Genovesio, A. Weakly supervised cross-modal learning in high-content screening. In ISBI, 2024. doi:10.1109/ISBI56570.2024.10635200

    2024

  56. [64]

    Cell PLM : Pre-training of cell language model beyond single cells

    Wen, H., Tang, W., Dai, X., Ding, J., Jin, W., Xie, Y., and Tang, J. Cell PLM : Pre-training of cell language model beyond single cells. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=BKXvPDekud

    2024

  57. [65]

    T., Bendidi, I., Russell, C., Hodgson, L., Mesbahi, Y

    Wenkel, F., Tu, W., Masschelein, C., Shirzad, H., Eastwood, C., Whitfield, S. T., Bendidi, I., Russell, C., Hodgson, L., Mesbahi, Y. E., Ding, J., Fay, M. M., Earnshaw, B., Noutahi, E., and Denton, A. K. Txpert: Leveraging biochemical relationships for out-of-distribution tran...

    2025

  58. [66]

    S., Wang, X., Chao, H., and Hu, H

    Wu, K., Peng, H., Zhou, Z., Xiao, B., Liu, M., Yuan, L., Xuan, H., Valenzuela, M., Chen, X. S., Wang, X., Chao, H., and Hu, H. Tinyclip: Clip distillation via affinity mimicking and weight inheritance. In ICCV, 2023

    2023

  59. [67]

    Propensity score alignment of unpaired multimodal data

    Xi, J., Osea, J., Xu, Z., and Hartford, J. Propensity score alignment of unpaired multimodal data. In NeurIPS, 2024

    2024

  60. [68]

    W., Perciani, C., MacParland, S., WANG, B., and Bader, G

    Xie, R., Pang, K., Chung, S. W., Perciani, C., MacParland, S., WANG, B., and Bader, G. Spatially resolved gene expression prediction from histology images via bi-modal contrastive learning. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https:...

    2023

  61. [70]

    Clip-kd: An empirical study of clip model distillation

    Yang, C., An, Z., Huang, L., Bi, J., Yu, X., Yang, H., Diao, B., and Xu, Y. Clip-kd: An empirical study of clip model distillation. In CVPR, 2024

    2024

  62. [71]

    scBERT as a large-scale pretrained deep language model for cell type annotation of single-cell RNA-seq data

    Yang, F., Wang, W., Wang, F., Fang, Y., Tang, D., Huang, J., Lu, H., and Yao, J. scBERT as a large-scale pretrained deep language model for cell type annotation of single-cell RNA-seq data. Nat. Mach. Intell., 4 0 (10): 0 852--866, September 2022

    2022

  63. [72]

    Weakly supervised set-consistency learning improves morphological profiling of single-cell images

    Yao, H., Hanslovsky, P., Huetter, J.-C., Hoeckendorf, B., and Richmond, D. Weakly supervised set-consistency learning improves morphological profiling of single-cell images. In CVPR Workshops, 2024

    2024

  64. [73]

    J., Neri, M., Yang, C., Kulkarni, T., Randhawa, R., Henault, M., Mostacci, N., Farmer, P., Renner, S., Ihry, R., Mansur, L., Keller, C

    Ye, C., Ho, D. J., Neri, M., Yang, C., Kulkarni, T., Randhawa, R., Henault, M., Mostacci, N., Farmer, P., Renner, S., Ihry, R., Mansur, L., Keller, C. G., McAllister, G., Hild, M., Jenkins, J., and Kaykas, A. DRUG-seq for miniaturized high-throughput transcriptome profiling in...

    2018

  65. [74]

    Sigmoid loss for language image pre-training

    Zhai, X., Mustafa, B., Kolesnikov, A., and Beyer, L. Sigmoid loss for language image pre-training. In ICCV, 2023

    2023

  66. [75]

    Zhou, Y., Badgery, H., Read, M., Bailey, J., and Davey, C. E. Dda: Dimensionality driven augmentation search for contrastive learning in laparoscopic surgery, 2024. URL https://arxiv.org/abs/2406.00907

    2024 arXiv

  67. [76]

    and Wang, Y

    Zhu, Y. and Wang, Y. Student customized knowledge distillation: Bridging the gap between student and teacher. In ICCV, 2021. doi:10.1109/ICCV48922.2021.00501

    2021

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.