Pith. sign in

REVIEW 4 major objections 5 minor 55 references

Towards Applying Large Language Models to Complement Single-Cell Foundation Models

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Large language models complement single-cell foundation models: text encoders carry marker-gene and rank-order signal, and fusing scGPT with Ember-V1 gives more consistent cell type classification.

desk verdict Useful empirical study of LLM embeddings for single-cell, but the central complementarity claim is undercut by a capacity confound in the fusion head versus unimodal baselines. read the letter →

arxiv 2507.10039 v1 pith:6223WH7X submitted 2025-07-14 cs.LG q-bio.GN

Steven Palayew , Bo Wang , Gary Bader This is my paper
classification cs.LGq-bio.GN
keywords single-cellfoundationmodelslargelanguagecelltypeclassificationsentencestextembeddingsmultimodalfusionmarkergenesinterpretability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that large language models, even frozen text encoders fed only simple “cell sentences,” contribute real biological signal to single-cell analysis, and that this signal can be combined with single-cell foundation models rather than replace them. The core claim is that text encoders leverage marker-gene knowledge and basic gene-expression ordering, which are complementary to scGPT’s learned representations. The paper introduces scMPT, a representation-level fusion of scGPT and the Ember-V1 text encoder, and reports stronger, more consistent cell type classification and disease phenotype prediction than either component alone. It also reports that reasoning-focused generative LLMs, when asked to arbitrate among scGPT’s top three candidate cell types, improve over scGPT alone on tested datasets.

What carries the argument

The cell sentence is a text sequence of gene names ranked by expression, which converts single-cell data into a form an LLM can encode while preserving essential information through gene identity and rank order. The ablation logic—replacing gene names with truncated SHA-256 hashes and shuffling order in stages—isolates how much of the model’s signal comes from semantics versus lexical and sequential similarity. scMPT is a representation-level fusion: frozen scGPT and Ember-V1 embeddings each pass through a dense ReLU layer, the outputs are concatenated, and a softmax output layer predicts the cell type; keeping both encoders frozen makes the architecture a probe for whether the two representation families actually complement each other.

What would settle it

Re-run the scMPT comparison on Aorta with Ember-V1 selected using only a development set that excludes Aorta (or using all other datasets); if the fusion advantage over the better component shrinks to noise, the reported complementarity on Aorta is not independent evidence.

Watch

Extended reading notes

Core claim

The paper’s central claim is that off-the-shelf text encoders applied to cell sentences—gene names written in descending expression order—produce representations with genuine biological content, and that this content is largely marker-gene knowledge plus simple expression-order statistics rather than deep regulatory understanding. This is supported by interpretability attributions (integrated gradients and LIME) that place known marker genes among the top features, by ablation results in which hashing gene names causes only a moderate drop while shuffling all genes or hashing each instance destroys performance, and by marker-gene embedding similarity statistics. The paper further claims that these text representations are complementary to scGPT: scMPT, a frozen-encoder fusion with a small trainable head, outperforms or matches the better component on cell type classification across seven datasets and on aneurysm phenotype prediction on Aorta, and generative reasoning models asked to arbitrate scGPT’s top three cell types outperform scGPT alone on the tested subsets.

Load-bearing premise

The evaluation assumes that judging candidate text encoders on the Aorta dataset before testing the fused model on that same dataset does not inflate the reported Aorta gains.

Editorial extensions

If this is right

  • If frozen text encoders carry marker-gene signal, cell type annotation can be bootstrapped from text alone on datasets where marker genes cleanly separate types, without retraining a single-cell foundation model.
  • Representation-level fusion such as scMPT is a cheap way to upgrade an existing single-cell foundation model: gains come from the complementary signal, not from added model capacity, because the encoders are frozen.
  • Because most of the encoder’s information is in the top 10% of in-context genes, shorter cell sentences may be sufficient for LLM-based annotation, lowering cost and input-length requirements.
  • Reasoning models can act as an arbitration layer over scGPT’s top-three predictions, and their advantage over standard LLMs suggests reasoning ability is a relevant axis for cell type classification.
  • The same fusion approach transfers to disease phenotype prediction, not only cell type classification.
  • The paper states its fusion evaluation is limited in scope and that only cell sentences were tested; a direct extension would evaluate scMPT on additional datasets and alternative textual representations while fixing the text encoder selection before benchmark evaluation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if marker-gene and rank-order signal are the mechanism, the fusion gain on a new dataset should be predictable from how cleanly marker genes separate cell types in the text-embedding space; ranking datasets by that measure and testing scMPT would check the mechanism.
  • Editorial inference: the near-total collapse under per-instance gene-name hashing suggests a portion of the gain is shallow lexical similarity, not semantics; a direct test would train the same MLP head on frequency-matched random gene tokens to measure how much semantic knowledge is actually required.
  • Editorial inference: because Ember-V1 was selected using Aorta performance, the cleanest confirmation of complementarity would be to freeze the encoder choice on a separate development set and then evaluate scMPT on Aorta; the paper’s architecture makes this a straightforward rerun.
  • Editorial inference: the paper’s own stated limitations—limited fusion-evaluation scope and a single textual representation—leave open whether the complementarity generalizes beyond cell sentences and beyond the datasets tested; expanding the benchmark suite would settle that directly.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies whether off-the-shelf LLM text encoders applied to "cell sentences" capture useful biological signal, and whether that signal can complement the single-cell foundation model scGPT. Using integrated gradients and LIME on a frozen Ember-V1 encoder with an MLP head, plus ablations that hash gene names or shuffle their order, the authors argue that Ember-V1 leverages marker-gene knowledge and simple expression patterns/lexical similarity. They introduce scMPT, a representation-level fusion of frozen scGPT and Ember-V1 embeddings through two 4096-unit dense layers, and report cell-type classification on seven datasets plus aneurysm phenotype prediction on Aorta, generally matching or exceeding unimodal baselines. They also propose a generative pipeline in which reasoning LLMs select among scGPT's top-3 KNN predictions, and report gains over scGPT top-1 on 100-cell subsets of Pancreas, Myeloid, and MS.

Significance. If the capacity confound is resolved, the paper would make a useful contribution: it provides one of the first systematic interpretations of why LLM text encoders work for cell sentences, and it demonstrates a concrete way to combine frozen text and expression embeddings without retraining the foundation models. The use of an external marker database (PanglaoDB) is a strength, as is the breadth of ablation conditions and the reporting of multiple metrics. The main scientific message—that text and expression are complementary modalities—is plausible and worth testing; the current evidence, however, is weakened by a classifier-capacity confound and by selection/validation overlap on Aorta, so the significance of the central claim is conditional on additional experiments.

major comments (4)
  1. [§3.2 and Appendix A.2] The claim that scMPT's architecture is simple enough to attribute gains to representation synergy is not supported, because the scMPT head uses dense layers with output dimension 4096 while unimodal baselines use sklearn's default MLP with a single 100-unit hidden layer. This comparison conflates modality complementarity with classifier capacity. Please add controlled baselines: (a) an MLP with the same 4096-4096 architecture trained on each encoder alone, and (b) an ensemble or concatenation of the two unimodal heads without representation-level fusion, matched in parameter count. If the equal-capacity unimodal head or the ensemble reproduces scMPT's performance, the central conclusion about complementarity would need to be substantially revised.
  2. [Appendix A.1 and Tables 6, 14, 20] Ember-V1 was selected based on its Aorta classification performance (Table 6), and Aorta is then used both as a main scMPT evaluation dataset (Table 14) and for the disease-phenotype experiment (Table 20). This makes the Aorta results non-independent evidence for Ember-V1's advantage. Please either select the encoder on a held-out dataset or using a selection criterion that does not involve Aorta, and then evaluate on Aorta, or report Aorta results as post-selection and re-run the main comparison on datasets not used for model selection.
  3. [Abstract and Section 4.2] The statement that scMPT shows "stronger, more consistent performance than either of its component models" is not supported by the reported tables. On Pancreas (Table 3), Ember-V1+MLP has higher accuracy (0.974 vs. 0.962), while on MS (Table 19), scGPT+MLP has higher accuracy (0.845 vs. 0.837) and higher F1 (0.726 vs. 0.704). scMPT is therefore not consistently stronger than the better unimodal baseline. Please either qualify the claim (e.g., "competitive or stronger on most datasets, with the largest gains on Bones and Myeloid") or provide paired statistical comparisons across datasets to support the consistency claim.
  4. [Section 4.1 and Table 5] The marker-gene evidence is suggestive but not quantitatively strong. The intra-vs-inter cosine similarity differences are small (0.644 vs. 0.623 and 0.667 vs. 0.653), no variance or significance testing is reported, and the interpretability lists in Table 1 and Table 30 are based on only ten cells per cell type. Please add bootstrap confidence intervals or permutation tests for Table 5, and report whether the marker-gene enrichment in the top-10 attribution lists is significant relative to a random-gene baseline. Without this, the discussion's claim that the results "strongly suggest" marker-gene knowledge is overstated.
minor comments (5)
  1. [Tables 13 and 17] "Tabula Sapeins" and "Tsapeins" are misspellings of "Tabula Sapiens"; also "classificaion" in Section 4.2 and "USed" in Figure 6 should be corrected.
  2. [Figures 2 and 3] The captions state that "scMPT outperforms scGPT on most datasets tested," but on MS, scGPT+MLP has higher accuracy and F1 than scMPT; please qualify the caption or plot the comparison so that the MS exception is visible.
  3. [Table 2 and Appendix A.4] The "Gene Name Per-Instance Ablation" column is described in the text as replacing each gene name with a random unique hexadecimal string per instance, but the table itself does not define this; please add a footnote to the table clarifying the difference from the standard gene-name ablation.
  4. [Appendix A.2] The grid search over learning rate, epochs, batch size, and decay rate is mentioned, but the final hyperparameter values for each dataset are not reported; please include them for reproducibility.
  5. [Section 4.2 and Appendix A.3] The generative fusion experiment evaluates only 100 cells per dataset, and the scGPT baseline is reported as a single number rather than a mean over repeated subsamples; please state whether the 100 cells are fixed or randomly resampled and provide variance estimates for the baseline.

Circularity Check

1 steps flagged · score 2.0 of 10

No derivation-level circularity: LLM marker-gene findings use external PanglaoDB and held-out benchmarks; scMPT is an empirical fusion result. Mild benchmark-selection leak on Aorta (encoder chosen on Aorta, then Aorta used in Tables 14/20) warrants a 2, not a higher score.

  1. fitted input called prediction [Appendix A.1, Table 6; Section 4.2, Tables 14 and 20]
    "We first compare the cell type classification and clustering performance of all encoders of potential interest on the Aorta dataset [29]. ... Ember-V1 performed particularly well, outperforming both of these encoders by a wide margin on this dataset. ... Ultimately, we find that Ember-V1 outperforms previously used text embedding models in generating cell embeddings from cell sentences, motivating us to select this encoder for our main experiments."

    The text encoder used in scMPT was selected by ranking encoders on the Aorta benchmark (Table 6). The same benchmark is then presented as evidence for scMPT's cell-type classification gain (Table 14) and disease-phenotype gain (Table 20). The Aorta results are therefore not independent of the model-selection procedure: the component encoder was chosen because it performed well on this exact benchmark. This is a selection-on-validation leak rather than a mathematical reduction, and it does not affect the six other datasets, so it is a mild circular-evaluation issue, not a forced prediction.

full rationale

The paper's two central claims are empirical rather than derivational. Claim (1) - LLMs capture marker-gene knowledge and simple expression patterns - is supported by integrated gradients and LIME attributions checked against the external PanglaoDB marker database, plus ablations (hash replacement, order shuffling) whose outcomes are measured on held-out test splits. No target quantity is defined in terms of the conclusion, and the cosine-similarity marker test uses frozen Ember-V1 embeddings and an external marker list. Claim (2) - scMPT fusion improves over scGPT and Ember-V1 - is a new architecture evaluated with train/test splits inherited from GenePT and CELLama; no fitted parameter is renamed as a prediction, and fusion success is not forced by construction. The main caveat is the Aorta selection leak described above: Ember-V1 was chosen after benchmarking on Aorta, and Aorta later appears in the main scMPT table and in the disease-phenotype experiment, so those particular rows are not independent. The skeptic's capacity concern (scMPT's 4096-unit dense layers vs. sklearn's default 100-unit MLP) is a genuine validity threat to the 'complementarity' interpretation, but it is not circularity: the paper does not define fusion performance as head capacity, and the comparison could be fixed by controlled capacity-matched baselines. Self-citations to scGPT [15] and the annotation tutorial [13] are from co-authors but are external published methods/guidelines, not a uniqueness theorem or an unverified premise that forbids alternatives. Overall the derivation chain is self-contained against external benchmarks; score 2 reflects the mild selection-on-Aorta issue, not derivation-level circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical or conceptual entities. Its central claims rest on a small number of selected hyperparameters and on domain assumptions about cell sentences, marker databases, and the semantic content of pretrained embeddings. No entity is invented beyond the scMPT architecture itself, which is a composite of existing encoders.

free parameters (4)
  • scMPT hyperparameters = not reported; chosen per dataset by grid search
    Initial learning rate, epochs, batch size, and decay rate are selected per dataset on a validation split, which means part of the reported test gains include tuning choices.
  • Dense layer output dimension = 4096
    The hidden dimension of the dense layers in scMPT is fixed at 4096 without a reported sensitivity analysis.
  • KNN k for scGPT and text encoders = 10
    The zero-shot classification baseline uses 10-nearest neighbours, following GenePT. The choice affects both the text encoder baselines and the scGPT candidate generation.
  • Generative fusion candidate list size = 3
    The LLM+scGPT pipeline restricts the reasoning model to choose among scGPT's top three cell types. This hand-picked number shapes the fusion accuracy.
assumptions (5)
  • domain assumption Cell sentence representation is sufficient for the downstream tasks
    The method relies on Cell2Sentence's claim that ranking genes by expression loses minimal information, cited in Section 3.1.
  • domain assumption PanglaoDB marker gene annotations are accurate for the cell types studied
    Used to judge whether top attribution genes are biological markers in Tables 1 and 30.
  • domain assumption Pretrained LLM embeddings encode useful biological semantics for cell sentences
    The entire approach assumes off-the-shelf text encoders generate meaningful cell representations without fine-tuning.
  • domain assumption Cosine similarity between gene-name embeddings measures biological relatedness
    Used in Table 5 to argue that same-type marker genes have higher embedding similarity than different-type markers.
  • domain assumption scGPT weights and evaluation setup from prior work are correct
    scGPT is treated as a fixed component and its reported baseline numbers are taken as given in several tables.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Applying Large Language Models to Complement Single-Cell Foundation Models." pith.science (2026). https://pith.science/paper/6223WH7X

@misc{pith2026250710039,
  author       = {Pith},
  title        = {Pith review of: Towards Applying Large Language Models to Complement Single-Cell Foundation Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6223WH7X}},
  note         = {Machine review of arXiv:2507.10039}
}
read the original abstract

Single-cell foundation models such as scGPT represent a significant advancement in single-cell omics, with an ability to achieve state-of-the-art performance on various downstream biological tasks. However, these models are inherently limited in that a vast amount of information in biology exists as text, which they are unable to leverage. There have therefore been several recent works that propose the use of LLMs as an alternative to single-cell foundation models, achieving competitive results. However, there is little understanding of what factors drive this performance, along with a strong focus on using LLMs as an alternative, rather than complementary approach to single-cell foundation models. In this study, we therefore investigate what biological insights contribute toward the performance of LLMs when applied to single-cell data, and introduce scMPT; a model which leverages synergies between scGPT, and single-cell representations from LLMs that capture these insights. scMPT demonstrates stronger, more consistent performance than either of its component models, which frequently have large performance gaps between each other across datasets. We also experiment with alternate fusion methods, demonstrating the potential of combining specialized reasoning models with scGPT to improve performance. This study ultimately showcases the potential for LLMs to complement single-cell foundation models and drive improvements in single-cell analysis.

Figures

Figures reproduced from arXiv: 2507.10039 by the authors.

Figure 1
Figure 1. The scMPT model architecture. Cell embeddings from scGPT and a text encoder are fed [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Comparison between cell type classification accuracy of scMPT and scGPT on all datasets [PITH_FULL_IMAGE:figures/full_fig_p017_2.png] view at source ↗
Figure 3
Figure 3. Comparison between cell type classification F1 score of scMPT and scGPT on all datasets [PITH_FULL_IMAGE:figures/full_fig_p018_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Prompt used to classify cell type using generative LLMs. [PITH_FULL_IMAGE:figures/full_fig_p019_4.png]
Figure 5
Figure 5. Figure 5: Prompt used to enable generative LLMs to complement scGPT. [PITH_FULL_IMAGE:figures/full_fig_p020_5.png]
Figure 6
Figure 6. Figure 6: Accuracy (Top), and F1 Score (Bottom) of Ember-V1 on Aorta Dataset with different cell [PITH_FULL_IMAGE:figures/full_fig_p022_6.png]
Figure 7
Figure 7. Figure 7: Accuracy (Top), and F1 Score (Bottom) of Ember-V1 on Bones Dataset with different cell [PITH_FULL_IMAGE:figures/full_fig_p023_7.png]
Figure 8
Figure 8. Figure 8: Accuracy (Top), and F1 Score (Bottom) of Ember-V1 on Pancreas Dataset with different [PITH_FULL_IMAGE:figures/full_fig_p024_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

55 extracted references · 35 canonical work pages

  1. [1]

    https://huggingface

    dunzhang/stella_en_400M_v5 · Hugging Face — huggingface.co. https://huggingface. co/dunzhang/stella_en_400M_v5. [Accessed 02-10-2024]

    work page 2024

  2. [2]

    https: //huggingface.co/sentence-transformers/all-MiniLM-L12-v2

    sentence-transformers/all-MiniLM-L12-v2 · Hugging Face — huggingface.co. https: //huggingface.co/sentence-transformers/all-MiniLM-L12-v2 . [Accessed 02-10- 2024]

    work page 2024

  3. [3]

    Deep learning using rectified linear units (relu)

    AF Agarap. Deep learning using rectified linear units (relu). arXiv preprint arXiv:1803.08375, 2018

    arXiv 2018

  4. [4]

    Deepseek ai / deepseek r1 (fast)

    Fireworks AI. Deepseek ai / deepseek r1 (fast). https://fireworks.ai/models/ fireworks/deepseek-r1, 1/20/2025. [Accessed 05-16-2025]

    work page 2025

  5. [5]

    Deepseek ai / deepseek v3

    Fireworks AI. Deepseek ai / deepseek v3. https://fireworks.ai/models/fireworks/ deepseek-v3, 12/30/2024. [Accessed 05-16-2025]

    work page 2024

  6. [6]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems , 35:23716–23736, 2022

    2022

  7. [7]

    Decoding the transcriptome of calcified atherosclerotic plaque at single-cell resolution

    Tom Alsaigh, Doug Evans, David Frankel, and Ali Torkamani. Decoding the transcriptome of calcified atherosclerotic plaque at single-cell resolution. Communications biology, 5(1):1084, 2022

    work page 2022

  8. [8]

    Genept: A simple but effective foundation model for genes and cells built from chatgpt

    Yiqun Chen and James Zou. Genept: A simple but effective foundation model for genes and cells built from chatgpt. bioRxiv, 2023

    work page 2023

Show all 55 references
  1. [9]

    A pan-cancer single-cell transcriptional atlas of tumor infiltrating myeloid cells

    Sijin Cheng, Ziyi Li, Ranran Gao, Baocai Xing, Yunong Gao, Yu Yang, Shishang Qin, Lei Zhang, Hanqiang Ouyang, Peng Du, et al. A pan-cancer single-cell transcriptional atlas of tumor infiltrating myeloid cells. Cell, 184(3):792–809, 2021

    2021

  2. [10]

    Cellama: Foundation model for single cell and spatial transcriptomics by cell embedding leveraging language model abilities

    Hongyoon Choi, Jeongbin Park, Sumin Kim, Jiwon Kim, Dongjoo Lee, Sungwoo Bae, Haenara Shin, and Daeseung Lee. Cellama: Foundation model for single cell and spatial transcriptomics by cell embedding leveraging language model abilities. bioRxiv, pages 2024–05, 2024

    2024

  3. [11]

    François Chollet et al. Keras. https://keras.io, 2015

    2015

  4. [12]

    Synovial cell cross-talk with cartilage plays a major role in the pathogenesis of osteoarthritis

    Ching-Heng Chou, Vaibhav Jain, Jason Gibson, David E Attarian, Collin A Haraden, Christo- pher B Yohn, Remi-Martin Laberge, Simon Gregory, and Virginia B Kraus. Synovial cell cross-talk with cartilage plays a major role in the pathogenesis of osteoarthritis. Scientific Reports...

    2020

  5. [13]

    Tutorial: guidelines for annotating single-cell transcriptomic maps using automated and manual methods

    Zoe A Clarke, Tallulah S Andrews, Jawairia Atif, Delaram Pouyabahar, Brendan T Innes, Sonya A MacParland, and Gary D Bader. Tutorial: guidelines for annotating single-cell transcriptomic maps using automated and manual methods. Nature protocols, 16(6):2749–2764, 2021

    2021

  6. [14]

    The tabula sapiens: A multiple-organ, single-cell transcriptomic atlas of humans

    The Tabula Sapiens Consortium*, Robert C Jones, Jim Karkanias, Mark A Krasnow, An- gela Oliveira Pisco, Stephen R Quake, Julia Salzman, Nir Yosef, Bryan Bulthaup, Phillip Brown, et al. The tabula sapiens: A multiple-organ, single-cell transcriptomic atlas of humans. Science, 3...

    2022

  7. [15]

    scgpt: toward building a foundation model for single-cell multi-omics using generative ai

    Haotian Cui, Chloe Wang, Hassaan Maan, Kuan Pang, Fengning Luo, Nan Duan, and Bo Wang. scgpt: toward building a foundation model for single-cell multi-omics using generative ai. Nature Methods, pages 1–11, 2024. 10

    2024

  8. [16]

    Deepseek-r1

    DeepSeek. Deepseek-r1. https://huggingface.co/deepseek-ai/DeepSeek-R1 . [Ac- cessed 05-16-2025]

    2025

  9. [17]

    The temperature parameter

    DeepSeek. The temperature parameter. https://api-docs.deepseek.com/quick_start/ parameter_settings. [Accessed 05-16-2025]

    2025

  10. [18]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  11. [19]

    How do large language models understand genes and cells

    Chen Fang, Yidong Wang, Yunze Song, Qingqing Long, Wang Lu, Linghui Chen, Pengfei Wang, Guihai Feng, Yuanchun Zhou, and Xin Li. How do large language models understand genes and cells. bioRxiv, pages 2024–03, 2024

    2024

  12. [20]

    Panglaodb: a web server for explo- ration of mouse and human single-cell rna sequencing data

    Oscar Franzén, Li-Ming Gan, and Johan LM Björkegren. Panglaodb: a web server for explo- ration of mouse and human single-cell rna sequencing data. Database, 2019:baz046, 2019

    2019

  13. [21]

    Bias and fairness in large language models: A survey

    Isabel O Gallegos, Ryan A Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K Ahmed. Bias and fairness in large language models: A survey. Computational Linguistics, pages 1–79, 2024

    2024

  14. [22]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  15. [23]

    Adam: A method for stochastic optimization

    Diederik P Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  16. [24]

    A multimodal deep learning model using text, image, and code data for improving issue classification tasks

    Changwon Kwak, Pilsu Jung, and Seonah Lee. A multimodal deep learning model using text, image, and code data for improving issue classification tasks. Applied Sciences, 13(16):9456, 2023

    2023

  17. [25]

    Explanatory predictive model for covid-19 severity risk employing machine learning, shapley addition, and lime

    Mariam Laatifi, Samira Douzi, Hind Ezzine, Chadia El Asry, Abdellah Naya, Abdelaziz Bouklouze, Younes Zaid, and Mariam Naciri. Explanatory predictive model for covid-19 severity risk employing machine learning, shapley addition, and lime. Scientific Reports, 13(1):5481, 2023

    2023

  18. [26]

    Open source strikes bread - new fluffy embeddings model, 2024

    Sean Lee, Aamir Shakir, Darius Koenig, and Julius Lipp. Open source strikes bread - new fluffy embeddings model, 2024

    2024

  19. [27]

    Cell2sentence: Teaching large language models the language of biology

    Daniel Levine, Sacha Lévy, Syed Asad Rizvi, Nazreen Pallikkavaliyaveetil, Xingyu Chen, David Zhang, Sina Ghadermarzi, Ruiming Wu, Zihe Zheng, Ivan Vrkic, et al. Cell2sentence: Teaching large language models the language of biology. bioRxiv, pages 2023–09, 2023

    2023

  20. [28]

    Angle-optimized text embeddings

    Xianming Li and Jing Li. Angle-optimized text embeddings. arXiv preprint arXiv:2309.12871, 2023

    2023 arXiv

  21. [29]

    Single-cell transcriptome analysis reveals dynamic cell populations and differential gene expression patterns in control and aneurysmal human aortic tissue

    Yanming Li, Pingping Ren, Ashley Dawson, Hernan G Vasquez, Waleed Ageedi, Chen Zhang, Wei Luo, Rui Chen, Yumei Li, Sangbae Kim, et al. Single-cell transcriptome analysis reveals dynamic cell populations and differential gene expression patterns in control and aneurysmal human ...

    2020

  22. [30]

    Towards general text embeddings with multi-stage contrastive learning

    Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. Towards general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308.03281, 2023

    2023 arXiv

  23. [31]

    Do language models know the way to rome? arXiv preprint arXiv:2109.07971, 2021

    Bastien Liétard, Mostafa Abdou, and Anders Søgaard. Do language models know the way to rome? arXiv preprint arXiv:2109.07971, 2021

    2021 arXiv

  24. [32]

    scelmo: Embeddings from language models are good learners for single-cell data analysis

    Tianyu Liu, Tianqi Chen, Wangjie Zheng, Xiao Luo, and Hongyu Zhao. scelmo: Embeddings from language models are good learners for single-cell data analysis. bioRxiv, pages 2023–12, 2023

    2023

  25. [33]

    Decoupled weight decay regularization

    I Loshchilov. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 11

    2017 arXiv

  26. [34]

    Benchmarking atlas-level data integration in single-cell genomics.Nature methods, 19(1):41–50, 2022

    Malte D Luecken, Maren Büttner, Kridsadakorn Chaichoompu, Anna Danese, Marta Interlandi, Michaela F Müller, Daniel C Strobl, Luke Zappia, Martin Dugas, Maria Colomé-Tatché, et al. Benchmarking atlas-level data integration in single-cell genomics.Nature methods, 19(1):41–50, 2022

    2022

  27. [35]

    Cellxgene: a performant, scalable exploration platform for high dimensional sparse matrices

    Colin Megill, Bruce Martin, Charlotte Weaver, Sidney Bell, Lia Prins, Seve Badajoz, Brian McCandless, Angela Oliveira Pisco, Marcus Kinsella, Fiona Griffin, et al. Cellxgene: a performant, scalable exploration platform for high dimensional sparse matrices. bioRxiv, pages 2021–04, 2021

    2021

  28. [36]

    Multi-modal classification using images and text

    Stuart J Miller, Justin Howard, Paul Adams, Mel Schwan, and Robert Slater. Multi-modal classification using images and text. SMU Data Science Review, 3(3):6, 2020

    2020

  29. [37]

    Mteb: Massive text embedding benchmark

    Niklas Muennighoff, Nouamane Tazi, Loïc Magne, and Nils Reimers. Mteb: Massive text embedding benchmark. arXiv preprint arXiv:2210.07316, 2022

    2022 arXiv

  30. [38]

    ember-v1: Sota embedding model, 2023

    Enrike Nur and Anar Aliyev. ember-v1: Sota embedding model, 2023

    2023

  31. [39]

    Embeddings

    OpenAI. Embeddings. https://platform.openai.com/docs/guides/embeddings. [Ac- cessed 02-10-2024]

    2024

  32. [40]

    Reasoning models

    OpenAI. Reasoning models. https://platform.openai.com/docs/guides/reasoning? api-mode=responses. [Accessed 05-15-2025]

    2025

  33. [41]

    Pedregosa, G

    F. Pedregosa, G. Varoquaux, A. Gramfort, V . Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V . Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learni...

    2011

  34. [42]

    Replicate

    Replicate. Replicate. https://replicate.com/home. [Accessed 05-16-2025]

    2025

  35. [43]

    why should i trust you?

    Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. " why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144, 2016

    2016

  36. [44]

    Scaling large language models for next-generation single-cell analysis

    Syed Asad Rizvi, Daniel Levine, Aakash Patel, Shiyang Zhang, Eric Wang, Sizhuang He, David Zhang, Cerise Tang, Zhuoyang Lyu, Rayyan Darji, et al. Scaling large language models for next-generation single-cell analysis. bioRxiv, pages 2025–04, 2025

    2025

  37. [45]

    Neuronal vulnerability and multilineage diversity in multiple sclerosis

    Lucas Schirmer, Dmitry Velmeshev, Staffan Holmqvist, Max Kaufmann, Sebastian Werneburg, Diane Jung, Stephanie Vistnes, John H Stockley, Adam Young, Maike Steindel, et al. Neuronal vulnerability and multilineage diversity in multiple sclerosis. Nature, 573(7772):75–82, 2019

    2019

  38. [46]

    Solatorio

    Aivin V . Solatorio. Gistembed: Guided in-sample selection of training negatives for text embedding fine-tuning. arXiv preprint arXiv:2402.16829, 2024

    2024 arXiv

  39. [47]

    Multimodal data fusion for cancer biomarker discovery with deep learning

    Sandra Steyaert, Marija Pizurica, Divya Nagaraj, Priya Khandelwal, Tina Hernandez-Boussard, Andrew J Gentles, and Olivier Gevaert. Multimodal data fusion for cancer biomarker discovery with deep learning. Nature machine intelligence, 5(4):351–362, 2023

    2023

  40. [48]

    Axiomatic attribution for deep networks

    Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International conference on machine learning, pages 3319–3328. PMLR, 2017

    2017

  41. [49]

    Exploring the performance and explainability of fine-tuned bert models for neuro- radiology protocol assignment

    Salmonn Talebi, Elizabeth Tong, Anna Li, Ghiam Yamin, Greg Zaharchuk, and Mohammad RK Mofrad. Exploring the performance and explainability of fine-tuned bert models for neuro- radiology protocol assignment. BMC Medical Informatics and Decision Making , 24(1):40, 2024

    2024

  42. [50]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017

    2017

  43. [51]

    Interpretable machine learning for personalized medical recommendations: A lime-based approach

    Yuanyuan Wu, Linfei Zhang, Uzair Aslam Bhatti, and Mengxing Huang. Interpretable machine learning for personalized medical recommendations: A lime-based approach. Diagnostics, 13(16):2681, 2023. 12

    2023

  44. [52]

    C-pack: Packaged resources to advance general chinese embedding, 2023

    Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. C-pack: Packaged resources to advance general chinese embedding, 2023

    2023

  45. [53]

    scbert as a large-scale pretrained deep language model for cell type annotation of single-cell rna-seq data

    Fan Yang, Wenchuan Wang, Fang Wang, Yuan Fang, Duyu Tang, Junzhou Huang, Hui Lu, and Jianhua Yao. scbert as a large-scale pretrained deep language model for cell type annotation of single-cell rna-seq data. Nature Machine Intelligence, 4(10):852–866, 2022

    2022

  46. [54]

    Scientific large language models: A survey on biological & chemical domains

    Qiang Zhang, Keyang Ding, Tianwen Lyv, Xinda Wang, Qingyu Yin, Yiwen Zhang, Jing Yu, Yuhao Wang, Xiaotong Li, Zhuoyi Xiang, et al. Scientific large language models: A survey on biological & chemical domains. arXiv preprint arXiv:2401.14656, 2024

    2024 arXiv

  47. [55]

    ""{cell sentence}

    Xin Zhang, Yanzhao Zhang, Dingkun Long, Wen Xie, Ziqi Dai, Jialong Tang, Huan Lin, Baosong Yang, Pengjun Xie, Fei Huang, et al. mgte: Generalized long-context text represen- tation and reranking models for multilingual text retrieval. arXiv preprint arXiv:2407.19669, 2024. A T...

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.