Pith. sign in

REVIEW 4 major objections 7 minor 59 references

scMamba: A Scalable Foundation Model for Single-Cell Multi-Omics Integration Beyond Highly Variable Feature Selection

T0 review · 4 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper builds scMamba, a single-cell multi-omics integration model that uses all genes and peaks without feature preselection and reports an average improvement of more than 10% over seven existing methods on the overall integration…

desk verdict A credible new architecture for no-HVG single-cell integration, but the headline 10% gain is confounded with feature count and the control is buried in the supplementary. read the letter →

arxiv 2506.20697 v1 pith:NMADQKK4 submitted 2025-06-25 q-bio.CB cs.LG

Zhen Yuan , Shaoqing Jiao , Yihang Xiao , Jiajie Peng This is my paper
classification q-bio.CBcs.LG
keywords single-cellmulti-omicsintegrationscRNA-seqandATAC-seqfoundationmodelstatespacedualitypatch-basedtokenizationcontrastivelearningatlas-scaledatacelltypeannotation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

scMamba is a model for integrating single-cell RNA expression, chromatin accessibility, and surface protein measurements taken from the same cells. The paper's central claim is that integration works best when the model consumes the full set of genes and peaks directly, skipping the common preprocessing step that keeps only a few thousand highly variable features. To make this feasible, the model orders genomic features by chromosome position, cuts each cell's profile into coordinate-based patches, and treats patches as tokens and cells as sentences in a state-space-duality sequence encoder. A contrastive objective with an added cosine-similarity term aligns the embeddings of the same cell across omics layers. Across three paired RNA/ATAC datasets the method is reported to outperform seven comparison methods, with an average gain of more than 10% in overall integration score, and to scale to a 377,134-cell human fetal atlas.

What carries the argument

The load-bearing mechanism is the patch-based tokenizer feeding a state-space-duality sequence encoder with alternating MLP blocks. Genes or peaks are sorted by genomic coordinates, split into patches, linearly projected, and summed with learned one-dimensional position embeddings; the encoder's causal mask restricts each patch token to preceding tokens, and the last token's representation is used as the cell embedding. The masked state-space operation $M = L \circ (C B^\top)$ omits softmax and multiplies the attention-like matrix by a 1-semiseparable mask $L$, which the paper argues avoids attention-sink behavior and lets the model weigh all relevant inputs. The objective $L = \lambda_{\mathrm{con}} L_{\mathrm{con}} + \lambda_{\mathrm{sim}} L_{\mathrm{sim}}$ combines the contrastive loss with the cosine-similarity regularizer; this combination is what allows tens of thousands of raw features to be processed and matched cells to be aligned at single-cell resolution.

What would settle it

Run the benchmark again with every comparison method given access to the same full gene and peak set as scMamba, or with scMamba restricted to the same 2,000–3,000 highly variable features; the central claim would be falsified if the reported more-than-10% overall integration-score gap disappeared or reversed.

Watch

Extended reading notes

Core claim

The discovery claimed is that dropping highly variable feature preselection is not merely computationally tolerable but beneficial: integration quality is highest when all genes and peaks are used. The patch-based cell tokenizer converts raw sparse count vectors into fixed-length patch embeddings ordered by genomic coordinates, and the last token of the causally masked sequence, after passing through a modality-specific head, becomes the cell embedding. The training objective combines a symmetric cross-modal contrastive loss—same-cell RNA/ATAC pairs positive, within-batch cross pairs negative—with a cosine-similarity regularizer that directly pulls matched cells' embeddings together. On the paper's benchmarks, this yields the best aggregate scores for biological-variation conservation and omics alignment, and the resulting embeddings improve clustering, cell type annotation, and trajectory reconstruction relative to the comparison methods.

Load-bearing premise

The benchmark feeds scMamba all genes and peaks while the comparison methods run with their default preselected sets of a few thousand highly variable features, so the claimed advantage depends on the assumption that this setup isolates the architecture rather than simply giving scMamba more input features.

Editorial extensions

If this is right

  • If the central claim holds, single-cell multi-omics pipelines can drop highly variable feature selection without sacrificing integration quality, preserving features that variable-gene filters would discard.
  • Atlas-scale integration of hundreds of thousands of cells becomes practical because the method's runtime and memory scale nearly linearly with cell number while keeping high integration scores.
  • The cell embeddings produced by the model are directly reusable for downstream tasks such as clustering, cell type annotation, and trajectory inference, so one integration step can feed multiple analyses.
  • Preserving genomic positional information via coordinate-ordered patches offers a way to integrate expression and chromatin data without constructing explicit gene regulatory network priors.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A matched-feature comparison—giving every comparison method the same full feature set as scMamba, or restricting scMamba to the same number of highly variable features—would separate an architecture effect from a pure input-size effect; the paper's own feature-count evidence sits in supplementary tables rather than in the main benchmark.
  • The same patch-token plus contrastive-and-cosine recipe should transfer to other paired modalities, such as RNA with surface protein or spatial transcriptomics, because the tokenizer only needs a genomic coordinate ordering for its features.
  • If the causal ordering along genomic coordinates is itself meaningful, it suggests a directional reading order for the genome that future encoders might further exploit or replace with bidirectional or graph-structured genomic encoders.
  • Since the overall integration score weights biology conservation at 0.6 and omics alignment at 0.4, a user whose goal is predominantly alignment should re-evaluate methods under a different weighting before choosing a tool.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The manuscript presents scMamba, a Mamba2-based encoder that tokenizes single-cell gene/peak matrices into genomic-region patches, trains modality-specific encoders with a contrastive loss plus a cosine-similarity regularization term, and benchmarks integration against seven existing methods on SHARE-seq, human brain, PBMC, fetal atlas, CITE-seq, and 10x Multiome BMMC datasets. The central claim is that processing all genes and peaks without prior highly-variable-feature selection yields 'an average improvement of more than 10% in the overall integration score' and consistent outperformance across benchmarks, while also providing atlas-scale scalability. The paper also reports downstream cell-type annotation and trajectory-reconstruction results.

Significance. If the quantitative claims are established, scMamba would be a useful contribution: most single-cell integration pipelines discard features via HVG/HVP selection, and a scalable method that preserves full-resolution genomic features while aligning modalities is of genuine interest. The manuscript has notable strengths: it uses multiple public datasets, a wide range of evaluation metrics including FOSCTTM and neighborhood overlap, an atlas-scale 377,134-cell experiment, and a clearly described architecture. However, the main comparative evidence does not currently isolate the architecture from the input feature count, key training hyperparameters are not reported, and the headline benchmark figures lack uncertainty quantification. The central claim is therefore defensible but not yet established as presented.

major comments (4)
  1. [§2.2, §4.2.2] The benchmark is asymmetric in input features: scMamba receives the full gene/peak matrices, whereas CVQVAE uses the top 3,000 HVGs plus PCA/LSI to 2,500 dimensions, GLUE uses 2,000 HVGs, and SCALEX/scVI/Harmony/Scanorama run their default HVG or gene-activity pipelines. Consequently, the reported >10% aggregate improvement and the 'consistently outperforms' claim confound the Mamba2/contrastive architecture with a 10-100x larger input feature set. The feature-count control is only summarized in §2.3, and Supplementary Tables 1-4 are not in the main text, so the claim that increasing features did not help baselines is not verifiable from the manuscript as presented. Please add a main-text controlled comparison in which all methods receive identical feature sets (for example, all genes/peaks, or matched HVG/HVP sizes), with per-dataset scores, and state explicitly whether scMamba's advantage persists under matched inputs.
  2. [§4.1] The Methods define the model in terms of free hyperparameters but never report their values: patch size P, number of patches C, embedding dimension D, number of scMamba blocks L, number of expression bins B, temperature sigma, and the loss weights lambda_con and lambda_sim in Eqs. (1), (7)-(9). Optimizer, learning rate, batch size, number of epochs, and hardware are also omitted, and no code availability statement is provided. Without these details the reported results cannot be independently reproduced, and the contribution of the architecture itself cannot be assessed.
  3. [§2.2, Fig. 2/Fig. 3] No repeated runs or statistical significance tests are reported for the headline benchmark figures. Fig. 2a and 2c appear to show single-run values, and the 'average improvement of more than 10%' in the overall integration score is given without variance or a statistical test across datasets and subsampling levels. Please add multiple seeds, report mean ± s.d., and provide paired significance tests (for example, Wilcoxon signed-rank or permutation tests) for the aggregate comparisons.
  4. [§4.1.4] The cosine-similarity regularization L_sim in Eq. (8) is presented as a novel component, but no ablation separates its contribution from the contrastive loss L_con in Eq. (7). With L2-normalized embeddings, the InfoNCE-style loss already maximizes positive-pair cosine similarity, so the added term may be redundant or may simply reinforce the same gradient. Please include an ablation with and without L_sim and with different lambda_sim values to support the claim that this module drives the reported alignment improvement.
minor comments (7)
  1. [§4.2.1, after Eq. (21)] The text says that 'M_bio contains the metrics OEMS, SAS, GC, and oASW'; the variable should be M_omics rather than M_bio.
  2. [§4.1.1] The phrase 'retrain positional information' should read 'retain positional information'.
  3. [§4.2.2] The baseline heading 'scCLP' should be 'scCLIP', and the spacing in 'CV A V AE' / 'CVQV AE' is inconsistent.
  4. [§2.4] The barplot comparing scMamba, CellPLM, and scGPT is cited as 'Fig. 5g' in the text but appears to be panel g of Fig. 4; please fix the cross-reference.
  5. [§4.1.1] The patch tokenization uses zero-padding to standardize token lengths, but the paper does not state how genomic regions with no measured features are handled after coordinate ordering; please clarify whether padding is applied per patch or per cell.
  6. [§1, §3] The term 'foundation model' may overstate the scope: the paper trains a fresh model on each dataset and does not demonstrate large-scale pretraining or transfer beyond the brain 3k annotation experiment. Consider softening the terminology.
  7. [§5] No code availability statement is provided; public code would greatly help replication and is increasingly expected for methods papers.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; scMamba's benchmark is externally validated, with only in-sample and feature-set confounds.

full rationale

The central derivation is self-contained: scMamba's embeddings are trained by Eqs. (7)-(9), a contrastive objective whose positive pairs are same-cell RNA/ATAC profiles, and the reported integration metrics (ARI, NMI, MAP, cASW, OEMS, SAS, GC, oASW, FOSCTTM, matching score, neighborhood overlap) are evaluated against external cell-type labels or known cell-to-cell correspondences, not against quantities that appear as fitted parameters in the model. No parameter is fitted to the evaluation metric, and no claim reduces by construction to an input value: the patch tokenization (Eq. 1), Mamba2 block (Eqs. 2-6), and loss (Eqs. 7-9) use standard building blocks cited from external work (ViT/ViM, Mamba2, CLIP), and the paper contains no uniqueness theorem or load-bearing self-citation. The only caveat is evaluative, not circular: the alignment metrics partly measure the same pairing signal used for training, and the main comparison gives scMamba all genes/peaks while baselines use default HVG sets (Methods 4.2.2), but these are benchmarking confounds rather than derivation-chain circularity.

Assumptions & free parameters 8 free parameters · 4 assumptions · 0 invented entities

The central claims rest on several unreported training hyperparameters and on domain assumptions about genomic order, pairing quality, and the adequacy of the benchmark metrics. None of these are validated in the main text, but none are circular.

free parameters (8)
  • Patch size P
    Determines how many genes or peaks form one token. Value not reported; central to tokenization.
  • Number of patches C
    Sequence length fed to the encoder; determined by profile length and patch size but not specified.
  • Embedding dimension D
    Dimension of patch projections and cell embeddings; not reported.
  • Number of scMamba blocks L
    Depth of each encoder; not reported.
  • Contrastive loss weight lambda_con
    Loss weight in Eq. 9; not reported despite being essential to training.
  • Cosine similarity regularization weight lambda_sim
    Loss weight in Eq. 9; not reported despite being one of the paper's main contributions.
  • Temperature sigma in contrastive loss
    Scales logits in Eq. 7; not reported.
  • Number of expression bins B
    Binning intervals for RNA expression values; not reported.
assumptions (4)
  • domain assumption Genomic order of genes and peaks carries information relevant to integration, and position embeddings over patches preserve it.
    The tokenization orders features by genomic coordinates and adds 1D position embeddings (Methods 4.1.1); no shuffled-order baseline is provided.
  • domain assumption Cells measured in paired multi-omics datasets are correctly matched, so contrastive positive pairs are reliable.
    Contrastive learning in Methods 4.1.4 assumes RNA and ATAC profiles from the same cell are true positives.
  • domain assumption Binarizing ATAC accessibility and binning RNA expression into discrete intervals preserves the information needed for integration.
    Methods 4.1.1 applies binarization and binning; no sensitivity analysis of these choices is in the main text.
  • domain assumption Biological variation and omics alignment are adequately measured by the selected eight metrics aggregated with 6:4 weighting and min-max scaling.
    The overall integration score is defined in Eqs. 19-22 using fixed metric weights; the choice affects all rankings.

how reviews work

0 comments
Cite this review

Pith. "Pith review of scMamba: A Scalable Foundation Model for Single-Cell Multi-Omics Integration Beyond Highly Variable Feature Selection." pith.science (2026). https://pith.science/paper/NMADQKK4

@misc{pith2026250620697,
  author       = {Pith},
  title        = {Pith review of: scMamba: A Scalable Foundation Model for Single-Cell Multi-Omics Integration Beyond Highly Variable Feature Selection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NMADQKK4}},
  note         = {Machine review of arXiv:2506.20697}
}
read the original abstract

The advent of single-cell multi-omics technologies has enabled the simultaneous profiling of diverse omics layers within individual cells. Integrating such multimodal data provides unprecedented insights into cellular identity, regulatory processes, and disease mechanisms. However, it remains challenging, as current methods often rely on selecting highly variable genes or peaks during preprocessing, which may inadvertently discard crucial biological information. Here, we present scMamba, a foundation model designed to integrate single-cell multi-omics data without the need for prior feature selection while preserving genomic positional information. scMamba introduces a patch-based cell tokenization strategy that treats genomics regions as words (tokens) and cells as sentences. Building upon the concept of state space duality, scMamba distills rich biological insights from high-dimensional, sparse single-cell multi-omics data. Additionally, our novel contrastive learning approach, enhanced with cosine similarity regularization, enables superior alignment across omics layers compared to traditional methods. Systematic benchmarking across multiple datasets demonstrates that scMamba significantly outperforms state-of-the-art methods in preserving biological variation, aligning omics layers, and enhancing key downstream tasks such as clustering, cell type annotation, and trajectory inference. Our findings position scMamba as a powerful tool for large-scale single-cell multi-omics integration, capable of handling large-scale atlases and advancing biological discovery.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

59 extracted references · 46 canonical work pages

  1. [1]

    Nature methods14(9), 865–868 (2017)

    Stoeckius, M., Hafemeister, C., Stephenson, W., Houck-Loomis, B., Chattopad- hyay, P.K., Swerdlow, H., Satija, R., Smibert, P.: Simultaneous epitope and transcriptome measurement in single cells. Nature methods14(9), 865–868 (2017)

    work page 2017

  2. [2]

    Nature biotechnology35(10), 936–939 (2017)

    Peterson, V.M., Zhang, K.X., Kumar, N., Wong, J., Li, L., Wilson, D.C., Moore, R., McClanahan, T.K., Sadekova, S., Klappenbach, J.A.: Multiplexed quantifi- cation of proteins and transcripts in single cells. Nature biotechnology35(10), 936–939 (2017)

    work page 2017

  3. [3]

    Cell183(4), 1103–1116 (2020)

    Ma, S., Zhang, B., LaFave, L.M., Earl, A.S., Chiang, Z., Hu, Y., Ding, J., Brack, A., Kartha, V.K., Tay, T.,et al.: Chromatin potential identified by shared single- cell profiling of rna and chromatin. Cell183(4), 1103–1116 (2020)

    work page 2020

  4. [4]

    Nature biotechnology37(12), 1452–1457 (2019)

    Chen, S., Lake, B.B., Zhang, K.: High-throughput sequencing of the transcrip- tome and chromatin accessibility in the same cell. Nature biotechnology37(12), 1452–1457 (2019)

    work page 2019

  5. [5]

    Nature Reviews Molecular Cell Biology24(10), 695–713 (2023) 24

    Baysoy, A., Bai, Z., Satija, R., Fan, R.: The technological landscape and applica- tions of single-cell multi-omics. Nature Reviews Molecular Cell Biology24(10), 695–713 (2023) 24

    work page 2023

  6. [6]

    BMC genomics26, 132 (2025)

    Ma, L., Liu, J., Sun, W., Zhao, C., Yu, L.: scmfg: a single-cell multi-omics integration method based on feature grouping. BMC genomics26, 132 (2025)

    work page 2025

  7. [7]

    Journal of Hematology & Oncology17(1), 72 (2024)

    Liu, X., Peng, T., Xu, M., Lin, S., Hu, B., Chu, T., Liu, B., Xu, Y., Ding, W., Li, L.,et al.: Spatial multi-omics: deciphering technological landscape of integration of multi-omics and its applications. Journal of Hematology & Oncology17(1), 72 (2024)

    work page 2024

  8. [8]

    Cell187(17), 4488–4519 (2024)

    Liu, L., Chen, A., Li, Y., Mulder, J., Heyn, H., Xu, X.: Spatiotemporal omics for biology and medicine. Cell187(17), 4488–4519 (2024)

    work page 2024

Show all 59 references
  1. [9]

    Experimental & Molecular Medicine56(3), 515–526 (2024)

    Lim, J., Park, C., Kim, M., Kim, H., Kim, J., Lee, D.-S.: Advances in single- cell omics and multiomics for high-resolution molecular profiling. Experimental & Molecular Medicine56(3), 515–526 (2024)

    2024

  2. [10]

    Nature Biotechnology, 1–12 (2024)

    He, Z., Hu, S., Chen, Y., An, S., Zhou, J., Liu, R., Shi, J., Wang, J., Dong, G., Shi, J., et al.: Mosaic integration and knowledge transfer of single-cell multimodal data with midas. Nature Biotechnology, 1–12 (2024)

    2024

  3. [11]

    International Journal of Molecular Sciences26(6), 2447 (2025)

    Guan, A., Quek, C.: Single-cell multi-omics: Insights into therapeutic innovations to advance treatment in cancer. International Journal of Molecular Sciences26(6), 2447 (2025)

    2025

  4. [12]

    Nature reviews genetics 20(5), 257–272 (2019)

    Stuart, T., Satija, R.: Integrative single-cell analysis. Nature reviews genetics 20(5), 257–272 (2019)

    2019

  5. [13]

    Nature Reviews Genetics24(8), 494–515 (2023)

    Vandereyken, K., Sifrim, A., Thienpont, B., Voet, T.: Methods and applications for single-cell and spatial multi-omics. Nature Reviews Genetics24(8), 494–515 (2023)

    2023

  6. [14]

    Nature methods16(12), 1289–1296 (2019)

    Korsunsky, I., Millard, N., Fan, J., Slowikowski, K., Zhang, F., Wei, K., Baglaenko, Y., Brenner, M., Loh, P.-r., Raychaudhuri, S.: Fast, sensitive and accurate integration of single-cell data with harmony. Nature methods16(12), 1289–1296 (2019)

    2019

  7. [15]

    Nature biotechnology37(6), 685–691 (2019)

    Hie, B., Bryson, B., Berger, B.: Efficient integration of heterogeneous single-cell transcriptomes using scanorama. Nature biotechnology37(6), 685–691 (2019)

    2019

  8. [16]

    Nature methods15(12), 1053–1058 (2018)

    Lopez, R., Regier, J., Cole, M.B., Jordan, M.I., Yosef, N.: Deep generative modeling for single-cell transcriptomics. Nature methods15(12), 1053–1058 (2018)

    2018

  9. [17]

    Nature biotechnology 40(1), 121–130 (2022) 25

    Lotfollahi, M., Naghipourfar, M., Luecken, M.D., Khajavi, M., B¨ uttner, M., Wagenstetter, M., Avsec, ˇZ., Gayoso, A., Yosef, N., Interlandi, M.,et al.: Mapping single-cell data to reference atlases by transfer learning. Nature biotechnology 40(1), 121–130 (2022) 25

    2022

  10. [18]

    Molecular systems biology17(1), 9620 (2021)

    Xu, C., Lopez, R., Mehlman, E., Regier, J., Jordan, M.I., Yosef, N.: Probabilis- tic harmonization and annotation of single-cell transcriptomics data with deep generative models. Molecular systems biology17(1), 9620 (2021)

    2021

  11. [19]

    Nature Communications13(1), 6118 (2022)

    Xiong, L., Tian, K., Li, Y., Ning, W., Gao, X., Zhang, Q.C.: Online single-cell data integration through projecting heterogeneous datasets into a common cell- embedding space. Nature Communications13(1), 6118 (2022)

    2022

  12. [20]

    arXiv preprint arXiv:1312.6114 (2013)

    Kingma, D.P.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)

    2013 arXiv

  13. [21]

    Genome biology20, 1–25 (2019)

    Chen, H., Lareau, C., Andreani, T., Vinyard, M.E., Garcia, S.P., Clement, K., Andrade-Navarro, M.A., Buenrostro, J.D., Pinello, L.: Assessment of computa- tional methods for the analysis of single-cell atac-seq data. Genome biology20, 1–25 (2019)

    2019

  14. [22]

    Nature biotechnology42(2), 284–292 (2024)

    Ghazanfar, S., Guibentif, C., Marioni, J.C.: Stabilized mosaic single-cell data integration using unshared features. Nature biotechnology42(2), 284–292 (2024)

    2024

  15. [23]

    Nature biotechnology40(5), 703–710 (2022)

    Lin, Y., Wu, T.-Y., Wan, S., Yang, J.Y., Wong, W.H., Wang, Y.R.: scjoint inte- grates atlas-scale single-cell rna-seq and atac-seq data with transfer learning. Nature biotechnology40(5), 703–710 (2022)

    2022

  16. [24]

    Cell184(13), 3573–3587 (2021)

    Hao, Y., Hao, S., Andersen-Nissen, E., Mauck, W.M., Zheng, S., Butler, A., Lee, M.J., Wilk, A.J., Darby, C., Zager, M.,et al.: Integrated analysis of multimodal single-cell data. Cell184(13), 3573–3587 (2021)

    2021

  17. [25]

    Nature methods18(3), 272–282 (2021)

    Gayoso, A., Steier, Z., Lopez, R., Regier, J., Nazor, K.L., Streets, A., Yosef, N.: Joint probabilistic modeling of single-cell multi-omic data with totalvi. Nature methods18(3), 272–282 (2021)

    2021

  18. [26]

    Genome biology23(1), 20 (2022)

    Li, G., Fu, S., Wang, S., Zhu, C., Duan, B., Tang, C., Chen, X., Chuai, G., Wang, P., Liu, Q.: A deep generative model for multi-view profiling of single-cell rna-seq and atac-seq data. Genome biology23(1), 20 (2022)

    2022

  19. [27]

    Cell177(7), 1873–1887 (2019)

    Welch, J.D., Kozareva, V., Ferreira, A., Vanderburg, C., Martin, C., Macosko, E.Z.: Single-cell multi-omic integration compares and contrasts features of brain cell identity. Cell177(7), 1873–1887 (2019)

    2019

  20. [28]

    Nature Biotechnology40(10), 1458–1466 (2022)

    Cao, Z.-J., Gao, G.: Multi-omics single-cell data integration and regulatory infer- ence with graph-linked embedding. Nature Biotechnology40(10), 1458–1466 (2022)

    2022

  21. [29]

    In: Machine Learning in Computational Biology, pp

    Liu, T., Greenberg, G., Shomorony, I.: Cvqvae: A representation learning based method for multi-omics single cell data integration. In: Machine Learning in Computational Biology, pp. 1–15 (2022). PMLR 26

    2022

  22. [30]

    In: NeurIPS 2023 AI for Science Workshop

    Xiong, L., Chen, T., Kellis, M.: scclip: Multi-modal single-cell contrastive learning integration pre-training. In: NeurIPS 2023 AI for Science Workshop

    2023

  23. [31]

    Nature biotechnology42(7), 1096–1106 (2024)

    Chen, S., Zhu, B., Huang, S., Hickey, J.W., Lin, K.Z., Snyder, M., Greenleaf, W.J., Nolan, G.P., Zhang, N.R., Ma, Z.: Integration of spatial and single-cell data across modalities with weakly linked features. Nature biotechnology42(7), 1096–1106 (2024)

    2024

  24. [32]

    arXiv preprint arXiv:2405.21060 (2024)

    Dao, T., Gu, A.: Transformers are ssms: Generalized models and efficient algo- rithms through structured state space duality. arXiv preprint arXiv:2405.21060 (2024)

    2024 arXiv

  25. [33]

    Nature Methods, 1–11 (2024)

    Cui, H., Wang, C., Maan, H., Pang, K., Luo, F., Duan, N., Wang, B.: scgpt: toward building a foundation model for single-cell multi-omics using generative ai. Nature Methods, 1–11 (2024)

    2024

  26. [34]

    Nature Machine Intelligence4(10), 852–866 (2022)

    Yang, F., Wang, W., Wang, F., Fang, Y., Tang, D., Huang, J., Lu, H., Yao, J.: scbert as a large-scale pretrained deep language model for cell type annotation of single-cell rna-seq data. Nature Machine Intelligence4(10), 852–866 (2022)

    2022

  27. [35]

    Nature methods21(8), 1481–1491 (2024)

    Hao, M., Gong, J., Zeng, X., Liu, C., Guo, Y., Cheng, X., Wang, T., Ma, J., Zhang, X., Song, L.: Large-scale foundation model on single-cell transcriptomics. Nature methods21(8), 1481–1491 (2024)

    2024

  28. [37]

    arXiv preprint arXiv:2401.09417 (2024)

    Zhu, L., Liao, B., Zhang, Q., Wang, X., Liu, W., Wang, X.: Vision mamba: Efficient visual representation learning with bidirectional state space model. arXiv preprint arXiv:2401.09417 (2024)

    2024 arXiv

  29. [38]

    Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologie...

    2019

  30. [39]

    Genomics, Proteomics & Bioinformatics20(5), 836–849 (2022)

    Stanojevic, S., Li, Y., Ristivojevic, A., Garmire, L.X.: Computational methods for single-cell multi-omics integration and alignment. Genomics, Proteomics & Bioinformatics20(5), 836–849 (2022)

    2022

  31. [40]

    Nature methods 19(1), 41–50 (2022)

    Luecken, M.D., B¨ uttner, M., Chaichoompu, K., Danese, A., Interlandi, M., M¨ uller, M.F., Strobl, D.C., Zappia, L., Dugas, M., Colom´ e-Tatch´ e, M.,et al.: Benchmarking atlas-level data integration in single-cell genomics. Nature methods 19(1), 41–50 (2022)

    2022

  32. [41]

    In: Proceedings of the 11th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, pp

    Singh, R., Demetci, P., Bonora, G., Ramani, V., Lee, C., Fang, H., Duan, Z., 27 Deng, X., Shendure, J., Disteche, C.,et al.: Unsupervised manifold alignment for single-cell multi-omics data. In: Proceedings of the 11th ACM International Conference on Bioinformatics, Computatio...

    2020

  33. [42]

    Nature Communications14(1), 223 (2023)

    Chen, J., Xu, H., Tao, W., Chen, Z., Zhao, Y., Han, J.-D.J.: Transformer for one stop interpretable cell type annotation. Nature Communications14(1), 223 (2023)

    2023

  34. [43]

    bioRxiv, 2023–10 (2023)

    Wen, H., Tang, W., Dai, X., Ding, J., Jin, W., Xie, Y., Tang, J.: Cellplm: pre- training of cell language model beyond single cells. bioRxiv, 2023–10 (2023)

    2023

  35. [44]

    Bioinformatics36(Supplement 1), 48–56 (2020)

    Cao, K., Bai, X., Hong, Y., Wan, L.: Unsupervised topological alignment for single-cell multi-omics integration. Bioinformatics36(Supplement 1), 48–56 (2020)

    2020

  36. [45]

    cell177(7), 1888–1902 (2019)

    Stuart, T., Butler, A., Hoffman, P., Hafemeister, C., Papalexi, E., Mauck, W.M., Hao, Y., Stoeckius, M., Smibert, P., Satija, R.: Comprehensive integration of single-cell data. cell177(7), 1888–1902 (2019)

    2019

  37. [46]

    arXiv preprint arXiv:2312.00752 (2023)

    Gu, A., Dao, T.: Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752 (2023)

    2023 arXiv

  38. [47]

    arXiv preprint arXiv:2403.14520 (2024)

    Zhao, H., Zhang, M., Zhao, W., Ding, P., Huang, S., Wang, D.: Cobra: Extending mamba to multi-modal large language model for efficient inference. arXiv preprint arXiv:2403.14520 (2024)

    2024 arXiv

  39. [48]

    arXiv preprint arXiv:2407.19832 (2024)

    Huang, W., Pan, J., Tang, J., Ding, Y., Xing, Y., Wang, Y., Wang, Z., Hu, J.: Ml-mamba: Efficient multi-modal large language model utilizing mamba-2. arXiv preprint arXiv:2407.19832 (2024)

    2024 arXiv

  40. [49]

    arXiv preprint arXiv: 2010.11929 (2020)

    Alexey, D.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv: 2010.11929 (2020)

    2020 arXiv

  41. [50]

    Advances in Neural Information Processing Systems (2017)

    Vaswani, A.: Attention is all you need. Advances in Neural Information Processing Systems (2017)

    2017

  42. [51]

    arXiv preprint arXiv:2309.17453 (2023)

    Xiao, G., Tian, Y., Chen, B., Han, S., Lewis, M.: Efficient streaming language models with attention sinks. arXiv preprint arXiv:2309.17453 (2023)

    2023 arXiv

  43. [52]

    arXiv preprint arXiv:2309.16588 (2023)

    Darcet, T., Oquab, M., Mairal, J., Bojanowski, P.: Vision transformers need registers. arXiv preprint arXiv:2309.16588 (2023)

    2023 arXiv

  44. [53]

    arXiv preprint arXiv:1312.4400 (2013)

    Lin, M., Chen, Q., Yan, S.: Network in network. arXiv preprint arXiv:1312.4400 (2013)

    2013 arXiv

  45. [54]

    In: International Conference on Machine Learning, pp

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J.,et al.: Learning transferable visual models from 28 natural language supervision. In: International Conference on Machine Learning, pp. 8748–8763 (2021). PMLR

    2021

  46. [55]

    arXiv preprint arXiv:2205.01917 (2022)

    Yu, J., Wang, Z., Vasudevan, V., Yeung, L., Seyedhosseini, M., Wu, Y.: Coca: Contrastive captioners are image-text foundation models. arXiv preprint arXiv:2205.01917 (2022)

    2022 arXiv

  47. [56]

    In: International Conference on Machine Learning, pp

    Jia, C., Yang, Y., Xia, Y., Chen, Y.-T., Parekh, Z., Pham, H., Le, Q., Sung, Y.-H., Li, Z., Duerig, T.: Scaling up visual and vision-language representation learning with noisy text supervision. In: International Conference on Machine Learning, pp. 4904–4916 (2021). PMLR

    2021

  48. [57]

    Journal of computational and applied mathematics20, 53–65 (1987)

    Rousseeuw, P.J.: Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of computational and applied mathematics20, 53–65 (1987)

    1987

  49. [58]

    Nature biotechnology36(5), 421–427 (2018)

    Haghverdi, L., Lun, A.T., Morgan, M.D., Marioni, J.C.: Batch effects in single- cell rna-sequencing data are corrected by matching mutual nearest neighbors. Nature biotechnology36(5), 421–427 (2018)

    2018

  50. [59]

    Nature biotechnology36(5), 411–420 (2018)

    Butler, A., Hoffman, P., Smibert, P., Papalexi, E., Satija, R.: Integrating single- cell transcriptomic data across different conditions, technologies, and species. Nature biotechnology36(5), 411–420 (2018)

    2018

  51. [60]

    Cell Genomics3(3) (2023) 29

    Anderson, A.G., Rogers, B.B., Loupe, J.M., Rodriguez-Nunez, I., Roberts, S.C., White, L.M., Brazell, J.N., Bunney, W.E., Bunney, B.G., Watson, S.J., et al.: Single nucleus multiomics identifies zeb1 and mafb as candidate regulators of alzheimer’s disease-specific cis-regulator...

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.