Scaling an Autoregressive Transformer for Single-Cell Generation

Aleksandr Sharipov; Igor Molybog; Yusif Mukhtarov

REVIEW 3 major objections 3 minor 14 references

Scaling an Autoregressive Transformer for Single-Cell Generation

T0 review · 3 major / 3 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A causal transformer over discrete single-cell codes follows a joint two-exponent scaling law, with compute-optimal training tilted toward data.

desk verdict A genuinely independent N×D scaling-law fit for a single-cell transformer, with strong held-out predictions; the unresolved tokenizer–validation-split question is the one load-bearing soft spot. read the letter →

arxiv 2608.02961 v1 pith:EHQSTJGW submitted 2026-08-03 cs.LG cs.AIq-bio.GN

Aleksandr Sharipov , Yusif Mukhtarov , Igor Molybog This is my paper · ORCID

classification cs.LGcs.AIq-bio.GN MSC 68T07

keywords single-cellfoundationmodelscalinglawsautoregressivetransformerRQ-VAEtokenizationcompute-optimalfrontiergeneexpressiongenerationChinchillaApproach3self-supervisedpretraining

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper seeks to establish that a single-cell gene-expression generator—a causal transformer fed eight discrete tokens per cell from a learned residual-quantized VAE—obeys the same kind of joint scaling law as large language models. On a grid of 22 points where model size $N$ (1.31M–83.9M parameters) and data budget $D$ (38.1M–362M cells) vary independently, the pretraining cross-entropy fits $L(N,D)=2.90+2.05\times10^4/N^{0.81}+4.28\times10^3/D^{0.57}$ with both exponents positive and $R^2\approx0.98$. Four runs held out of the fit are predicted within 0.5 percent, and the inverted law says compute-optimal training spends more on data than on parameters ($D^*\propto C^{0.59}$, $N^*\propto C^{0.41}$). If true, this turns single-cell foundation-model pretraining into a budget-allocation problem and reframes earlier no-data-scaling plateaus as capacity bottlenecks rather than inherent limits.

What carries the argument

The load-bearing mechanism is a tokenized sequence-modeling pipeline. An RQ-VAE compresses each 18,080-gene log-normalized expression vector into a 256-dimensional latent and quantizes it across eight residual codebooks of 256 entries each, turning one cell into a block of eight integer tokens; a causal transformer of the LLaMA family, trained from scratch by next-token cross-entropy, models sentences of 32 same-cell-type cells. The transformer adds a learned within-cell position embedding shared across cells and a learned bias separating cell tokens from separators, so the discrete structure of a cell is explicit. The scaling result comes from fitting the two-term parametric form by nonlinear least squares on log-residuals with a Huber loss and a multi-start initialization grid, then inverting the fit to obtain the compute-optimal model size $N^*(C)$.

What would settle it

Retrain the RQ-VAE on only the training split, excluding the 20 percent validation cells per cell type, and refit Eq. 2 on the same 22-point grid; the law is falsified if the exponents move beyond the reported fit uncertainty or if the four held-out predictions fall outside 0.5 percent.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the self-supervised pretraining loss of an RQ-VAE-tokenized autoregressive transformer trained on single-cell expression follows the Chinchilla-style form $L(N,D)=E+A/N^\alpha+B/D^\beta$ over an independently varied $N\times D$ grid, with fitted values $E\approx2.90$, $A\approx2.05\times10^4$, $\alpha\approx0.81$, $B\approx4.28\times10^3$, and $\beta\approx0.57$. Both exponents are clearly positive; the in-sample fit has $R^2\approx0.98$; and four runs deliberately excluded from the fit, one at each model size, are predicted within 0.5 percent. Because returns to parameters fall faster than returns to data, the compute-optimal frontier tilts toward data in this regime. The same model, conditioned on held-out vectors of a cell type, generates expression populations whose per-gene mean vectors sit just below empirical-replicate baselines (Pearson 0.960 vs 0.964, MAE 0.041 vs 0.038), with cell-type discriminability 0.990, and this generation quality tracks the pretraining loss across checkpoints.

Load-bearing premise

The central claim depends on the validation loss being measured on cells that neither the transformer nor the frozen tokenizer ever saw; the paper does not state that the tokenizer's training set excluded the per-cell-type validation cells.

Editorial extensions

If this is right

The pretraining loss of a new run, and by proxy its generation quality, can be predicted before training from $N$ and $D$, so compute can be allocated to hit a target loss.
At the scales tested, adding cells is more compute-efficient than adding parameters: the optimal allocation follows $D^*\propto C^{0.59}$ against $N^*\propto C^{0.41}$.
A fixed-$N$ slice of the fitted surface plateaus as $D$ grows, which explains reported no-data-scaling results as model-capacity bottlenecks rather than a failure of scaling.
Generation fidelity tracks the pretraining loss monotonically (checkpoint correlations of $r=-0.99$ for Pearson and $r=+0.98$ for MAE), so the scaling law is a practical proxy for biological fidelity.
The law can be reproduced with modest compute, since the largest grid run is 83.9M parameters on a corpus of roughly 200M cells, making scaling-law planning accessible to small teams.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The fitted floor $E\approx2.90$ nats is partly set by the lossy RQ-VAE; one testable extension is to separate tokenizer reconstruction error from irreducible biological variability by re-fitting the law with more codebooks or larger codebooks.
The checkmark-shaped held-out loss on perturbation-benchmark control cells, which rises while the main validation loss falls, suggests that a perturbation-response finetune may need early checkpoint selection or corpus reweighting; this is the authors' open question, not a demonstrated result.
If the two-exponent form transfers to continuous-embedding single-cell models, scaling-law planning would generalize across architecture families; the paper does not test that transfer.
The law is fit in the 1.31M–83.9M parameter range, so extrapolation to billion-parameter scales is untested; a direct extension would add one large run and check whether $\alpha$ and $\beta$ hold.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The core result is a Chinchilla-style two-exponent scaling law, L(N,D) = 2.90 + 2.05e4/N^0.81 + 4.28e3/D^0.57, fit to a 22-point grid where model size and data budget are varied independently for a discrete-tokenized single-cell transformer. That is new for this subfield: prior work mostly sweeps one axis or confounds model size with other factors. The paper is transparent about reusing Hoffmann et al.'s Approach 3 fitting procedure, and the diagnostics are genuinely good: R^2 = 0.977, small symmetric residuals, and four completely held-out runs predicted within 0.5%. The generation-quality probe is also useful, showing that lower pretraining loss tracks better biological fidelity. I believe the central claim is solid within its stated scope, and the authors are careful to bound that scope in the limitations section.

The main soft spot is the tokenizer. The 20% validation split is introduced only for the transformer, and the RQ-VAE is described as "trained on the scBaseCount atlas" with no statement that validation cells were excluded from its training. If the tokenizer saw the validation cells, then the validation loss is partly in-sample, and the fitted exponents—especially the data exponent—could be biased. The four held-out runs share the same frozen tokenizer, so their agreement does not test this. This is not fatal, but it is load-bearing: the claim that this is a truly held-out loss needs either a clear statement that the tokenizer was trained on the 80% pool only, a re-fit with a split-aware tokenizer, or an auxiliary experiment showing the tokenizer's reconstruction loss does not differ between training and validation cells. The authors should also add uncertainty estimates for the exponents; a simple bootstrap would do. My only other complaint is a single sentence in the conclusion that locates other groups' plateaus as "a downstream, not an upstream, phenomenon," which overreaches; the body's scope statement already says the right thing.

Who this is for: practitioners building single-cell foundation models, especially those deciding how to allocate compute between parameters and data. The paper deserves a serious referee; the issues are concrete and fixable, and the result is useful. I would send it out.

Referee Report

3 major / 3 minor

Summary. The paper studies conditional generation of single-cell gene expression vectors using a causal transformer over RQ-VAE discrete tokens. Its principal contribution is a joint two-exponent scaling law, L(N,D)=2.90+2.05e4/N^0.81+4.28e3/D^0.57, fit to a controlled 22-point N x D grid (1.31M-83.9M parameters, 38.1M-362M cells) and validated by four held-out runs predicted within 0.5%. The authors also evaluate generation quality against the pretraining loss, discuss why single-axis sweeps can look like data-scaling plateaus, and sketch a Stage 2 perturbation-response finetuning plan.

Significance. If the held-out status of the loss can be established, this is the first jointly fit two-exponent scaling law and compute-optimal frontier for a discrete-tokenized single-cell transformer. The paper's strengths are its controlled shape-preserving model sweep, independent N and D variation, proportional pool sampling, multi-start Approach-3 fitting with residual diagnostics, explicit four-point out-of-sample prediction, and released code. The framing against the diversity-over-scale literature is fair, and the limitations section is unusually candid. The two concerns below—tokenizer exposure to validation cells and ambiguity about the number of fit points—are central because both bear directly on whether L(N,D) in Eq. (2) is a true held-out loss and whether the reported R² and prediction errors support the stated law.

major comments (3)

[Section 4.3; Appendix A.1; Appendix B] The validation split is introduced only for transformer training: Section 4.3 and Appendix B state that within each cell type 20% of cells are reserved and no transformer run trains on them, but Appendix A.1 says the RQ-VAE tokenizer is trained on the scBaseCount atlas and frozen, with no statement that validation cells were excluded from tokenizer training. Because L(N,D) in Eq. (2) is the cross-entropy on tokens generated by this tokenizer, any tokenizer exposure to validation cells makes the fitted loss partly in-sample, and the bias need not cancel across N and D since the same frozen tokenizer is used at every grid point. Please clarify whether validation cells were excluded from tokenizer training; if they were not, retrain the tokenizer on the 80% training pool and refit the law, and report the change in α and β.
[Section 4.4; Section 4.5; Appendix E.1] The paper is internally inconsistent about whether the four held-out runs are part of the 22 grid points used in the fit. Section 4.4 says the law is fit over the 22 (N,D) points and calls the R²=0.977 in-sample, while Section 4.5 says four runs were held completely out of the fit, and Appendix E.1 describes residuals over the 22 grid points. If the four runs are excluded from parameter estimation, the fit uses 18 points and the in-sample R² and residual diagnostics should be reported for those 18 points, with the four held-out runs reported only as prediction errors; if they are included, the label 'held out' is inaccurate. This distinction is load-bearing for the out-of-sample claim in Section 4.5.
[Section 4.4; Eq. (2); Eq. (9)] The exponents α≈0.81 and β≈0.57, and consequently the frontier exponents D*∝C^0.59 and N*∝C^0.41 in Eq. (9), are reported without confidence intervals or standard errors. With five free parameters and at most 22 (or 18) points, the estimate depends on the initialization grid and on individual grid points; please provide bootstrap or refit-based intervals so the reader can judge whether the data-tilt of the frontier is statistically supported.

minor comments (3)

[Throughout] The rendered full text contains numerous missing spaces and mangled words (e.g., 'Westudya' in the abstract and 'LLaMAarchitecture' in Section 3.4); please run a careful spelling and formatting pass before resubmission.
[Section 4.1; Table 4] The baseline values are reported as 0.964/0.038/0.998 in Table 1 but 0.9637/0.0382/0.9978 in Table 4; please reconcile the rounding consistently in the text and table captions.
[Section D.4] Since C≈6ND counts D in cells rather than tokens, the constant offset is acknowledged in the Limitations, but stating explicitly in Section D.4 that this constant factor does not change the frontier exponents would avoid reader confusion.

Circularity Check

0 steps flagged · score 1.0 of 10

No material circularity: the scaling-law fit and held-out predictions are genuine out-of-sample estimates; the only self-citation is non-load-bearing, and the RQ-VAE split concern is a validity issue, not circularity.

full rationale

The central derivation chain is not circular. The paper takes the Hoffmann et al. (2022) two-term functional form as an external parametric assumption, fits its five parameters by nonlinear least squares to 22 measured grid losses, reports residuals, and then checks generalization on four runs explicitly excluded from the fit. Nothing in Eq. 2 is stipulated to equal the fitted losses by construction, and the four held-out predictions are genuine extrapolation/interpolation checks rather than refits of excluded points. The compute-optimal frontier of Eq. 9 is the analytic inversion of the fitted law, which is standard use of a fitted scaling law rather than a separate first-principles claim being derived from itself. The only self-citation is to REAL for the mean-pooling mechanism used to form cell-line embeddings (Section 3.4 and Appendix A.2); that mechanism is not load-bearing for the scaling-law result, and the paper does not rely on a self-citation to justify the exponents or the frontier. One non-circular data-hygiene concern should be flagged: Appendix A.1 states the RQ-VAE tokenizer is trained on the scBaseCount atlas and frozen, while Appendix B introduces the 20% validation split only in the context of transformer runs and does not state that tokenizer training excluded validation cells. If the tokenizer saw those cells, L(N,D) would be partly in-sample, potentially biasing the fitted exponents; however, this is a contamination/validity risk, not a circularity, because no claimed result is equal to its inputs by construction.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on five fitted constants, the assumed parametric form, a compute proxy, and the integrity of the held-out split. There are no invented physical entities. The RQ-VAE and transformer are compositions of published components.

free parameters (5)

E = 2.90 nats
Irreducible-loss floor in the fitted scaling law, Eq. 2. It is estimated from the same 22 grid points and absorbs most of the loss magnitude.
A = 2.05e4
Coefficient of the model-size power-law term in Eq. 2, fitted to the grid.
alpha = 0.81
Exponent for model size in Eq. 2. Determines the compute-optimal frontier along with beta; no confidence interval is reported.
B = 4.28e3
Coefficient of the data power-law term in Eq. 2, fitted to the grid.
beta = 0.57
Exponent for data in Eq. 2. Its positive value is the central data-scaling claim; no confidence interval is reported.

assumptions (4)

domain assumption The Chinchilla two-term parametric form L(N,D)=E+A/N^alpha+B/D^beta (Eq. 1) is assumed to describe the loss surface of this architecture.
Adopted from Hoffmann et al. (2022) as the fitting model; the paper does not test alternative functional forms. Invoked in Section 4.4 and Appendix D.3.
domain assumption Compute is approximated as C=6ND with D counted in cells rather than tokens.
Stated in Appendix D.4 and acknowledged in Appendix F as offsetting FLOP counts by a roughly constant factor. Affects the absolute location of the compute-optimal frontier, not the exponents.
domain assumption Proportional pool sampling keeps corpus composition fixed and prevents memorization, so validation loss tracks training loss.
Appendix B and E.2 motivate this via a control run with uniform sampling that overfits. The scaling grid relies on this to interpret D as data quantity rather than repeated memorization.
domain assumption The frozen RQ-VAE is a fixed, lossy interface whose reconstruction error bounds achievable fidelity.
Section 3.4 and Appendix A.5 use this to separate the transformer's contribution from tokenizer error. Assumes the tokenizer does not leak validation information, which is not stated.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Scaling an Autoregressive Transformer for Single-Cell Generation." pith.science (2026). https://pith.science/paper/EHQSTJGW

@misc{pith2026260802961,
  author       = {Pith},
  title        = {Pith review of: Scaling an Autoregressive Transformer for Single-Cell Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EHQSTJGW}},
  note         = {Machine review of arXiv:2608.02961}
}

read the original abstract

We study a self-supervised generation task for single-cell gene expression vectors: given a set of vectors from a cell type, we aim to generate additional gene expression vectors of that cell type. For this task we characterize both the biological fidelity of the generated gene expression vectors and the scaling behavior of the pretraining loss. The model is a causal transformer paired with a learned quantized VAE tokenizer, trained with a cross-entropy loss. To evaluate the model, we condition it on held-out gene expression vectors of a cell type and generate vectors of gene expression, comparing the resulting distribution over gene expression vectors to the ground truth distribution of that cell type. We study the scaling properties of the proposed architecture by varying the number of trained parameters and the amount of training data. To our knowledge, we find the first jointly-fit two-exponent scaling law and compute-optimal frontier for a single-cell foundation model. Finally, we discuss how this pretrained model could be finetuned for perturbation response prediction.

Figures

Figures reproduced from arXiv: 2608.02961 by the authors.

**Figure 1.** Evaluation of single-cell gene expression vector generation quality metrics. Each panel plots a generation-quality [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗

**Figure 2.** Fitted scaling surface for the self-supervised pretraining loss. Iso-loss contours of Eq. 2 are shown in the [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** Distribution of log-residuals log Lˆ − log L of the Approach-3 fit over the 22 grid points. Residuals are small (≲ 2%) and roughly centered on zero, with no systematic structure. analogue of Section B) and substituting D = C/6N into Eq. 1, the loss-minimizing model size at fixed compute has the closed form N ∗ (C) = αA βB 1 α+β C 6 β α+β , (9) equivalently the locus where the marginal returns to N and D bala… view at source ↗

Figures from the paper (2 more)

**Figure 5.** Figure 5: Two losses from the same pretraining run, repre [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

**Figure 1.** Figure 1: generation quality for the 4.43M model as its pretraining validation loss falls over training. F Limitations Our conclusions are bounded by the extent of the grid and by the objective we fit. The parameter axis spans 1.3–83.9M, well below the largest models in [PITH_…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

14 extracted references · 7 canonical work pages

[1]

Adduri, A. K.; Gautam, D.; Bevilacqua, B.; Imran, A.; Shah, R.; Naghipourfar, M.; Teyssier, N.; Ilango, R.; Na- garaj, S.; Dong, M.; Ricci-Tam, C.; Carpenter, C.; Sub- ramanyam, V.; Winters, A.; Tirukkovular, S.; Sullivan, J.; Plosky, B. S.; Eraslan, B.; Youngblut, N. D.; Leskovec, J.; Gilbert,L.A.;Konermann,S.;Hsu,P.D.;Dobin,A.;Burke, D.P.;Goodarzi,H.;an...

work page 2025
[7]

Scaling Laws for Masked-Reconstruction Transformers on Single-Cell Transcriptomics

Scaling laws for masked- reconstruction transformers on single-cell transcriptomics. arXiv:2602.15253. Lee, D.; Kim, C.; Kim, S.; Cho, M.; and Han, W.-S

work page Pith review arXiv
[8]

Lopez, R.; Regier, J.; Cole, M

Cell2Sentence: Teaching large language modelsthelanguageofbiology.bioRxiv2023.11.08.566117. Lopez, R.; Regier, J.; Cole, M. B.; Jordan, M. I.; and Yosef, N.2018. Deepgenerativemodelingforsingle-celltranscrip- tomics.Nature Methods, 15: 1053–1058. Lotfollahi, M.; Susmelj, A. K.; Donno, C. D.; Roohani, Y.; and Theis, F. J

work page 2018
[10]

arXiv:2302.13971

LLaMA: Open and efficient foundation language models. arXiv:2302.13971. Wang, C.; Karimzadeh, M.; Ravindra, N. G.; Bounds, L. R.; Alerasool,N.;Huang,A.C.;Ma,S.;Gulbranson,D.R.;Cui, H.;Lee,Y.;...,A.A.;MacKrell,E.J.;Wilken,M.S.;Chen,J.; Herken,B.W.;Weber,J.A.;Onesto,M.M.;Gonzalez-Teran, B.;Leung,N.F.;Shi,S.Y.;Smith,B.J.;Lam,S.K.;Barner, A.;Wright,P.;Rumsey,...

arXiv 2026
[11]

PRiMeFlow: Capturing Complex Expression Heterogeneity in Perturbation Response Modelling

PRiMeFlow: Capturing complex expression heterogeneity in perturbation response modelling. arXiv:2604.13986. Zhang, H.; Yuan, G.-H.; Yuan, C.; Xu, T.; Bian, T.; Cheng, H.; Huang, W.; Zhao, D.; and Rong, Y

work page Pith review arXiv
[12]

arXiv:2603.25240

Lingshu- Cell: A Generative Cellular World Model for Transcriptome Modeling Toward Virtual Cells. arXiv:2603.25240. Zhang, H.; Zhao, X.; Molybog, I.; and Zhang, J

arXiv
[14]

Formally, writeC= 131for the number of cell types and G= 18,080for the number of genes

—1when each generated population’s mean vector is nearest the em- piricalmeanvectorofitsowncelltypeamongalltypes,and ≈0.5under random guessing — which measures whether generation is cell-type-specific rather than mode-collapsed across types. Formally, writeC= 131for the number of cell types and G= 18,080for the number of genes. For cell typeclet ¯xc∈R G b...

work page 2026
[2020]

arXiv:2001.08361

Scaling laws for neural language models. arXiv:2001.08361. Kendiukhov, I

arXiv 2001

Show all 14 references

[2021]

arXiv:2112.11446

Scaling language models: Methods, analysis & insights from training Gopher. arXiv:2112.11446. Rizvi,S.A.;Levine,D.;Patel,A.;Zhang,S.;Wang,E.;Perry, C. J.; Vrkic, I.; Constante, N. M.; Fu, Z.; He, S.; Zhang, D.; Tang,C.;Lyu,Z.;Darji,R.;Li,C.;Sun,E.;Jeong,D.;Zhao, L.; Kwan, J.; ...

arXiv
[2022]

arXiv:2203.15556

Training compute-optimal large language models. arXiv:2203.15556. Kaplan, J.; McCandlish, S.; Henighan, T.; Brown, T. B.; Chess, B.; Child, R.; Gray, S.; Radford, A.; Wu, J.; and Amodei, D

arXiv
[2023]

arXiv:2312.00785

Sequential modeling enables scalable learning for large vision models. arXiv:2312.00785. Chen, H.; Venkatesh, M. S.; Ortega, J. G.; Mahesh, S. V.; Nandi, T. N.; Madduri, R. K.; Pelka, K.; and Theodoris, C. V

arXiv
[2024]

arXiv:2409.17169

REAL: Response Embedding-based Alignment for LLMs. arXiv:2409.17169. A Method Ourmodelhastwocomponents,trainedinsequenceandthen composed: (i) a RQ-VAE that learns a discrete tokenizer for single-cell expression profiles, and (ii) a causal transformer that models sequences of t...

2022 arXiv
[2025]

bioRxiv 2025.10.23.683759

Tahoe-x1: scaling perturbation-trained single- cell foundation models to 3 billion parameters. bioRxiv 2025.10.23.683759. Hoffmann, J.; Borgeaud, S.; Mensch, A.; Buchatskaya, E.; Cai, T.; Rutherford, E.; de Las Casas, D.; Hendricks, L. A.; Welbl, J.; Clark, A.; Hennigan, T.; N...

2025
[2026]

bioRxiv 2026.02.04.703804

Virtual cells need context, not just scale. bioRxiv 2026.02.04.703804. Gandhi, S.; Javadi, F.; Svensson, V.; Khan, U.; Jones, M. G.; Yu, J.; Merico, D.; Goodarzi, H.; and Alidoust, N

2026

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Adduri, A. K.; Gautam, D.; Bevilacqua, B.; Imran, A.; Shah, R.; Naghipourfar, M.; Teyssier, N.; Ilango, R.; Na- garaj, S.; Dong, M.; Ricci-Tam, C.; Carpenter, C.; Sub- ramanyam, V.; Winters, A.; Tirukkovular, S.; Sullivan, J.; Plosky, B. S.; Eraslan, B.; Youngblut, N. D.; Leskovec, J.; Gilbert,L.A.;Konermann,S.;Hsu,P.D.;Dobin,A.;Burke, D.P.;Goodarzi,H.;an...

work page 2025

[2] [7]

Scaling Laws for Masked-Reconstruction Transformers on Single-Cell Transcriptomics

Scaling laws for masked- reconstruction transformers on single-cell transcriptomics. arXiv:2602.15253. Lee, D.; Kim, C.; Kim, S.; Cho, M.; and Han, W.-S

work page Pith review arXiv

[3] [8]

Lopez, R.; Regier, J.; Cole, M

Cell2Sentence: Teaching large language modelsthelanguageofbiology.bioRxiv2023.11.08.566117. Lopez, R.; Regier, J.; Cole, M. B.; Jordan, M. I.; and Yosef, N.2018. Deepgenerativemodelingforsingle-celltranscrip- tomics.Nature Methods, 15: 1053–1058. Lotfollahi, M.; Susmelj, A. K.; Donno, C. D.; Roohani, Y.; and Theis, F. J

work page 2018

[4] [10]

arXiv:2302.13971

LLaMA: Open and efficient foundation language models. arXiv:2302.13971. Wang, C.; Karimzadeh, M.; Ravindra, N. G.; Bounds, L. R.; Alerasool,N.;Huang,A.C.;Ma,S.;Gulbranson,D.R.;Cui, H.;Lee,Y.;...,A.A.;MacKrell,E.J.;Wilken,M.S.;Chen,J.; Herken,B.W.;Weber,J.A.;Onesto,M.M.;Gonzalez-Teran, B.;Leung,N.F.;Shi,S.Y.;Smith,B.J.;Lam,S.K.;Barner, A.;Wright,P.;Rumsey,...

arXiv 2026

[5] [11]

PRiMeFlow: Capturing Complex Expression Heterogeneity in Perturbation Response Modelling

PRiMeFlow: Capturing complex expression heterogeneity in perturbation response modelling. arXiv:2604.13986. Zhang, H.; Yuan, G.-H.; Yuan, C.; Xu, T.; Bian, T.; Cheng, H.; Huang, W.; Zhao, D.; and Rong, Y

work page Pith review arXiv

[6] [12]

arXiv:2603.25240

Lingshu- Cell: A Generative Cellular World Model for Transcriptome Modeling Toward Virtual Cells. arXiv:2603.25240. Zhang, H.; Zhao, X.; Molybog, I.; and Zhang, J

arXiv

[7] [14]

Formally, writeC= 131for the number of cell types and G= 18,080for the number of genes

—1when each generated population’s mean vector is nearest the em- piricalmeanvectorofitsowncelltypeamongalltypes,and ≈0.5under random guessing — which measures whether generation is cell-type-specific rather than mode-collapsed across types. Formally, writeC= 131for the number of cell types and G= 18,080for the number of genes. For cell typeclet ¯xc∈R G b...

work page 2026

[8] [2020]

arXiv:2001.08361

Scaling laws for neural language models. arXiv:2001.08361. Kendiukhov, I

arXiv 2001