REVIEW 3 major objections 5 minor 35 references
Multimodal Modeling of CRISPR-Cas12 Activity Using Foundation Models and Chromatin Accessibility Data
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Frozen RNA foundation-model embeddings, a light CNN, and an optional chromatin-accessibility flag reach Spearman 0.78 for CRISPR-Cas12 gRNA activity, beating the quoted DeepCpf1 score of 0.71.
desk verdict Frozen RNA-FM embeddings beat published DeepCpf1 numbers on Cas12 gRNA activity, but the margin is unverified because the baselines were quoted, not re-run. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing setup is probing: frozen RNA-FM embeddings of 20-, 34-, or 50-nucleotide windows around the cleavage site are fed into a compact CNN regressor copied from the DeepCpf1 architecture. For the multimodal variant, a binary chromatin-accessibility label (derived by aligning ATAC-seq reads, calling peaks, and intersecting with the guide's genomic locus) is projected by an MLP and combined with the sequence features by element-wise multiplication before the regression head.
What would settle it
Re-run DeepCpf1 and the linear/boosting baselines in the same codebase, with the same windows, strand handling, HT1-1/HT1-2 split, and Spearman evaluation as Cas-FM; if DeepCpf1 then reports a score at or above 0.76, the claimed foundation-model advantage reduces to a preprocessing artifact.
Extended reading notes
Core claim
The central discovery is that embeddings from RNA-FM, a transformer pretrained on transcriptomes, transfer to CRISPR-Cas12 gRNA activity prediction without any gRNA-specific fine-tuning or pretraining. With the same DeepCpf1-style convolutional head, Cas-FM (RNA-FM) scores 0.76 Spearman on the HT1-2 test split and 0.78 when a binarized ATAC-seq accessibility label is projected and multiplied into the sequence feature map; the quoted DeepCpf1 baseline is 0.71. The paper also finds that a DNA-pretrained transformer, DNABERT-2, underperforms at 0.49, which it interprets as showing that domain alignment—RNA rather than DNA pretraining—is what carries the advantage.
Load-bearing premise
The load-bearing premise is that the published baseline numbers were computed under the same sequence windows, strand conventions, train/test split, and Spearman metric as the new Cas-FM runs, since the paper quotes those baselines directly instead of re-running them in its own pipeline.
Editorial extensions
If this is right
- Frozen RNA-FM embeddings alone outperform every non-FM baseline the paper compares against, with a 0.10 Spearman margin over the best handcrafted-feature baseline (0.76 vs 0.66).
- Adding the binarized chromatin accessibility label improves the same architecture from Spearman 0.76 to 0.78.
- A 34-nucleotide sequence window is the best input length; both the 20-nt core guide and the 50-nt extended window underperform it.
- The underperformance of DNABERT-2 (0.49) implies that choosing a foundation model whose pretraining domain matches the gRNA modality matters more than the mere availability of a large FM.
- The curated ATAC-seq accessibility annotations for the roughly 15,000 training guides and 1,290 test guides are released as a new resource for multimodal gRNA modeling.
Reading between the lines
- If the margin over DeepCpf1 survives a same-pipeline re-run, the probing recipe should transfer to other CRISPR nucleases, letting a new Cas variant be scored without a new large-scale screen.
- The 34-nucleotide optimum may be an artifact of RNA-FM's tokenization or positional encoding rather than a biological boundary; comparing several RNA transformers on identical windows would separate the two explanations.
- The multimodal branch currently reduces accessibility to one bit at a fixed signal threshold; using the continuous ATAC signal or the distance to the nearest peak is a direct extension that might extract more of the orthogonal signal.
- The wide gap between RNA-FM and DNABERT-2 predicts that transcriptome-trained foundation models will generalize to RNA-guided nuclease tasks better than genome-trained ones, independent of model scale.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Cas-FM, a framework that uses frozen embeddings from pre-trained RNA (RNA-FM) and DNA (DNABERT-2) foundation models as input to a CNN regression head for predicting CRISPR-Cas12 gRNA activity. It further integrates a binary chromatin accessibility label derived from HEK293T ATAC-seq data via an MLP and element-wise multiplication. Evaluated on the Kim et al. (2018a) Cpf1 dataset with the HT1-1/HT1-2 split, the authors report Spearman correlations of 0.76 for Cas-FM (RNA-FM) and 0.78 with chromatin accessibility (Cas-FM-CA), compared with 0.71 for DeepCpf1 and lower values for other baselines quoted from the original paper. They also study context window lengths (20/34/50 nt).
Significance. If the reported gains are genuine, the result would be a useful demonstration that transcriptomic foundation models transfer to gRNA activity prediction without domain-specific pre-training, and that chromatin accessibility adds complementary signal. The probing framework is clean and the paper is clearly written. The curated ATAC-seq alignment for the gRNA loci is a modest but potentially reusable resource. However, the central empirical claim is currently unsupported: all baseline numbers are quoted from the original paper without re-running in the same pipeline, and no uncertainty quantification is provided for the Cas-FM correlations. The claimed margins (0.05–0.07 over DeepCpf1) are small relative to likely preprocessing and sampling variability.
major comments (3)
- [Section 4, Table 1] The central claim that Cas-FM (RNA-FM) 'substantially outperforms' DeepCpf1 is not supported by the comparison as presented, because the baseline values are 'directly taken from (Kim et al., 2018a)' rather than computed in the same pipeline. The manuscript does not verify that the input sequence windows (34-nt/50-nt contexts vs. DeepCpf1's 34-bp window), strand/PAM handling, ATAC-seq thresholding, and train/test splits are identical. In particular, Cas-FM is evaluated with Spearman rank correlation (Appendix A.4), whereas DeepCpf1's reported 0.71 in the original publication is a Pearson correlation; if so, the 0.05–0.07 gap is a metric mismatch. The authors should re-run all baselines on the same preprocessed inputs and report the same correlation metric for all methods.
- [Section 4, Table 1 and Figure 2] All reported correlations are single point estimates with no error bars, confidence intervals, or significance tests. With 1,290 test guides and one split, the 0.76 vs. 0.71 difference over DeepCpf1 could be within sampling noise. The authors should report bootstrap confidence intervals or standard errors across multiple training seeds, and, after re-running DeepCpf1, a paired comparison.
- [Section 3.2 and Appendix A.2] The claimed improvement from chromatin accessibility (0.78 vs. 0.76) rests on a binary label derived from a single normalized-signal threshold of 0.001, with no sensitivity analysis. Since the CA contribution is a stated contribution, the authors should show that the improvement is robust to the threshold choice and that the binary label adds signal beyond the sequence embeddings (e.g., an ablation with the continuous signal or with the CA label as an additional input feature).
minor comments (5)
- [Section 5] The sentence 'Finally, we introduce a curated dataset of demonstrate of chromatin accessibility features' contains a typo ('of demonstrate of') that should be corrected.
- [Appendix A.2] The paragraph beginning 'The raw data were preprocessed through a standard ATAC-seq pipeline' is duplicated verbatim, which should be fixed.
- [Appendix A.2] BLAT is cited as '(?)' with no reference; a proper citation should be added.
- [Figure 2] The y-axis label 'Performance' is vague; it should specify 'Spearman rank correlation' and ideally include error bars.
- [Section 3.1] The exact definitions of the 20-, 34-, and 50-nt contexts are not given; the authors should specify coordinates relative to the PAM or cleavage site to ensure reproducibility.
Circularity Check
No circular derivation: frozen RNA-FM embeddings plus a supervised CNN head on a held-out external split; minor non-load-bearing self-citations only.
full rationale
I found no circular step in which a predicted quantity is equivalent by construction to an input or fitted parameter. The Cas-FM models use RNA-FM/DNABERT-2 embeddings from frozen backbones, and the CNN regression head is trained on the HT1-1 split and evaluated on HT1-2, the same external split used by Kim et al. (2018a). The headline gains over DeepCpf1 are against baseline numbers 'directly taken from (Kim et al., 2018a)', which is an external benchmark, not a parameter fitted in this paper; whether those numbers are strictly comparable in preprocessing is a correctness/comparability risk, not circularity. The chromatin-accessibility modality is an independent ATAC-seq-derived binary feature, and its contribution is measured by comparing Cas-FM-CA to the sequence-only model under the same training protocol. The only self-citations (Yazdani-Jahromi et al., 2024; Prakash et al., 2024) are used to name the probing setup and to support the choice of RNA-FM/DNABERT-2, but the reported Spearman values do not depend on those citations; they are computed directly from the external Kim et al. data with a standard supervised split. Hence no load-bearing circular step.
Assumptions & free parameters
free parameters (2)
- ATAC-seq normalized signal threshold =
0.001
- gRNA sequence context window length =
34 nucleotides
assumptions (4)
- domain assumption RNA-FM embeddings are informative for short, out-of-distribution gRNA sequences that the model was not pretrained on.
- domain assumption The HEK293T ATAC-seq sample (GSM2902624) captures the chromatin state relevant to the Kim et al. HT1 gRNA activity experiment.
- domain assumption BLAT alignment of gRNA sequences to hg38 correctly identifies the intended target loci.
- domain assumption Baseline correlations quoted from Kim et al. 2018 are directly comparable to the Cas-FM results.
Cite this review
Pith. "Pith review of Multimodal Modeling of CRISPR-Cas12 Activity Using Foundation Models and Chromatin Accessibility Data." pith.science (2026). https://pith.science/paper/ALWP7RGQ
@misc{pith2026250611182,
author = {Pith},
title = {Pith review of: Multimodal Modeling of CRISPR-Cas12 Activity Using Foundation Models and Chromatin Accessibility Data},
year = {2026},
howpublished = {\url{https://pith.science/paper/ALWP7RGQ}},
note = {Machine review of arXiv:2506.11182}
}
read the original abstract
Predicting guide RNA (gRNA) activity is critical for effective CRISPR-Cas12 genome editing but remains challenging due to limited data, variation across protospacer adjacent motifs (PAMs-short sequence requirements for Cas binding), and reliance on large-scale training. We investigate whether pre-trained biological foundation model originally trained on transcriptomic data can improve gRNA activity estimation even without domain-specific pre-training. Using embeddings from existing RNA foundation model as input to lightweight regressor, we show substantial gains over traditional baselines. We also integrate chromatin accessibility data to capture regulatory context, improving performance further. Our results highlight the effectiveness of pre-trained foundation models and chromatin accessibility data for gRNA activity prediction.
Figures
Reference graph
Works this paper leans on
-
[1]
Pre-trained language models for biological sequences at the scale of the human genome
Brandes, N., Ofer, D., Peleg, Y., Rappoport, N., and Linial, M. Pre-trained language models for biological sequences at the scale of the human genome. Nature Communications, 13 0 (1): 0 1--13, 2022
work page 2022
-
[2]
G., Ku, J., Poli, M., Brockman, G., Chang, D., Gonzalez, G
Brixi, G., Durrant, M. G., Ku, J., Poli, M., Brockman, G., Chang, D., Gonzalez, G. A., King, S. H., Li, D. B., Merchant, A. T., et al. Genome modeling and design across all domains of life with evo 2. BioRxiv, pp.\ 2025--02, 2025
2025
-
[3]
Rna foundation model enables prediction of transcriptome and structural dynamics
Chen, J., Hou, T., Zhang, B., Zhou, X., Zhang, X., and Liu, X. Rna foundation model enables prediction of transcriptome and structural dynamics. Nature Machine Intelligence, 4 0 (12): 0 1046--1058, 2022
work page 2022
-
[4]
Chen, K., Zhou, Y., Ding, M., Wang, Y., Ren, Z., and Yang, Y. Self-supervised learning on millions of pre-mrna sequences improves sequence-based rna splicing prediction. BioRxiv, pp.\ 2023--01, 2023
work page 2023
-
[5]
Chen, T. and Guestrin, C. Xgboost: A scalable tree boosting system. pp.\ 785--794, 2016
work page 2016
-
[6]
H., Oteri, F., Dallago, C., Trop, E., de Almeida, B
Dalla-Torre, H., Gonzalez, L., Mendoza-Revilla, J., Lopez Carranza, N., Grzywaczewski, A. H., Oteri, F., Dallago, C., Trop, E., de Almeida, B. P., Sirelkhatim, H., et al. Nucleotide transformer: building and evaluating robust foundation models for human genomics. Nature Methods, 22 0 (2): 0 287--297, 2025
work page 2025
-
[7]
G., Fusi, N., Sullender, M., Hegde, M., Vaimberg, E
Doench, J. G., Fusi, N., Sullender, M., Hegde, M., Vaimberg, E. W., Donovan, K. F., Smith, I., Tothova, Z., Wilen, C., Orchard, R., et al. Optimized sgrna design to maximize activity and minimize off-target effects of crispr-cas9. Nature Biotechnology, 34 0 (2): 0 184--191, 2016
work page 2016
-
[8]
Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., et al. Prottrans: Towards cracking the language of life’s code through self-supervised deep learning and high performance computing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021
work page 2021
Show all 35 references
-
[9]
Deeper learning of human regulatory variants from large-scale genome sequencing data
Frazer, J., Notin, P., Dias, M., Gomez, A., Min, J., Brock, K., Gal, Y., Birney, E., and Lunter, G. Deeper learning of human regulatory variants from large-scale genome sequencing data. Nature Genetics, 53 0 (8): 0 1141--1149, 2021
2021
-
[10]
S., Wang, X., Li, H., and Zhou, J
Gao, R., Wang, Y., Shah, P., Zeng, Z., Liu, S.-W., Xie, J., Freemont, P. S., Wang, X., Li, H., and Zhou, J. Unfolded rna structures using deep learning reveal sequence-specific mechanisms of antisense oligonucleotide activity. Bioinformatics, 38 0 (12): 0 3231--3239, 2022
2022
-
[11]
Hoerl, A. E. and Kennard, R. W. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12 0 (1): 0 55--67, 1970
1970
-
[12]
D., Scott, D
Hsu, P. D., Scott, D. A., Weinstein, J. A., Ran, F. Z., Konermann, S., Agarwala, V., Li, Y., Fine, E. J., Wu, X., Shalem, O., et al. Dna targeting specificity of rna-guided cas9 nucleases. Nature Biotechnology, 31 0 (9): 0 827--832, 2013
2013
-
[13]
Ji, Y., Zhou, Z., Liu, H., and Davuluri, R. V. Dnabert: pre-trained bidirectional encoder representations from transformers model for dna-language in genome. Bioinformatics, 37 0 (15): 0 2112--2120, 2021
2021
-
[14]
K., Song, M., Lee, J., Menon, A
Kim, H. K., Song, M., Lee, J., Menon, A. V., Jung, S., Kang, Y.-M., Choi, J. W., Woo, E., Koh, H. C., Nam, J.-W., et al. In vivo high-throughput profiling of crispr--cpf1 activity. Nature methods, 14 0 (2): 0 153--159, 2017
2017
-
[15]
K., Min, S., Song, M., Jung, S., Choi, J
Kim, H. K., Min, S., Song, M., Jung, S., Choi, J. W., Kim, Y., Lee, S., Yoon, S., and Kim, H. H. Deep learning improves prediction of crispr--cpf1 guide rna activity. Nature biotechnology, 36 0 (3): 0 239--241, 2018 a
2018
-
[16]
W., Kim, Y.-H., Lee, S., Yoon, S., and Kim, H
Kim, H.-K., Min, S., Song, M., Jung, S., Choi, J. W., Kim, Y.-H., Lee, S., Yoon, S., and Kim, H. Deep learning improves prediction of crispr--cpf1 guide rna activity. Nature Biotechnology, 36 0 (3): 0 239--241, 2018 b
2018
-
[17]
P., Tsai, S
Kleinstiver, B. P., Tsai, S. Q., Prew, M. S., Nguyen, N. T., Welch, M. M., Lopez, J. M., McCaw, Z. R., Aryee, M. J., and Joung, J. K. Genome-wide specificities of crispr-cpf1 nucleases in human cells. Nature Biotechnology, 34 0 (8): 0 869--874, 2016
2016
-
[18]
Crafting papers on machine learning
Langley, P. Crafting papers on machine learning. In Langley, P. (ed.), Proceedings of the 17th International Conference on Machine Learning (ICML 2000), pp.\ 1207--1216, Stanford, CA, 2000. Morgan Kaufmann
2000
-
[19]
and Salzberg, S
Langmead, B. and Salzberg, S. L. Fast gapped-read alignment with bowtie 2. Nature methods, 9 0 (4): 0 357--359, 2012
2012
-
[20]
Evolutionary-scale prediction of atomic-level protein structure with a language model
Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science, 379 0 (6637): 0 1123--1130, 2023
2023
-
[21]
A., Vejnar, C
Moreno-Mateos, M. A., Vejnar, C. E., Beaudoin, J.-D., Fernandez, J. P., Mis, E. K., Khokha, M. K., and Giraldez, A. J. Crisprscan: designing highly efficient sgrnas for crispr-cas9 targeting in vivo. Nature Methods, 12 0 (10): 0 982--988, 2015
2015
-
[22]
G., Kang, B., Katrekar, D., Li, D
Nguyen, E., Poli, M., Durrant, M. G., Kang, B., Katrekar, D., Li, D. B., Bartie, L. J., Thomas, A. W., King, S. H., Brixi, G., et al. Sequence modeling and design from molecular to genome scale with evo. Science, 386 0 (6723): 0 eado9336, 2024
2024
-
[23]
J., Vla s i \'c , T., Huber, R
Peni \'c , R. J., Vla s i \'c , T., Huber, R. G., Wan, Y., and S iki \'c , M. Rinalmo: General-purpose rna language models can generalize well on structure prediction tasks. arXiv preprint arXiv:2403.00043, 2024
2024 arXiv
-
[24]
A., Combs, S., Mansi, T., Scheer, J., and Liao, R
Prakash, M., Moskalev, A., DiMaggio, P. A., Combs, S., Mansi, T., Scheer, J., and Liao, R. Bridging biomolecular modalities for knowledge transfer in bio-language models. bioRxiv, pp.\ 2024--10, 2024
2024
-
[25]
Quinlan, A. R. and Hall, I. M. Bedtools: a flexible suite of utilities for comparing genomic features. Bioinformatics, 26 0 (6): 0 841--842, 2010
2010
-
[26]
Regression shrinkage and selection via the lasso
Tibshirani, R. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58 0 (1): 0 267--288, 1996
1996
-
[27]
A., Wu, Q., Wu, D., Cong, L., Zhang, F., Liu, X
Xu, H., Xiao, T., Chen, C.-H., Li, W., Meyer, C. A., Wu, Q., Wu, D., Cong, L., Zhang, F., Liu, X. S., et al. Sequence determinants of improved crispr sgrna design. Genome Research, 25 0 (8): 0 1147--1157, 2015
2015
-
[28]
Deciphering 3'utr mediated gene regulation using interpretable deep representation learning
Yang, Y., Li, G., Pang, K., Cao, W., Zhang, Z., and Li, X. Deciphering 3'utr mediated gene regulation using interpretable deep representation learning. Advanced Science, 11 0 (39): 0 2407013, 2024
2024
-
[29]
Helm: Hierarchical encoding for mrna language modeling
Yazdani-Jahromi, M., Prakash, M., Mansi, T., Moskalev, A., and Liao, R. Helm: Hierarchical encoding for mrna language modeling. arXiv preprint arXiv:2410.12459, 2024
2024 arXiv
-
[30]
S., Abudayyeh, O
Zetsche, B., Gootenberg, J. S., Abudayyeh, O. O., Slaymaker, I. M., Makarova, K. S., Essletzbichler, P., Volz, S. E., Joung, J., van der Oost, J., Regev, A., et al. Cpf1 is a single rna-guided endonuclease of a class 2 crispr-cas system. Cell, 163 0 (3): 0 759--771, 2015
2015
-
[31]
A., Eeckhoute, J., Johnson, D
Zhang, Y., Liu, T., Meyer, C. A., Eeckhoute, J., Johnson, D. S., Bernstein, B. E., Nusbaum, C., Myers, R. M., Brown, M., Li, W., and Liu, X. S. Model-based analysis of chip-seq (macs). Genome biology, 9 0 (9): 0 R137, 2008
2008
-
[32]
grna-fm: Foundation models for crispr-cas9 grna representation
Zhou, Z., Ji, Y., et al. grna-fm: Foundation models for crispr-cas9 grna representation. bioRxiv, 2023. Preprint: https://doi.org/10.1101/2023.10.17.562702
2023 doi
-
[33]
V., and Liu, H
Zhou, Z., Ji, Y., Li, W., Dutta, P., Davuluri, R. V., and Liu, H. DNABERT -2: Efficient foundation model and benchmark for multi-species genomes. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=oMLQB4EZE1
2024
-
[34]
and Hastie, T
Zou, H. and Hastie, T. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67 0 (2): 0 301--320, 2005
2005
-
[35]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.