{"id":"db524466-7272-4569-a0dd-39fc1af218b6","arxiv_id":"2506.11182","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"RNA-FM embeddings, fed into a lightweight CNN with binarized ATAC-seq accessibility labels, outperform published Cas12 gRNA activity baselines on the Kim et al. HT1 test set.","lead":"This paper tests whether embeddings from pre-trained RNA and DNA foundation models can predict CRISPR-Cas12 guide RNA activity better than existing models. It reports that RNA-FM embeddings plus chromatin accessibility signals reach the highest correlation on a public Cas12 dataset, but no code or data is released.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed margin over DeepCpf1 rests on baseline numbers quoted from the original paper; until those baselines are re-run under identical windows, strand handling, splits, and metric, the 0.05–0.07 Spearman advantage is unverified.","rationale":"The reader's verdict (CONDITIONAL) is the right calibration: the paper's central claim is plausible and the RNA-FM vs DNABERT-2 comparison is likely robust, but the headline margin over DeepCpf1 is only as reliable as the quoted baseline. I could not find an internal contradiction that would justify REJECT; the issue is missing evidence at the comparison boundary. The reader's weakest_assumption names precisely the same comparability risk, so I agree. I would leave the verdict unchanged: conditional acceptance pending re-running the baselines under identical conditions and reporting variance.","tokens_in":9827,"tokens_out":8708,"duration_ms":104241,"concrete_test":"Download the public DeepCpf1 implementation and rerun it inside the authors' preprocessing code using the same HT1-1/HT1-2 split, the same 34-nt centered windows, the same strand/PAM orientation, and the same Spearman routine; also rerun at least the Ridge (L2) and Boosted RT baselines on those windows. If the reproduced DeepCpf1 Spearman is ≥0.75, or if the gap between Cas-FM (RNA-FM) and DeepCpf1 shrinks below ~0.02, the claimed superiority is not established. As a secondary check, bootstrap 1,000 resamples of HT1-2 or train with 5 seeds for Cas-FM and Cas-FM-CA to see whether the 0.76-vs-0.78 CA gain is within the noise band.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The manuscript's central, headline-forming comparison (Cas-FM (RNA-FM) 0.76 / Cas-FM-CA 0.78 vs DeepCpf1 0.71) is made entirely against baseline numbers that are 'directly taken from (Kim et al., 2018a)' rather than computed in the same pipeline. The paper states that the same HT1-1/HT1-2 splits are used, but it does not establish that the inputs match: Cas-FM consumes 34-/50-nt contexts through RNA-FM/DNABERT tokenizers, while the quoted DeepCpf1 number was generated with its own 34-bp one-hot input, its own PAM/strand handling, and a chromatin-accessibility feature that is not re-run here (the paper's ATAC-seq processing, threshold, and binary labeling may differ from the published baseline). Even a modest preprocessing or metric mismatch could produce a 0.05 Spearman shift, which is the entire claimed advantage. No confidence interval or seed variation is reported for the 0.76/0.78 test-set values, so the advantage over a point-estimate baseline is unquantified. This is a support/comparability problem rather than an internal contradiction.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Cas-FM, a framework that uses frozen embeddings from pre-trained RNA (RNA-FM) and DNA (DNABERT-2) foundation models as input to a CNN regression head for predicting CRISPR-Cas12 gRNA activity. It further integrates a binary chromatin accessibility label derived from HEK293T ATAC-seq data via an MLP and element-wise multiplication. Evaluated on the Kim et al. (2018a) Cpf1 dataset with the HT1-1/HT1-2 split, the authors report Spearman correlations of 0.76 for Cas-FM (RNA-FM) and 0.78 with chromatin accessibility (Cas-FM-CA), compared with 0.71 for DeepCpf1 and lower values for other baselines quoted from the original paper. They also study context window lengths (20/34/50 nt).","tokens_in":10057,"tokens_out":6219,"duration_ms":64610,"significance":"If the reported gains are genuine, the result would be a useful demonstration that transcriptomic foundation models transfer to gRNA activity prediction without domain-specific pre-training, and that chromatin accessibility adds complementary signal. The probing framework is clean and the paper is clearly written. The curated ATAC-seq alignment for the gRNA loci is a modest but potentially reusable resource. However, the central empirical claim is currently unsupported: all baseline numbers are quoted from the original paper without re-running in the same pipeline, and no uncertainty quantification is provided for the Cas-FM correlations. The claimed margins (0.05–0.07 over DeepCpf1) are small relative to likely preprocessing and sampling variability.","major_comments":[{"comment":"The central claim that Cas-FM (RNA-FM) 'substantially outperforms' DeepCpf1 is not supported by the comparison as presented, because the baseline values are 'directly taken from (Kim et al., 2018a)' rather than computed in the same pipeline. The manuscript does not verify that the input sequence windows (34-nt/50-nt contexts vs. DeepCpf1's 34-bp window), strand/PAM handling, ATAC-seq thresholding, and train/test splits are identical. In particular, Cas-FM is evaluated with Spearman rank correlation (Appendix A.4), whereas DeepCpf1's reported 0.71 in the original publication is a Pearson correlation; if so, the 0.05–0.07 gap is a metric mismatch. The authors should re-run all baselines on the same preprocessed inputs and report the same correlation metric for all methods.","section":"Section 4, Table 1"},{"comment":"All reported correlations are single point estimates with no error bars, confidence intervals, or significance tests. With 1,290 test guides and one split, the 0.76 vs. 0.71 difference over DeepCpf1 could be within sampling noise. The authors should report bootstrap confidence intervals or standard errors across multiple training seeds, and, after re-running DeepCpf1, a paired comparison.","section":"Section 4, Table 1 and Figure 2"},{"comment":"The claimed improvement from chromatin accessibility (0.78 vs. 0.76) rests on a binary label derived from a single normalized-signal threshold of 0.001, with no sensitivity analysis. Since the CA contribution is a stated contribution, the authors should show that the improvement is robust to the threshold choice and that the binary label adds signal beyond the sequence embeddings (e.g., an ablation with the continuous signal or with the CA label as an additional input feature).","section":"Section 3.2 and Appendix A.2"}],"minor_comments":[{"comment":"The sentence 'Finally, we introduce a curated dataset of demonstrate of chromatin accessibility features' contains a typo ('of demonstrate of') that should be corrected.","section":"Section 5"},{"comment":"The paragraph beginning 'The raw data were preprocessed through a standard ATAC-seq pipeline' is duplicated verbatim, which should be fixed.","section":"Appendix A.2"},{"comment":"BLAT is cited as '(?)' with no reference; a proper citation should be added.","section":"Appendix A.2"},{"comment":"The y-axis label 'Performance' is vague; it should specify 'Spearman rank correlation' and ideally include error bars.","section":"Figure 2"},{"comment":"The exact definitions of the 20-, 34-, and 50-nt contexts are not given; the authors should specify coordinates relative to the PAM or cleavage site to ensure reproducibility.","section":"Section 3.1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's main finding hinges on a single table comparing against external baseline numbers. In addition to requiring the authors to re-run baselines and add confidence intervals, the editor may wish to ask for the code and data to be released for reproducibility, since the paper does not mention code availability."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper shows that frozen RNA-FM embeddings plus a DeepCpf1-style CNN head reach Spearman 0.76 on Cas12 gRNA activity prediction, beating the published DeepCpf1 value of 0.71, and adding binarized ATAC-seq accessibility gets 0.78. That is a plausible and potentially useful result, but the margin over DeepCpf1 is not verified because the baseline numbers are quoted from the original paper rather than re-run in this pipeline.\n\nWhat is actually new: this is the first application of a frozen transcriptomic foundation model to Cas12 gRNA activity prediction. The RNA-FM versus DNABERT-2 comparison is internally consistent—both use the same head, splits, and evaluation—and the large gap (0.76 vs 0.49) is a genuinely interesting finding about domain alignment. The ATAC-seq curation and the observation that accessible-target guides have higher activity is a nice community resource, and the architecture is described in enough detail to reproduce.\n\nSoft spots, in order of importance. First, Table 1's baseline numbers are 'directly taken from Kim et al. 2018a'. That means no re-run under identical sequence windows, strand handling, train/test splits, or metric. Cas-FM uses 34- or 50-nt contexts through FM tokenizers; DeepCpf1's 0.71 was produced with its own 34-bp one-hot input and its own chromatin accessibility processing. A 0.05-0.07 Spearman shift could plausibly come from preprocessing, so the headline advantage is unquantified. Second, there are no confidence intervals, error bars, or seed variations for the 0.76/0.78 test values, so we don't know if the gap over DeepCpf1 is meaningful. Third, DNABERT-2's 0.49 is suspiciously low—worse than Lasso or Ridge—and the paper doesn't investigate why. That doesn't undermine the RNA-FM result, but it does raise a question about whether the FM comparison is fair. Minor: the appendix has duplicated paragraphs and a 'BLAT (?)' placeholder, and the Discussion has a garbled sentence ('curated dataset of demonstrate'). These are cosmetic but suggest a hurried draft.\n\nBottom line: this is a subfield-level contribution for CRISPR gRNA prediction and foundation-model probing. The central claim is plausible, non-circular, and reproducible in principle. It deserves a serious referee, but the referee should ask for re-runs of DeepCpf1 and the linear baselines under identical conditions, variance reporting, and code/data release. If the 0.05-0.07 advantage survives that, it's a solid result.","headline":"Frozen RNA-FM embeddings beat published DeepCpf1 numbers on Cas12 gRNA activity, but the margin is unverified because the baselines were quoted, not re-run.","tokens_in":10667,"tokens_out":3621,"would_cite":false,"duration_ms":33116,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Frozen RNA foundation-model embeddings, a light CNN, and an optional chromatin-accessibility flag reach Spearman 0.78 for CRISPR-Cas12 gRNA activity, beating the quoted DeepCpf1 score of 0.71.","keywords":["CRISPR-Cas12","guide RNA activity prediction","foundation models","RNA-FM","chromatin accessibility","ATAC-seq","transfer learning","probing"],"falsifier":"Re-run DeepCpf1 and the linear/boosting baselines in the same codebase, with the same windows, strand handling, HT1-1/HT1-2 split, and Spearman evaluation as Cas-FM; if DeepCpf1 then reports a score at or above 0.76, the claimed foundation-model advantage reduces to a preprocessing artifact.","tokens_in":9611,"feed_emoji":"🧬","tokens_out":11077,"duration_ms":102492,"temperature":0.7,"pith_summary":"Cas-FM's claim is that a general-purpose RNA foundation model can substitute for task-specific training data in CRISPR-Cas12 guide RNA scoring. On the standard 15,000-guide Cas12a dataset, frozen RNA-FM embeddings plus a small CNN reach Spearman 0.76 on the held-out HT1-2 split, above the 0.71 the paper quotes for DeepCpf1; adding a binary ATAC-seq-based chromatin accessibility label raises the score to 0.78. The paper treats this as evidence that transcriptomic pretraining captures gRNA-relevant sequence biology even though guide RNAs are short and outside the FM's training distribution. If the result holds, guide design for new Cas variants could be bootstrapped from off-the-shelf embeddings instead of large new screens, with regulatory context adding further signal.","feed_headline":"Frozen RNA embeddings beat CRISPR-Cas12 guide scoring at 0.78","feed_subtitle":"Frozen RNA embeddings plus chromatin-accessibility beat the cited DeepCpf1 score on 1,290 guides.","key_machinery":"The load-bearing setup is probing: frozen RNA-FM embeddings of 20-, 34-, or 50-nucleotide windows around the cleavage site are fed into a compact CNN regressor copied from the DeepCpf1 architecture. For the multimodal variant, a binary chromatin-accessibility label (derived by aligning ATAC-seq reads, calling peaks, and intersecting with the guide's genomic locus) is projected by an MLP and combined with the sequence features by element-wise multiplication before the regression head.","core_discovery":"The central discovery is that embeddings from RNA-FM, a transformer pretrained on transcriptomes, transfer to CRISPR-Cas12 gRNA activity prediction without any gRNA-specific fine-tuning or pretraining. With the same DeepCpf1-style convolutional head, Cas-FM (RNA-FM) scores 0.76 Spearman on the HT1-2 test split and 0.78 when a binarized ATAC-seq accessibility label is projected and multiplied into the sequence feature map; the quoted DeepCpf1 baseline is 0.71. The paper also finds that a DNA-pretrained transformer, DNABERT-2, underperforms at 0.49, which it interprets as showing that domain alignment—RNA rather than DNA pretraining—is what carries the advantage.","pith_inferences":["If the margin over DeepCpf1 survives a same-pipeline re-run, the probing recipe should transfer to other CRISPR nucleases, letting a new Cas variant be scored without a new large-scale screen.","The 34-nucleotide optimum may be an artifact of RNA-FM's tokenization or positional encoding rather than a biological boundary; comparing several RNA transformers on identical windows would separate the two explanations.","The multimodal branch currently reduces accessibility to one bit at a fixed signal threshold; using the continuous ATAC signal or the distance to the nearest peak is a direct extension that might extract more of the orthogonal signal.","The wide gap between RNA-FM and DNABERT-2 predicts that transcriptome-trained foundation models will generalize to RNA-guided nuclease tasks better than genome-trained ones, independent of model scale."],"forward_implications":["Frozen RNA-FM embeddings alone outperform every non-FM baseline the paper compares against, with a 0.10 Spearman margin over the best handcrafted-feature baseline (0.76 vs 0.66).","Adding the binarized chromatin accessibility label improves the same architecture from Spearman 0.76 to 0.78.","A 34-nucleotide sequence window is the best input length; both the 20-nt core guide and the 50-nt extended window underperform it.","The underperformance of DNABERT-2 (0.49) implies that choosing a foundation model whose pretraining domain matches the gRNA modality matters more than the mere availability of a large FM.","The curated ATAC-seq accessibility annotations for the roughly 15,000 training guides and 1,290 test guides are released as a new resource for multimodal gRNA modeling."],"supporting_citations":[{"why":"Supplies the HT1-2 dataset, the train/test split, and the baseline numbers quoted in Table 1, including the DeepCpf1 score the paper claims to beat.","marker":"(Kim et al., 2018a)"},{"why":"Provides the DeepCpf1 CNN architecture reused, unchanged, as the Cas-FM downstream regression head.","marker":"(Kim et al., 2018b)"},{"why":"Supplies the frozen RNA-FM transformer whose embeddings carry the main transfer-learning claim.","marker":"(Chen et al., 2022)"},{"why":"Supplies DNABERT-2, the contrasting DNA-pretrained backbone whose poor score underlines the domain-alignment argument.","marker":"(Zhou et al., 2024)"},{"why":"Alignment step in the ATAC-seq pipeline that produces the chromatin accessibility modality.","marker":"(Langmead & Salzberg, 2012)"},{"why":"Peak calling step used to define accessible regions for the binary CA labels.","marker":"(Zhang et al., 2008)"},{"why":"Intersection of gRNA loci with ATAC-seq peaks that assigns each guide its accessibility label.","marker":"(Quinlan & Hall, 2010)"}],"fun_headline_variants":["Cross-domain RNA embeddings lift CRISPR-Cas12 guide scores","ATAC-seq context pushes Cas12 scoring past DeepCpf1","RNA-FM beats DNA-BERT2 for Cas12 guide activity","Frozen RNA features top CRISPR-Cas12 prediction benchmarks","Chromatin accessibility gives Cas12 scoring an extra edge"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the published baseline numbers were computed under the same sequence windows, strand conventions, train/test split, and Spearman metric as the new Cas-FM runs, since the paper quotes those baselines directly instead of re-running them in its own pipeline.","fun_headline_variants_meta":{"raw":{"variants":["Cross-domain RNA embeddings lift CRISPR-Cas12 guide scores","ATAC-seq context pushes Cas12 scoring past DeepCpf1","RNA-FM beats DNA-BERT2 for Cas12 guide activity","Frozen RNA features top CRISPR-Cas12 prediction benchmarks","Chromatin accessibility gives Cas12 scoring an extra edge"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000617,"raw_usage":{"total_tokens":2800,"prompt_tokens":819,"completion_tokens":1981,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":435,"completion_tokens_details":{"reasoning_tokens":1897}},"tokens_in":435,"tokens_out":1981,"duration_ms":18682,"temperature":1.0,"reasoning_tokens":1897,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T04:16:13.039328+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run DeepCpf1 and the linear/boosting baselines in the same codebase, with the same windows, strand handling, HT1-1/HT1-2 split, and Spearman evaluation as Cas-FM; if DeepCpf1 then reports a score at or above 0.76, the claimed foundation-model advantage reduces to a preprocessing artifact.","supporting_citations":[{"cited_title":"Rna foundation model enables prediction of transcriptome and structural dynamics","cited_arxiv_id":null,"evidence_quote":"Supplies the frozen RNA-FM transformer whose embeddings carry the main transfer-learning claim."},{"cited_title":"V., and Liu, H","cited_arxiv_id":null,"evidence_quote":"Supplies DNABERT-2, the contrasting DNA-pretrained backbone whose poor score underlines the domain-alignment argument."},{"cited_title":"and Salzberg, S","cited_arxiv_id":null,"evidence_quote":"Alignment step in the ATAC-seq pipeline that produces the chromatin accessibility modality."},{"cited_title":"A., Eeckhoute, J., Johnson, D","cited_arxiv_id":null,"evidence_quote":"Peak calling step used to define accessible regions for the binary CA labels."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Intersection of gRNA loci with ATAC-seq peaks that assigns each guide its accessibility label."}],"review_version":1}