{"id":"a1e13b46-ba8f-4e2a-86ec-7a4a028446d6","arxiv_id":"2507.08542","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"CircFormerMoE is a transformer and mixture-of-experts model for detecting and pairing circular RNA splice sites in plant genomes, reporting high in-domain accuracy but lacking whole-genome and experimental validation.","lead":"A new deep learning model, CircFormerMoE, claims to spot circular RNA splice sites directly from plant DNA and then pair them into candidate circles. The paper reports very high accuracy on ten species, but the evaluation only tests windows centered on known sites, so the genome-wide discovery claim is unsubstantiated.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported 99.2% AP/AR is measured on 5001 bp windows centered on known splice sites; no whole-chromosome scan is evaluated, so the central genome-wide discovery claim is unsupported.","rationale":"The reader's weakest assumption identifies exactly the same load-bearing weakness: performance on evaluation windows centered on known splice sites is not shown to transfer to whole-chromosome scans. The paper's abstract and introduction claim end-to-end prediction directly from genomic DNA and de novo discovery of circRNAs, but the only quantitative evidence is windowed classification around annotated sites. This is not a dispute with the field's consensus; it is an internal gap between the stated capability and the experimental protocol. A single whole-genome scan with validation would settle the concern. Because the reader's REJECT verdict is warranted by this gap, and no additional concern is needed, the verdict should remain unchanged. The paper could be reconsidered if the authors supply genome-wide scanning results, validated novel circRNAs, and proper negative controls.","tokens_in":11224,"tokens_out":3124,"duration_ms":45365,"concrete_test":"Run CircFormerMoE with the specified 5001 bp sliding window and one-third overlap across at least one full chromosome (e.g., Glycine max chromosome 1), apply the SSD peak detector and SSP pairing, and compare predicted circRNA sites to PlantCircRNA annotations and RNA-seq back-splice evidence. If genome-wide precision/recall is substantially below the Table II/III centered-window metrics, the central discovery claim collapses.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section II.A states that Task 1 sequences are 'generated by truncating each splicing site and 2500 bases on both sides,' meaning every positive evaluation window has an annotated splice site at the center, while negatives are sampled from non-splicing regions. Table II and Fig. 4 therefore measure discrimination on centered windows, not genome-wide detection. The Discussion's proposed sliding-window inference is never run end-to-end on chromosomes, and no predicted circRNAs are validated by RNA-seq or RT-PCR. Consequently, the two headline claims—predicting circRNAs 'directly from plant genomic DNA' and 'discovering previously unannotated circRNAs'—rest entirely on an untested extrapolation from centered windows to whole-genome scanning. A model could achieve high AP by exploiting the center-position prior and local sequence motifs without learning to find splice sites at arbitrary positions in a sliding scan. Additionally, the top-k accuracy defined as recovery of true sites among top-k peaks in centered test windows does not imply useful genome-wide precision, where true sites are rare and off-center. The saliency analysis showing A/T-stretch importance further suggests the model relies on local composition rather than demonstrating genome-wide validity.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript presents CircFormerMoE, a two-stage transformer/MoE model for circular RNA splice site detection (SSD) and pairing (SSP) in 10 plant species. SSD is framed as position-wise classification on 5001 bp windows, and SSP as binary classification on concatenated ±500 bp windows around candidate splice-site pairs. The authors report near-perfect SSD metrics (AP/AR/F1 ≈ 99%) and 97.35% SSP accuracy, and use saliency maps to claim polyA/T importance. The abstract and discussion further claim that the model predicts circRNAs directly from plant genomic DNA and can discover previously unannotated circRNAs, with code and data released.","tokens_in":11371,"tokens_out":6184,"duration_ms":74328,"significance":"If the genome-wide claim were established, this would be a useful contribution: it would enable DNA-only circRNA screening in plants and reduce dependence on RNA-seq. Strengths include the MoE design for cross-species transfer, the multi-species plant dataset, the released code/data/trained models, and the reported inference speed of over one million bases per second. However, the current evaluation does not demonstrate the central discovery claim, because all metrics are computed on windows that are centered on known splice sites rather than on a whole-genome scan.","major_comments":[{"comment":"The SSD evaluation is performed exclusively on 5001 bp windows centered on annotated splice sites, so every positive window contains a true splice site at the center, while negatives are sampled from non-splicing regions. A model can attain high AP/AR by learning to output a peak at the window center for gene-like windows, without learning to locate splice sites at arbitrary positions in a genome-wide sliding-window scan. No whole-chromosome evaluation is reported, so the metrics in Table II and Fig. 4 do not support the abstract's claim of predicting circRNAs directly from genomic DNA or discovering unannotated circRNAs. I request a genome-wide scan evaluation with per-chromosome precision/recall and top-k site recovery, and validation of predicted circRNAs against independent RNA-seq or RT-PCR data.","section":"II.A and III.A, Table II, Fig. 4"},{"comment":"The SSP negatives are constructed by concatenating splice-site pairs from different circRNAs. These random pairs are not representative of the candidate pairs that arise in a genome-wide scan, and the model may rely on trivial biases such as site-to-site distance or annotation-derived pairing statistics. The paper does not report the distance distribution or genomic context of the negative pairs. Please characterize the negative distribution and evaluate SSP on genome-wide candidate pairs or on held-out pairs from the same experimental pipeline.","section":"II.A and III.B"},{"comment":"The interpretability analysis is self-referential: saliency maps are computed on the training sequences, and the same sequences are then used to report polyA/T enrichment (43.7% versus 17.4% in random sequences). This does not independently validate the model's biological basis. Please compute saliency on held-out or independent sequences and compare polyA/T enrichment against matched genomic background, or explicitly present the saliency results as descriptive only.","section":"II.D and III.C"}],"minor_comments":[{"comment":"There are several typos, including 'exprimental', 'seperately', and 'gonomes'; the manuscript should be proofread.","section":"Throughout"},{"comment":"The gradient notation appears reversed: Eq. (7) writes ϕ_G = ∂x / ∂y but the text and Eq. (8) require ∂y / ∂x; please correct.","section":"II.B, Eq. (7)"},{"comment":"Architecture hyperparameters (number of layers, hidden dimension, number of attention heads, random feature count m, learning rate, fine-tuning epochs) are not specified in the paper; the code release mitigates this, but the manuscript should include the key settings for reproducibility.","section":"II.B and II.C"},{"comment":"The perfect 100% scores for Lactuca sativa are suspicious and should be investigated for possible data leakage or label duplication rather than attributed only to 'cleaner annotations'.","section":"III.B, Table III"},{"comment":"Reference [11] concerns intronic branch point constraints and does not appear to support the statement about GT-AG motif usage in human pre-mRNA; please verify the citation.","section":"I, reference [11]"},{"comment":"The term 'end-to-end' may be misleading because the method is presented as two separate subtasks; consider clarifying the pipeline structure.","section":"Title and Abstract"}],"recommendation":"major_revision","confidential_remarks":"The missing whole-genome validation is the decisive issue. On reading the manuscript, the reader's concern about the centered-window shortcut is confirmed by the data construction in Section II.A. I would recommend major revision rather than outright rejection because the architecture and multi-species dataset are potentially useful, and a properly designed genome-wide evaluation could establish or refute the central claim. If the authors cannot provide such validation, rejection should follow."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe short version: this is a legitimate model-building paper with a sensible two-stage pipeline, but the headline claim—predicting circRNAs directly from plant genomic DNA and discovering unannotated circRNAs—is not supported by the evaluation. Every positive test window is centered on a known splice site, so the 99% AP/AR numbers measure discrimination on easy, position-anchored examples, not genome-wide detection. That's the thing to know before reading further.\n\nWhat's new and useful: decomposing circRNA prediction into splice site detection and splice site pairing is a reasonable way to handle ultra-long circularization events. The MoE design with per-species heads sharing a backbone is a sensible use of the architecture, and the Performer-style linear attention keeps the 5001 bp windows tractable. The dataset covers ten plant species and is a real resource. Comparing against SpliceAI and a no-MoE baseline is the right instinct.\n\nThe soft spots are consequential. Section II.A says positive windows are made by taking each splice site plus 2500 bases on each side. A model can get near-perfect recall by simply predicting the center of the window; nothing in the metrics rules that out. The paper never runs the proposed sliding-window scan on whole chromosomes, and the 'discovery' of new circRNAs comes with no RNA-seq or RT-PCR validation. The interpretability section finds A/T-stretch enrichment in the same training data used to fit the model—that's self-referential, not evidence of a biological signal. The SSP negatives are random pairs from different circRNAs, which is fine for a proof of concept but doesn't reflect genome-wide pair space. And there are no error bars anywhere; the perfect 100% scores for Lactuca sativa suggest something trivial or leaked in that split.\n\nThe central argument as stated does not hold up. But the underlying idea is worth testing properly. This paper should go to peer review, with the expectation that the authors add whole-genome scanning, validate a sample of predictions, include error bars and proper baselines, and release code with a reproducible commit. If those are added, it could be a useful contribution. As it stands, it's a promising negative result about evaluation design.\n\nMy recommendation: send it to review, but tell the authors the genome-wide claim is the make-or-break.","headline":"A sensible two-stage circRNA model whose headline genome-wide discovery claim is unsupported by center-anchored evaluation; worth reviewing with clear revision requirements.","tokens_in":11950,"tokens_out":2523,"would_cite":false,"duration_ms":29690,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CircFormerMoE claims circular RNA splice sites and their pairing can be predicted directly from plant genomic DNA, reporting near-99% detection accuracy and 97.35% pairing accuracy across ten species.","keywords":["circular RNA","splice site detection","splice site pairing","plant genomes","transformer","mixture of experts","deep learning","genomic DNA prediction"],"falsifier":"Run CircFormerMoE on complete chromosomes with a sliding window and compare its predicted novel splice sites and circRNA pairs against independent RNA-seq back-splice-junction calls or RT-PCR on the same plant tissue; a large drop from the reported 99.2% detection precision, or failure of novel candidates to validate, would falsify the DNA-only discovery claim.","tokens_in":10940,"feed_emoji":"🧬","tokens_out":8213,"duration_ms":89974,"temperature":0.7,"pith_summary":"CircFormerMoE is an attempt to make circular RNA (circRNA) discovery in plants a DNA-sequence prediction problem rather than an RNA-sequencing problem. The paper breaks the task into two stages: detecting individual splice sites along 5001-base genomic windows, then deciding whether pairs of detected sites form a circle by classifying their surrounding sequence contexts. On annotated data from ten plant species it reports near-99% precision and recall for splice-site detection and 97.35% pairing accuracy, with the species-specific mixture-of-experts design adding measurable gains over a shared-backbone baseline. The payoff, if the result holds, is a fast genome-wide screening tool for a class of RNAs whose known abundance is likely underestimated and whose splice signals deviate from canonical GT-AG motifs.","feed_headline":"Plant circRNA sites predicted from raw DNA","feed_subtitle":"Transformer-plus-experts model reports 99% splice-site detection and 97% pairing, no RNA-seq needed.","key_machinery":"The load-bearing object is the CircFormerMoE architecture: an input encoder for one-hot DNA (A, C, G, T, N, and M for concatenation), a local encoder of one-dimensional convolutional residual blocks, a global encoder using lightweight multi-head self-attention based on Performer's FAVOR+ random-feature kernel, and species-specific mixture-of-experts heads on a shared backbone. Task 1 uses a per-position 1x1 convolution head to output a splicing score at every base; Task 2 pools the shared representation and passes it through a per-species linear head to classify pairs. The MoE heads with hard routing are what let the model share transferable sequence features across species while specializing each expert; the linear attention is what keeps the 5001-base window and long-range pairing computationally feasible. The two-task decomposition converts the intractable whole-genome problem into local windows plus one classification step.","core_discovery":"On the paper's own terms, the central claim is that circRNA formation signals in plants are learnable from raw genomic sequence. CircFormerMoE is presented as an end-to-end framework that first labels every nucleotide as splice-site or not and then classifies candidate site pairs as joined or not, with shared transformer representations and per-species expert heads. The paper reports that CircFormerMoE reaches 99.2% average precision and recall on splice-site detection across ten species, reports scores higher than the SpliceAI baseline by 7.5 to 8.2 percentage points on those metrics, and achieves 97.35% accuracy on splice-site pairing. It also reports that saliency analysis concentrates importance at the junction and reveals polyA/polyT stretches that occur in 43.7% of real sequences near splice sites versus 17.4% expected at random. From this the paper infers that the model can discover previously unannotated circRNAs directly from plant genomes.","pith_inferences":["The evaluation windows are centered on known splice sites, so the strongest unexamined risk is that genome-wide scanning inflates false positives; a chromosome-scale test with threshold calibration is the natural next experiment.","The negative pairs in Task 2 are drawn from different circRNAs; constructing harder negatives from nearby splice sites or from sites on the same transcript would give a stricter measure of pairing specificity.","A direct benchmark against RNA-seq back-splicing tools on matched plant samples, measuring overlap and validating novel candidates, would turn the discovery claim into an operational one.","If polyA/T motifs are causal rather than correlational, perturbing those stretches near validated splice sites could provide a CRISPR-based test of the model's learned mechanism."],"forward_implications":["If the reported accuracy transfers to unseen genomes, circRNA screening in plants can begin from DNA alone and reserve RNA-seq for validation instead of discovery.","The species-specific expert design means adding a new plant species requires only a small new head plus fine-tuning, not a full retraining.","Because the global encoder scales linearly with sequence length, the architecture is suited to whole-chromosome scans at the reported inference speed of over one million bases per second on a consumer GPU.","The detected enrichment of polyA/polyT near splice sites gives a concrete sequence-level feature that can be tested for a role in plant circRNA biogenesis."],"supporting_citations":[{"why":"Defines the human splice-site prediction task and top-k evaluation metric and serves as the main baseline the paper must beat.","marker":"[15]"},{"why":"Supplies the annotated circRNA splice-site positions for ten plant species that form the training and test labels.","marker":"[19]"},{"why":"Supplies the plant genome sequences used as model input.","marker":"[20]"},{"why":"Provides the FAVOR+ random-feature linear attention that keeps the transformer's cost linear in sequence length.","marker":"[23]"}],"fun_headline_variants":["CircRNA splice sites read straight from plant DNA","Transformer model finds circRNA junctions in raw genomes","No RNA-seq needed: Deep learning predicts plant circRNAs","Genome-only circRNA detection hits 99% in plants","CircFormerMoE maps circRNA pairing from DNA alone"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"That accuracy measured on 5001-base windows centered on known splice sites carries over to sliding across whole chromosomes, where negative positions dominate and the model has not been evaluated.","fun_headline_variants_meta":{"raw":{"variants":["CircRNA splice sites read straight from plant DNA","Transformer model finds circRNA junctions in raw genomes","No RNA-seq needed: Deep learning predicts plant circRNAs","Genome-only circRNA detection hits 99% in plants","CircFormerMoE maps circRNA pairing from DNA alone"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000179,"raw_usage":{"total_tokens":1345,"prompt_tokens":1034,"completion_tokens":311,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":650,"completion_tokens_details":{"reasoning_tokens":233}},"tokens_in":650,"tokens_out":311,"duration_ms":3717,"temperature":1.0,"reasoning_tokens":233,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T18:16:38.098245+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run CircFormerMoE on complete chromosomes with a sliding window and compare its predicted novel splice sites and circRNA pairs against independent RNA-seq back-splice-junction calls or RT-PCR on the same plant tissue; a large drop from the reported 99.2% detection precision, or failure of novel candidates to validate, would falsify the DNA-only discovery claim.","supporting_citations":[{"cited_title":"Predicting splicing from primary se uence with deep learning,","cited_arxiv_id":null,"evidence_quote":"Defines the human splice-site prediction task and top-k evaluation metric and serves as the main baseline the paper must beat."},{"cited_title":"PlantCircRNA: A comprehensive database for plant circular RNAs,","cited_arxiv_id":null,"evidence_quote":"Supplies the annotated circRNA splice-site positions for ten plant species that form the training and test labels."},{"cited_title":"Ensembl 2021,","cited_arxiv_id":null,"evidence_quote":"Supplies the plant genome sequences used as model input."}],"review_version":1}