{"id":"d33d9f06-9082-49f4-af6f-19c7219f7a79","arxiv_id":"2511.09026","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"DeepVRegulome fine-tunes 464 DNABERT models on ENCODE/GENCODE regulatory regions and reports thousands of recurrent, survival-associated non-coding mutations in glioblastoma.","lead":"The authors built DeepVRegulome, a toolkit of 464 DNABERT-based models that scores how short mutations affect gene-regulatory regions, and applied it to 190 glioblastoma genomes to flag recurrent, survival-linked non-coding mutations. A generalist would read it to judge whether fine-tuned genomic language models are now ready to prioritize non-coding cancer variants end-to-end.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Abstract promises SNP-SELEX benchmark and four-predictor comparison, but the main text contains no such analysis; the Discussion's limitations even state validation is confined to in-silico methods, so the central external-validation claim is unsupported.","rationale":"The paper has real assets: 462 TF/RBP models with high classification accuracy, 87% JASPAR motif match, and open code/dashboard. The internal engineering is credible. However, the core product is variant effect prediction, and the only external check promised is absent. The reader's weakest assumption about the 0.5 threshold is germane, but the more specific load-bearing issue is the missing benchmark: it is the sole evidence that would calibrate the threshold. The internal inconsistency between abstract and Discussion is a red flag that cannot be dismissed as a pipeline artifact. A conditional acceptance is appropriate since the benchmark could be added; but without it, the paper does not support its headline claim.","tokens_in":14715,"tokens_out":4022,"duration_ms":37339,"concrete_test":"Obtain the SNP-SELEX data used in the abstract's claim (e.g., from Yan et al. 2021) and run the corresponding DeepVRegulome TF models on each reference/mutant pair. Compute the AUROC or Spearman correlation between the predicted log-odds ratio or delta-probability and the measured binding fold-change; include the four named predictors for comparison. If the AUROC is significantly above 0.5, the external-validation claim is supported; if not, the core claim fails and the survival results become uninterpretable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that DeepVRegulome 'accurately captures variant effects on baseline binding status' rests entirely on a promised benchmark against SNP-SELEX allele-specific binding data and comparison to four variant-effect predictors. This analysis is stated in the Abstract but does not appear in the Results, Methods, or any supplementary section of the posted manuscript; no dataset, protocol, or comparison table is provided. The Discussion's Limitations further state that 'functional validation is confined to in-silico motif disruption, RNA-seq isoform shifts, and survival correlations,' which internally contradicts the abstract's assertion of independent experimental benchmarking. Without a shown SNP-SELEX evaluation, the model's variant-impact scores (log-odds ratios, delta-probabilities) are not demonstrated to correspond to real binding changes; the JASPAR motif match (§3) confirms that models learn recognized motifs, but it does not validate that a probability drop from 0.6 to 0.4 predicts loss of binding. All downstream statistics — recurrent counts, survival associations, hazard ratios — are derived from the uncalibrated 0.5 posterior-crossover rule, so the absence of the benchmark undermines every quantitative conclusion.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces DeepVRegulome, a framework that fine-tunes DNABERT models on ENCODE/GENCODE regulatory regions to predict the functional impact of short genomic variants in splice sites, transcription-factor binding sites, and histone-mark regions. The pipeline scores variants using log-odds ratios and probability changes, extracts interpretable motifs via attention and k-mer enrichment, and connects predicted regulatory disruptions to survival outcomes in 190 glioblastoma WGS samples. The authors report high classification accuracy across hundreds of TF models, JASPAR motif recovery for 87% of high-confidence models, thousands of recurrent predicted functional variants, and multiple survival associations. The abstract further claims independent benchmarking against SNP-SELEX data and comparison with four established variant-effect predictors, but this benchmark is absent from the body of the manuscript.","tokens_in":14893,"tokens_out":4167,"duration_ms":41567,"significance":"If the central claims were fully supported, DeepVRegulome would be a valuable resource: it is a large-scale, openly available framework with a plausible modular design, and the JASPAR motif validation (Results §3) is a genuinely independent check that the models learn known regulatory grammar. The reported classifier metrics (Accuracy 94–96% for splice sites; AUC > 0.97 for top TF models) are strong, and the public code, model checkpoints, and interactive portal are concrete contributions to reproducibility. However, the manuscript's main downstream claim—that the method 'accurately captures variant effects on baseline binding status'—rests on a benchmark that is promised in the abstract but not present in the Results, Methods, or supplement. The JASPAR comparison validates motif learning, not the calibration of the 0.5-threshold disruption rule on which all variant counts, recurrence statistics, and survival analyses depend. The clinical and biological conclusions are therefore currently supported only by in-silico associations, which the authors themselves acknowledge in the Limitations.","major_comments":[{"comment":"The abstract states that DeepVRegulome was 'benchmarked ... against an independent experimental assay of allele-specific transcription factor binding (SNP-SELEX) data and compared its performance to four established variant-effect predictors.' No such benchmark or comparison appears anywhere in the Results, Methods, figures, tables, or supplementary sections of the posted manuscript. The Discussion's Limitations instead state that 'functional validation is confined to in-silico motif disruption, RNA-seq isoform shifts, and survival correlations.' This is an internal contradiction on the paper's central validation claim. Because the 0.5-threshold posterior-crossover rule is the sole basis for every functional-impact call, the absence of the promised benchmark leaves the accuracy of all downstream variant-effect predictions unestablished. The authors must either supply the benchmark (with","section":"Abstract vs Results/Methods"},{"comment":"The functional-disruption rule—classify a variant as disruptive if p(s) > 0.5 and p(s') < 0.5—is an ad-hoc threshold with no demonstrated relationship to actual regulatory activity. The JASPAR motif matching in Results §3 shows that models learn motifs, not that a probability drop from 0.6 to 0.4 (or 0.99 to 0.31) corresponds to loss of TF binding or splicing function. All downstream numbers (35,867 indels, 19,709 SNVs, recurrence counts, survival associations) are counts of variants crossing this threshold. The authors should calibrate the threshold against an external quantitative assay (e.g., the promised SNP-SELEX data), or at minimum provide sensitivity analyses showing that conclusions are robust to threshold choice.","section":"Methods, Variant Impact Prediction and Scoring"},{"comment":"The score-change definition, S = Δp = (p(s') − p(s)) + max(p(s'), p(s)), does not behave as described. For a large drop from p(s)=0.99 to p(s')=0.31, S = 0.31; for a small drop from p(s)=0.51 to p(s')=0.49, S = 0.51. Thus the term 'max' does not 'amplify the strong effects'—it yields lower scores for stronger disruptions when the wild-type probability is high. This is a load-bearing scoring formula used to rank variants. The authors should correct the formula or clarify the intended definition, and re-run any affected analyses.","section":"Methods, Eq. (1) — score change"},{"comment":"The survival analysis tests a large number of variants/regions (1,352 mutations and 563 regulatory regions are reported as significant) but no multiple-testing correction is described. With thousands of Cox/KM tests, raw p-values of 10^-2 to 10^-4 are expected by chance and do not establish robust associations. The authors should report FDR-adjusted q-values, or a permutation-based procedure, and discuss how many associations survive correction. Without this, the clinical-stratification claim is not quantitatively supported.","section":"Results §4, Survival Analysis"}],"minor_comments":[{"comment":"The number of fine-tuned models is inconsistent across the manuscript: the arXiv abstract says 464 models (458 TF, 4 histone, 2 splice), the full-text abstract says 700, Results §1 reports 667 TFs and 33 histone marks, and the high-confidence set is later 462. Please standardize the model counts and clarify which models were used in variant scoring.","section":"Abstract and Results §1"},{"comment":"The phrase 'recapitulated ~97% (2256 of 2346) of the curated matrices in JASPAR' is difficult to interpret, because 2346 seems to be the total number of JASPAR matrices, not the subset relevant to the 462 ENCODE TF models. Clarify the denominator and how 'coverage' is defined.","section":"Results §3"},{"comment":"The isoform-expression analysis is mentioned for a few examples but not described in Methods. Please provide the data source, statistical test, and multiple-testing context for the reported isoform p-values.","section":"Figure 4C and Results §4"}],"recommendation":"major_revision","confidential_remarks":"The abstract/body mismatch on the SNP-SELEX benchmark is the most serious issue and must be resolved before the paper can be considered further. The paper may be salvageable if the authors can supply the benchmark or explicitly reframe the claims as purely in-silico predictions. I also noted the score-change formula appears mathematically inconsistent with its stated purpose; this needs correction and re-analysis. I saw no indication of misconduct, but the manuscript requires substantial additional evidence and revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know about this one: it's a genuinely useful engineering effort wrapped in an abstract that overpromises. The core resource is a set of 464 DNABERT fine-tuned models for splice sites and ENCODE TFBSs, with code, trained weights, and a live dashboard all available. The strongest evidence in the paper is the JASPAR comparison: 87% of the TF/RBP models learn motifs that significantly match a known JASPAR profile, and 72% of the direct name matches are identical. That is real, reproducible validation that the models are capturing meaningful sequence preferences, not just memorizing training labels. The application to 190 GBM WGS also produces a substantial catalog of recurrent candidate regulatory variants, and the attention-based interpretation examples (e.g., NR2F2/MIDN) are illustrative.\n\nThe soft spots are not subtle. The abstract states the framework was benchmarked against SNP-SELEX allele-specific binding data and four established variant-effect predictors. That analysis appears nowhere in the main text, methods, or available supplementary. The Discussion's limitations even say functional validation is confined to in-silico motif disruption, RNA-seq shifts, and survival correlations—an internal contradiction. The abstract also reports headline counts (572 splice-disrupting, 9,837 TFBS-altering mutations) that don't match Table 1's sums, so the reader can't trust the numbers as stated. The survival analysis tests 1,352 mutations with unadjusted p-values and no multiple-testing correction or multivariate Cox model; most of those associations would not survive FDR. Finally, the disruptive-variant rule is an uncalibrated 0.5 posterior crossover, and the score-change formula's max-amplification term looks ad hoc.\n\nTo be fair, the authors are transparent about the limitations—the Discussion is honest about the in-silico nature of the functional validation. That honesty makes the abstract's SNP-SELEX claim stranger; it's not a malicious overstatement but a clear inconsistency that must be fixed before publication. The JASPAR validation and the model collection give the paper a solid, partially validated core. The missing benchmark and survival analysis corrections are addressable, not fatal.\n\nThis deserves a serious referee. The resource is valuable to the variant-effect prediction community, and the JASPAR validation alone is worth publishing. But the referee should push for the benchmark to be either properly reported or removed from the abstract, counts reconciled, and the survival analysis re-done with FDR or at least a sensitivity analysis. I'd send it to peer review with major-revision expectations.\n\nIf you work on non-coding variant prioritization, this is worth knowing about, but don't cite the survival-specific claims until they're fixed.","headline":"A substantial model resource and GBM variant catalog whose central external-validation claim is missing from the body; worth refereeing, but only after the authors reconcile the abstract with the results.","tokens_in":15538,"tokens_out":2182,"would_cite":true,"duration_ms":23738,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DeepVRegulome claims that hundreds of element-specific DNA language models can predict which non-coding mutations matter, and in glioblastoma it links thousands of them to patient survival.","keywords":["non-coding variant effect prediction","DNA language models","transcription factor binding sites","splice site prediction","glioblastoma","survival analysis","regulome","variant prioritization"],"falsifier":"Compile a panel of variants DeepVRegulome calls disruptive (posterior crosses 0.5 downward) and a matched panel that stays above 0.5, then measure allele-specific transcription-factor binding or splicing in a reporter assay; if the disruptive panel is not enriched for experimentally lost binding or splicing relative to the controls, the 0.5-crossing rule is falsified.","tokens_in":14472,"feed_emoji":"🧬","tokens_out":12206,"duration_ms":104159,"temperature":0.7,"pith_summary":"DeepVRegulome aims to make the non-coding genome interpretable at single-variant resolution. The paper's central claim is that fine-tuning a DNA-sequence transformer separately on hundreds of regulatory element classes—splice sites, transcription-factor binding regions, and histone-mark regions—yields models whose reference-versus-mutant probability changes distinguish functional non-coding mutations from neutral ones. Applied to whole-genome sequencing of 190 glioblastoma tumors, the framework identifies thousands of recurrent splice-disrupting and binding-site-altering mutations, including 1,352 mutations and 563 disrupted regulatory regions associated with patient survival, and uses them to stratify patients. The authors also show the fine-tuned models rediscover known transcription-factor motifs, arguing that the variant scores reflect real regulatory grammar. If right, the framework is a general, genome-wide path from raw sequencing data to clinically prioritizable non-coding variants.","feed_headline":"DeepVRegulome maps 10,000+ recurrent non-coding mutations in GBM","feed_subtitle":"Survival-linked mutations expand the search for non-coding drivers in glioblastoma.","key_machinery":"The load-bearing mechanism is the paired-sequence scoring scheme: for each variant, the reference and mutated regulatory windows are run through the same fine-tuned DNA language model, and the difference in predicted binding or splicing probability—quantified as a score change and a log-odds ratio—is treated as functional impact. The 0.5-crossing rule converts those continuous probability shifts into a binary disruptive call, and that binary call feeds all downstream recurrence counts and survival analyses. Around this core sit attention-based motif extraction, which recovers known and candidate binding motifs from the models' internal weights, and a survival module that links disruptive cal","core_discovery":"The discovery is that a single framework can turn a large collection of element-specific DNA language models into a quantitative variant-effect predictor for the human regulome. DeepVRegulome fine-tunes one model per regulatory element—acceptor/donor splice sites, individual transcription factors, and histone marks—then scores every short variant by comparing the model's predicted probability on the reference sequence to its probability on the mutated sequence. A variant is called disruptive when the posterior crosses from above 0.5 to below 0.5, with log-odds ratios and attention-score changes quantifying the magnitude. In 190 glioblastoma genomes this yields roughly 59,000 predicted high-i","pith_inferences":["Editorial: the abstract states that DeepVRegulome was benchmarked against an independent allele-specific binding assay and four established predictors, but the body does not report that comparison; until it appears, the quantitative accuracy claim rests on motif rediscovery and in-silico plausibility rather than direct experimental validation of the scores.","A natural next step is to calibrate the 0.5 posterior-crossing threshold against allele-specific binding measurements; if the threshold changes, the reported counts and hazard ratios would shift.","Because the models only see ±150 base pairs, distal enhancer mutations that act through long-range looping are invisible; extending to long-context transformers or chromatin-contact maps is the obvious follow-up.","The survival-linked loci, such as the NR2F2-MIDN regulatory variant and the PARPBP splice-site mutation, are directly testable with reporter assays or CRISPR base editing, giving an experimental route to confirm or refute the framework's causal claims."],"forward_implications":["More than 10,000 recurrent, predicted-disruptive regulatory variants in glioblastoma—most absent from clinical databases—become concrete candidates for experimental follow-up.","The 1,352 survival-linked mutations and 563 disrupted regulatory regions define a non-coding mutation signature that separates patients by overall survival.","Because about 87% of the transcription-factor models recover known binding motifs, the attention weights can point to the specific broken motif behind a variant.","Splice-site models flag 572 recurrent splice-disrupting mutations, and several show isoform-expression shifts consistent with the predicted splicing change.","The same paired-sequence scoring pipeline applies to any whole-genome cohort, extending the approach to other cancers, germline rare disease, and population-scale sequencing."],"fun_headline_variants":["DeepVRegulome ties 10k recurrent non-coding mutations to GBM","DNABERT framework flags 1,352 survival-linked GBM mutations","One model per element: DeepVRegulome predicts variant impact","10,000+ recurrent non-coding mutations mapped in glioblastoma","DeepVRegulome: survival-linked mutations in non-coding GBM"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that a fine-tuned model's posterior falling from above 0.5 to below 0.5 measures real loss of regulatory function—a premise the paper supports mainly through motif rediscovery, since the allele-specific binding benchmark promised in the abstract does not appear in the results.","fun_headline_variants_meta":{"raw":{"variants":["DeepVRegulome ties 10k recurrent non-coding mutations to GBM","DNABERT framework flags 1,352 survival-linked GBM mutations","One model per element: DeepVRegulome predicts variant impact","10,000+ recurrent non-coding mutations mapped in glioblastoma","DeepVRegulome: survival-linked mutations in non-coding GBM"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000234,"raw_usage":{"total_tokens":1352,"prompt_tokens":783,"completion_tokens":569,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":527,"completion_tokens_details":{"reasoning_tokens":475}},"tokens_in":527,"tokens_out":569,"duration_ms":5970,"temperature":1.0,"reasoning_tokens":475,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T22:42:31.032530+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compile a panel of variants DeepVRegulome calls disruptive (posterior crosses 0.5 downward) and a matched panel that stays above 0.5, then measure allele-specific transcription-factor binding or splicing in a reporter assay; if the disruptive panel is not enriched for experimentally lost binding or splicing relative to the controls, the 0.5-crossing rule is falsified.","supporting_citations":[],"review_version":1}