{"id":"8f4b835a-4dce-4df3-af4a-e1fcb707a45b","arxiv_id":"2607.20323","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A multi-modal transformer fusing raw current traces, wavelet images, and catch22 features classifies nanopore peptide events with 92.6% macro accuracy—more than 10 points above the best single-modality baseline.","lead":"A new deep-learning model reads nanopore electrical signals in three formats at once—raw current, time-frequency images, and statistical descriptors—and identifies which peptide molecule produced each signal. It outperforms single-format methods by more than 10 percentage points on a 42-peptide test and reaches near-perfect accuracy on a 20-amino-acid dataset.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline >10pp margin depends on untested comparability of Table 1 baseline numbers; need same-split recomputation.","rationale":"The paper's strongest and most checkable claim is the '>10 percentage points' improvement on the 42-peptide benchmark. Computing 92.6 - 81.7 = 10.9 makes the claim turn on a single baseline number. The text never states that the prior-work baselines in Table 1 were run on the identical held-out split described in Section 4; instead it calls them a 'comparison to prior work.' Since refs [17] and [30] are prior papers by the same group, the infrastructure to recompute exists, but the preprint does not report that this was done. If those earlier numbers were produced under a different split or preprocessing, the 10.9pp margin is not a controlled comparison, and the abstract's central quantitative claim would not be established. This is exactly the reader's weakest assumption, and I agree with it. I also note that no error bars or seeds are reported, so even a true 10.9pp gap has unknown variance. Other possible concerns (feature selection via SHAP on the same dataset, inclusion of test data in pretraining) are real but would require additional evidence; the baseline comparability issue is the first-order question. If the recomputation test passes, CONDITIONAL is appropriate; if it fails, the central claim would need to be reworded.","tokens_in":14063,"tokens_out":10477,"duration_ms":87824,"concrete_test":"Obtain the original code/data for refs [17] and [30] (or request from authors) and run the ResNet18-wavelet and MLP-catch22 baselines on the exact 15% held-out test split and preprocessing pipeline used for the multi-modal model, with the same event filtering and wavelet/catch22 parameters. Record macro- and micro-accuracies and per-class minima. If the best rerun baseline macro accuracy is ≤82.6% (i.e., at least 10pp below 92.6%), the headline claim survives; if it exceeds 82.6%, the >10pp claim should be revised. Also report accuracy over at least 3 seeds to bound seed variance.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is the >10 percentage point improvement on the 42-peptide benchmark (92.6% vs 81.7% macro, Table 1). The margin is only 10.9pp. The text does not state that the ResNet18-Wavelet [30] and MLP-Catch22 [17] baselines were re-evaluated on the same 70/15/15 split described in Section 4; Table 1 says 'comparison to prior work,' suggesting those numbers may be taken from the original papers. If prior papers used a different train/validation/test split or preprocessing (wavelet parameters, catch22 feature computation, event filtering), the 10.9pp margin could shrink below 10pp or change direction. No error bars or seed information are reported for any number in Table 1, so the margin's robustness to run-to-run variance is also unknown. This assumption is load-bearing because the abstract's headline 'more than 10 percentage points' is exactly the difference between 92.6 and 81.7.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a multi-modal transformer for classifying nanopore blockade events, combining three input representations: raw ionic-current time series, wavelet-transform images, and a small set of catch22-derived static features. The model uses branch-specific transformer encoders with cross-attention, plus a masked-autoencoder pretraining stage. On a 42-peptide ladder dataset (350,000 events), the authors report 92.6% macro / 92.1% micro accuracy, improving the best published baseline by more than 10 percentage points, and raising the worst-class accuracy from 58.7% to 77.2%. A transfer experiment to a 20-amino-acid XR7 dataset reports 99.0% macro accuracy when fine-tuning from the peptide-ladder model, with faster convergence and a narrower per-class accuracy distribution than training from scratch. Attention analysis is used to argue that time-series and wavelet modalities focus on complementary signal attributes.","tokens_in":14327,"tokens_out":4051,"duration_ms":39814,"significance":"If the results are reproducible, this would be a substantial advance for nanopore-based protein/peptide identification and for multi-modal learning on single-molecule signals. The architecture is generic and could be extended to additional input modalities, and the transfer result across analyte sets with the same pore type is practically important. The paper is explicit about its limitations (same pore architecture, only 42 peptide classes) and the authors provide a reasonably detailed appendix on model and training specifications. However, the central benchmark comparison currently rests on unverified baseline comparability, the headline improvement lacks uncertainty quantification, and the catch22 feature selection procedure appears to use the full dataset. The manuscript's code/data availability statement is also deferred until publication, which hampers verification.","major_comments":[{"comment":"Table 1 labels the ResNet18-Wavelet and MLP-Catch22 rows as 'comparison to prior work,' but the text does not state whether these numbers were recomputed on the same 70/15/15 split and identical preprocessing pipeline used for the multi-modal model. The headline 'more than 10 percentage points' is exactly the margin between 92.6% and 81.7%; if the prior numbers used a different split or wavelet/catch22 parameters, the margin could shrink or change direction. The authors should either rerun the baselines on the exact same test events and preprocessing, or explicitly report the original settings and justify comparability.","section":"§4, Table 1"},{"comment":"No error bars, confidence intervals, or multiple seeds are reported for any accuracy number. The claim of a 10.9pp macro improvement (and the 18.5pp worst-class improvement) is the central result, yet the run-to-run variance of transformer training is often substantial. At minimum, 3–5 independent seeds should be run and mean±std reported for each model, with per-class bootstrap intervals if possible. Without this, the magnitude of the improvement is not statistically grounded.","section":"§2, Table 1 and §4 (Model training)"},{"comment":"The manuscript states: 'To remove redundant data, we reduced this set to the five most relevant features identified in [17] by performing SHAP analysis on the same peptide ladder dataset used in this work.' This indicates the catch22 feature subset was selected using the full dataset, including the test partition, and before model training. This is a selection leak that can bias the reported accuracy optimistically. It is also an unfair advantage over the MLP-Catch22 baseline if that baseline uses the full feature set. The feature selection must be performed inside the training folds, or justified as using only training labels and samples.","section":"§4, 'Experimental data and model input'"},{"comment":"The paper states that 'Approximately 3–5% of these events are mislabeled due to limitations in the labeling process' and later uses this to argue that the multi-modal model 'approaches the ceiling attainable on this data set.' No method, reference, or measurement is given for this estimate. If the mislabel rate is different or if it is not independently estimated, the 'near ceiling' claim is unsupported. Please provide the estimation procedure or rephrase the claim as a qualitative speculation.","section":"§4, 'Experimental data and model input'; §3 Discussion"}],"minor_comments":[{"comment":"'More than 10 percentage points' is a fine summary, but the actual margins are 10.9pp macro and 10.6pp micro; given the baseline-comparability concern, the wording is somewhat optimistic. Suggest 'about 10 percentage points' or a qualified statement until baselines are recomputed.","section":"Abstract"},{"comment":"The expression 'p D2/h' should read 'sqrt(D2/h)' or be typeset properly; the current inline notation is ambiguous.","section":"§4, Equation (2)"},{"comment":"Typo: 'thehhhatwavelet' should presumably be 'the hhhat wavelet' (or 'the HHHat wavelet').","section":"§4, 'Wavelet images'"},{"comment":"The 'adjusted validation loss' subtracts an 'irreducible offset' from label smoothing, but the offset value and its calculation are not specified. Please state how it was computed so the plot is reproducible.","section":"Figure 5"},{"comment":"Availability 'upon publication' is standard, but for review purposes the withholding of code/data makes it impossible to verify the central benchmark. Please provide reviewer access or a detailed model/config file in the supplement.","section":"Data and code availability"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's heavy reliance on references [17] and [30] from the same group is not itself problematic, but it makes the unverified baseline comparison more delicate. If the authors can rerun those baselines on the same split and report seeds/error bars, the central claim may well survive. The feature-selection leakage is the most serious methodological issue and needs to be fixed or explicitly argued away."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid engineering result with a real accuracy jump, and the architecture is sensible. The paper is not a conceptual breakthrough — every component is borrowed from existing work — but the combination is new for nanopore classification and the empirical gain is large. It deserves a real referee. The main caveat is the comparison protocol, not the model.\n\nWhat's new: fusing three views of the same event — raw current trace, wavelet image, and catch22 feature vector — in a multi-branch cross-attention transformer. The paper shows fusion beats each single modality, and the attention analysis gives a plausible explanation: the raw and wavelet branches attend to different structures in the same signal. Training details are unusually complete, with architecture, hyperparameters, and pretraining procedures in the appendices. The transfer experiment includes a from-scratch baseline, which is the right control. The authors also acknowledge the main limitations of their study honestly.\n\nSoft spots:\n\n1. The 'more than 10 percentage points' claim in the abstract is 10.9 pp over the ResNet18-Wavelet baseline in Table 1. The table says 'comparison to prior work,' and the text never states that those numbers were recomputed on the same held-out split and preprocessing pipeline. If they were taken from the original papers, the margin could change. Since the margin is only just above 10, this is load-bearing for the headline. That said, the authors' own time-series transformer baseline (77.0% macro) is trained under the same protocol, and the multi-modal model beats it by a much wider margin, so the fusion gain itself is not in doubt.\n\n2. There are no error bars and no multiple-seed results anywhere. For a claim of 92.6% on a 42-class benchmark, run-to-run variance could matter, especially with the claimed 3–5% label noise.\n\n3. The catch22 feature subset was selected by SHAP on the same dataset used for evaluation. This is a mild selection leak. It is a minor issue for five simple features, but it should be stated as such.\n\n4. Code and data are promised only upon publication, so the current preprint cannot be independently checked.\n\nThe stress-test note is right that the headline margin is fragile. I would not call the paper unreliable, though: the multi-modal model's advantage over the authors' own single-modality transformer is substantial, and the worst-class improvement is consistent with the fusion story.\n\nWho is this for: researchers applying deep learning to nanopore event classification, and experimental groups looking for a strong baseline architecture. It should go to peer review, with requests for same-split baseline recomputation, multiple seeds, and code/data release.","headline":"A credible empirical win for fusing raw, wavelet, and catch22 features in nanopore classification, but the '>10pp' headline rests on baseline numbers that should have been recomputed on the same split.","tokens_in":77,"tokens_out":3020,"would_cite":true,"duration_ms":54094,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By jointly processing the raw ionic-current trace, a wavelet image, and a statistical descriptor vector of the same nanopore blockade event in one transformer, this paper reports 92.6% macro-averaged accuracy across 42 peptide classes — mor","keywords":["nanopore sensing","peptide classification","multi-modal transformer","cross-attention","wavelet transform","catch22 features","transfer learning","single-molecule detection"],"falsifier":"Recompute the ResNet18-wavelet and catch22-MLP baselines on the exact test split and preprocessing used for the multi-modal model. If the multi-modal model no longer leads by 10+ percentage points, the headline margin is not reproducible; additionally, re-labeling a sample of the test set to remove the estimated 3–5% label noise and re-evaluating would test whether the model is truly at the attainable ceiling.","tokens_in":13971,"feed_emoji":"🧬","tokens_out":6084,"duration_ms":50392,"temperature":0.7,"pith_summary":"This paper tries to establish that a single deep-learning model which processes several representations of the same nanopore blockade event at once can classify peptides far more accurately and uniformly than any method that looks at one representation alone. On a 42-peptide benchmark, the multi-modal transformer reaches 92.6% macro-averaged accuracy, more than 10 percentage points above the best single-modality baseline, and it lifts the worst-performing class from 58.7% to 77.2%. The same architecture, pre-trained on the large benchmark and fine-tuned on a smaller 20-amino-acid dataset, reaches 99.0% macro accuracy and converges faster than training from scratch. If these results hold, they would make nanopore single-molecule sensing substantially more reliable for diagnostics, where the least-well-resolved analyte limits the whole assay, and suggest that pre-trained models can reduce the cost of applying machine learning to new nanopore analytes.","feed_headline":"Multi-modal transformer tops nanopore peptide classifiers by 10 points","feed_subtitle":"Raw current traces, wavelet images, and statistical features together push worst-class accuracy from 58.7% to 77.2%.","key_machinery":"The key mechanism is a multi-branch transformer encoder built on the Vision Transformer. Each modality forms its own branch: the raw current trace is split into segments, the wavelet image into patches, and the catch22 descriptor vector is passed through an MLP to create a classification token. Cross-attention transfers information between branches — once from the descriptor token to the time-series and image branches before the encoder, and then alternately with standard transformer blocks inside the encoder. Masked-autoencoder pretraining, with classification tokens included and different masking ratios per branch (50% of time-series tokens, 70% of wavelet tokens), forces the branches to s","core_discovery":"The central claim is that combining three complementary views of the same blockade current signal — the raw time series, a wavelet-transformed image, and a vector of catch22 statistical descriptors — within one multi-branch transformer yields qualitatively better peptide classification than any single view. The model achieves 92.6% macro / 92.1% micro accuracy on 42 peptide classes from a peptide-ladder experiment, versus 81.7%/81.5% for the best prior single-modality model; its worst-class accuracy jumps from 58.7% to 77.2%, and the best class reaches 100%. Attention analysis shows the representations emphasize different structures — entry/exit phases and deep short blockades for the time s","pith_inferences":["Both datasets come from the same aerolysin pore, so the paper demonstrates transfer across analytes, not across pore types; testing on a structurally different pore would show whether the model is a general nanopore foundation model.","The label-noise ceiling (3–5%) is estimated, not measured; generating a cleanly labeled subset of the peptide-ladder test set and re-evaluating would reveal whether the model's apparent ceiling is real or an artifact of noisy labels.","The attention patterns are observational; a targeted perturbation — e.g., zeroing the high-frequency band or the entry/exit phases during inference — would directly test whether those features are causally responsible for the wavelet and time-series branches' contributions.","The multi-branch design is agnostic to input type, so joint electro-optical readouts or other time-series derived representations can be plugged in; one could even imagine using the same architecture for classification of other single-molecule translocation signals."],"forward_implications":["If the 92.6% macro accuracy is correct, the model operates near the ceiling set by the estimated 3–5% label noise, so further gains would require cleaner labels, not better architectures.","Worst-class accuracy of 77.2% (up from 58.7%) means a clinical assay built on this classifier would not be dragged down by a single poorly resolved peptide.","Transfer to the 20-amino-acid set at 99.0% macro accuracy, with faster convergence after pretraining, implies that nanopore signal features generalize across analytes and that pretraining can cut the labeled-data cost for new sensing tasks.","Because the architecture accepts any number and type of branches, joint analysis with additional readouts (for instance optical signals recorded alongside ionic current) can be added without redesigning the model."],"fun_headline_variants":["Multi-modal transformer lifts nanopore peptide accuracy to 92.6%","Joint wavelet, raw, and stat inputs push nanopore peptide classifier past 10-point gap","Worst-class nanopore peptide accuracy jumps 58.7% to 77.2% with transformer","Three-signal-view transformer beats single-modality on 42-peptide nanopore set"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The headline 'more than 10 percentage points' improvement rests on the assumption that the published baseline accuracies were measured on the same test split and with the same preprocessing as the multi-modal model; the paper does not state that the baselines were recomputed on identical held-out events.","fun_headline_variants_meta":{"raw":{"variants":["Multi-modal transformer lifts nanopore peptide accuracy to 92.6%","Joint wavelet, raw, and stat inputs push nanopore peptide classifier past 10-point gap","Worst-class nanopore peptide accuracy jumps 58.7% to 77.2% with transformer","Three-signal-view transformer beats single-modality on 42-peptide nanopore set"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000395,"raw_usage":{"total_tokens":1886,"prompt_tokens":702,"completion_tokens":1184,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":446,"completion_tokens_details":{"reasoning_tokens":1091}},"tokens_in":446,"tokens_out":1184,"duration_ms":10124,"temperature":1.0,"reasoning_tokens":1091,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T10:07:47.402412+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the ResNet18-wavelet and catch22-MLP baselines on the exact test split and preprocessing used for the multi-modal model. If the multi-modal model no longer leads by 10+ percentage points, the headline margin is not reproducible; additionally, re-labeling a sample of the test set to remove the estimated 3–5% label noise and re-evaluating would test whether the model is truly at the attainable ceiling.","supporting_citations":[],"review_version":1}