{"id":"987a2a93-2631-4207-8b44-87822a88c538","arxiv_id":"2607.03653","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.5,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A three-branch CNN-wavelet-ViT ensemble with soft voting reaches 98.01% accuracy and 0.9742 weighted F1 on Malimg, with wavelet features improving discrimination of similar families.","lead":"ThreatVisionAI combines a raw-image CNN, a wavelet CNN, and a Vision Transformer to classify malware families from binary images, reaching 98.01% accuracy on Malimg. The wavelet branch measurably helps separate visually similar families that pure spatial models confuse.","discovery_kind":"extension","skeptic_critique":{"model":"grok-4.5","headline":"Wavelet gains rest on tiny Swizzor test counts; the headline family-level claim is not yet statistically secured.","rationale":"The Reader correctly flags single-dataset (2011 Malimg) evaluation as the principal external-validity risk and assigns CONDITIONAL. That concern is real but secondary: even inside Malimg the paper’s own strongest internal claim—that the wavelet branch measurably improves discrimination of visually similar families—is statistically under-powered. The Swizzor numbers that carry the narrative are based on tiny absolute counts and lack any uncertainty quantification. If those gains disappear under a simple bootstrap or paired test, the architectural novelty collapses to a 0.001 F1 ensemble lift that is already acknowledged as marginal. The concrete bootstrap/McNemar check is cheap, uses only the existing split, and would either solidify or retract the central differentiator without requiring new datasets. Because the Reader already landed on CONDITIONAL for related reasons, the verdict itself need not change; the load-bearing soft spot is simply more precise and more immediately falsifiable than the generalization worry alone.","tokens_in":10240,"tokens_out":588,"duration_ms":5260,"concrete_test":"Recompute per-class F1 for Swizzor.gen!E and Swizzor.gen!I on the exact same train/val/test split, then obtain 1 000 bootstrap resamples of the test set (or a paired McNemar test of raw-CNN vs wavelet-CNN predictions). If the 95 % CI for the F1 difference includes zero or the McNemar p-value exceeds 0.05, the wavelet-gain claim is unsupported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim that wavelet-domain features provide measurable gains on minority and visually similar families rests almost entirely on two Swizzor variants (Table II and surrounding text). Swizzor.gen!E F1 rises from 0.7368 to 0.8571 and Swizzor.gen!I from 0.5263 to 0.8276 when the wavelet CNN replaces the raw CNN. Malimg is heavily imbalanced (80–2 949 samples per family) and the 15 % test split is stratified, so each of these families contributes only a handful of test images (likely ~15–20). With such small denominators, F1 swings of 0.12–0.30 are expected under ordinary binomial sampling variance; no confidence intervals, bootstrap, or McNemar/paired test is reported. Consequently the paper’s strongest differentiator—frequency-domain discrimination of closely related families—could be noise rather than a reliable architectural effect. The overall hybrid gain over the best two-branch ensemble is only +0.0009 weighted F1, reinforcing that the headline improvement is fragile.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.5","summary":"ThreatVisionAI proposes a three-branch hybrid for image-based malware family classification on Malimg: a raw-image ResNet-18 CNN, a Haar-wavelet CNN (four sub-bands stacked as channels into a second ResNet-18), and a ViT-Tiny branch that also consumes the wavelet tensor. Branches are trained independently with class-weighted cross-entropy and fused by weighted soft voting (0.50/0.40/0.10) chosen by validation grid search. On a single stratified 70/15/15 split the hybrid reports 98.01% accuracy and 0.9742 weighted F1. Ablations attribute most of the lift over a pure spatial CNN to the wavelet branch (especially Swizzor.gen!E/I F1), with a smaller additional gain from ViT. Grad-CAM is used to argue that the persistent Autorun.K→Yuner.A collapse is dataset identity rather than model failure; FGSM shows sharp accuracy drop under white-box perturbation.","tokens_in":10515,"tokens_out":853,"duration_ms":12012,"significance":"If the frequency-domain contribution is real and transferable, the work supplies a concrete, easy-to-reproduce architectural increment over existing pixel-space CNN–Transformer hybrids: a dedicated wavelet branch that targets directional texture differences among visually similar families. The ablation table cleanly isolates each branch, Grad-CAM is applied appropriately to the known hard pair, and the authors candidly report adversarial fragility and Malimg’s age. Those strengths make the manuscript useful to the image-based malware community even if the absolute accuracy numbers are not state-of-the-art. The contribution remains incremental and currently rests on a single 2011 benchmark with severe imbalance and no multi-seed statistics.","major_comments":[{"comment":"The central differentiator—wavelet-domain gains on closely related families—rests almost entirely on Swizzor.gen!E (F1 0.7368→0.8571) and Swizzor.gen!I (0.5263→0.8276) when comparing raw CNN to wavelet CNN (text after Table II). With Malimg’s imbalance and a 15% stratified test split, each family contributes only a handful of test images. No confidence intervals, bootstrap, McNemar, or paired test is reported. Under ordinary binomial variance those F1 swings are compatible with noise; the headline claim that frequency features provide “measurable gains” is therefore not yet statistically secured.","section":null},{"comment":"All reported numbers (Table II) come from a single 70/15/15 split with no cross-validation, no multiple random seeds, and no error bars. Soft-voting weights themselves are free parameters fitted on the same validation fold. The full-hybrid improvement over the best two-branch ensemble is only +0.0009 weighted F1 (0.9733→0.9742). Without multi-run statistics it is impossible to judge whether this increment, or the ViT’s 0.10 weight, is stable.","section":null},{"comment":"Table I and the literature review cite recent hybrids that report higher Malimg accuracies (MalSort 98.28%, Enhanced ViT 99.35%, ConvNeXt+Swin, etc.), yet the experimental section never re-evaluates those methods under the authors’ own split and protocol. Without a controlled head-to-head, the claim that the three-branch design “improves” image-based family classification relative to the current hybrid literature remains unanchored.","section":null},{"comment":"Evaluation is confined to Malimg (2011, 9 465 images). The Conclusion correctly flags that MaleVis, BIG2015, and modern corpora are unvalidated; given that the paper’s weakest assumption is transfer of the wavelet lift, at least one additional public benchmark (or a clear multi-dataset plan with preliminary numbers) is needed before the architectural claim can be considered established.","section":null}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The new piece is the three-branch stack: raw ResNet-18, Haar-wavelet ResNet-18, and ViT-Tiny fused by fixed soft voting (0.50/0.40/0.10). Earlier hybrids stay in pixel space; earlier frequency CNNs lack the ViT. That combination, plus the reported Malimg numbers, is not in the cited literature.\n\nWhat they do well is the ablation hygiene. Table II isolates each branch cleanly. Wavelet alone beats raw CNN (0.9733 vs 0.9662 weighted F1). Adding ViT to the two CNN branches gives a further +0.0009. Grad-CAM on the Autorun.K/Yuner.A collapse is honest: the images are near-identical, so the model is not inventing the error. They also report FGSM degradation and parameter/latency numbers without overselling. Citations cover the recent hybrids (LeViT-MC, MalSort, ConvNeXt-Swin, etc.) fairly.\n\nSoft spots are real but proportionate. The strongest differentiator—wavelet lift on Swizzor.gen!E/I—rests on a handful of test images from a stratified 15 % split of a heavily imbalanced 2011 set. F1 jumps of 0.12–0.30 on ~15–20 samples are inside ordinary binomial noise; no CIs, bootstrap, or paired test appears. Overall hybrid gain over the best two-branch ensemble is only +0.0009 weighted F1. Single split, no code, no MaleVis/BIG2015, and white-box FGSM accuracy collapses to 7.8 % at ε=0.05. Soft-voting weights are free parameters chosen on validation F1, which is fine but not free of selection.\n\nThis is for people already working image-based malware family classification who want a cleanly documented frequency-domain branch. It is not a new detection paradigm and will not change operational practice. Math and data handling are standard and internally consistent; nothing circular. I would send it to referees—they can demand multi-dataset numbers, error bars on the Swizzor claim, and artifacts—but I would not cite it myself until those appear. Worth a reading-group slot only if someone is already deep in the niche.","headline":"Clean three-branch hybrid with honest ablations and a real wavelet idea, but the headline family-level gains sit on tiny Swizzor counts and a single 2011 split.","tokens_in":11125,"tokens_out":570,"would_cite":false,"duration_ms":4639,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Adding a wavelet frequency branch to a CNN–ViT hybrid measurably improves malware family classification on Malimg, especially for look-alike families.","keywords":["malware family classification","image-based malware detection","wavelet CNN","Vision Transformer","hybrid ensemble","Haar decomposition","Malimg","Grad-CAM"],"falsifier":"Retrain and evaluate the identical three-branch architecture on MaleVis or BIG2015 (or any post-2015 multi-family malware-image set) and check whether the wavelet branch still produces a statistically significant F1 gain over the raw-CNN-plus-ViT baseline, especially on the most confusable family pairs.","tokens_in":11125,"feed_emoji":"🛡️","tokens_out":683,"duration_ms":5809,"temperature":0.7,"pith_summary":"Malware binaries can be turned into grayscale images and classified by family using computer-vision models. Existing hybrids that combine convolutional nets and transformers still work only in ordinary pixel space, so they miss frequency-domain texture that separates families that look almost identical. ThreatVisionAI adds a third branch that feeds Haar-wavelet coefficient maps into a second CNN, then fuses the three soft-max outputs by fixed weighted voting. On the classic Malimg benchmark the full hybrid reaches 98.01 percent accuracy and a weighted F1 of 0.9742; the largest gains appear on the Swizzor variants and other minority classes that pure spatial models routinely confuse. Grad-CAM further shows that the single remaining systematic error (Autorun.K versus Yuner.A) is caused by near-identical visual structure in the data itself rather than by model failure. The work therefore claims that deliberately injecting multi-scale frequency features, not just deeper spatial or attention layers, is a practical route to better family-level triage of obfuscated malware.","feed_headline":"Wavelet branch lifts malware image classifier to 98% on Malimg","feed_subtitle":"Frequency features separate look-alike families that pure spatial CNN–ViT hybrids still confuse","key_machinery":"The wavelet CNN branch: a single-level 2-D Haar decomposition that turns each malware image into a four-channel (approximation + three detail) tensor, processed by a ResNet-18 trained from scratch and later fused by soft voting with weights 0.50 / 0.40 / 0.10.","core_discovery":"A three-branch hybrid that jointly extracts raw spatial features (ResNet-18), multi-scale Haar-wavelet frequency features (second ResNet-18), and global self-attention features (ViT-Tiny), then combines their probability vectors by weighted soft voting, yields higher family-classification accuracy and F1 on Malimg than any two-branch subset, with the wavelet branch supplying the decisive lift on visually similar families.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["Wavelet-CNN-ViT hybrid hits 98% malware family accuracy on Malimg","Three-branch hybrid reaches 98.01% on Malimg malware images","Wavelet features lift CNN-ViT malware classifier to 98% on Malimg","Frequency-aware hybrid CNN-ViT scores 98% on Malimg families","CNN-wavelet-ViT fusion classifies malware images at 98% accuracy"],"cache_read_input_tokens":128,"weakest_assumption_plain":"That the accuracy and discrimination gains measured on the 2011 Malimg collection of 9,465 grayscale images will still hold for larger, more recent malware corpora that the paper itself has not yet tested.","fun_headline_variants_meta":{"raw":{"variants":["Wavelet-CNN-ViT hybrid hits 98% malware family accuracy on Malimg","Three-branch hybrid reaches 98.01% on Malimg malware images","Wavelet features lift CNN-ViT malware classifier to 98% on Malimg","Frequency-aware hybrid CNN-ViT scores 98% on Malimg families","CNN-wavelet-ViT fusion classifies malware images at 98% accuracy"]},"model":"grok-4.5","effort":"low","cost_usd":0.004696,"raw_usage":{"total_tokens":1316,"prompt_tokens":704,"num_sources_used":0,"completion_tokens":92,"cost_in_usd_ticks":46960000,"prompt_tokens_details":{"text_tokens":704,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":520,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":704,"tokens_out":92,"duration_ms":4112,"temperature":1.0,"reasoning_tokens":520,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-12T00:53:40.015498+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Retrain and evaluate the identical three-branch architecture on MaleVis or BIG2015 (or any post-2015 multi-family malware-image set) and check whether the wavelet branch still produces a statistically significant F1 gain over the raw-CNN-plus-ViT baseline, especially on the most confusable family pairs.","supporting_citations":[],"review_version":1}