{"id":"faec1f39-92a2-44a3-8a3a-0ff32dfbb62b","arxiv_id":"2508.20193","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A reconstruction-pretrained Vision Transformer reportedly recognizes modulation types with 15% labeled data at 71% accuracy, surpassing its fully supervised counterpart, though data handling errors undercut the claim.","lead":"The paper trains a Vision Transformer to reconstruct distorted radio signals, then fine-tunes it to recognize modulation types with only a fraction of the labels. The authors report that reconstruction-based pretraining beats contrastive learning when labels are scarce, but inconsistent data reporting and missing code undercut the result.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Dataset construction arithmetic is internally inconsistent: 16 classes × 1,000 samples/class/SNR cannot total 220,000 for the stated SNR ranges, so the reported accuracy tables rest on an undefined dataset.","rationale":"The paper's central claim is that reconstruction-only pretraining gives the strongest low-label accuracy and that the semi-supervised ViT approaches ResNet-level performance with only 15–20% labels. All evidence for this claim is experimental, reported in Tables 4–7. Those tables are defined over a dataset whose construction is described inconsistently and arithmetically impossible as stated: 16 classes × 1,000 samples per class per SNR cannot yield 220,000 total samples for any integer number of SNR levels in the stated ranges (−2 to +21, −2 to +20, or −2 to +22 dB). If the actual subsampling differs from the description—which it must—then the class balance, SNR coverage, train/validation/test split, and per-SNR evaluation all change, and the reported accuracies are not reproducible or interpretable. This is not a minor typo; it undermines the validity of every quantitative comparison in the paper. The duplicate 68.21 value in two different experimental settings is a further red flag, suggesting possible mislabeling or repeated results. Since the paper provides no code, no data-release link, and no formal verification, there is no independent check on any of the reported numbers. The reader's verdict of REJECT with moderate confidence is appropriate; my stress-test identifies the same load-bearing concern and does not change that verdict.","tokens_in":11894,"tokens_out":2923,"duration_ms":32493,"concrete_test":"Obtain or reconstruct the exact RML2018.01A subset used in the paper (e.g., via the authors' code or a precise file list). Count the number of samples per modulation class per SNR level and the total number of samples. If the total is not 220,000, or if any class does not have exactly 1,000 samples at each SNR in the stated range, then every reported accuracy in Tables 4–7 must be recomputed from the correctly defined subset. Also rerun the two settings that produced 68.21% (Scenario 1 at 10% labels and supervised ViT at 100% labels) to check whether the duplicate is real or a typographical/computational error.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5.2 states: 'For each class and SNR, 1,000 samples are selected, resulting in a total of 220,000 signals' over 16 classes. This requires 16,000 samples per SNR level, so 220,000 would imply 13.75 SNR levels—not an integer. The SNR range is given as −2 to +21 dB in §5.1/§5.2, −2 to +20 dB in the Table 4 caption and §5.3, and −2 to +22 dB in the Figure 4 caption. For any integer SNR count in these ranges (22–25 levels), the total would be 352,000–400,000, not 220,000. Even 14 levels would give 224,000, not 220,000. The actual filtered subset is therefore not the one described. This is load-bearing because every reported accuracy—including the central claim that reconstruction-only pretraining reaches 68.21% at 10% labels and that ViT (15% semi) reaches 71.01% versus ResNet's 78.50%—depends on exactly how classes and SNR levels were subsampled and split. The test-set size (~55,000 samples in §5.3.5) and the '700 test samples per SNR' in §5.3.6 further depend on this undefined construction. Additionally, the identical accuracy 68.21 appears in Table 4 (Scenario 1, 10% labels) and Table 5 (fully supervised ViT, 100% labels), which is consistent with a data/computation mix-up and further undermines confidence. No code or data-release link is provided, so the numbers cannot be externally audited.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a unified ViT-based framework for automatic modulation recognition (AMR) that combines supervised classification with self-supervised reconstruction and, optionally, contrastive learning. A ViT encoder processes I/Q signals as patches; a lightweight convolutional decoder reconstructs augmented signals back to their original forms; a linear head on mean-pooled patch embeddings performs classification. The authors claim that reconstruction-only pretraining gives the strongest low-label performance, that the semi-supervised ViT with 15% labels outperforms fully supervised CNN and ViT baselines and approaches ResNet accuracy, and that the method is robust across SNR levels on RML2018.01A.","tokens_in":1306,"tokens_out":1241,"duration_ms":64031,"significance":"If the empirical claims were reliable, the paper would make a useful, simple contribution to low-label AMR: the architecture is clearly described, avoids teacher-student complexity, and the reconstruction-to-original anchoring is a sensible pretext task. The pseudo-label self-training loop does not touch the held-out test evaluation, so there is no derivation-level circularity. Unfortunately, the experimental validation is undermined by multiple internal inconsistencies in the dataset construction and in the reported numbers; no code or data-release link is provided. As a result, the central empirical claims cannot currently be accepted.","major_comments":[{"comment":"The dataset construction is arithmetically impossible as stated. Section 5.2 says 16 classes, 1,000 samples per class per SNR, and a total of 220,000 signals. That would require 13.75 SNR levels. The SNR range is given inconsistently as -2 to +21 dB (Section 5.2), -2 to +20 dB (Table 4 caption and Section 5.3), and -2 to +22 dB (Figure 4 caption); no integer number of SNR levels in any of these ranges gives 220,000 total samples (e.g., 24 levels would give 384,000). The 70/10/20 split implies a test set of about 44,000 samples, but Section 5.3.5 reports about 55,000 test samples, and Section 5.3.6 reports 700 test samples per SNR. These numbers are mutually incompatible. Because every reported accuracy depends on the actual subsample and split, this is a load-bearing error: the results are attached to an undefined dataset.","section":"Section 5.2, Table 4 caption, Section 5.3.5, Figure 4 caption"},{"comment":"The value 68.21 appears twice: as Scenario 1 (Reconstruction Only) with 10% labels in Table 4, and as the fully supervised ViT with 100% labels in Table 5. The paper's central claim is label efficiency, so a fully supervised model using ten times as many labels should not produce an identical overall accuracy to the 10%-label model. This strongly suggests a table or computation mix-up and further erodes confidence in the other reported numbers.","section":"Tables 4 and 5"},{"comment":"The claim that the proposed model 'outperforms supervised CNN and ViT baselines in low-label regimes' is not supported by the reported experiments. Table 5 gives supervised CNN, ResNet, and ViT results only at 100% labels. Table 4 compares three pretraining scenarios against each other at 10%, 15%, and 20% labels but contains no CNN, ResNet, or supervised ViT baselines at those same label fractions. Without matching-label-fraction baseline comparisons, the headline low-label superiority claim is not demonstrated.","section":"Section 5.3, Tables 4 and 5"}],"minor_comments":[{"comment":"The t-SNE visualization is explicitly for 10 modulation classes, while the experiments use 16 classes. The claim of well-separated clusters under low-label regimes is therefore only verified on a subset; please provide the full 16-class visualization or qualify the claim.","section":"Figure 3"},{"comment":"The SNR range is stated inconsistently across the paper. Please unify: Section 5.1 says -2 to +21 dB, Section 5.3 and Table 4 say -2 to +20 dB, and Figure 4 says -2 to +22 dB.","section":"Section 5.1 / 5.3"},{"comment":"No code or data-release link is provided, so the reported numbers cannot be externally audited. Also, reference [22] is incompletely formatted ('A. names not provided in abstract'), and Section 2.4 includes several citations that are not clearly connected to the AMR problem; please tighten.","section":"Section 5.2 / Related Work"},{"comment":"The text 'our ViT-based framework outperforms both CNN and ResNet baselines across all labeling ratios' is stronger than the data shown; Table 5 has only 100%-label baselines, and the semi-supervised ViT (71.01%) is below supervised ResNet (78.50%). Please align the wording with what is actually tabulated.","section":"Section 5.3.2 / Table 5"}],"recommendation":"reject","confidential_remarks":"The paper's core idea is plausible, but the empirical section contains arithmetic impossibilities in the dataset description, an unexplained identical accuracy in two key tables, and missing baseline comparisons at matched label fractions. Without code or a corrected, re-run experimental protocol, the central claims cannot be trusted. This is not a case of mere presentation issues; the load-bearing numbers themselves are inconsistent."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Briefly: the paper asks a legitimate empirical question—does reconstruction-only pretraining beat contrastive-only or joint objectives on I/Q modulation recognition under limited labels—and the per-class breakdown is thorough. If the numbers were trustworthy, the finding that reconstruction is the strongest low-label objective would be a useful incremental contribution. The writing is organized and the method is clearly specified.\n\nThe numbers are not trustworthy. The dataset description is arithmetically impossible: 16 classes × 1,000 samples/class/SNR cannot total 220,000 for any integer SNR count in the stated ranges, which themselves disagree (−2 to +21 in §5.1/5.2, −2 to +20 in Table 4 and §5.3, −2 to +22 in Figure 4). The identical accuracy 68.21 appears as both Scenario 1 at 10% labels (Table 4) and supervised ViT at 100% labels (Table 5). The t-SNE shows 10 classes while the paper claims 16, and the test set is said to be roughly 55,000 samples, but 20% of 220,000 is 44,000. These are not peripheral typos; the central claims—reconstruction-only beats contrastive-only, and 15% labels approach ResNet—rest on this undefined dataset. No code or data release is provided, so the numbers cannot be checked independently.\n\nThere is also a citation gap: the paper does not compare against the semi-supervised AMR methods it cites (SSCL-AMC, SemiAMC, GAF-MAE, etc.), so the implied state-of-the-art improvement is unsupported even on its own terms.\n\nThe core idea is plausible and the paper is structured sensibly, but the internal contradictions make the evidence unusable. A corrected version with a consistent dataset description, matching SNR ranges, an explanation of the duplicated 68.21, and ideally code or a public data split would be worth revisiting. As written, it is not ready for peer review; a serious editor should desk reject it and invite resubmission after the numbers are fixed. It could still serve as a reading-group case study on why dataset arithmetic matters.","headline":"Plausible idea and a legitimate empirical question, but the dataset construction is arithmetically impossible and the reported numbers don't hang together; the evidence as presented is not usable.","tokens_in":12808,"tokens_out":3334,"would_cite":false,"duration_ms":34812,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A reconstruction-only pretraining objective, rather than contrastive or joint objectives, gives a Vision Transformer the best accuracy on automatic modulation recognition with 10–20% labeled data, approaching fully supervised ResNet perform","keywords":["automatic modulation recognition","Vision Transformer","semi-supervised learning","self-supervised pretraining","reconstruction loss","I/Q signals","label efficiency","SNR robustness"],"falsifier":"Re-run Scenario 1 (reconstruction-only) and the baselines on a subset with an explicitly documented sample count, for instance 500 samples per class per SNR for a stated set of integer SNR levels, and check whether Table 4's ordering and the 71.0% figure at 15% labels reproduce. A second check is to zero out the reconstruction loss while keeping the decoder and augmentations, which should lower low-label accuracy if reconstruction is the active ingredient.","tokens_in":11808,"feed_emoji":"📡","tokens_out":7433,"duration_ms":78117,"temperature":0.7,"pith_summary":"This paper tries to show that a Vision Transformer can do automatic modulation recognition with very few labels if its pretraining objective is reconstruction rather than contrastive invariance. The central question is whether forcing the encoder to map augmented I/Q signals back to their original form teaches features that transfer to classification under data scarcity. On the RML2018.01A benchmark, the paper finds that reconstruction-only pretraining beats both contrastive-only and joint reconstruction+contrastive objectives at 10%, 15%, and 20% labels, and that a semi-supervised ViT with 15% labels approaches a fully supervised ResNet. If true, this gives wireless systems a single-stage recipe for label-efficient spectrum monitoring.","feed_headline":"Reconstruction beats contrastive for low-label signal recognition","feed_subtitle":"A semi-supervised Vision Transformer matches fully supervised ResNet on modulation recognition with a fifth of the labels.","key_machinery":"The reconstruction-to-original branch: a lightweight convolutional decoder maps augmented I/Q patches back to the original signal, while the ViT encoder's patch embeddings are averaged (rather than using a class token) for both reconstruction projection and classification. This forces the encoder to preserve fine phase and amplitude structure instead of learning only contrastive invariance, and the reconstruction loss can be trained jointly with or without a contrastive loss, letting the paper isolate the contribution of reconstruction.","core_discovery":"On RML2018.01A with 16 modulation classes and SNR −2 to +20 dB, a ViT pretrained by reconstructing augmented I/Q signals back to their originals, then fine-tuned on 10–20% of labels, reaches 68.2–73.7% accuracy. At 15% labels it reaches 71.0%, beating a fully supervised CNN (61.8%) and a fully supervised ViT (68.2%) and approaching a fully supervised ResNet (78.5%). Reconstruction-only pretraining outperforms reconstruction+contrastive and contrastive-only across every tested label ratio, with the largest gains on high-order QAM and APSK classes, where contrastive-only learning largely collapses.","pith_inferences":["The reconstruction-to-original objective can be read as denoising in I/Q space: it likely forces the encoder to keep phase and amplitude detail that rotation and flip contrastive views discard. A direct test would be ablating each augmentation family to see which one drives the low-label gain.","The same pretraining should transfer across I/Q datasets better than contrastive invariance; evaluating on a different modulation benchmark or on hardware-varied channel data would test this directly.","A masked-autoencoder variant that reconstructs only masked I/Q patches, rather than full augmented signals, is a natural next step that could push the 10%-label regime lower.","The absolute accuracy figures are likely tied to the exact class/SNR balance, so comparing them across papers requires reproducing that balance; the paper's own stated counts are internally inconsistent on this point."],"forward_implications":["Practitioners can drop the contrastive branch: reconstruction-only pretraining outperforms reconstruction+contrastive and contrastive-only at every label ratio (68.2/71.0/73.7% vs 66.4/70.2/71.4% and 53.4/54.4/56.4%).","Label budgets for AMR can shrink to 10–20%: with 15% labels the semi-supervised ViT reaches 71.0%, above the fully supervised CNN (61.8%) and ViT (68.2%) and within about 7.5 points of the fully supervised ResNet (78.5%).","High-order constellations are where reconstruction helps most: at 15% labels, reconstruction-only beats contrastive-only by large margins on 32APSK, 64APSK, 128APSK, 16QAM, 32QAM, and 128QAM.","The single-stage pipeline avoids teacher–student architectures and multi-stage pretraining, simplifying deployment for cognitive radio and spectrum monitoring.","SNR robustness is preserved: the framework stays above roughly 70% accuracy for SNR ≥ 5 dB, so the low-label gains are not limited to clean signals."],"supporting_citations":[{"why":"Supplies the RML2018.01A benchmark dataset used for every experiment and comparison.","marker":"[4]"},{"why":"Defines the rotation, flip, and Gaussian-noise augmentations that create the distorted views the reconstruction branch must undo.","marker":"[30]"},{"why":"Defines the scaling, time-warping, and magnitude-warping augmentations used in the same pretraining pipeline.","marker":"[31]"},{"why":"Provides the contrastive Transformer-based semi-supervised AMR approach that the single-stage reconstruction design is compared against conceptually.","marker":"[3]"},{"why":"Shows that reconstruction-based pretraining via masked autoencoders already helps AMR with very few labels, motivating the paper's reconstruction baseline.","marker":"[15]"},{"why":"Introduces a rotation-based self-contrastive model for semi-supervised radio modulation classification, representing the contrastive-only family.","marker":"[7]"},{"why":"Establishes the CNN-based supervised deep-learning approach to modulation classification that serves as a baseline.","marker":"[1]"},{"why":"Teacher-student contrastive framework whose multi-stage complexity the paper argues its unified pipeline avoids.","marker":"[9]"}],"fun_headline_variants":["ViT with reconstruction nears ResNet using only 15% labels","Reconstruction, not contrastive, unlocks label-efficient modulation recognition","Self-supervised reconstruction tops contrastive for scarce-label radio signals","Reconstruction-driven ViT rivals ResNet with 80% fewer labels"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The load-bearing premise is that the RML2018.01A subset is built exactly as described; Section 5.2 claims 1,000 samples per class per SNR across 16 classes and a total of 220,000, which no integer number of SNR levels in the stated −2 to +21 dB range can produce, so the true subsampling is the assumption on which all reported accuracies rest.","fun_headline_variants_meta":{"raw":{"variants":["ViT with reconstruction nears ResNet using only 15% labels","Reconstruction, not contrastive, unlocks label-efficient modulation recognition","Self-supervised reconstruction tops contrastive for scarce-label radio signals","Reconstruction-driven ViT rivals ResNet with 80% fewer labels"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001148,"raw_usage":{"total_tokens":4583,"prompt_tokens":714,"completion_tokens":3869,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":458,"completion_tokens_details":{"reasoning_tokens":3794}},"tokens_in":458,"tokens_out":3869,"duration_ms":31058,"temperature":1.0,"reasoning_tokens":3794,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T15:13:30.802924+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run Scenario 1 (reconstruction-only) and the baselines on a subset with an explicitly documented sample count, for instance 500 samples per class per SNR for a stated set of integer SNR levels, and check whether Table 4's ordering and the 71.0% figure at 15% labels reproduce. A second check is to zero out the reconstruction loss while keeping the decoder and augmentations, which should lower low-label accuracy if reconstruction is the active ingredient.","supporting_citations":[{"cited_title":"Data augmentation for deep learning-based radio modulation classification,","cited_arxiv_id":null,"evidence_quote":"Defines the rotation, flip, and Gaussian-noise augmentations that create the distorted views the reconstruction branch must undo."},{"cited_title":"Improving modulation recognition using time series data augmentation via a spatiotemporal multi-channel framework,","cited_arxiv_id":null,"evidence_quote":"Defines the scaling, time-warping, and magnitude-warping augmentations used in the same pretraining pipeline."},{"cited_title":"A transformer-based contrastive semi-supervised learning framework for automatic modula- tionrecognition,","cited_arxiv_id":null,"evidence_quote":"Provides the contrastive Transformer-based semi-supervised AMR approach that the single-stage reconstruction design is compared against conceptually."},{"cited_title":"Gaf-mae: A self-supervised automatic modulation classification method based on gramian angular field and masked autoencoder,","cited_arxiv_id":null,"evidence_quote":"Shows that reconstruction-based pretraining via masked autoencoders already helps AMR with very few labels, motivating the paper's reconstruction baseline."},{"cited_title":"Self-contrastive learn- ing based semi-supervised radio modulation classification,","cited_arxiv_id":null,"evidence_quote":"Introduces a rotation-based self-contrastive model for semi-supervised radio modulation classification, representing the contrastive-only family."},{"cited_title":"A deep learning method based on convolutional neural network for automatic modulation classi- fication of wireless signals,","cited_arxiv_id":null,"evidence_quote":"Establishes the CNN-based supervised deep-learning approach to modulation classification that serves as a baseline."},{"cited_title":"Sscl-amc: A self-supervised automatic modulation classification method via dynamic augmentation and ensemble learning,","cited_arxiv_id":null,"evidence_quote":"Teacher-student contrastive framework whose multi-stage complexity the paper argues its unified pipeline avoids."}],"review_version":1}