{"id":"97a58e2a-843c-4057-a0bf-1368bf2fc997","arxiv_id":"2505.01632","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A transfer-learned ResNet is reported to achieve 98.94% clean and 91.21% noisy digit recognition accuracy on Aurora-2, beating CNN and LSTM baselines, but the comparison is under-specified.","lead":"This paper applies a pre-trained ResNet model, fine-tuned with transfer learning, to recognize isolated digits from the Aurora-2 speech dataset in clean and noisy conditions. It reports higher accuracy than CNN and LSTM baselines, at 98.94% clean and 91.21% noisy, but provides little experimental detail.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table II baseline comparison is not controlled: the four competing accuracies are attributed to [17], an intrusion-detection paper, with no Aurora-2 protocol given for them.","rationale":"The paper's central claim is empirical: a fine-tuned ResNet achieves higher digit-recognition accuracy than four baselines on Aurora-2. For that claim to hold, the baselines must be evaluated on the same task, data split, features, and training setup. The paper never says this, and the sole attribution for the baselines is [17], a paper whose title indicates IoT intrusion detection. The natural reading is that the baseline numbers were taken from an unrelated domain. The Aurora-2 description in Section V-C is also inconsistent with the public corpus, so even the proposed-model results cannot be checked against a known benchmark. No code, data split, or feature specification is provided. Absence of code is not by itself a flaw, but together with the cross-domain baseline attribution it leaves the headline result without a controlled comparison. The reader and I converge on the same weakest assumption. A single reproducibility check—rerunning the four baselines on the actual Aurora-2 setup used for ResNet—would settle the issue, but until then the claim is unsupported. I would keep the reader's REJECT verdict, hence UNCHANGED.","tokens_in":7464,"tokens_out":5151,"duration_ms":52261,"concrete_test":"Obtain [17] and check whether the CNN/LSTM/BiLSTM/CNN-LSTM numbers in Table II match those reported for the intrusion-detection task. Then request or reconstruct the exact Aurora-2 split, feature representation, and training setup used for ResNet, and rerun all four baseline architectures on those same data. If the baselines cannot be reproduced on speech data, or if their accuracies shift by more than a percentage point under a controlled comparison, the claimed ResNet improvement is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that ResNet outperforms CNN, LSTM, BiLSTM, and CNN-LSTM rests entirely on Table II. The text provides no experimental protocol for those baselines: no feature front-end, no train/validation/test partition, no sample counts, and no training details beyond 'SGD with learning rate 0.001.' The only attribution is in the contributions list: 'Compare the results of ResNet with CNN, LSTM, BiLSTM, and a concatenated CNN-LSTM, as employed in [17]' — and [17] is an IoT intrusion-detection paper, not a speech recognition paper. Nothing in Section V states that the Table II baseline numbers were produced on the Aurora-2 task, or even on speech data. If the baseline accuracies are carried over from a different domain or dataset, the 1.98 percentage-point clean margin and 0.49 percentage-point noisy margin are meaningless. This is compounded by Section V-C, whose Aurora-2 description (4,824 isolated files, 40% test, clean/noisy binary split) does not match the standard Aurora-2 corpus (8,440 training utterances, 4,004 test utterances, 20 noise/SNR combinations), so even the proposed-model results are not tied to a reproducible public benchmark.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a transfer-learning-based ResNet-50 system for isolated digit recognition in clean and noisy conditions, evaluated on a dataset the authors identify as Aurora-2. The authors report 98.94% clean and 91.21% noisy accuracy for their model, and claim these results are significantly higher than those of CNN, LSTM, BiLSTM, and a concatenated CNN-LSTM model. The paper provides architectural descriptions of the source and target models, a brief related-work survey, and an experimental section with two accuracy tables and one WER figure.","tokens_in":7712,"tokens_out":4254,"duration_ms":41783,"significance":"If the reported accuracy were obtained under a well-defined, reproducible protocol with properly matched baselines, the result would be a modest but potentially useful data point on transferring image-pretrained residual networks to small-vocabulary noisy speech recognition. The paper does not establish this, however: the baseline accuracies are attributed to an intrusion-detection paper rather than to experiments on Aurora-2, the dataset description is internally inconsistent, and no statistical analysis is provided. The claimed significance is therefore not currently supported.","major_comments":[{"comment":"The central claim that ResNet outperforms CNN, LSTM, BiLSTM, and CNN-LSTM rests entirely on Table II, yet the four baseline accuracies are attributed to reference [17], which is a paper on IoT intrusion detection, not speech recognition. Section V-C describes the experimental protocol only for the proposed ResNet; no information is given about the feature front-end, data partition, noise conditions, or training details for the baselines. Without evidence that these numbers were produced on the same Aurora-2 task and setup, the margins in Table II (1.98 points clean, 0.49 points noisy) are meaningless and cannot support the claimed superiority.","section":"Table II and Section V-C"},{"comment":"The dataset description is irreconcilable between two sections. Section V-A describes the standard Aurora-2 corpus with 8,440 clean training utterances and 4,004 test utterances, while Section V-C states that the experiments used a total of 4,824 isolated digit files, split equally into 2,412 clean and 2,412 noisy, with about 40% used for testing. These numbers cannot both describe the same corpus. The manuscript never clarifies which split was actually used, how the 11-class softmax model is applied to Aurora-2's connected-digit task, or how the binary clean/noisy split relates to the standard multi-condition training mode. As a result, even the proposed-model results are not tied to a reproducible public benchmark.","section":"Sections V-A and V-C"},{"comment":"All reported accuracies are single numbers with no error bars, no number of runs, and no statistical significance tests. Given the small differences in Table II, particularly the 0.49 percentage-point noisy-mode margin between ResNet (91.21%) and CNN-LSTM (90.72%), the improvements could easily be within run-to-run variance. The authors should report means and standard deviations over multiple independent runs and perform a significance test, for example a paired test across test sets, before claiming a significant improvement.","section":"Tables I and II"}],"minor_comments":[{"comment":"The phrase 'the authors evaluate' in the abstract should be 'we evaluate', and the sentence 'Addressing the detrimental impact... has been a persistent and significant research focus' is repeated nearly verbatim in the introduction; this should be streamlined.","section":"Abstract and Introduction"},{"comment":"The sentence introducing ResNet as 'introduced by He et al. in 2015' is cited to reference [22], which is a bird-call recognition paper, not the original ResNet paper. The correct citation is reference [10] (He et al., 2016).","section":"Section II, reference [22]"},{"comment":"The final sentence of Section IV-B says the output dense layer uses softmax 'to classify the input 11 classes into one of several classes'; this is confusing and should be rewritten to say the softmax layer produces a probability distribution over the 11 digit classes.","section":"Section IV-B"},{"comment":"The caption reads 'WER (%) Recognition rates obtained by CNN, LSTM, BiLSTM, and ResNet', but word error rate and recognition rate are distinct metrics and should not be conflated; the caption and the figure need to state clearly which quantity is plotted.","section":"Figure 4"}],"recommendation":"reject","confidential_remarks":"This manuscript appears to be a lightly revised conference paper. The use of an intrusion-detection paper as the source of speech-recognition baselines is a serious problem that undermines the experimental comparison, and the internal dataset discrepancy suggests the experimental section was not carefully prepared. I do not see a path to acceptance without re-running the entire comparison under a clearly defined Aurora-2 protocol, which goes beyond a normal revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take on arXiv:2505.01632. The headline claim is a fine-tuned ResNet hitting 98.94% clean / 91.21% noisy on Aurora-2, beating CNN, LSTM, BiLSTM, and CNN-LSTM. The problem is the comparison is not controlled. The baseline numbers in Table II are attributed to [17], which is an IoT intrusion-detection paper, not speech recognition. Nothing in the text says those baselines were re-run on Aurora-2. If they were carried over from a different domain, the claimed margins are meaningless. The reader's stress-test nailed this, and I verified it against the full text: the contributions list says 'as employed in [17]', and [17] is indeed 'Enhancing IoT Security with CNN and LSTM-Based Intrusion Detection Systems.' No protocol for the baselines is given.\n\nOn top of that, the dataset description contradicts itself. Section V-A describes the standard Aurora-2 with 8,440 clean training utterances and 4,004 test utterances. Section V-C says the dataset has 4,824 isolated digit files, 40% test, split into clean/noisy. Those cannot both be true. So even the proposed-model numbers are not anchored to a reproducible public benchmark.\n\nWhat's good? The architecture is clearly described: a ResNet-50 with the early layers kept, a small custom stack of residual blocks, dropout, and an 11-way softmax. That is a reasonable engineering setup. The idea of applying transfer learning to Aurora-2 digit recognition is not new — the paper itself cites several works that do the same — but it is a legitimate exercise. The accuracy is plausible for this task. So it's not nonsense; it's just unverified.\n\nThe flaws are load-bearing. No code, no error bars, no statistical tests, no details on features beyond 'Mel-frequency', no SNR-wise breakdown. That would be acceptable for a short conference paper if the comparison were sound, but it's not. I would want the authors to re-run the baselines on the same Aurora-2 setup, report the split and features, and give error bars or at least a few runs.\n\nWho is this for? Someone who wants a quick look at how a transfer-learned ResNet does on a small-vocabulary noisy digit task. But the numbers can't be trusted until the protocol is fixed. I wouldn't cite it.\n\nShould it get peer review? Yes — a serious referee could identify the baseline problem in thirty minutes and the authors could fix it. The paper is not beyond repair, but it needs major revision, not acceptance. If the authors redo the experiments properly and the result holds, there is a modest conference-level paper here.\n\nFor a reading group, maybe, as a case study in how baseline borrowing goes wrong. Not for the science.\n\nHope that helps.","headline":"Transfer-learned ResNet on Aurora-2 gives plausible numbers, but the baseline comparison is borrowed from an intrusion-detection paper and the dataset description contradicts itself, so the claim is unverified.","tokens_in":8228,"tokens_out":4225,"would_cite":false,"duration_ms":39268,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A transfer-learned ResNet reaches 98.94% clean and 91.21% noisy digit accuracy on Aurora-2, beating four deep baselines.","keywords":["speech recognition","transfer learning","ResNet","Aurora-2","noisy speech","Mel-frequency features","residual learning","digit recognition"],"falsifier":"Re-run CNN, LSTM, BiLSTM, and CNN-LSTM on the same Aurora-2 clean and multi-condition training sets, using the same Mel-frequency features, optimizer, learning rate, and train/test split as the ResNet. If any of these baselines reaches or exceeds 98.94% in clean conditions or 91.21% in noisy conditions, the claimed superiority would not stand.","tokens_in":7292,"feed_emoji":"🎙️","tokens_out":10663,"duration_ms":84366,"temperature":0.7,"pith_summary":"The paper claims that a ResNet-50 adapted through transfer learning to Mel-frequency feature inputs is a stronger acoustic frontend than CNN, LSTM, BiLSTM, and CNN-LSTM for the Aurora-2 digit-recognition task. On that benchmark the fine-tuned ResNet reaches 98.94% accuracy in clean conditions and 91.21% under multi-condition noise, improving on its own from-scratch training by about four and eight points respectively. If the result holds, it would show that residual connections plus ImageNet-pretrained weights transfer well to noisy small-vocabulary speech, a setting where labeled data are often scarce.","feed_headline":"Fine-tuned ResNet beats CNN, LSTM on noisy Aurora-2 digits","feed_subtitle":"An ImageNet-pretrained ResNet reaches 98.94% clean, 91.21% noisy; baseline numbers need a same-setup check.","key_machinery":"The load-bearing object is the residual block with skip connections, formulated as $y = f(z) + z$, where $f(z)$ is the layer output and $z$ the block input. The target model stacks three such blocks (64, 128, then 256 filters), each followed by max-pooling, and feeds the flattened output into a 128-unit dense layer with dropout 0.5 and an 11-way softmax. Transfer learning supplies the initial weights: the source ResNet-50, pretrained on ImageNet, is adapted to the speech task, so the first convolutional layers keep generic feature detectors while the final layers are replaced and fine-tuned. This combination is what the paper credits for the accuracy gain in both clean and noisy modes.","core_discovery":"The central discovery is that a 50-layer residual network, after being pretrained on ImageNet and then fine-tuned on Mel-frequency spectral features of Aurora-2 digits, recognizes isolated English digits with 98.94% accuracy in clean conditions and 91.21% in noisy conditions spanning four noise types and SNRs from 20 dB to 5 dB plus clean. The paper presents this as an improvement over four deep baselines—CNN, LSTM, BiLSTM, and concatenated CNN-LSTM—and over the same ResNet trained without transfer learning. The authors interpret the residual skip connections, which let the network learn the residual $y = f(z) + z$, together with transfer learning from a large visual domain, as the mechanism that makes deep training stable and keeps the model robust when noise is added.","pith_inferences":["Because the baselines are imported from an unrelated intrusion-detection study, the fair comparison would require re-evaluating those architectures on the same Aurora-2 split; until then the reported margins over baselines are best read as indicative rather than measured.","The same transfer-learning recipe could be tested on larger vocabulary or continuous speech benchmarks such as Aurora-4 or CHiME to see whether the ImageNet-pretrained frontend advantage persists beyond isolated digits.","A direct ablation varying the input feature between MFCC and log-Mel spectrogram would clarify whether the gain comes from the residual architecture, the pretrained weights, or the feature choice."],"forward_implications":["For small-vocabulary digit recognition, transfer learning from an ImageNet-pretrained ResNet raises accuracy over training from scratch by roughly four points in clean and eight points in noisy conditions.","The same fine-tuned ResNet yields lower word error rates than CNN, LSTM, BiLSTM, and CNN-LSTM in both clean and noisy modes, with the largest gap in noise.","Multi-condition training with four noise types and several SNRs is sufficient for the transferred ResNet to generalize to unseen noise conditions in the Aurora-2 test sets.","The 11-class softmax output makes the model directly usable as a frontend for connected-digit recognition tasks without modification."],"supporting_citations":[{"why":"supplies the residual learning architecture and skip-connection design central to the model.","marker":"[10]"},{"why":"provides the CNN, LSTM, BiLSTM, and CNN-LSTM baseline accuracies that the claimed improvement is measured against.","marker":"[17]"},{"why":"supplies the transfer-learning fine-tuning procedure used to adapt the pretrained model.","marker":"[21]"},{"why":"gives the residual formulation with skip connections that defines each block of the target network.","marker":"[32]"}],"fun_headline_variants":["ResNet transfer learning hits 98.94% clean, 91.21% noisy on Aurora-2","98.94% clean, 91.21% noisy: ImageNet-pretrained ResNet on digits","Transfer-learned ResNet outdoes CNN, LSTM on noisy speech recognition","ResNet fine-tuned from ImageNet: 98.94% clean, 91.21% noisy","ImageNet-pretrained ResNet lifts Aurora-2 digit accuracy to 98.94%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that ResNet beats the four baselines assumes the baseline accuracies were measured on the same Aurora-2 data partition, features, and training conditions as the ResNet; the paper instead cites those numbers from a separate study, so the comparison may not be controlled.","fun_headline_variants_meta":{"raw":{"variants":["ResNet transfer learning hits 98.94% clean, 91.21% noisy on Aurora-2","98.94% clean, 91.21% noisy: ImageNet-pretrained ResNet on digits","Transfer-learned ResNet outdoes CNN, LSTM on noisy speech recognition","ResNet fine-tuned from ImageNet: 98.94% clean, 91.21% noisy","ImageNet-pretrained ResNet lifts Aurora-2 digit accuracy to 98.94%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000699,"raw_usage":{"total_tokens":3132,"prompt_tokens":896,"completion_tokens":2236,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":512,"completion_tokens_details":{"reasoning_tokens":2110}},"tokens_in":512,"tokens_out":2236,"duration_ms":14158,"temperature":1.0,"reasoning_tokens":2110,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:13:34.127421+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run CNN, LSTM, BiLSTM, and CNN-LSTM on the same Aurora-2 clean and multi-condition training sets, using the same Mel-frequency features, optimizer, learning rate, and train/test split as the ResNet. If any of these baselines reaches or exceeds 98.94% in clean conditions or 91.21% in noisy conditions, the claimed superiority would not stand.","supporting_citations":[{"cited_title":"Enhancing iot security with cnn and lstm-based intrusion detection systems,","cited_arxiv_id":null,"evidence_quote":"provides the CNN, LSTM, BiLSTM, and CNN-LSTM baseline accuracies that the claimed improvement is measured against."},{"cited_title":"A customized residual neural network and bi-directional gated recurrent unit-based automatic speech recognition model,","cited_arxiv_id":null,"evidence_quote":"gives the residual formulation with skip connections that defines each block of the target network."}],"review_version":1}