{"id":"a82698a9-8774-4fb3-97c1-d3ee711d3f48","arxiv_id":"2411.18636","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":0.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"A review that reports Conformer as the most accurate (5.27% error) and CNN as the lightest of four architectures on Voxlingua6, but contributes no new experiments or derivations.","lead":"This paper surveys four convolution-based neural network architectures and compares them for speech recognition, speaker identification, emotion detection, and speech enhancement. A reader might check it for comparative accuracy and model size numbers for these speech models, but the comparison is based on one unpublished source.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The comparative accuracy claim in Section IV-B rests entirely on tables copied from the unpublished Easy Chair preprint [9], with no experimental methodology reproduced here; the conclusion's statement 'We tested these models' is contradicted by the manuscript's own content.","rationale":"The reader's verdict is REJECT with high confidence, and the weakest assumption is exactly the comparability and provenance of the numerical results in Tables II-III. My independent stress-test identifies the same load-bearing concern: the paper's only quantitative contribution is a comparison lifted from an unpublished preprint, and the paper itself provides no experimental methodology or independent evidence to validate those numbers. The claim 'We tested these models' in the conclusion is contradicted by the absence of any experiment in the manuscript, making the support for the central claim internally inconsistent rather than merely weak. I considered whether the standard equations for CNN, Conformer, ResNet, and CRNN could count as a parameter-free derivation supporting the survey's value; however, those equations are textbook material and do not establish the comparative accuracy claim. The concrete test I propose, an independent reimplementation with controlled conditions, would settle whether Conformer really is the best among the four architectures on Voxlingua6. Until that check is performed, rejection remains appropriate, and my read does not change the reader's verdict.","tokens_in":9713,"tokens_out":1919,"duration_ms":23043,"concrete_test":"Independently reproduce the comparison by training CNN, CRNN, ResNet, and Conformer on VoxForge with an identical feature extraction pipeline (e.g., 80-dimensional log-Mel filterbanks) and identical training budget (optimizer, learning rate schedule, epochs, seed) and evaluating on the Voxlingua6 Dev set. If the resulting error rates do not place Conformer first at approximately 5.27% with CNN at 7.18%, CRNN at 11.35%, and ResNet at 8.56%, the central comparative claim in Section IV-B is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of the paper is the quantitative ranking in Section IV-B: Conformer outperforms CNN, CRNN, and ResNet on the Voxlingua6 Dev set (error rates 5.27%, 7.18%, 11.35%, 8.56% respectively, Table III), with parameter counts from Table II. Every comparative conclusion in the abstract and conclusion depends on these numbers. The manuscript provides no experimental protocol: no feature extraction details, no training hyperparameters, no data splits beyond the speaker counts in Table I, and no code or logs. The cited source [9] is an unpublished Easy Chair preprint whose contents are not reproduced or independently verified. Moreover, Section V states 'We tested these models on training cost, model size, accuracy, and inference speed,' which is not supported by any experiment described in the paper; the only numeric results are attributed to [9], not to work by the authors. This is an internal inconsistency, not merely a disagreement with consensus. Additional red flags, such as the unresolved citation '[?]' in Eq. (26)-(31) area and the placeholder author 'J. Doe' in reference [14], further indicate that the manuscript has not undergone careful verification. For the central claim to hold, Tables II and III must be accurate, fairly comparable measurements produced under identical conditions for all four architectures. That condition is load-bearing and currently unverified. The paper provides no independent evidence—no derivation, no reproducible code, no new experiments—that would substitute for this missing verification.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript presents a survey-style account of four convolution-based speech processing architectures (CNN, Conformer, ResNet, and CRNN). It gives standard mathematical definitions for convolution, pooling, normalization, attention, residual connections, and LSTM cells, then discusses applications in speech recognition, speaker identification, and emotion detection. The paper's main empirical claim appears in Section IV, where it reports parameter counts and error rates for the four models on the Voxlingua6 Dev set, attributing these numbers to the unpublished preprint [9]. Section V states that the authors tested the models on training cost, model size, accuracy, and inference speed, but no experimental setup, training details, evaluation protocol, or code is provided in the manuscript.","tokens_in":10001,"tokens_out":4595,"duration_ms":49935,"significance":"If the comparative ranking in Section IV-B were properly supported, it would be a useful reference for practitioners choosing among CNN, Conformer, ResNet, and CRNN for speech tasks. The manuscript also provides a readable compilation of standard equations for these architectures, and it correctly identifies relevant application areas. These strengths are, however, outweighed by the fact that the central quantitative claim is not supported by any experiment reported here: the accuracy and parameter-count tables are taken from an unpublished external source, and the manuscript does not reproduce the experimental conditions. As a survey, it lacks critical synthesis and contains unresolved placeholders. The paper therefore does not currently meet the standard for archival publication.","major_comments":[{"comment":"The central empirical claim, that the Conformer has the lowest error rate (5.27%) on the Voxlingua6 Dev set and outperforms CNN, CRNN, and ResNet, rests entirely on numbers attributed to [9], an unpublished Easy Chair preprint. The manuscript reports no feature extraction details, training hyperparameters, optimizer choices, data splits, evaluation protocol, code, or logs. Section V states that 'We tested these models on training cost, model size, accuracy, and inference speed,' but no such testing is described in the paper. This internal inconsistency makes the comparative conclusion unverifiable from the manuscript alone.","section":"IV-B and V"},{"comment":"The training-cost and speed comparisons are asserted without supporting measurements. Section IV-A equates training cost with parameter count, but Table II only lists parameter values; no wall-clock training time, FLOPs, memory usage, or hardware details are given. Section IV-C makes qualitative speed claims (CNN is most efficient, Conformer balances speed and accuracy) but no inference latency or throughput numbers appear anywhere in the paper or in the figures. Since Section V claims these quantities were tested, the absence of such data is a load-bearing gap.","section":"IV-A and IV-C"},{"comment":"The provenance and comparability of the accuracy numbers are unclear. Table I reports speaker counts by language for train, validation, and test sets, but it does not specify whether these counts refer to VoxForge, Voxlingua6, or both, nor how the Voxlingua6 Dev set used in Table III is constructed. Additionally, the text says the Conformer 'on average outperforms' the other architectures, but Table III provides a single error rate per model with no standard deviation, number of runs, or significance test. Without this information, the ranking in Table III cannot be evaluated.","section":"IV-B and Table I"}],"minor_comments":[{"comment":"The LSTM equations are followed by the unresolved citation '[ ?]'; this placeholder should be replaced with a specific reference or removed.","section":"II-D, Eq. (27)-(31)"},{"comment":"Reference [14] lists 'J. Doe' as an author; this is a placeholder and must be corrected before any resubmission.","section":"References"},{"comment":"The title contains a grammatical error: 'and it’s Applications' should be 'and its Applications'.","section":"Title"},{"comment":"Several equations are repeated verbatim across sections, for example H_t = LSTM(CNN(X_t), H_{t-1}) appears as Eqs. (37), (41), and (46), and Y = F(X, W) + X appears as Eqs. (38), (42), and (47). Consolidating these repetitions would improve readability.","section":"III-A to III-C"},{"comment":"The phrase describing P(X), the evidence, as having 'the possibility to be set to zero during decoding' is unclear and should be rephrased.","section":"III-A, Eq. (33)"}],"recommendation":"reject","confidential_remarks":"The manuscript appears to be an early draft: it contains placeholder citations, a placeholder author name, and its main quantitative conclusions are imported from an unpublished preprint without any experimental verification. The central comparative claim is not independently reproducible from the manuscript, and the internal inconsistency between Section V and the absence of experimental details is a serious soundness issue. I would not invite a revision unless the authors can supply the full experimental protocol and verify the reported numbers."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a survey, not a research contribution. If you need a clean statement of the standard equations for CNN, Conformer, ResNet, CRNN, attention, and Bayesian decision theory for speech tasks, the first half is serviceable. The authors organize known material clearly and the equations are standard and correct. That is the extent of the credit.\n\nThe problem is the quantitative core. Section IV presents a comparison of the four architectures on VoxForge and Voxlingua6, with parameter counts and error rates in Tables II and III, and the conclusion says 'We tested these models on training cost, model size, accuracy, and inference speed.' But the manuscript contains no experimental protocol whatsoever: no feature extraction, no hyperparameters, no training details, no code, no logs. The tables are attributed to reference [9], an unpublished 'Easy Chair' preprint. So the central comparative claim—Conformer best at 5.27% error on the Voxlingua6 Dev set, CNN lightest at 6M parameters—rests entirely on another paper's numbers that the reader cannot check. That is not a minor omission; the ranking is the only non-textbook content in the paper, and it is unverifiable here.\n\nThere are also signs of poor quality control: an unresolved '[?]' citation near the LSTM equations (Section II-D) and a placeholder author 'J. Doe' in reference [14]. Those alone would not sink a paper, but they compound the credibility problem.\n\nI'll note that the survey parts do a decent job of laying out the statistical framing of convolution, Bayesian inference, and the four architectures. If the authors had positioned this as a tutorial and dropped the experimental claims, it could be a mild contribution for beginners. As is, the internal contradiction between the conclusion's 'We tested' and the absence of experiments is load-bearing. The paper needs major revision or a clear repositioning before it is credible.\n\nRecommendation: desk-reject or send back for a fundamental reframing. Not worth a full referee cycle in its current form. If the authors can release the underlying comparison protocol or replace it with genuinely reproduced experiments, then it might be worth a look. For now, I would not cite it or bring it to reading group.","headline":"A survey-style manuscript that overclaims its own experiments; the comparative rankings are borrowed from an unpublished preprint and the paper never actually tests anything itself.","tokens_in":10506,"tokens_out":2443,"would_cite":false,"duration_ms":23690,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A comparison of four convolution-based speech architectures finds the Conformer most accurate at 5.27% error on the Voxlingua6 development set, with the CNN the smallest model at 6.0 million parameters.","keywords":["speech signal processing","convolutional neural network","Conformer","ResNet","CRNN","speaker identification","emotion recognition","Voxlingua6"],"falsifier":"Re-run the four architectures on the Voxlingua6 development set with matched preprocessing, training data, and hyperparameter budget and compare the error rates; if the Conformer does not achieve the lowest error, or the CNN does not post 6.0 million parameters, the paper's central trade-off claim fails.","tokens_in":9518,"feed_emoji":"🎙️","tokens_out":5337,"duration_ms":48859,"temperature":0.7,"pith_summary":"This survey paper compares four convolution-based neural architectures—CNNs, Conformers, CRNNs, and ResNets—as tools for speech signal processing. Its central empirical claim is that on the Voxlingua6 development set, the Conformer reaches the lowest error rate (5.27%) while the plain CNN is the lightest model (6.0 million parameters), with CRNN and ResNet in between on both axes. The authors interpret this as a speed–accuracy trade-off: Conformers give the best accuracy, CNNs the best efficiency, and the other two architectures fall between. If the comparison holds under fair conditions, it gives practitioners a concrete rule of thumb for choosing a default architecture depending on whether accuracy or resource constraints matter.","feed_headline":"Conformer posts lowest error rate of four speech architectures","feed_subtitle":"Conformer scores 5.27% error on Voxlingua6; CNN the lightest at 6.0 million parameters.","key_machinery":"The shared machinery is the discrete convolution operation, which the paper presents as the statistical signal-processing core of all four models. For each architecture it writes the block equations—CNN's convolution, pooling and batch normalisation; the Conformer's half-step feed-forward, multi-head self-attention and depthwise-separable convolution; the ResNet's residual mapping; and the CRNN's convolution-plus-LSTM—and then ties the comparison to two quantities: parameter count and dev-set error rate. These two quantities, taken from an external comparison, are what actually carry the paper's trade-off argument.","core_discovery":"The paper's central claim is that, on the Voxlingua6 development set used by its comparison, the Conformer—a Transformer with convolutional modules—attains an average error rate of 5.27%, outperforming the CNN (7.18%), the Residual Network (8.56%), and the CRNN (11.35%). It further reports that the CNN has 6.0 million parameters, the Conformer 15.5 million, the CRNN 19.5 million, and the ResNet 23.5 million. From these two tables the paper concludes that Conformers dominate on accuracy, CNNs dominate on model size and speed, and the other two architectures sit in between, so the choice of architecture depends on whether a deployment prioritises accuracy or resource efficiency.","pith_inferences":["If the ranking transfers to other datasets, Conformers would be the default pick for accuracy-critical speech systems, but the paper only reports one dev set.","The parameter counts imply the Conformer is not only more accurate but also more parameter-efficient than the ResNet (15.5 vs 23.5 million parameters), suggesting that convolution plus attention is doing more with less than depth alone.","A natural follow-up the paper does not run is an uncertainty or calibration check: the reported error rates say nothing about how confident each model is when it is wrong."],"forward_implications":["For accuracy-critical multilingual deployment, the Conformer is the best default among the four architectures.","For low-resource or real-time settings, the CNN is the best default because of its 6.0 million parameters and speed.","CRNN and ResNet offer no clear advantage on this comparison: they are both larger than the Conformer and less accurate, so they would only be chosen for other reasons, such as temporal modeling or very deep feature hierarchies.","The trade-off between accuracy and model size motivates hybrid work, such as combining convolution with self-supervised learning, which the paper names as a future direction."],"supporting_citations":[{"why":"Defines the Conformer architecture and reports its speech-recognition results, the model the paper ranks first.","marker":"[1]"},{"why":"Supplies the parameter counts and Voxlingua6 dev-set error rates that form the paper's comparison tables.","marker":"[9]"},{"why":"Introduces the CNN-plus-RNN combination that the paper describes as the CRNN.","marker":"[11]"},{"why":"Provides CNN speech-recognition results that ground the paper's discussion of CNNs for speech tasks.","marker":"[22]"},{"why":"Introduces residual connections, the mechanism that defines the ResNet in the comparison.","marker":"[25]"}],"fun_headline_variants":["Conformer hits 5.27% error, best of four speech models","CNN is lightest, Conformer most accurate among speech nets","Conformer wins accuracy; CNN wins size in speech AI showdown","Speech model face-off: Conformer 5.27% error, CNN 6M params","Conformer beats CNN, ResNet, CRNN in speech error rates"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The accuracy ranking rests entirely on the assumption that the error rates quoted from the cited comparison were measured under fair and identical conditions for all four models, with the same data, features, and tuning budget.","fun_headline_variants_meta":{"raw":{"variants":["Conformer hits 5.27% error, best of four speech models","CNN is lightest, Conformer most accurate among speech nets","Conformer wins accuracy; CNN wins size in speech AI showdown","Speech model face-off: Conformer 5.27% error, CNN 6M params","Conformer beats CNN, ResNet, CRNN in speech error rates"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000591,"raw_usage":{"total_tokens":2690,"prompt_tokens":781,"completion_tokens":1909,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":397,"completion_tokens_details":{"reasoning_tokens":1811}},"tokens_in":397,"tokens_out":1909,"duration_ms":13479,"temperature":1.0,"reasoning_tokens":1811,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:35:31.381034+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the four architectures on the Voxlingua6 development set with matched preprocessing, training data, and hyperparameter budget and compare the error rates; if the Conformer does not achieve the lowest error, or the CNN does not post 6.0 million parameters, the paper's central trade-off claim fails.","supporting_citations":[{"cited_title":"Gulati, Y","cited_arxiv_id":null,"evidence_quote":"Defines the Conformer architecture and reports its speech-recognition results, the model the paper ranks first."},{"cited_title":"Bazazo, M","cited_arxiv_id":null,"evidence_quote":"Supplies the parameter counts and Voxlingua6 dev-set error rates that form the paper's comparison tables."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the CNN-plus-RNN combination that the paper describes as the CRNN."},{"cited_title":"Abdel-Hamid, A.-R","cited_arxiv_id":null,"evidence_quote":"Provides CNN speech-recognition results that ground the paper's discussion of CNNs for speech tasks."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces residual connections, the mechanism that defines the ResNet in the comparison."}],"review_version":1}