{"id":"7e0f0658-16d2-4a00-9345-0fe4ec2ee1cc","arxiv_id":"2505.03378","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Simulator experiments show QuanNN is broadly more noise-robust than QCNN, while QCNN sometimes beats its own noiseless version under bit-flip, phase-flip, and phase-damping noise.","lead":"This paper measures how two hybrid quantum-classical image classifiers, QuanNN and QCNN, behave when quantum noise is injected into their circuits at strengths from 10% to 100%. The two architectures respond very differently, and some noise types can even improve accuracy.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim that QuanNN is more robust than QCNN for all noise types and datasets is not yet supported: rankings come from single unseeded runs with no error bars and no defined robustness metric.","rationale":"The reader's weakest assumption correctly identifies the single most load-bearing gap: the architecture-level ranking is read off from one training trajectory per condition, with no seeds, error bars, or released code/data. My review confirms that this is not a minor reproducibility nicety but the empirical foundation of the paper's headline claim. The curves for QCNN in particular are acknowledged in the text to be highly fluctuating and trend-free, so a single run can easily invert apparent robustness comparisons. I also note that the term 'robustness' is never given a quantitative definition; adding a metric such as area under the accuracy-versus-noise curve is necessary before the claim can be checked at all. This does not make the paper's conclusion wrong, but it is currently unverifiable from the reported material. The reader's conditional verdict remains appropriate: the claim is plausible and the experimental design is clearly described, but the missing replication and metric definition require either additional experiments or a softened conclusion before full acceptance. No internal inconsistency or unsupported theoretical step was found beyond this empirical evidentiary gap.","tokens_in":16233,"tokens_out":4594,"duration_ms":51070,"concrete_test":"Rerun the full grid (2 architectures x 5 noise channels x 10 probabilities x 2 datasets) with at least 10 fixed random seeds, holding Table II hyperparameters fixed, and report mean +/- standard deviation of final validation accuracy. Pre-register a robustness score, e.g., mean validation accuracy over p=0.1-1.0 minus the noise-free baseline accuracy, per architecture and dataset. Then check whether QuanNN's score exceeds QCNN's for every noise channel and both datasets and whether the margin is larger than the pooled standard error; if any ranking reverses or falls within run-to-run noise, the Section IV-D claim must be weakened to a seed-dependent observation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The comparative conclusion in Section IV-D ('QuanNN exhibits greater robustness than the QCNN across all examined noise types and task demands') rests entirely on the single training/validation trajectories shown in Figs. 4-7. Section IV-A reports learning rates optimized separately for MNIST and Fashion-MNIST using noise-free models, but no random seed or repeated-run information is given, and no error bars or aggregate tables accompany the curves. The QCNN curves are repeatedly described as 'highly irregular' and having 'no discernible pattern' (Section IV-C-b/d, Figs. 6-7), so a single run can reflect optimizer or initialization luck rather than architecture-level robustness. Moreover, 'greater robustness' is never operationalized: no metric such as mean accuracy across noise levels, drop relative to the noise-free baseline, or failure threshold is defined, and the text notes cases where QCNN reaches high absolute accuracies at specific noise levels (e.g., 90% under depolarization 0.1 on MNIST). Without replicated runs and a pre-registered robustness measure, the claimed universal ranking is not falsifiable from the reported data.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents an empirical comparison of two hybrid quantum-classical image classifiers, a Quanvolutional Neural Network (QuanNN) and a Quantum Convolutional Neural Network (QCNN), under five quantum noise channels (bit flip, phase flip, depolarization, phase damping, amplitude damping) with probabilities from 0.1 to 1.0. Experiments use four-class subsets of MNIST and Fashion-MNIST, with 500 training samples and 150 validation samples per condition, and report training and validation accuracy over 20 epochs. The central claim, stated in Section IV-D, is that the QuanNN exhibits greater robustness than the QCNN across all examined noise types and task demands, with the QuanNN robust to phase-related noise and to bit-flip noise at p=1.0, and the QCNN sometimes outperforming its noise-free baseline at specific noise levels. The paper concludes with guidelines for architecture selection under different device noise profiles.","tokens_in":16497,"tokens_out":4519,"duration_ms":44783,"significance":"If the claimed architecture-dependent robustness ranking were established, the paper would provide practically useful guidance for choosing between QuanNN and QCNN architectures on NISQ devices. The study has real strengths: it covers a broad matrix of five noise channels, ten probability values, two architectures, and two datasets; it extends prior 2-qubit noise studies to 4-qubit multi-class settings; and it compares against explicit noise-free baselines. However, the central comparative claim is not currently supported by the evidence as presented: every condition is a single unseeded training run, no robustness metric is defined, and several stated conclusions conflict with the reported curves. The paper is a reasonable starting point for an empirical study, but it requires additional statistical grounding and a tightening of the claims before it can be accepted as a comprehensive analysis.","major_comments":[{"comment":"The central claim that \"QuanNN exhibits greater robustness than the QCNN across all examined noise types and task demands\" is not supported because each reported condition is a single unseeded run. Table II lists no random seed or repetition count, and Section IV-A states only that learning rates were tuned separately on noise-free models. Given that the text itself describes QCNN curves as \"highly irregular\" and \"with no discernible pattern\" (Section IV-C-b and IV-C-d), a single trajectory can reflect initialization or optimizer luck rather than architecture-level robustness. The authors should rerun each condition with multiple seeds and report mean accuracy with error bars or interquartile ranges, and should provide per-condition summary tables rather than relying only on curves.","section":"Section IV-D and Figures 4-7"},{"comment":"\"Noise robustness\" is never operationalized. The paper does not define a metric such as mean accuracy across noise levels, accuracy drop relative to the noise-free baseline, area under the noise-accuracy curve, or a failure threshold. The ranking in Section IV-D is therefore a qualitative reading of visual curves. A pre-specified robustness measure is needed before the paper can claim that one architecture is more robust than another; otherwise the conclusion is not falsifiable from the reported data.","section":"Section III-C / Section IV-D"},{"comment":"The abstract states that the QCNN \"tends to benefit from noise injection by outperforming noise-free models for bit flip, phase flip, and phase damping at high noise probabilities,\" but the body reports the opposite for phase damping: Section IV-C-d states that the QCNN \"fails to learn effectively\" at phase damping levels 0.7-1.0 on MNIST and that Fashion-MNIST shows \"no discernible pattern,\" with only p=0.4 on MNIST outperforming the noise-free model. The abstract overstates the findings and should be corrected to match the reported results.","section":"Abstract and Section IV-C-d"},{"comment":"At p=1.0, the bit-flip and phase-flip channels become deterministic unitary operations (Pauli X and Pauli Z, respectively), not stochastic noise. Presenting these results as \"robustness to noise\" or \"benefiting from noise\" conflates adaptation to a fixed unitary gate with resilience to stochastic errors. The authors should either analyze the p<1 cases separately from the deterministic p=1 cases or explicitly relabel the p=1 results as deterministic gate operations rather than noise robustness.","section":"Section II-B and Sections IV-B-a/IV-C-a"},{"comment":"The experimental setup uses only 500 training samples and 150 validation samples from four classes, with dataset-specific learning rates. With a balanced four-class subset, 25% accuracy is the chance level, yet the text describes curves flattening at around 25% without noting this is failure to learn. The small sample size also makes the \"outperforms noise-free\" observations (e.g., 90% validation accuracy under depolarization 0.1 on MNIST in Section IV-C-c) hard to interpret without confidence intervals. Please report balanced-class confirmation, confidence intervals for the accuracies, and the chance-level threshold.","section":"Section III-A and Table II"}],"minor_comments":[{"comment":"The captions contain typos: \"Trainnig Accuracy\" should be \"Training Accuracy\", and \"mot fluctuation\" should be \"more fluctuation\".","section":"Figure 4 and Figure 5 captions"},{"comment":"Phase damping and amplitude damping are parameterized by gamma in the Kraus operators, but the text refers to the same quantity as \"probability p\". Please clarify the relation between gamma and the probability values reported in the experiments.","section":"Section II-B, Equations (4)-(5)"},{"comment":"The noise-injection description says noise gates are placed \"after every parametric gate block and entanglement block,\" but it is not explicit whether a noise channel is applied after every individual gate or after each layer, and whether the same channel is applied to all qubits. Please specify the exact insertion rule.","section":"Section III-B"},{"comment":"The bullet point stating that QuanNN shows robustness to bit flip \"against non-deterministic noise\" is contradicted by the fact that p=1.0 is a deterministic X operation; please rephrase to avoid the inconsistency.","section":"Section IV-D"},{"comment":"For reproducibility, please include the PennyLane version, the random seed(s), the number of repeated runs, and the exact hyperparameter search procedure used to select the two learning rates.","section":"Table II and Section IV-A"}],"recommendation":"major_revision","confidential_remarks":"This is an empirical benchmark paper for IJCNN, and the scope is appropriate. The main concern is evidentiary: the headline comparative claim is based on single unseeded runs and no defined robustness metric, which could be remedied with a moderate amount of additional experimentation and analysis. I would also encourage the editor to ask the authors to reconcile the abstract with the body text regarding QCNN performance under phase damping, since the overstatement is likely to mislead readers. No code or data repository is mentioned, which further limits reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: this is a modest empirical extension of the authors' earlier 2-qubit binary noise study [48], and the qualitative message — noise type and architecture interact — is plausible. But the headline claim, \"QuanNN exhibits greater robustness than the QCNN across all examined noise types and task demands,\" is not backed by the experiments as reported.\n\nWhat's new: they take two popular HQNN architectures, QuanNN and QCNN, use their own best-performing configurations from prior work, and sweep five noise channels at ten intensities on 4-class MNIST and Fashion-MNIST with a 4-qubit circuit. That systematic comparison is genuinely new as a measurement, even though the noise models and architectures are established. The writing is clear, the noise-channel exposition is solid, and the authors deserve credit for showing the messy QCNN curves rather than hiding them.\n\nThe soft spots are serious. No robustness metric is defined — \"greater robustness\" is never operationalized as mean accuracy, drop from baseline, or failure threshold — so the universal ranking isn't falsifiable from the data. Every condition appears to be a single unseeded training run. With 500 training samples, 150 test samples, 20 epochs, and batch size 5, accuracy estimates are noisy to begin with. The QCNN curves are repeatedly called \"highly irregular,\" which the authors attribute to circuit design, but that variance could just as easily be initialization or optimizer luck. The learning rates are optimized separately per dataset on the noise-free models, which is defensible, but it means the architectures are compared under different optimization conditions. No seeds, error bars, or released code/data. So the specific rankings in Figs. 4–7 should be read as anecdotal until replicated.\n\nThe qualitative direction of the main claim is likely correct: phase noise barely affects QuanNN, depolarization is destructive for both, and bit flip at p=1.0 is effectively a deterministic gate the model can adapt to. Those patterns are visible even in single runs. But \"comprehensive analysis\" is overselling what is, by the paper's own scope, a small-scale pilot.\n\nRecommendation: send it to peer review — the question matters and the setup is sensible — but it needs major revision before acceptance: define the robustness metric, add repeated seeds with error bars, soften the universal conclusion, and release code and data. A practitioner deciding between QuanNN and QCNN on a specific noisy device gets a suggestive map here, not a reliable guide.","headline":"Worth refereeing but not accepting as is: a plausible small-scale noise-robustness benchmark whose headline ranking rests on single unseeded runs and no defined robustness metric.","tokens_in":16965,"tokens_out":3744,"would_cite":false,"duration_ms":35920,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["81P68","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"Under five quantum noise channels and two image tasks, the QuanNN classifier survives better than the QCNN.","keywords":["hybrid quantum neural networks","quantum convolutional neural network","quanvolutional neural network","noise robustness","quantum noise channels","NISQ devices","MNIST","Fashion-MNIST"],"falsifier":"Re-run every architecture--noise-channel--probability--dataset condition with at least ten random seeds under fixed hyperparameters and report accuracy distributions; if the QuanNN-over-QCNN ranking and the beneficial-noise windows, such as bit flip at $p=1.0$ and phase flip or phase damping at high probabilities, do not consistently recur, the central claim is refuted. A sharper check for the bit-flip case is to verify whether a 100% bit-flip channel, being a deterministic X applied after every gate block, merely reparameterizes the circuit into an equivalent noise-free circuit; if so, the observed robustness is a learned compensation rather than genuine noise tolerance.","tokens_in":16063,"feed_emoji":"⚛️","tokens_out":6577,"duration_ms":58938,"temperature":0.7,"pith_summary":"This paper tries to establish that architecture choice, not just noise level, determines whether hybrid quantum-classical image classifiers survive noisy quantum hardware. To show this, it trains two four-qubit architectures, the Quanvolutional Neural Network and the Quantum Convolutional Neural Network, on four-class MNIST and Fashion-MNIST under five quantum noise channels at probabilities from 0.1 to 1.0. It reports that the QuanNN is more robust overall, staying accurate under phase flip and phase damping at all tested levels and under bit flip at probability 1.0, while the QCNN can outperform its noise-free version for bit flip, phase flip, and phase damping at high noise probabilities. If these findings hold, a device's noise profile should guide which HQNN architecture a practitioner chooses, and some noise can be tolerated or even exploited rather than merely mitigated.","feed_headline":"QuanNN is more noise-robust than QCNN, five-noise study finds","feed_subtitle":"Four-qubit classifiers on MNIST and Fashion-MNIST: phase noise is survivable, depolarization is not, and architecture decides the…","key_machinery":"The load-bearing object is the noise-injected variational quantum circuit: a four-qubit parameterized circuit with rotation gates and entangling layers, with a quantum noise gate inserted after every parametric-gate block and entanglement block. The QuanNN uses three basic-entangling layers as a sliding quantum filter feeding classical post-processing, while the QCNN uses three strongly-entangling layers with classical pre-processing to downsize the input. Five noise channels, namely bit flip, phase flip, phase damping, amplitude damping, and depolarizing noise, are implemented as Kraus-operator channels at ten probabilities from 0.1 to 1.0, and models are trained and validated under the injected noise on a mixed-state simulator. This setup lets the paper attribute accuracy differences to noise type, intensity, and architecture rather than to hardware-specific error correlations.","core_discovery":"On the authors' own terms, the central discovery is a conditional robustness ranking: across all five noise channels, both datasets, and all noise probabilities studied, the QuanNN exhibits greater robustness than the QCNN. The QuanNN is insensitive to phase flip and phase damping noise at every intensity, and at 100% bit flip probability it returns to near-noise-free training accuracy, apparently because deterministic X gates become a learnable transformation. It degrades sharply under depolarizing and amplitude damping noise once the probability passes roughly 0.3 to 0.5. The QCNN, by contrast, benefits from noise in specific windows, namely bit flip at 1.0, phase flip at several high probabilities, and phase damping at 0.4, but otherwise degrades, and on Fashion-MNIST its accuracy fluctuates with no consistent trend. The paper reads these patterns as evidence that noise resilience is a joint property of circuit architecture, noise channel, and task complexity.","pith_inferences":["A testable extension of the bit-flip result: a 100% bit-flip channel applies a deterministic X to every qubit after every gate block, which is equivalent to a fixed unitary transformation of the circuit; checking whether the trained weights simply rotate to compensate would separate 'learning despite noise' from 'learning the noise.'","The phase-damping robustness of the QuanNN may be a measurement-basis artifact: if the final measurement is in the Z basis and phase damping only kills off-diagonal coherences, the measurement probabilities are unchanged; swapping in X-basis or Y-basis measurements would test whether robustness disappears.","The paper's single-run design suggests a natural safeguard: future comparisons should report seed-averaged accuracies, since the ranking claim depends on differences that may be smaller than optimization variance.","If the noise-benefit windows for the QCNN are reproducible, they could be exploited deliberately by injecting matched noise during training as a regularizer, rather than treating all noise as something to remove."],"forward_implications":["A device whose dominant errors are phase flip or phase damping should favor the QuanNN architecture, since it stays near noise-free accuracy at all tested intensities.","A device with strong bit-flip errors at high probability is not necessarily unusable: both architectures adapt at $p=1.0$, with the QuanNN returning to near-baseline accuracy.","Depolarizing and amplitude damping noise are the main failure modes for both architectures, with collapse typically beginning between $0.3$ and $0.5$ probability, so mitigation effort should target these channels first.","Task complexity amplifies noise sensitivity: the same architecture and noise level that works on MNIST can fluctuate badly on Fashion-MNIST, so noise-robustness claims should be benchmarked on more than one task.","QCNN deployment should be noise-profile-aware, since the architecture is robust only at select noise levels and noise types, not across the board."],"supporting_citations":[{"why":"Supplies the best-performing QuanNN and QCNN architectures and the noise-free baselines that this study adopts.","marker":"[42]"},{"why":"Introduces the quanvolutional filter architecture used as one of the two HQNNs under test.","marker":"[53]"},{"why":"Introduces the quantum convolutional neural network architecture used as the other HQNN under test.","marker":"[55]"},{"why":"Establishes the five-channel noise influence analysis on a two-qubit variational circuit that this work extends to four-qubit multi-class tasks.","marker":"[48]"},{"why":"Provides evidence that noise can be beneficial or detrimental depending on the learning task, the conceptual basis for the noise-benefit observations.","marker":"[31]"},{"why":"Earlier readout-error analysis for HQNNs that this work extends by injecting noise after every parametric and entangling block.","marker":"[50]"},{"why":"Supplies the simulation and automatic-differentiation environment used to train the quantum circuits and to inject noise gates.","marker":"[26]"},{"why":"Provides the MNIST dataset used for the simpler classification task.","marker":"[56]"},{"why":"Provides the Fashion-MNIST dataset used as the higher-complexity task to test whether noise robustness depends on task demands.","marker":"[57]"}],"fun_headline_variants":["QuanNN beats QCNN in most noise tests","Depolarizing noise kills QuanNN, bit flip does not","QCNN sometimes improves with noise, QuanNN rarely","Quantum noise resilience: architecture is the deciding factor"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The robustness ranking rests on the assumption that the single training run per noise condition is representative, so that accuracy differences reflect architecture and noise channel rather than random initialization, optimizer noise, or the separately tuned learning rates.","fun_headline_variants_meta":{"raw":{"variants":["QuanNN beats QCNN in most noise tests","Depolarizing noise kills QuanNN, bit flip does not","QCNN sometimes improves with noise, QuanNN rarely","Quantum noise resilience: architecture is the deciding factor"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00045,"raw_usage":{"total_tokens":2333,"prompt_tokens":1073,"completion_tokens":1260,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":689,"completion_tokens_details":{"reasoning_tokens":1194}},"tokens_in":689,"tokens_out":1260,"duration_ms":12213,"temperature":1.0,"reasoning_tokens":1194,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:52:54.148286+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run every architecture--noise-channel--probability--dataset condition with at least ten random seeds under fixed hyperparameters and report accuracy distributions; if the QuanNN-over-QCNN ranking and the beneficial-noise windows, such as bit flip at $p=1.0$ and phase flip or phase damping at high probabilities, do not consistently recur, the central claim is refuted. A sharper check for the bit-flip case is to verify whether a 100% bit-flip channel, being a deterministic X applied after every gate block, merely reparameterizes the circuit into an equivalent noise-free circuit; if so, the observed robustness is a learned compensation rather than genuine noise tolerance.","supporting_citations":[{"cited_title":"Investigating the effect of noise on the training performance of hybrid quantum neural networks,","cited_arxiv_id":null,"evidence_quote":"Establishes the five-channel noise influence analysis on a two-qubit variational circuit that this work extends to four-qubit multi-class tasks."},{"cited_title":"Taking advantage of noise in quantum reservoir computing,","cited_arxiv_id":null,"evidence_quote":"Provides evidence that noise can be beneficial or detrimental depending on the learning task, the conceptual basis for the noise-benefit observations."},{"cited_title":"The mnist database of handwritten digit images for machine learning research [best of the web],","cited_arxiv_id":null,"evidence_quote":"Provides the MNIST dataset used for the simpler classification task."},{"cited_title":"Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,","cited_arxiv_id":null,"evidence_quote":"Provides the Fashion-MNIST dataset used as the higher-complexity task to test whether noise robustness depends on task demands."}],"review_version":1}