{"id":"252361a1-68f7-4fe1-ac89-b2c368248d2f","arxiv_id":"2501.14531","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"On CIFAR-10, quantization-aware training with large constant scaling factors improves noise robustness, but noisy training (injecting matching Gaussian noise during training) gives far larger robustness gains, and quantization adds little on top of it.","lead":"This paper tests two ways to make neural networks survive the extra noise that analog computing hardware adds to their calculations. It finds that training a network with realistic noise injected during training beats simply quantizing the network, especially for deeper models.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central QAT-vs-noisy-training ranking rests on the unvalidated assumption that i.i.d. additive Gaussian activation noise is the relevant noise model; under weight-drift or multiplicative hardware noise the ranking could reverse.","rationale":"The reader's weakest-assumption analysis correctly identifies the synthetic activation-noise model as the load-bearing premise. My stress-test converges on the same concern: the paper's headline conclusions are empirical comparisons made under one specific noise-injection protocol, and the transferability of those conclusions to real analog hardware is not established. This is a genuine soft spot because the paper itself cites alternative, weight-based noise models in Related Work, making the choice of activation noise a contestable modeling decision rather than a settled fact. However, the concern is not fatal to the paper's internal logic: the paper is explicitly an empirical study with clearly stated experimental conditions, and the claims are framed within those conditions. The abstract's wording ('mimics the noise encountered during inference') is somewhat overstrong, since the inference noise is also synthetic, but this does not invalidate the comparative results as reported. The appropriate response is a conditional acceptance requiring the noise-model sensitivity analysis (or at least a clear statement of scope) rather than rejection. The missing code and data, noted by the reader, weaken independent verification but do not constitute a separate load-bearing concern for the central claim. Thus I recommend the verdict remain CONDITIONAL, consistent with the reader's assessment, since the identified concern is substantial enough to require follow-up but not enough to overturn the reported findings.","tokens_in":10706,"tokens_out":2405,"duration_ms":25377,"concrete_test":"Re-run the main experiments (LeNet-5, VGG-11, ResNet-18 on CIFAR-10) using the weight-noise model of BayesFT (multiplicative Gaussian drift on each weight, w' = w·e^λ, λ~N(0,σ^2)) and the weight-additive model of Noisy Machines, instead of activation noise. Compute midpoint noise levels for: (a) fp32 baseline, (b) QAT with constant scaling factors, and (c) noisy training with the noise type matched to the evaluation noise. If the ranking changes—for example, if QAT becomes competitive or noisy training loses its advantage—the central claim is not robust to noise model. As a secondary check, vary the training noise intensity (σ_train = 0.5σ_inference, 1.0σ_inference, 2.0σ_inference) under the original activation-noise model to test whether the exact matching assumption drives the result.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claims—\"noisy training emerges as the superior approach\" and \"quantization-aware training with constant scaling factors enhances robustness\"—are both established using a single synthetic noise model: zero-mean i.i.d. Gaussian noise added to every layer's activations, with the same standard deviation at all layers and with training noise exactly matched to inference noise. The authors justify this in Section IV with \"As additive noise is often the primary type of noise in accelerators,\" but this is asserted without evidence and conflicts with the weight-noise models they cite in Related Work: Noisy Machines (additive Gaussian noise on weights) and BayesFT (multiplicative drift w' = w·e^λ, λ~N(0,σ^2)). If real analog hardware exhibits weight-dependent, multiplicative, correlated, or layer-specific noise, the measured robustness ranking may not transfer. In particular, QAT changes the weight/activation statistics substantially (large constant scaling factors inject large quantization noise), so its apparent robustness under activation-noise injection may be an artifact of that specific perturbation; under weight-drift noise, the quantized models could degrade more, while noisy training trained on activation noise may not generalize across noise types. The fairness of the comparison is also tied to the matching assumption: noisy training is tested under the exact noise distribution and intensity used in training, whereas QAT is not given this advantage; other training-noise schedules (e.g., σ_train ≠ σ_inference, or noise annealed over training) are untested and could change the ranking. Since the abstract and conclusion generalize to \"analog noise\" broadly, the load-bearing premise is that the chosen activation-noise model is representative enough; this is the least secure support for the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper empirically compares two strategies for making DNNs robust to noisy analog computations: quantization-aware training (QAT) with constant or dynamic activation scaling, and noisy training with Gaussian activation-noise injection. Using the midpoint noise level μ from the authors' prior work, they evaluate LeNet-5, VGG-11, and ResNet-18 on CIFAR-10. They report that constant-scaling QAT improves robustness relative to an fp32 baseline and dynamic scaling, that noisy training leads to substantially higher μ, and that quantization does not further improve already noisy-trained models. The central claims are that QAT with constant scaling enhances robustness and that noisy training is the superior robustness strategy, especially for deeper architectures.","tokens_in":10979,"tokens_out":5705,"duration_ms":57215,"significance":"The questions addressed here are practically relevant: if the results hold, they provide direct guidance for deploying DNNs on analog accelerators and for choosing between quantization-based and noise-injection-based defenses. The paper has several strengths: it uses a quantitative robustness metric, compares three architectures with different depths, sweeps bit widths and scaling factors, and reports both accuracy and robustness so that trade-offs are at least partially visible. The main limitation is that the entire comparison is conducted under one synthetic noise model—zero-mean i.i.d. Gaussian noise added to activations with identical strength at all layers—and the training/inference noise matching for noisy training is fixed to exact equality. The empirical ranking may therefore be an artifact of this specific perturbation and of the matched-noise evaluation protocol. The paper does not release code or trained models, which limits reproducibility of the quantitative claims.","major_comments":[{"comment":"The central comparison rests on a single synthetic noise model: zero-mean i.i.d. Gaussian noise added to every layer's activations, with the same standard deviation at all layers and with training noise exactly equal to inference noise. The paper's own Related Work describes alternative hardware noise models—Noisy Machines uses additive Gaussian weight noise, and BayesFT uses multiplicative drift w' = w·e^λ—so the sentence 'As additive noise is often the primary type of noise in accelerators' is an unsupported assertion, not a demonstrated premise. The ranking of QAT versus noisy training could plausibly reverse under weight-dependent, multiplicative, or layer-heterogeneous noise. Please add experiments with at least one weight-noise model (e.g., the BayesFT multiplicative drift) and with a sweep over training-noise intensity (σ_training = c·σ_inference for several c, including c=0 and c=2), or explicitly and prominently restrict the conclusions to the case of matched additive activation noise.","section":"Section IV, first paragraph; Section II"},{"comment":"The headline comparison is not apples-to-apples: noisy training is evaluated under the exact noise distribution and intensity used during training (σ_training = σ_inference), whereas the QAT models are given no advantage of being trained under the evaluation noise. This makes the statement 'noisy training emerges as the superior approach' a statement about a specially privileged evaluation condition rather than about general robustness. The paper should state this as a boundary condition and report how the ranking changes when the training noise intensity is not perfectly matched, for example by training with a smaller or larger σ_training and evaluating across the same noise range.","section":"Section IV-B, Fig. 3"},{"comment":"The key figures contain no error bars or confidence intervals, even though Section IV-A1 states that each inference is repeated 10 times on different randomly initialized networks and Eq. (2) defines a fit with uncertainty. This matters for the central conclusion: the LeNet-5 differences between the fp32 baseline (μ=0.286) and dynamic scaling (μ=0.242, μ=0.233) are small, and the 'closely matched, almost overlapping' dashed curves in Fig. 3 are used to support the negative claim that quantization does not improve noisy-trained models. Without uncertainty estimates, these comparisons are not quantitatively supported. Please plot confidence bands or report fit uncertainties for every curve in Figs. 2 and 3.","section":"Section IV-A1, Figs. 2 and 3"},{"comment":"The selection of scaling factors is not systematic. For LeNet-5 the paper sweeps s ∈ {0.5, 2, 8, 64, 512}; for VGG-11 and ResNet-18 the tested sets are different and smaller, and the text does not state a criterion for choosing 'representative' configurations when making claims such as 'constant scaling enhances robustness' or '8-bit outperforms 4-bit'. A fair cross-architecture comparison should use the same candidate scaling-factor grid for all architectures and report the entire Pareto frontier of peak accuracy versus μ, as is done only for LeNet-5 in Fig. 2(c). As written, the comparison reflects the choice of scaling-factor grid as much as the methods themselves.","section":"Table I and Fig. 2(c)"}],"minor_comments":[{"comment":"The abstract says 'both two methods increase tolerance against noise,' but the dynamic-scaling QAT results in Fig. 2 have lower μ than the fp32 baseline; please qualify the claim to constant-scaling QAT and noisy training.","section":"Abstract and Section I"},{"comment":"The statement 'Experiments with models quantized to 16-bit did not yield better results than 8-bit' is unsupported by any figure or table; please include the data or remove the claim.","section":"Section IV-A1"},{"comment":"Table II lists 'Midpoint noise level μ w/o Noisy Training' values that appear to be the fp32 baselines (e.g., VGG-11 0.154), but the caption does not clarify that these are baseline values rather than QAT values, and the table does not include the best QAT configurations from Table I. Please clarify the table's content or add the QAT rows so the overall comparison is complete.","section":"Table II"},{"comment":"The symbol δa is used both in the definition 'δa = (amax - amin)/2' and as a free parameter in Eq. (1); please use distinct notation to avoid ambiguity.","section":"Section III-A, Eq. (1)"},{"comment":"There are several typos and small errors, including 'quanitzation,' 'extend,' 'is has been shown,' and 'it is shown previously that that quantization'; please proofread.","section":"Throughout"},{"comment":"No code, configuration files, or trained models are released; please provide at least the Brevitas configuration and hyperparameter details to make the experiments reproducible.","section":"Section IV"}],"recommendation":"major_revision","confidential_remarks":"The paper is likely to be of interest to the analog-computing and hardware-robustness community, but the central empirical claim is currently tied to a single noise model and a matched-training-noise protocol. The missing error bars in the key figures are also a reproducibility concern. I do not see a fundamental flaw in the experimental design, so major revision rather than rejection seems appropriate if the authors can add the requested noise-model robustness checks and uncertainty quantification. The self-citation of [3] for the μ metric is appropriate, since the metric is introduced there, but the paper should state this more explicitly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a plausible, novel empirical comparison of constant-scaling QAT against noisy training for robustness to additive Gaussian activation noise. The ranking (noisy training wins, QAT helps but costs accuracy) is credible, but the paper generalizes too far from one synthetic noise model and ships no artifacts. It deserves a serious referee, but needs revision.\n\nWhat's actually new: the direct head-to-head of QAT with static constant scaling versus noisy training for analog-noise robustness across three architectures. I don't know of another paper that does this. The methods are standard, but the question is well-posed. The main negative result—quantization adds nothing on top of noisy training once you train with matched noise—is a useful, non-obvious finding for hardware folks. The use of the midpoint noise level µ from their own prior work is fine; it's not circular since the central comparison doesn't depend on that definition.\n\nWhere it's soft: the load-bearing premise is that i.i.d. additive Gaussian noise on activations, same sigma in every layer, is a representative model of analog hardware noise. That's asserted in Section IV without evidence, and it sits uncomfortably next to the weight-drift and multiplicative noise models cited in Related Work (Noisy Machines, BayesFT). If real hardware noise is weight-based or layer-specific, the measured ranking could change. Since the abstract and conclusion talk about 'analog noise' broadly, this is a real limitation, not a nitpick. Second, there are no error bars in Figs. 2 and 3 despite ten random runs for LeNet-5; Table I gives uncertainties for µ but the curves need them too. Third, no code or trained models are released, which is a problem for a purely empirical paper that asks for a strong claim. Fourth, the abstract overstates: 'enhances robustness' should be qualified as 'under matched activation noise' and should acknowledge the accuracy trade-off more explicitly.\n\nThe scaling-factor sweep is ad hoc but that's acceptable for an exploratory study; the paper honestly flags that dynamic scaling is worse, which is interesting. The lack of a systematic criterion for choosing s is minor.\n\nBottom line: the paper is useful for the analog-ML community as an empirical data point. The central ranking might hold up under other noise models, but that's untested. With released code and either one additional noise model (e.g., weight noise) or a careful reframing to activation-additive noise, this could be a solid workshop or short-conference paper. I'd send it to an editor for peer review, but with a clear request for artifacts and a scoped abstract.","headline":"A genuinely novel empirical comparison of constant-scaling QAT versus noisy training for analog-noise robustness, but the central ranking rests on one unvalidated noise model and the paper ships no artifacts.","tokens_in":11556,"tokens_out":2275,"would_cite":false,"duration_ms":21183,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Training with the same additive Gaussian noise analog hardware produces during inference is a stronger defense than quantization-aware training, especially for deep networks.","keywords":["analog noise","robustness","quantization","quantization-aware training","noisy training","midpoint noise level","CIFAR-10","deep neural networks"],"falsifier":"Run the same quantization and noisy-training recipes on a real analog accelerator, such as a resistive-memory or photonic chip, with its native noise measured, and compare which model keeps accuracy at the device's operating noise; if noisy training stops winning under weight-dependent or multiplicative noise, the paper's ranking does not carry over to that hardware.","tokens_in":10508,"feed_emoji":"🔊","tokens_out":7674,"duration_ms":61329,"temperature":0.7,"pith_summary":"The paper claims that for deep neural networks running on noisy analog hardware, the most dependable way to keep accuracy high is to train the network in the presence of the same additive Gaussian noise it will encounter during inference. It also claims that quantization-aware training with constant scaling factors improves tolerance to such noise, but at a noticeable cost in clean accuracy, and that this benefit disappears once noisy training is used. If correct, the practical implication is that hardware developers should match training-time noise to inference-time noise rather than rely on quantization as the main robustness mechanism.","feed_headline":"Noisy training beats quantization for analog-noise DNNs","feed_subtitle":"Matching training noise to inference noise gives deeper networks the biggest robustness boost.","key_machinery":"The central object is the midpoint noise level $\\mu$, a metric that fits a scaled and shifted logistic function to the accuracy-versus-noise curve and reports the noise level at which accuracy drops halfway; larger $\\mu$ means more robust. The mechanism that produces the results is uniform quantization with a constant scaling factor $s$, implemented through quantization-aware training with straight-through gradient estimation, plus a noise-injection module added after each layer that applies forward-path Gaussian noise matching the noise used during inference when noisy training is enabled.","core_discovery":"On CIFAR-10, across LeNet-5, VGG-11, and ResNet-18, the authors measure robustness by the midpoint noise level $\\mu$, the injected noise standard deviation at which validation accuracy falls halfway from its maximum to random chance. They find that quantized models with dynamic scaling are less robust than full-precision models, while constant scaling factors shift the accuracy-versus-noise curve to the right at the price of lower peak accuracy. Pairing quantization-aware training with noisy training yields almost no additional robustness over noisy training alone, yet quantized noisy-trained models match the robustness of floating-point noisy-trained models, so smaller models can be deployed without losing noise tolerance.","pith_inferences":["Inference: the equal-strength matching of training and inference noise may be overly strict; a sweep of training noise levels above and below the inference level could reveal a cheaper or more robust schedule than exact matching, which the paper does not test.","Inference: if real device noise is multiplicative or weight-dependent, as memristor drift models suggest, the ranking of defenses found here could change, because quantization's clipped and bounded activations might help more under multiplicative noise than under additive noise.","Inference: the result suggests a practical design rule: treat quantization as a compression step that is nearly free in robustness once noisy training is applied, and focus engineering effort on measuring and reproducing the accelerator's noise profile during training."],"forward_implications":["Quantization-aware training with constant scaling factors raises the midpoint noise level of LeNet-5, VGG-11, and ResNet-18 compared with dynamic scaling or full precision, at the cost of lower peak accuracy.","Noisy training with training noise strength equal to inference noise outperforms quantization-aware training alone on all tested architectures, and the gap grows for deeper networks.","Adding quantization-aware training on top of noisy training does not meaningfully improve robustness, but quantized noisy-trained models keep the same robustness as floating-point noisy-trained models, making quantized deployment nearly free in robustness once noisy training is used.","Deeper architectures are more vulnerable to error amplification under quantization, with VGG-11 collapsing at a scaling factor of 3, while ResNet-18's skip connections reduce error accumulation and improve robustness.","Dynamic scaling quantization is consistently the least robust option, performing worse than full-precision models in every architecture tested."],"supporting_citations":[{"why":"defines the midpoint noise level $\\mu$ used to score robustness.","marker":"[3]"},{"why":"demonstrates accurate inference on noisy phase-change memory, motivating robustness against analog noise as a goal.","marker":"[14]"},{"why":"explains the error amplification effect of quantization in deep networks, used to interpret VGG-11's collapse at large scaling factors.","marker":"[21]"},{"why":"supplies the quantization-aware training implementation used in the experiments.","marker":"[25]"},{"why":"models memristor perturbation noise and proposes noise injection variants, one of the noisy training baselines.","marker":"[33]"},{"why":"shows that Gaussian noise injection during training improves resilience of analog networks, the method compared here.","marker":"[34]"},{"why":"guides the per-channel and per-tensor quantization setup adopted in the experiments.","marker":"[17]"},{"why":"introduces the straight-through estimator used to backpropagate through the non-differentiable quantization operator.","marker":"[1]"}],"fun_headline_variants":["Noisy training beats quantization for analog DNNs","Match training noise to inference noise for robust DNNs","Quantization can't beat noisy training for analog robustness","For robust analog DNNs, inject noise during training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that adding the same strength of additive Gaussian noise to every layer's activations is a faithful proxy for the noise of real analog accelerators.","fun_headline_variants_meta":{"raw":{"variants":["Noisy training beats quantization for analog DNNs","Match training noise to inference noise for robust DNNs","Quantization can't beat noisy training for analog robustness","For robust analog DNNs, inject noise during training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000678,"raw_usage":{"total_tokens":3026,"prompt_tokens":830,"completion_tokens":2196,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":446,"completion_tokens_details":{"reasoning_tokens":2131}},"tokens_in":446,"tokens_out":2196,"duration_ms":14039,"temperature":1.0,"reasoning_tokens":2131,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T15:03:18.602941+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same quantization and noisy-training recipes on a real analog accelerator, such as a resistive-memory or photonic chip, with its native noise measured, and compare which model keeps accuracy at the device's operating noise; if noisy training stops winning under weight-dependent or multiplicative noise, the paper's ranking does not carry over to that hardware.","supporting_citations":[{"cited_title":"Walking Noise: On Layer-Specific Robustness of Neu- ral Architectures against Noisy Computations and Asso- ciated Characteristic Learning Dynamics","cited_arxiv_id":null,"evidence_quote":"defines the midpoint noise level $\\mu$ used to score robustness."},{"cited_title":"Accurate deep neural network infer- ence using computational phase-change memory","cited_arxiv_id":null,"evidence_quote":"demonstrates accurate inference on noisy phase-change memory, motivating robustness against analog noise as a goal."},{"cited_title":"Defensive Quan- tization: When Efficiency Meets Robustness","cited_arxiv_id":null,"evidence_quote":"explains the error amplification effect of quantization in deep networks, used to interpret VGG-11's collapse at large scaling factors."},{"cited_title":"Improving the robustness of analog deep neural networks through a Bayes-optimized noise injection approach","cited_arxiv_id":null,"evidence_quote":"models memristor perturbation noise and proposes noise injection variants, one of the noisy training baselines."},{"cited_title":"Noisy Machines: Understanding Noisy Neural Networks and Enhancing Robustness to Analog Hardware Errors Using Distillation","cited_arxiv_id":"2001.04974","evidence_quote":"shows that Gaussian noise injection during training improves resilience of analog networks, the method compared here."}],"review_version":1}