{"id":"5bef0cda-67ff-46fc-8c14-5da9d213a801","arxiv_id":"2608.02700","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"NANQ constructs non-uniform quantization levels from the measured magnitude-dependent noise of an eFlash CIM array and uses precision-saturation thresholds to assign layer bit-widths, improving low-bit on-chip inference beyond PowerQuant.","lead":"Analog compute-in-memory chips suffer from read and programming noise, so quantizing neural networks to low bits is risky. NANQ measures each chip's noise curve, concentrates quantization levels where noise is low, and chooses per-layer bit-widths that stop adding precision once the hardware noise floor saturates.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Readback-derived noise profile may not match inference-time MAC noise, undermining the noise-awareness mechanism; empirical results on the specific chip survive, but transferability and the method's core explanation require validation.","rationale":"The reader's weakest assumption is exactly the noise-model fidelity, and I agree. The empirical results on the specific chip are real measurements; even if the noise model is imperfect, NANQ still outperforms baselines, so the central empirical claim (improvements on this SoC) is not falsified. However, the paper's motivating contribution is the noise-floor-aware mechanism, and that mechanism rests on the unverified assumption that readback noise equals MAC noise. This is a genuine gap: the experimental protocol does not test it, and the Discussion acknowledges the approximation. Secondary concerns (lack of error bars, unclear calibration/test split, absence of CIMQ/CIM2PQ baselines) are protocol weaknesses that contribute to the CONDITIONAL verdict but are not as load-bearing. A direct comparison of readback and MAC-level noise would settle the main concern. If the profiles match, the method's explanation is credible; if not, the method's advantage may be due to the flexibility of per-layer γ search and non-uniform density rather than true noise awareness, diminishing the generality of the contribution. Since the reader already judged the paper CONDITIONAL for these reasons, my assessment does not move the verdict; it confirms it.","tokens_in":10792,"tokens_out":7197,"duration_ms":64707,"concrete_test":"On the same eFlash CIM SoC, program cells to a set of known conductance values spanning the full magnitude range used in the paper. For each programmed value, perform N repeated matrix-vector multiplications with randomly sampled signed input vectors (e.g., standard Gaussian) and record the output error relative to the ideal digital computation. Compute the empirical error variance as a function of weight magnitude, σ_MAC(w), and compare it to the readback-derived σ(w) used in Section 'Adaptive Quantization Algorithm'. If the relative variance profiles differ by more than 20% at any magnitude bin, or a formal equality test (e.g., F-test on variances in each bin) rejects at 5% significance, then the readback noise profile is not an accurate proxy for inference-time error, and the noise-aware density reshaping is misaligned.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central mechanism of NANQ is the inverse-noise density ρ(w)=1/(σ(w)+ε)^γ in Eq. (5), which redistributes quantization levels based on a magnitude-dependent noise profile σ(w) obtained from repeated on-chip readback of programmed conductance states. The paper's own Discussion limits this: 'the current framework approximates CIM non-idealities as magnitude-dependent weight perturbations.' During actual matrix-vector multiplications, the dominant error sources may include input-dependent cell-current nonlinearity, IR drop, and peripheral readout noise, none of which are captured by weight-readback measurements. If σ(w) from readback does not accurately describe the variance of errors that corrupt the MAC outputs, then the density reshaping is misaligned with the true error process. In that case, the empirical improvements in Table 1 could arise primarily from the non-uniform density matching the bell-shaped weight distribution (NANQ concentrates levels near zero, as do APoT and PowerQuant) rather than from noise-aware allocation. The theoretical SNR analysis (Eqs. 2-4) is explicitly motivational and the paper concedes it 'does not uniquely determine the optimal quantization density,' so the specific form of Eq. (5) is not independently justified. The load-bearing empirical claim therefore depends on the unverified equivalence between readback noise and inference-time noise; without this, the method's contribution is not the 'noise-floor-aware' mechanism advertised, and its generalizability to other CIM technologies (e.g., RRAM, MRAM) is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"NANQ proposes a training-free, noise-aware mixed-precision non-uniform quantization framework for analog compute-in-memory (CIM) inference. The method measures a magnitude-dependent weight-noise profile sigma(w) from repeated on-chip readbacks, converts it into a non-uniform quantization density rho(w)=1/(sigma(w)+epsilon)^gamma, and constructs quantization boundaries by equal-partitioning the cumulative density. A second stage assigns layer-wise bit-widths by comparing each layer's hardware-aware loss curve against a unified saturation threshold tau. On the authors' eFlash CIM SoC, NANQ is evaluated on four vision models and five language models at magnitude bit-widths from 2 to 6, and in a mixed-precision mode with matched equivalent bit-width budgets. The central empirical claims are that at 2-bit magnitude quantization NANQ improves vision accuracy by 8.05 percentage points and reduces language-model perplexity by 54.7% on average over PowerQuant, and that mixed-precision NANQ achieves 3.2-3.8 equivalent bits while capturing most of the gains of higher-precision allocation.","tokens_in":11046,"tokens_out":9442,"duration_ms":71475,"significance":"If the empirical claims hold, this is a practically valuable result for low-bit analog CIM inference: the method is training-free, requires no retraining or combinatorial search, and is evaluated on real hardware across nine model families. The paper's strengths include the use of genuine on-chip measurements, a matched-budget mixed-precision protocol, and sensitivity analyses for the gamma and tau hyperparameters. The 38-of-45 claim is consistent with the numbers in Table 1. The main unresolved issue is attribution: the paper's advertised mechanism is 'noise-floor awareness,' but the reported gains could partly arise from the non-uniform density simply matching the bell-shaped weight distribution, especially because gamma and tau are tuned on calibration/full-model trials. Additional ablations and clearer experimental protocol details are needed to separate these explanations.","major_comments":[{"comment":"The central mechanism assumes that the readback-derived sigma(w) in Eq. (5) characterizes the error process that actually corrupts weights during analog MAC operations, but the Discussion states that 'the current framework approximates CIM non-idealities as magnitude-dependent weight perturbations.' Because a density concentrating levels near zero also matches typical bell-shaped weight distributions, the reported improvements could in principle come from conventional non-uniform quantization rather than from noise-floor awareness. Please add an ablation that replaces sigma(w) with a mismatched profile (e.g., a constant or a reversed profile) or that compares against a density derived from the weight distribution alone, and show that the measured sigma(w) is responsible for the gains.","section":"Adaptive Quantization Algorithm, Eq. (5)"},{"comment":"The mixed-precision procedure selects gamma via Eq. (10) using a calibration set D_cal and selects tau via full-model trials, but the paper does not state the size of D_cal, whether D_cal and the full-model validation set are disjoint from the test sets used in Table 1, or how tau is chosen on the full model. If these sets overlap with the reported test sets, the selection of gamma and tau is optimistically biased and Table 1's numbers are not a clean held-out evaluation. Please specify the calibration/validation/test split and report the performance of the final configuration on a fully held-out test set.","section":"Noise-Aware Mixed-Precision Allocation, Eq. (10)"},{"comment":"Table 1 reports results averaged over three on-chip runs but gives no standard deviations or confidence intervals, and several entries are close to the best baseline (e.g., ResNet-20 at Bm=5 and OPT-1.3B at Bm=6 differ from the best baseline by about 0.25 and 0.01, respectively). Without error bars the claim of 38/45 best configurations is not fully supported. Please provide run-to-run variability for Table 1, at least for the near-tie entries.","section":"Experiments, Table 1"},{"comment":"The method as written in Eqs. (9)-(10) refers to a measured noise distribution p_hw and samples xi from it, but only the pointwise sigma(w) is defined in Eq. (5). It is unclear how p_hw is estimated and how 'hardware noise only to this layer' is implemented during layer-wise evaluation, i.e., whether the layer is actually programmed on chip or whether synthetic noise is added offline. Please clarify the noise sampling procedure so the experiments are reproducible.","section":"Methodology, Eqs. (9)-(10)"}],"minor_comments":[{"comment":"Eq. (2) treats quantization error and hardware noise as independent additive variances; please justify this assumption, since device nonlinearity can create correlations between the two terms.","section":"Theoretical Analysis, Eq. (2)"},{"comment":"Table 2 presents a single gamma per Bm, whereas Eq. (10) defines a per-layer gamma*_l(b); clarify whether the table reports an aggregated or representative value.","section":"Ablation Study, Table 2"},{"comment":"Please specify whether the SNR in Fig. 8 is measured from the chip output or computed from a simulation using the measured noise profile, and how the 'quantization only' setting is produced.","section":"Experiments, Fig. 8"},{"comment":"The caption of Fig. 9 should list the exact equivalent bit-width values for each model rather than only the range 3.2-3.8.","section":"Experiments, Fig. 9"},{"comment":"Please define the total bit-width including the sign bit explicitly; saying '2-bit weight-magnitude quantization' could be misread as 2-bit total.","section":"Experiment Setup"},{"comment":"Eq. (3) uses a proportionality to 2^(-2b), but for non-uniform quantizers the local step at a given weight does not necessarily scale exactly as 2^(-b); a brief justification or caveat would be helpful.","section":"Theoretical Analysis, Eq. (3)"}],"recommendation":"major_revision","confidential_remarks":"The 38/45 count checks out against Table 1. The main risk is attribution: with gamma and tau tuned on calibration/full-model trials and no error bars, the empirical claim is real but its interpretation as noise-floor awareness needs the proposed ablation. The missing data-split description is a straightforward fix and should be addressed before any acceptance decision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, this one is worth a look. NANQ does something genuinely new: instead of designing quantization levels to minimize ideal reconstruction error, it takes a measured magnitude-dependent noise profile from an eFlash CIM array and builds a non-uniform quantizer by inverse-noise-weighted density and cumulative-density partitioning. It also assigns per-layer bit-widths by detecting each layer's precision saturation point under hardware noise. The key empirical claim is that under 2-bit weight-magnitude quantization it beats PowerQuant by 8.05 accuracy points on vision models and reduces LM PPL by 54.7% on the real chip. That is a concrete, useful result for CIM hardware developers.\n\nWhat's good: the method is training-free, simple, has one main hyperparameter, and the CDF construction is clean. The layer-wise bit allocation is intuitive and avoids combinatorial search. The results are on actual silicon, not simulation. The sensitivity analysis for gamma and tau is honest and shows the trend you'd hope for. The paper explicitly says the theory is motivational and doesn't overclaim it. The discussion acknowledges that the framework treats all non-idealities as magnitude-dependent weight perturbations, which is a real limitation and they flag it.\n\nSoft spots, in order of concern. First, the main Table 1 reports means over three runs but no error bars; for low-bit cases where PPL is ~300, run-to-run variance matters. They show std dev in Tables 2 and 3, so they have the data; means alone are hard to interpret. Second, the selection of gamma and tau is under-specified. Gamma is chosen per layer per bit-width on a calibration set, but how many inference trials, how it interacts with the chosen quantizer, and whether the calibration set is disjoint from the test set isn't stated clearly. Tau is chosen by full-model trials without reporting the candidate set or the data split. This is fixable, not fatal.\n\nThe stress-test worry is that the noise profile measured by readback may not match the noise that actually corrupts the MAC output during inference, so the density reshaping might just be another way of concentrating levels near zero, which APoT and PowerQuant also do. That is a fair concern, but I don't think it kills the paper. The gains are real on the chip, and even if the mechanism is not exactly as advertised, the method is a useful empirical recipe. The transferability to RRAM/MRAM is unproven, but they don't claim it. I'd want to see at least one experiment that varies the noise environment (say, temperature or voltage) and checks whether the quantization levels track it.\n\nWho it's for: people doing quantization for analog CIM, hardware-aware model compression. It deserves a serious referee; the main things to tighten are error bars, protocol clarity, and ideally a second device or noise-condition experiment. Send it out.","headline":"Real-chip, training-free non-uniform quantizer for analog CIM that works; the noise-awareness story is plausible but only partially pinned down.","tokens_in":11636,"tokens_out":2309,"would_cite":true,"duration_ms":20503,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that when quantizing neural networks for analog compute-in-memory, the hardware's magnitude-dependent noise floor—not ideal reconstruction error—should dictate where quantization bits go, and that doing so recovers most…","keywords":["analog compute-in-memory","non-uniform quantization","mixed precision","noise floor","eFlash","noise-aware quantization","low-bit neural networks","quantization saturation"],"falsifier":"Run the same NANQ configuration on a CIM device where the effective inference-time weight noise is measured directly (e.g., by differencing ideal and actual MAC outputs over many runs) and compare it with the readback-derived $\\sigma(w)$; if the two profiles differ significantly, NANQ's density reshaping will be misaligned and its gains over PowerQuant should shrink or reverse.","tokens_in":10577,"feed_emoji":"🎯","tokens_out":5964,"duration_ms":49115,"temperature":0.7,"pith_summary":"This paper claims that analog compute-in-memory (CIM) inference is best quantized by matching precision to the hardware's magnitude-dependent noise floor rather than to ideal reconstruction error. The authors propose NANQ, a training-free framework that converts a chip-measured noise profile into a non-uniform quantization density, and they show on an eFlash CIM system that this recovers most of the accuracy lost by aggressive 2-bit weight quantization. In their on-chip experiments, NANQ improves vision-model top-1 accuracy by 8.05 percentage points and reduces language-model perplexity by 54.7% on average over PowerQuant at 2-bit weights. They also introduce a layer-wise saturation rule that picks each layer's bit-width from hardware-aware loss curves, reaching 3.2-3.8 equivalent bits with mixed-precision gains matching far higher resource use.","feed_headline":"Noise-aware quantizer lifts 2-bit CIM vision by 8 points","feed_subtitle":"By measuring each chip's noise floor, NANQ puts quantization bits where they count and beats standard methods at 2-bit weights.","key_machinery":"The load-bearing mechanisms are: (1) the inverse-noise quantization density $\\rho(w)=1/(\\sigma(w)+\\epsilon)^\\gamma$, which reshapes the level spacing so that low-noise weight magnitudes get finer resolution and noise-dominated magnitudes get coarser intervals; (2) cumulative-density partitioning $F(e_i)=\\frac{i}{Q}F(w_{\\max})$ that converts this continuous density into discrete boundaries without iterative optimization; and (3) a layer-wise saturation criterion $R_l(b)$ that measures the fraction of recoverable loss reduction still available at bit-width $b$ and picks the smallest $b$ where $R_l(b)\\le\\tau$. Together these turn a measured hardware noise profile directly into a quantizer and a bit-width allocation, with no retraining and no combinatorial search.","core_discovery":"The central discovery is a precision-saturation effect: once local quantization error falls below the hardware noise variance, adding bits no longer raises output SNR, so quantization resolution should be concentrated in low-noise weight regions. NANQ implements this by measuring a magnitude-dependent noise profile $\\sigma(w)$ from repeated on-chip readbacks, defining an inverse-noise density $\\rho(w)=1/(\\sigma(w)+\\epsilon)^\\gamma$, and partitioning the cumulative density into equal-probability intervals to set non-uniform boundaries. On the authors' eFlash CIM SoC, this yields the best quantized result in 38 of 45 model-bit-width configurations, with the largest gains at 2-3 bits. For mixed precision, NANQ computes each layer's marginal loss reduction per added bit, fits a monotone loss curve, and selects the smallest bit-width at which the remaining recoverable loss reduction falls below a unified threshold $\\tau$; across models this lands at 3.2-3.8 equivalent bits while matching or beating uniform, APoT, PowerQuant, and PNMQ under matched budgets.","pith_inferences":["If the readback-derived noise profile drifts with temperature, aging, or operating point, NANQ's density reshaping would need periodic recalibration; the paper acknowledges this in its limitation discussion.","The same cumulative-density construction could be applied to activation noise or output noise if those profiles were measured, effectively extending NANQ from weight-only to full-stack quantization.","Because the density concentrates levels where the weight distribution and low noise overlap, NANQ's benefit should scale with the kurtosis of the weight distribution; heavy-tailed models may need a different $\\gamma$ per layer than the uniform $\\gamma^*$ currently searched.","A direct test of the central mechanism would be to hold the quantizer fixed and artificially inflate the measured $\\sigma(w)$ in simulation: NANQ's advantage should shrink monotonically as the simulated noise profile diverges from the hardware's true profile."],"forward_implications":["At 2-bit weight magnitude, NANQ lifts vision accuracy by 7 to 24 points over PowerQuant on individual models and cuts language-model perplexity by up to 73.3%.","Mixed-precision NANQ with $\\tau=0.1$ reaches 3.2-3.8 equivalent bits while matching or exceeding the performance of uniform, APoT, PowerQuant, and PNMQ at the same budget.","The precision-saturation analysis implies that any further reduction in hardware noise (e.g., improved programming or readout) directly translates into more usable bits per layer.","Layer-wise saturation points vary widely; allocating uniform precision to all layers wastes bits on saturated layers and starves unsaturated ones.","NANQ is training-free and requires only a few inference trials for parameter search, making it practical for deployment on existing CIM accelerators."],"supporting_citations":[{"why":"PowerQuant serves as the primary non-uniform quantization baseline; NANQ's headline 2-bit accuracy and perplexity gains are reported against it.","marker":"Yvinec et al. 2023"},{"why":"APoT is the structured non-uniform baseline; NANQ is compared against its on-chip SNR and end-to-end results.","marker":"Li et al. 2020"},{"why":"PNMQ provides the mixed-precision allocation baseline that NANQ must beat under matched equivalent bit-widths.","marker":"Chikin and Antiukh 2022"},{"why":"Describes the eFlash CIM SoC used for on-chip inference and noise characterization.","marker":"Wang et al. 2023"},{"why":"WikiText-2 is the language-model evaluation corpus used for perplexity comparisons.","marker":"Merity et al. 2016"}],"fun_headline_variants":["Noise-floor-aware quantizer gains 8.05 points at 2-bit CIM","Why 2-bit CIM fails: precision saturation beats uniform quantizers","NANQ: density-aware bits cut CIM noise, lift accuracy by 8","Mixed-precision NANQ matches full precision at 3.2-3.8 bits","Quantization bits saturate: 2-bit CIM gains via noise-floor density"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The quantizer assumes that the noise profile measured by reading back programmed conductance states is the same noise that corrupts weights during real matrix-vector multiplications, and that activation and readout noise are negligible.","fun_headline_variants_meta":{"raw":{"variants":["Noise-floor-aware quantizer gains 8.05 points at 2-bit CIM","Why 2-bit CIM fails: precision saturation beats uniform quantizers","NANQ: density-aware bits cut CIM noise, lift accuracy by 8","Mixed-precision NANQ matches full precision at 3.2-3.8 bits","Quantization bits saturate: 2-bit CIM gains via noise-floor density"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000807,"raw_usage":{"total_tokens":3562,"prompt_tokens":985,"completion_tokens":2577,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":601,"completion_tokens_details":{"reasoning_tokens":2467}},"tokens_in":601,"tokens_out":2577,"duration_ms":16140,"temperature":1.0,"reasoning_tokens":2467,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:02:08.695987+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same NANQ configuration on a CIM device where the effective inference-time weight noise is measured directly (e.g., by differencing ideal and actual MAC outputs over many runs) and compare it with the readback-derived $\\sigma(w)$; if the two profiles differ significantly, NANQ's density reshaping will be misaligned and its gains over PowerQuant should shrink or reverse.","supporting_citations":[{"cited_title":"2023 , pages=","cited_arxiv_id":null,"evidence_quote":"Describes the eFlash CIM SoC used for on-chip inference and noise characterization."}],"review_version":2}