Pith. sign in

REVIEW 4 major objections 6 minor 32 references

NANQ: Noise-Floor-Aware Mixed-Precision Non-Uniform Quantization for Analog Compute-in-Memory

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper argues that when quantizing neural networks for analog compute-in-memory, the hardware's magnitude-dependent noise floor—not ideal reconstruction error—should dictate where quantization bits go, and that doing so recovers most…

desk verdict Real-chip, training-free non-uniform quantizer for analog CIM that works; the noise-awareness story is plausible but only partially pinned down. read the letter →

arxiv 2608.02700 v1 pith:FVPXXRDM submitted 2026-08-03 cs.LG cs.AI

Yizhe Chen , Wenshuai Yao , Saiya Wang , Yuannuo Feng , Wenbo Qi , Kechao Tang , Ngai Wong , Wenyong Zhou
show 1 more author
Wang Kang
This is my paper · ORCID
classification cs.LGcs.AI
keywords analogcompute-in-memorynon-uniformquantizationmixedprecisionnoiseflooreFlashnoise-awarelow-bitneuralnetworkssaturation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that analog compute-in-memory (CIM) inference is best quantized by matching precision to the hardware's magnitude-dependent noise floor rather than to ideal reconstruction error. The authors propose NANQ, a training-free framework that converts a chip-measured noise profile into a non-uniform quantization density, and they show on an eFlash CIM system that this recovers most of the accuracy lost by aggressive 2-bit weight quantization. In their on-chip experiments, NANQ improves vision-model top-1 accuracy by 8.05 percentage points and reduces language-model perplexity by 54.7% on average over PowerQuant at 2-bit weights. They also introduce a layer-wise saturation rule that picks each layer's bit-width from hardware-aware loss curves, reaching 3.2-3.8 equivalent bits with mixed-precision gains matching far higher resource use.

What carries the argument

The load-bearing mechanisms are: (1) the inverse-noise quantization density $\rho(w)=1/(\sigma(w)+\epsilon)^\gamma$, which reshapes the level spacing so that low-noise weight magnitudes get finer resolution and noise-dominated magnitudes get coarser intervals; (2) cumulative-density partitioning $F(e_i)=\frac{i}{Q}F(w_{\max})$ that converts this continuous density into discrete boundaries without iterative optimization; and (3) a layer-wise saturation criterion $R_l(b)$ that measures the fraction of recoverable loss reduction still available at bit-width $b$ and picks the smallest $b$ where $R_l(b)\le\tau$. Together these turn a measured hardware noise profile directly into a quantizer and a bit-width allocation, with no retraining and no combinatorial search.

What would settle it

Run the same NANQ configuration on a CIM device where the effective inference-time weight noise is measured directly (e.g., by differencing ideal and actual MAC outputs over many runs) and compare it with the readback-derived $\sigma(w)$; if the two profiles differ significantly, NANQ's density reshaping will be misaligned and its gains over PowerQuant should shrink or reverse.

Watch

Extended reading notes

Core claim

The central discovery is a precision-saturation effect: once local quantization error falls below the hardware noise variance, adding bits no longer raises output SNR, so quantization resolution should be concentrated in low-noise weight regions. NANQ implements this by measuring a magnitude-dependent noise profile $\sigma(w)$ from repeated on-chip readbacks, defining an inverse-noise density $\rho(w)=1/(\sigma(w)+\epsilon)^\gamma$, and partitioning the cumulative density into equal-probability intervals to set non-uniform boundaries. On the authors' eFlash CIM SoC, this yields the best quantized result in 38 of 45 model-bit-width configurations, with the largest gains at 2-3 bits. For mixed precision, NANQ computes each layer's marginal loss reduction per added bit, fits a monotone loss curve, and selects the smallest bit-width at which the remaining recoverable loss reduction falls below a unified threshold $\tau$; across models this lands at 3.2-3.8 equivalent bits while matching or beating uniform, APoT, PowerQuant, and PNMQ under matched budgets.

Load-bearing premise

The quantizer assumes that the noise profile measured by reading back programmed conductance states is the same noise that corrupts weights during real matrix-vector multiplications, and that activation and readout noise are negligible.

Editorial extensions

If this is right

  • At 2-bit weight magnitude, NANQ lifts vision accuracy by 7 to 24 points over PowerQuant on individual models and cuts language-model perplexity by up to 73.3%.
  • Mixed-precision NANQ with $\tau=0.1$ reaches 3.2-3.8 equivalent bits while matching or exceeding the performance of uniform, APoT, PowerQuant, and PNMQ at the same budget.
  • The precision-saturation analysis implies that any further reduction in hardware noise (e.g., improved programming or readout) directly translates into more usable bits per layer.
  • Layer-wise saturation points vary widely; allocating uniform precision to all layers wastes bits on saturated layers and starves unsaturated ones.
  • NANQ is training-free and requires only a few inference trials for parameter search, making it practical for deployment on existing CIM accelerators.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the readback-derived noise profile drifts with temperature, aging, or operating point, NANQ's density reshaping would need periodic recalibration; the paper acknowledges this in its limitation discussion.
  • The same cumulative-density construction could be applied to activation noise or output noise if those profiles were measured, effectively extending NANQ from weight-only to full-stack quantization.
  • Because the density concentrates levels where the weight distribution and low noise overlap, NANQ's benefit should scale with the kurtosis of the weight distribution; heavy-tailed models may need a different $\gamma$ per layer than the uniform $\gamma^*$ currently searched.
  • A direct test of the central mechanism would be to hold the quantizer fixed and artificially inflate the measured $\sigma(w)$ in simulation: NANQ's advantage should shrink monotonically as the simulated noise profile diverges from the hardware's true profile.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. NANQ proposes a training-free, noise-aware mixed-precision non-uniform quantization framework for analog compute-in-memory (CIM) inference. The method measures a magnitude-dependent weight-noise profile sigma(w) from repeated on-chip readbacks, converts it into a non-uniform quantization density rho(w)=1/(sigma(w)+epsilon)^gamma, and constructs quantization boundaries by equal-partitioning the cumulative density. A second stage assigns layer-wise bit-widths by comparing each layer's hardware-aware loss curve against a unified saturation threshold tau. On the authors' eFlash CIM SoC, NANQ is evaluated on four vision models and five language models at magnitude bit-widths from 2 to 6, and in a mixed-precision mode with matched equivalent bit-width budgets. The central empirical claims are that at 2-bit magnitude quantization NANQ improves vision accuracy by 8.05 percentage points and reduces language-model perplexity by 54.7% on average over PowerQuant, and that mixed-precision NANQ achieves 3.2-3.8 equivalent bits while capturing most of the gains of higher-precision allocation.

Significance. If the empirical claims hold, this is a practically valuable result for low-bit analog CIM inference: the method is training-free, requires no retraining or combinatorial search, and is evaluated on real hardware across nine model families. The paper's strengths include the use of genuine on-chip measurements, a matched-budget mixed-precision protocol, and sensitivity analyses for the gamma and tau hyperparameters. The 38-of-45 claim is consistent with the numbers in Table 1. The main unresolved issue is attribution: the paper's advertised mechanism is 'noise-floor awareness,' but the reported gains could partly arise from the non-uniform density simply matching the bell-shaped weight distribution, especially because gamma and tau are tuned on calibration/full-model trials. Additional ablations and clearer experimental protocol details are needed to separate these explanations.

major comments (4)
  1. [Adaptive Quantization Algorithm, Eq. (5)] The central mechanism assumes that the readback-derived sigma(w) in Eq. (5) characterizes the error process that actually corrupts weights during analog MAC operations, but the Discussion states that 'the current framework approximates CIM non-idealities as magnitude-dependent weight perturbations.' Because a density concentrating levels near zero also matches typical bell-shaped weight distributions, the reported improvements could in principle come from conventional non-uniform quantization rather than from noise-floor awareness. Please add an ablation that replaces sigma(w) with a mismatched profile (e.g., a constant or a reversed profile) or that compares against a density derived from the weight distribution alone, and show that the measured sigma(w) is responsible for the gains.
  2. [Noise-Aware Mixed-Precision Allocation, Eq. (10)] The mixed-precision procedure selects gamma via Eq. (10) using a calibration set D_cal and selects tau via full-model trials, but the paper does not state the size of D_cal, whether D_cal and the full-model validation set are disjoint from the test sets used in Table 1, or how tau is chosen on the full model. If these sets overlap with the reported test sets, the selection of gamma and tau is optimistically biased and Table 1's numbers are not a clean held-out evaluation. Please specify the calibration/validation/test split and report the performance of the final configuration on a fully held-out test set.
  3. [Experiments, Table 1] Table 1 reports results averaged over three on-chip runs but gives no standard deviations or confidence intervals, and several entries are close to the best baseline (e.g., ResNet-20 at Bm=5 and OPT-1.3B at Bm=6 differ from the best baseline by about 0.25 and 0.01, respectively). Without error bars the claim of 38/45 best configurations is not fully supported. Please provide run-to-run variability for Table 1, at least for the near-tie entries.
  4. [Methodology, Eqs. (9)-(10)] The method as written in Eqs. (9)-(10) refers to a measured noise distribution p_hw and samples xi from it, but only the pointwise sigma(w) is defined in Eq. (5). It is unclear how p_hw is estimated and how 'hardware noise only to this layer' is implemented during layer-wise evaluation, i.e., whether the layer is actually programmed on chip or whether synthetic noise is added offline. Please clarify the noise sampling procedure so the experiments are reproducible.
minor comments (6)
  1. [Theoretical Analysis, Eq. (2)] Eq. (2) treats quantization error and hardware noise as independent additive variances; please justify this assumption, since device nonlinearity can create correlations between the two terms.
  2. [Ablation Study, Table 2] Table 2 presents a single gamma per Bm, whereas Eq. (10) defines a per-layer gamma*_l(b); clarify whether the table reports an aggregated or representative value.
  3. [Experiments, Fig. 8] Please specify whether the SNR in Fig. 8 is measured from the chip output or computed from a simulation using the measured noise profile, and how the 'quantization only' setting is produced.
  4. [Experiments, Fig. 9] The caption of Fig. 9 should list the exact equivalent bit-width values for each model rather than only the range 3.2-3.8.
  5. [Experiment Setup] Please define the total bit-width including the sign bit explicitly; saying '2-bit weight-magnitude quantization' could be misread as 2-bit total.
  6. [Theoretical Analysis, Eq. (3)] Eq. (3) uses a proportionality to 2^(-2b), but for non-uniform quantizers the local step at a given weight does not necessarily scale exactly as 2^(-b); a brief justification or caveat would be helpful.

Circularity Check

0 steps flagged · score 0.0 of 10

No load-bearing circularity: the noise profile is an independent measured input, the density form is an acknowledged ansatz, and the reported gains are empirical chip comparisons.

full rationale

NANQ's derivation chain is not circular. The noise profile σ(w) is an independently measured device characterization, obtained 'directly from repeated on-chip readback measurements at different programmed weight magnitudes,' rather than being defined in terms of the target loss or the reported accuracy. The density ρ(w)=1/(σ(w)+ε)^γ is explicitly introduced as a motivated ansatz, and the paper states that the earlier SNR analysis 'does not uniquely determine the optimal quantization density,' so no claim is made that the functional form follows from the theory. Quantizer boundaries and mixed-precision bit-widths are constructed from this measured profile and from loss curves evaluated with real-chip or noise-sampled responses; the hyperparameters γ and τ are calibrated on validation loss rather than hidden fits presented as predictions. The central empirical claims—2-bit accuracy/PPL improvements over PowerQuant and mixed-precision gains under matched budgets—are on-chip measurements against external baselines, not derivations from the method's own definitions. The only definitional element is that τ=0.1 means 90% of the layer-wise recoverable loss reduction is captured, but the paper is transparent about this and separately verifies end-to-end saturation, so the abstract's 'captures most of the gains' is a summary of a calibrated operating point, not a parameter-free prediction. No load-bearing self-citation or imported uniqueness theorem is present. The Discussion's caveat that the framework 'approximates CIM non-idealities as magnitude-dependent weight perturbations' is a transferability limitation, not a circularity.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

NANQ's construction relies on the measured noise profile σ(w), which is an empirical input rather than a derived quantity. The meaningful free parameters are γ (density exponent) and τ (saturation threshold), both selected by calibration/full-model trials, plus the unspecified stability constant ε. The theoretical analysis (Eqs. 2-4) is motivational and does not determine γ, ε, or τ, so the method's performance is an empirical result. No new physical entities are introduced. The layer-wise saturation curves assume single-layer perturbation is representative of the full-network error.

free parameters (3)
  • γ (noise-adaptation exponent) = per-layer, per-bit-width; e.g., 0.93 to 1.42 for Pythia-410M in Table 2
    Controls how strongly the quantization density follows the measured noise profile; selected by minimizing calibration loss in Eq. (10).
  • τ (saturation threshold) = 0.10 used in main mixed-precision results
    Determines the layer bit-width at which R_l(b) ≤ τ; selected by evaluating candidate thresholds on the full model.
  • ε (numerical stability constant) = not specified; small positive
    Prevents division by zero in Eq. (5) and avoids level concentration where σ(w) ≈ 0; its value is not reported.
assumptions (5)
  • domain assumption Total error variance is the sum of quantization-error variance and hardware-noise variance, Eq. (2).
    Justifies the SNR-saturation argument used as motivation; independence of the two noise sources is assumed for this approximation.
  • standard math Local quantization-error variance scales as Δ_b(w)^2/12 ∝ 2^{-2b}, Eq. (3).
    Well-known for uniform quantizers; used only to motivate diminishing returns, not to construct NANQ's density.
  • domain assumption Readback-measured σ(w) represents inference-time weight noise, Eq. (5).
    Central premise for converting noise into quantization density; the Discussion notes the framework approximates non-idealities as magnitude-dependent weight perturbations.
  • domain assumption Layer-wise quantization effects are independent; one-layer-at-a-time loss estimates combine through a single global threshold τ, Eq. (10).
    Cross-layer error accumulation is handled only by tuning τ; the paper does not model interactions explicitly.
  • domain assumption The fitted layer-wise loss curve is monotonically non-increasing.
    Monotonicity is imposed to define R_l(b); real validation loss with finite calibration data may be non-monotonic.

how reviews work

0 comments
Cite this review

Pith. "Pith review of NANQ: Noise-Floor-Aware Mixed-Precision Non-Uniform Quantization for Analog Compute-in-Memory." pith.science (2026). https://pith.science/paper/FVPXXRDM

@misc{pith2026260802700,
  author       = {Pith},
  title        = {Pith review of: NANQ: Noise-Floor-Aware Mixed-Precision Non-Uniform Quantization for Analog Compute-in-Memory},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FVPXXRDM}},
  note         = {Machine review of arXiv:2608.02700}
}
read the original abstract

Analog compute-in-memory (CIM) enables energy-efficient neural network inference, but device variation and read noise can severely degrade low-bit quantized models. Existing CIM-oriented quantization methods mainly minimize ideal quantization error, ignoring the hardware noise floor and thus causing inefficient precision allocation. We propose NANQ, a noise-aware mixed-precision non-uniform quantization framework for analog CIM. NANQ models magnitude-dependent weight noise from measured responses of an eFlash CIM array and converts the noise profile into an adaptive quantization density, assigning finer resolution to low-noise regions while avoiding ineffective precision in noise-dominated regions. It further assigns layer-wise bit-widths by identifying each layer's precision saturation point under hardware noise using a unified threshold. On-chip experiments on an eFlash CIM SoC show that, under 2-bit weight-magnitude quantization, NANQ improves vision-model accuracy by 8.05 percentage points and reduces language-model PPL by 54.7% on average over PowerQuant. Mixed-precision NANQ captures most of the gains obtainable from additional quantization resources with only 3.2-3.8 equivalent bits.

Figures

Figures reproduced from arXiv: 2608.02700 by the authors.

Figure 1
Figure 1. Fixed quantization levels spread into noisy distribu [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 3
Figure 3. Output SNR versus weight quantization bit-width [PITH_FULL_IMAGE:figures/full_fig_p002_3.png] view at source ↗
Figure 4
Figure 4. Layer-wise marginal gains from increasing weight [PITH_FULL_IMAGE:figures/full_fig_p003_4.png] view at source ↗
Figures from the paper (4 more)
Figure 5
Figure 5. Figure 5: Overview of our NANQ method. Building on existing quantization methods, NANQ adjusts quantization points based [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]
Figure 6
Figure 6. Figure 6: For CNNs, convolutional and fully connected layers [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 8
Figure 8. Figure 8: Single-MAC output SNR under (a) quantization [PITH_FULL_IMAGE:figures/full_fig_p005_8.png]
Figure 9
Figure 9. Figure 9: Comparison of mixed-precision quantization meth [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 25 canonical work pages

  1. [1]

    Zhou, Aojun and others , booktitle =

    work page

  2. [2]

    Li, Yuhang and others , booktitle =

    work page

  3. [3]

    IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , volume=

    Bai, Jinyu and others , title=. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , volume=. 2023 , publisher=

    work page 2023

  4. [4]

    IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , volume=

    Sun, Sifan and others , title=. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , volume=. 2024 , publisher=

    work page 2024

  5. [5]

    2024 IEEE International Conference on Integrated Circuits, Technologies and Applications (ICTA) , pages=

    Chen, Yizhe and others , title=. 2024 IEEE International Conference on Integrated Circuits, Technologies and Applications (ICTA) , pages=. 2024 , organization=

    work page 2024

  6. [6]

    Moore and More , volume=

    Chen, Yizhe and others , title=. Moore and More , volume=. 2025 , publisher=

    work page 2025

  7. [7]

    and others , booktitle=

    Guo, X. and others , booktitle=. 2017 , pages=

    work page 2017

  8. [8]

    2025 , pages=

    Mao, Wanru and others , booktitle =. 2025 , pages=

    work page 2025

Show all 32 references
  1. [9]

    2025 , pages=

    Liu, Hanjie and others , booktitle=. 2025 , pages=

    2025

  2. [10]

    2025 , pages=

    Wang, Guangyao and others , journal=. 2025 , pages=

    2025

  3. [11]

    2021 , volume=

    Jhang, Chuan-Jia and others , journal=. 2021 , volume=

    2021

  4. [12]

    2022 , volume=

    Pan, Biao and others , journal=. 2022 , volume=

    2022

  5. [13]

    2019 , pages=

    He, Zhezhi and others , booktitle=. 2019 , pages=

    2019

  6. [14]

    2019 , pages=

    Liu, Tao and others , booktitle=. 2019 , pages=

    2019

  7. [15]

    2023 , pages=

    Wang, Guangyao and others , booktitle=. 2023 , pages=

    2023

  8. [16]

    Xiang, Y. C. and others , booktitle=. 2019 , pages=

    2019

  9. [17]

    and others , title=

    Ikotun, Abiodun M. and others , title=. Information Sciences , volume=. 2023 , issn=. doi:10.1016/j.ins.2022.11.139 , keywords=

    2023 doi

  10. [18]

    Binary Neural Networks: A Survey , journal =

    Haotong Qin and others , keywords =. Binary Neural Networks: A Survey , journal =. 2020 , issn =. doi:10.1016/j.patcog.2020.107281 , url =

    2020

  11. [19]

    Ternary Weight Networks , year=

    Liu, Bin and others , booktitle=. Ternary Weight Networks , year=

  12. [20]

    HAQ: Hardware-Aware Automated Quantization With Mixed Precision , year=

    Wang, Kuan and others , booktitle=. HAQ: Hardware-Aware Automated Quantization With Mixed Precision , year=

  13. [21]

    HAWQ: Hessian AWare Quantization of Neural Networks With Mixed-Precision , year=

    Dong, Zhen and others , booktitle=. HAWQ: Hessian AWare Quantization of Neural Networks With Mixed-Precision , year=

  14. [22]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Data-free network compression via parametric non-uniform mixed precision quantization , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  15. [23]

    Applied Physics Letters , volume=

    Experimental and modeled analysis of source-line loading effects in NOR flash compute-in-memory arrays , author=. Applied Physics Letters , volume=. 2025 , publisher=

    2025

  16. [24]

    and Markov, Stanislav and Vil

    Georgiev, Vihar P. and Markov, Stanislav and Vil. IEEE Transactions on Electron Devices , title=. 2014 , volume=

    2014

  17. [25]

    Noise-Aware Adaptive Sampling for Robust Diffusion Models on Analog Compute-in-Memory , year=

    Feng, Yuannuo and Zhou, Wenyong and Lv, Yuexi and Liu, Hanjie and Wang, Guangyao and Liu, Zhengwu and Wong, Ngai and Kang, Wang , booktitle =. Noise-Aware Adaptive Sampling for Robust Diffusion Models on Analog Compute-in-Memory , year=

  18. [26]

    arXiv preprint arXiv:2301.09858 , year=

    Powerquant: Automorphism search for non-uniform quantization , author=. arXiv preprint arXiv:2301.09858 , year=

    arXiv

  19. [27]

    arXiv preprint arXiv:2306.07629 , year=

    Squeezellm: Dense-and-sparse quantization , author=. arXiv preprint arXiv:2306.07629 , year=

    arXiv

  20. [28]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Ompq: Orthogonal mixed precision quantization , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  21. [29]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Infoq: Mixed-precision quantization via global information flow , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  22. [30]

    Learning multiple layers of features from tiny images.(2009) , author=

    2009

  23. [31]

    International Journal of Computer Vision , volume =

    ImageNet Large Scale Visual Recognition Challenge , author =. International Journal of Computer Vision , volume =. 2015 , publisher =. doi:10.1007/s11263-015-0816-y , url =

    2015 doi

  24. [32]

    arXiv preprint arXiv:1609.07843 , year=

    Pointer sentinel mixture models , author=. arXiv preprint arXiv:1609.07843 , year=

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.