Pith. sign in

REVIEW 5 major objections 5 minor 25 references

A Fully Hardware Implemented Accelerator Design in ReRAM Analog Computing without ADCs

T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read RACA strips ADCs and per-layer DACs out of ReRAM inference by turning device thermal noise into the activation function, reaching 96.7% MNIST accuracy.

desk verdict Clever ADC-less ReRAM accelerator idea, but the WTA-to-softmax claim is asserted, not derived—the headline accuracy is not credible. read the letter →

arxiv 2412.19869 v1 pith:QLBSLQ5M submitted 2024-12-27 cs.AR cs.AI

classification cs.ARcs.AI
keywords ReRAMcrossbarComputing-in-Memorystochasticbinarizationnoise-drivenactivationADC-freeacceleratorwinner-take-allsoftmaxapproximationMNISTinference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

RACA is a ReRAM-based analog computing-in-memory accelerator that replaces the usual ADC readout, per-layer DACs, and explicit sigmoid/softmax activation circuits with simple comparators driven by device noise. The core idea is to lower the crossbar's signal-to-noise ratio until thermal noise currents dominate the comparison, so each neuron's binary output is stochastic with a sigmoid activation probability. The final classification layer uses a winner-take-all race among binary stochastic neurons, and repeated trials are accumulated to approximate a softmax distribution. On a fully connected MNIST network, this restores 96.7% accuracy after majority voting while cutting energy by 58.29% and area by 38.43% relative to a 1-bit-ADC baseline.

What carries the argument

The central mechanism is the binary stochastic neuron built from a trans-impedance amplifier, a reference-subtraction circuit, and a voltage comparator. ReRAM column current minus reference current is compared against a threshold while the device's Nyquist noise $i_{\text{RMS}}=\sqrt{4kT G \Delta f}$ injects randomness; tuning the read voltage, the weight-to-conductance scale, the readout bandwidth, and the number of devices per column sets the effective signal-to-noise ratio. The output layer replaces the comparator threshold with an adaptive winner-take-all threshold, and a counter accumulates the winning neuron across trials. This one circuit does the work that conventional accelerators assign to DACs, ADCs, and digital activation units.

What would settle it

Repeat a fixed input many times through the winner-take-all classifier, record which neuron wins each trial, and compare the empirical win fractions with the softmax probabilities of the same logits; a statistically significant mismatch would show that the hardware classification layer is not implementing the softmax distribution the network was trained with.

Watch

Extended reading notes

Core claim

The paper's central discovery is that the nonlinear activations of a neural network can be implemented physically rather than computed. When the read voltage is reduced so that the signal current leaving a ReRAM column is comparable to the device's Nyquist thermal noise $i_{\text{RMS}} = \sqrt{4kT G \Delta f}$, a comparator's decision is stochastic, and the activation probability takes the sigmoid form $1/(1+e^{-\sum_i W_{ij}x_i})$ (Eq. 13). In the output layer, a winner-take-all circuit with an adaptive threshold lets only one neuron fire per trial, and accumulating those trials across repeated experiments yields a cumulative probability distribution that the paper treats as the hardware's softmax approximation, $e^{\sum_i w_{ij} x_i}/\sum_k e^{\sum_i w_{ik} x_i}$ (Eq. 14). This lets the hidden and output layers run without DACs or ADCs and without any separate activation-function block; a simple counter tallies the classifier trials. The design is demonstrated on a [784,500,300,10] fully connected MNIST network, where it reaches 96.7% accuracy with repeated stochastic inference and majority voting.

Load-bearing premise

The load-bearing premise is that the winner-take-all race among binary stochastic neurons produces exactly the softmax distribution the network was trained with; if the race's cumulative probabilities differ from softmax, the final classification layer no longer implements the intended classifier.

Editorial extensions

If this is right

  • With comparator-based stochastic neurons, hidden and output layers no longer need ADCs or per-layer DACs, directly removing the largest reported contributors to energy and area in ReRAM accelerators.
  • According to the paper's device-to-algorithm simulations, the RACA design lowers energy by 58.29%, area by 38.43%, and raises energy efficiency from 61.3 to 148.58 TOPS/W on the benchmark network.
  • Because accuracy improves with repeated stochastic inference and majority voting, the architecture trades a small amount of latency for accuracy and has a tunable operating point through the read voltage and comparator threshold.
  • The mixed-signal design remains compatible with digital peripheral devices because the stochastic output is already binary, so no precision ADC is needed between layers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test of the softmax approximation would measure the empirical winner-take-all probabilities over many trials and compare them with $e^{\sum w x}/\sum e^{\sum w x}$; if the race distribution differs from softmax, retraining the final layer on the hardware's actual distribution could restore accuracy on datasets beyond MNIST.
  • The same stochastic-binarization mechanism may extend to convolutional layers and deeper networks, since the comparator circuit is not specific to fully connected layers, but the paper only demonstrates a 784-500-300-10 multilayer perceptron.
  • Because thermal noise scales with temperature, inference accuracy could drift with operating temperature; a temperature-compensated read voltage or a calibration pass would be needed before deployment in uncontrolled environments.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The manuscript proposes RACA, a ReRAM crossbar-based accelerator that replaces per-layer DACs, ADCs, and explicit activation-function units with stochastic binarization circuits driven by the intrinsic thermal noise of the ReRAM devices. It claims that comparator-thresholded currents yield sigmoid activation probabilities (Eq. 13) and that a winner-take-all (WTA) race among output neurons implements softmax (Eq. 14). Using a [784,500,300,10] fully connected network on MNIST, it reports 96.7% accuracy with repeated stochastic inference and majority voting, together with NeuroSim-based estimates of 58% energy and 38% area reduction relative to a 1-bit ADC baseline.

Significance. The idea of reusing device noise as a free stochastic source is attractive, and the reported energy/area gains would be important if the accuracy claim were supported. The paper presents a concrete architecture, qualitative circuit simulations, and a first-order NeuroSim comparison, and it explicitly acknowledges at least one missing component (the cumulative-probability counter). However, the central mathematical link between the stochastic comparator mechanism and softmax is not established, and the evaluation lacks a same-network baseline. The contribution is therefore not yet supported at the level required for publication.

major comments (5)
  1. [Section III-B, Eq. (14)] The WTA-to-softmax equivalence is asserted without derivation. If each neuron fires with the sigmoid probability of Eq. (13), then a WTA race among independent Bernoulli neurons selects neuron j with probability sigma(Z_j) / sum_k sigma(Z_k), not softmax. For two neurons with logits 10 and 0, this gives a winner probability of about 0.667 for the first neuron, whereas softmax gives about 0.99995. The paper provides no argument that the adaptive-threshold first-crossing mechanism produces Gumbel-distributed race times or any other distribution that would yield softmax. Because the network was trained with standard softmax cross-entropy, the deployed classifier may not implement the intended decision rule. The authors should either derive the race distribution from the circuit dynamics (including the noise statistics in Eq. (11)) or characterize it empirically and show that it matches softmax over the operating range; otherwise the 96.7% accuracy claim lacks a valid mechanism.
  2. [Section III-A, Eq. (13)] The approximation of the Gaussian CDF by a logistic sigmoid requires the noise standard deviation to be proportional to the scale parameter Vr*G0 in the sigmoid exponent. However, the variance in Eq. (11) is 4kT*Delta_f * sum_i(G_ij + G_ref), which depends on the conductance values and hence on the weights. No argument or bound is given for why this weight-dependent variance can be absorbed into the fixed scale Vr*G0. The paper should quantify the approximation error over the actual range of programmed conductances used in the MNIST experiments and state what assumptions about weight/conductance statistics are needed.
  3. [Section III-C] The text states that 'the hardware for computing the cumulative probability distribution of the classifier is not provided here, but it can be easily implemented at the output end with a simple counter.' This admitted omission directly contradicts the title and abstract's claim of a 'fully hardware implemented' accelerator. The counter is part of the classification readout and must be specified (including its area, energy, and timing overhead) before the architecture can be considered complete. The claim should be revised or the counter hardware should be included in the design and in Table I.
  4. [Section IV-C and Table I] No baseline accuracy for the same [784,500,300,10] FCNN with standard activations is reported, so the claim that RACA achieves 'without compromising inference accuracy' is unsubstantiated. The reader cannot tell whether 96.7% (with repeated trials and majority voting) matches the software-trained softmax accuracy for this architecture. In addition, Table I is based on a 'modified NeuroSim' but the modifications are not described; energy and area numbers for the missing counter hardware, the input DAC, and the WTA threshold circuitry are not itemized.
  5. [Abstract and Section III-C] The abstract and introduction claim that the design removes both DACs and ADCs, but Section III-C explicitly states that 'a DAC is used at the input stage to preserve the integrity of input data features.' The title says 'without ADCs,' which is consistent with keeping an input DAC, but the broader claim in the abstract overstates the contribution. The authors should state precisely which DACs/ADCs are eliminated and which remain.
minor comments (5)
  1. [Section III-A, Eq. (8)] The threshold variable p in Eq. (8) is not clearly connected to the comparator threshold Vth1 used later; aligning these notations would improve readability.
  2. [Section IV-C] In the sentence 'with set to 0.05V', the parameter name (presumably Vth0) is missing; please correct the missing symbol.
  3. [Section II-B, Eqs. (4)-(7)] The variable Ncol is mentioned in Section IV-A as an SNR-tuning knob but is not defined in the weight-mapping equations; please clarify how multiple devices per column enter the current summation.
  4. [Section IV-A] The text says the statistical probabilities in Fig. 4(c)-(f) are 'solely used for analysis and comparison.' This is ambiguous: it is not clear whether these curves come from SPICE-level circuit simulation, from a behavioral model, or from the analytic Eq. (13). Please specify the simulation methodology.
  5. [Section IV-C, Table I] The NeuroSim estimation should state the technology node, the ADC resolution and sampling rate assumptions for the baseline, and whether the baseline already includes the activation-function hardware whose area/energy is being removed.

Circularity Check

0 steps flagged · score 0.0 of 10

No input-equivalent circularity; Eq. 14 is an unsupported softmax assertion, not a circular reduction.

full rationale

The paper's derivation chain is self-contained with respect to the quantities it predicts. Eq. 13 derives the neuron activation probability from the physical Nyquist noise model (Eq. 11) and a Gaussian-CDF-to-logistic approximation; the target sigmoid is an external software activation, not a value fitted from the reported accuracy, and Vr, G0, Δf, and Ncol are described as SNR-tuning degrees of freedom rather than as parameters fitted to the final accuracy. The central efficiency claims (removal of DACs and ADCs) are structural circuit claims benchmarked in Table I via NeuroSim, not predictions that reduce to fitted inputs. No load-bearing argument is justified by a citation from the present authors; WTA is cited to Makhzani and Frey [12], an external source. The serious weakness is Eq. 14: the WTA-to-softmax relation is asserted, not derived from Eq. 13, and for two logits (10,0) a Bernoulli sigmoid race gives a winner probability of about 0.667 rather than softmax's 0.99995. That is an unsupported mathematical and empirical claim and a correctness risk, not a circular reduction: the paper never defines the hardware output in terms of the softmax result, and no fitted parameter is renamed as a prediction. The missing cumulative-probability counter hardware and absent baseline accuracy are completeness limitations rather than input-equivalent reasoning.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The central claim relies on three unverified premises: Gaussian device noise, the erf-to-logistic approximation, and the WTA-to-softmax mapping. The first two are physically plausible but not validated; the third is both unproved and mathematically suspect for sigmoid probabilities.

free parameters (5)
  • Read voltage Vr
    Amplitude of input voltage pulses; adjusted to set signal-to-noise ratio so that comparator output probability approximates sigmoid (Sec. IV-A, Fig. 4(c)).
  • Readout bandwidth Δf
    Bandwidth of readout circuit; adjusted to set noise level and sigmoid slope (Sec. IV-A, Fig. 4(e)).
  • Conductance scaling G0
    Maps weights to conductances (Eq. 4); sets the signal scale per column.
  • Columns per neuron Ncol
    Number of ReRAM devices per column; affects noise and signal averaging (Fig. 4(f)).
  • Rest threshold voltage Vth0 = 0.05 V
    Initial adaptive threshold in WTA circuit; set to 0.05 V in evaluation to balance sigmoid approximation and activation probability (Sec. IV-C).
assumptions (3)
  • domain assumption Thermal noise current in ReRAM devices is Gaussian with variance 4kT G Δf and independent across devices
    Used in Eq. 11 to derive the sigmoid activation probability; device noise properties are not verified experimentally in this paper.
  • standard math The Gaussian cumulative distribution function can be approximated by a logistic sigmoid
    Eq. 13 replaces erf(...) with 1/(1+e^{-x}); a known approximation but introduces uncontrolled error, and the variance term depends on weights.
  • ad hoc to paper Winner-take-all competition among stochastic neurons yields a softmax distribution
    Eq. 14 asserts the WTA probability equals e^{Σ w x}/Σe^{Σ w x} without derivation from the sigmoid activation probabilities of Eq. 13.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Fully Hardware Implemented Accelerator Design in ReRAM Analog Computing without ADCs." pith.science (2026). https://pith.science/paper/QLBSLQ5M

@misc{pith2026241219869,
  author       = {Pith},
  title        = {Pith review of: A Fully Hardware Implemented Accelerator Design in ReRAM Analog Computing without ADCs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QLBSLQ5M}},
  note         = {Machine review of arXiv:2412.19869}
}
read the original abstract

Emerging ReRAM-based accelerators process neural networks via analog Computing-in-Memory (CiM) for ultra-high energy efficiency. However, significant overhead in peripheral circuits and complex nonlinear activation modes constrain system energy efficiency improvements. This work explores the hardware implementation of the Sigmoid and SoftMax activation functions of neural networks with stochastically binarized neurons by utilizing sampled noise signals from ReRAM devices to achieve a stochastic effect. We propose a complete ReRAM-based Analog Computing Accelerator (RACA) that accelerates neural network computation by leveraging stochastically binarized neurons in combination with ReRAM crossbars. The novel circuit design removes significant sources of energy/area efficiency degradation, i.e., the Digital-to-Analog and Analog-to-Digital Converters (DACs and ADCs) as well as the components to explicitly calculate the activation functions. Experimental results show that our proposed design outperforms traditional architectures across all overall performance metrics without compromising inference accuracy.

Figures

Figures reproduced from arXiv: 2412.19869 by the authors.

Figure 1
Figure 1. The conventional CiM architecture. To address the above challenges, we propose a novel ac￾celerator architecture, RACA, and its corresponding hardware implementation. RACA consists of ReRAM crossbar arrays and hardware-implemented Sigmoid and SoftMax activation circuits. The Sigmoid function utilizes noise current from ReRAM devices as a random number generator, while each layer’s output is activated through stochas… view at source ↗
Figure 2
Figure 2. The implementation of the binary stochastic Sigmoid neurons by [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The implementation of the binary stochastic SoftMax neurons by [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: (a) presents illustrative traces of the output voltage over time, compared with the adaptive threshold voltage, for ten WTA binary stochastic neurons in three consecutive decision experiments. This indicates that only one neuron is activated in each decision experiment…
Figure 6
Figure 6. Figure 6: Experimental results of RACA. (a) Test results for varying the SNR [PITH_FULL_IMAGE:figures/full_fig_p004_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 24 canonical work pages

  1. [1]

    Equivalent-accuracy accelerated neural-network training using analogue memory,

    S. Ambrogio, P. Narayanan, H. Tsai, R. M. Shelby, I. Boybat, C. Di Nolfo, S. Sidler, M. Giordano, M. Bodini, N. C. Farinha, et al. , “Equivalent-accuracy accelerated neural-network training using analogue memory,” Nature, vol. 558, no. 7708, pp. 60–67, 2018

  2. [2]

    A compute-in-memory chip based on resistive random-access memory,

    W. Wan, R. Kubendran, C. Schaefer, S. B. Eryilmaz, W. Zhang, D. Wu, S. Deiss, P. Raina, H. Qian, B. Gao, et al., “A compute-in-memory chip based on resistive random-access memory,” Nature, vol. 608, no. 7923, pp. 504–512, 2022

  3. [3]

    Hardware implementation of memristor-based artificial neural networks,

    F. Aguirre, A. Sebastian, M. Le Gallo, W. Song, T. Wang, J. J. Yang, W. Lu, M.-F. Chang, D. Ielmini, Y . Yang, et al. , “Hardware implementation of memristor-based artificial neural networks,” Nature communications, vol. 15, no. 1, p. 1974, 2024

  4. [4]

    Towards a formally verified hardware root-of-trust for data-oblivious computing,

    L. Deutschmann, J. M ¨uller, M. R. Fadiheh, D. Stoffel, and W. Kunz, “Towards a formally verified hardware root-of-trust for data-oblivious computing,” in Proceedings of the 59th ACM/IEEE Design Automation Conference, pp. 727–732, 2022

  5. [5]

    Design guidelines of rram based neural-processing-unit: A joint device- circuit-algorithm analysis,

    W. Zhang, X. Peng, H. Wu, B. Gao, H. He, Y . Zhang, S. Yu, and H. Qian, “Design guidelines of rram based neural-processing-unit: A joint device- circuit-algorithm analysis,” in Proceedings of the 56th Annual Design Automation Conference 2019 , pp. 1–6, 2019

  6. [6]

    Fpcim: A fully- parallel robust reram cim processor for edge ai devices,

    Y .-C. Guo, W.-T. Lin, T.-H. Hou, and T.-S. Chang, “Fpcim: A fully- parallel robust reram cim processor for edge ai devices,” in 2023 IEEE International Symposium on Circuits and Systems (ISCAS) , pp. 1–5, IEEE, 2023

  7. [7]

    Prime: A novel processing-in-memory architecture for neural network computation in reram-based main memory,

    P. Chi, S. Li, C. Xu, T. Zhang, J. Zhao, Y . Liu, Y . Wang, and Y . Xie, “Prime: A novel processing-in-memory architecture for neural network computation in reram-based main memory,” ACM SIGARCH Computer Architecture News, vol. 44, no. 3, pp. 27–39, 2016

  8. [8]

    Neurosim+: An integrated device- to-algorithm framework for benchmarking synaptic devices and array architectures,

    P.-Y . Chen, X. Peng, and S. Yu, “Neurosim+: An integrated device- to-algorithm framework for benchmarking synaptic devices and array architectures,” in 2017 IEEE International Electron Devices Meeting (IEDM), pp. 6–1, IEEE, 2017

Show all 25 references
  1. [9]

    Compute-in-memory technologies and architectures for deep learning workloads,

    M. Ali, S. Roy, U. Saxena, T. Sharma, A. Raghunathan, and K. Roy, “Compute-in-memory technologies and architectures for deep learning workloads,” IEEE Transactions on Very Large Scale Integration (VLSI) Systems, vol. 30, no. 11, pp. 1615–1630, 2022

  2. [10]

    Fully hardware-implemented memristor convolutional neural network,

    P. Yao, H. Wu, B. Gao, J. Tang, Q. Zhang, W. Zhang, J. J. Yang, and H. Qian, “Fully hardware-implemented memristor convolutional neural network,” Nature, vol. 577, no. 7792, pp. 641–646, 2020

  3. [11]

    Enabling highly-efficient dna sequence mapping via reram-based tcam,

    Y .-S. Lai, S.-H. Chen, and Y .-H. Chang, “Enabling highly-efficient dna sequence mapping via reram-based tcam,” in 2023 IEEE/ACM Inter- national Symposium on Low Power Electronics and Design (ISLPED) , pp. 1–6, IEEE, 2023

  4. [12]

    Winner-take-all autoencoders,

    A. Makhzani and B. J. Frey, “Winner-take-all autoencoders,” Advances in neural information processing systems , vol. 28, 2015

  5. [13]

    Thermal agitation of electricity in conductors,

    J. B. Johnson, “Thermal agitation of electricity in conductors,” Physical review, vol. 32, no. 1, p. 97, 1928

  6. [14]

    Swipe: Enhancing robustness of reram crossbars for in-memory computing,

    S. K. Gonugondla, A. D. Patil, and N. R. Shanbhag, “Swipe: Enhancing robustness of reram crossbars for in-memory computing,” inProceedings of the 39th International Conference on Computer-Aided Design , pp. 1– 9, 2020

  7. [15]

    Mixed-signal computing for deep neural network in- ference,

    B. Murmann, “Mixed-signal computing for deep neural network in- ference,” IEEE Transactions on Very Large Scale Integration (VLSI) Systems, vol. 29, no. 1, pp. 3–13, 2020

  8. [16]

    Towards adc-less compute-in- memory accelerators for energy efficient deep learning,

    U. Saxena, I. Chakraborty, and K. Roy, “Towards adc-less compute-in- memory accelerators for energy efficient deep learning,” in 2022 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 624– 627, IEEE, 2022

  9. [17]

    Analog-to-digital conversion with reconfig- urable function mapping for neural networks activation function accel- eration,

    M. Giordano, G. Cristiano, K. Ishibashi, S. Ambrogio, H. Tsai, G. W. Burr, and P. Narayanan, “Analog-to-digital conversion with reconfig- urable function mapping for neural networks activation function accel- eration,” IEEE Journal on Emerging and Selected Topics in Circuits a...

  10. [18]

    X. Sun, X. Peng, P.-Y . Chen, R. Liu, J.-s. Seo, and S. Yu, “Fully parallel rram synaptic array for implementing binary neural network with (+ 1,-

  11. [19]

    574–579, IEEE, 2018

    weights and (+ 1, 0) neurons,” in 2018 23rd Asia and South Pacific Design Automation Conference (ASP-DAC) , pp. 574–579, IEEE, 2018

  12. [20]

    Improving robustness of reram-based spiking neural network accelerator with stochastic spike- timing-dependent-plasticity,

    X. She, Y . Long, and S. Mukhopadhyay, “Improving robustness of reram-based spiking neural network accelerator with stochastic spike- timing-dependent-plasticity,” in 2019 International Joint Conference on Neural Networks (IJCNN) , pp. 1–8, IEEE, 2019

  13. [21]

    Regularized binary network training,

    S. Darabi, “Regularized binary network training,” arXiv:1812.11800, 2018

  14. [22]

    Rram-based spiking nonvolatile computing-in-memory processing engine with precision- configurable in situ nonlinear activation,

    B. Yan, Q. Yang, W.-H. Chen, K.-T. Chang, J.-W. Su, C.-H. Hsu, S.- H. Li, H.-Y . Lee, S.-S. Sheu, M.-S. Ho, et al. , “Rram-based spiking nonvolatile computing-in-memory processing engine with precision- configurable in situ nonlinear activation,” in 2019 Symposium on VLSI Tech...

  15. [23]

    Stochastic phase-change neurons,

    T. Tuma, A. Pantazi, M. Le Gallo, A. Sebastian, and E. Eleftheriou, “Stochastic phase-change neurons,” Nature nanotechnology , vol. 11, no. 8, pp. 693–699, 2016

  16. [24]

    Simple statistical gradient-following algorithms for con- nectionist reinforcement learning,

    R. J. Williams, “Simple statistical gradient-following algorithms for con- nectionist reinforcement learning,” Machine learning , vol. 8, pp. 229– 256, 1992

  17. [25]

    Versatile stochastic dot product circuits based on nonvolatile memories for high perfor- mance neurocomputing and neurooptimization,

    M. Mahmoodi, M. Prezioso, and D. Strukov, “Versatile stochastic dot product circuits based on nonvolatile memories for high perfor- mance neurocomputing and neurooptimization,” Nature communica- tions, vol. 10, no. 1, p. 5113, 2019

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.