REVIEW 5 major objections 5 minor 25 references
A Fully Hardware Implemented Accelerator Design in ReRAM Analog Computing without ADCs
T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read RACA strips ADCs and per-layer DACs out of ReRAM inference by turning device thermal noise into the activation function, reaching 96.7% MNIST accuracy.
desk verdict Clever ADC-less ReRAM accelerator idea, but the WTA-to-softmax claim is asserted, not derived—the headline accuracy is not credible. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is the binary stochastic neuron built from a trans-impedance amplifier, a reference-subtraction circuit, and a voltage comparator. ReRAM column current minus reference current is compared against a threshold while the device's Nyquist noise $i_{\text{RMS}}=\sqrt{4kT G \Delta f}$ injects randomness; tuning the read voltage, the weight-to-conductance scale, the readout bandwidth, and the number of devices per column sets the effective signal-to-noise ratio. The output layer replaces the comparator threshold with an adaptive winner-take-all threshold, and a counter accumulates the winning neuron across trials. This one circuit does the work that conventional accelerators assign to DACs, ADCs, and digital activation units.
What would settle it
Repeat a fixed input many times through the winner-take-all classifier, record which neuron wins each trial, and compare the empirical win fractions with the softmax probabilities of the same logits; a statistically significant mismatch would show that the hardware classification layer is not implementing the softmax distribution the network was trained with.
Extended reading notes
Core claim
The paper's central discovery is that the nonlinear activations of a neural network can be implemented physically rather than computed. When the read voltage is reduced so that the signal current leaving a ReRAM column is comparable to the device's Nyquist thermal noise $i_{\text{RMS}} = \sqrt{4kT G \Delta f}$, a comparator's decision is stochastic, and the activation probability takes the sigmoid form $1/(1+e^{-\sum_i W_{ij}x_i})$ (Eq. 13). In the output layer, a winner-take-all circuit with an adaptive threshold lets only one neuron fire per trial, and accumulating those trials across repeated experiments yields a cumulative probability distribution that the paper treats as the hardware's softmax approximation, $e^{\sum_i w_{ij} x_i}/\sum_k e^{\sum_i w_{ik} x_i}$ (Eq. 14). This lets the hidden and output layers run without DACs or ADCs and without any separate activation-function block; a simple counter tallies the classifier trials. The design is demonstrated on a [784,500,300,10] fully connected MNIST network, where it reaches 96.7% accuracy with repeated stochastic inference and majority voting.
Load-bearing premise
The load-bearing premise is that the winner-take-all race among binary stochastic neurons produces exactly the softmax distribution the network was trained with; if the race's cumulative probabilities differ from softmax, the final classification layer no longer implements the intended classifier.
Editorial extensions
If this is right
- With comparator-based stochastic neurons, hidden and output layers no longer need ADCs or per-layer DACs, directly removing the largest reported contributors to energy and area in ReRAM accelerators.
- According to the paper's device-to-algorithm simulations, the RACA design lowers energy by 58.29%, area by 38.43%, and raises energy efficiency from 61.3 to 148.58 TOPS/W on the benchmark network.
- Because accuracy improves with repeated stochastic inference and majority voting, the architecture trades a small amount of latency for accuracy and has a tunable operating point through the read voltage and comparator threshold.
- The mixed-signal design remains compatible with digital peripheral devices because the stochastic output is already binary, so no precision ADC is needed between layers.
Reading between the lines
- A direct test of the softmax approximation would measure the empirical winner-take-all probabilities over many trials and compare them with $e^{\sum w x}/\sum e^{\sum w x}$; if the race distribution differs from softmax, retraining the final layer on the hardware's actual distribution could restore accuracy on datasets beyond MNIST.
- The same stochastic-binarization mechanism may extend to convolutional layers and deeper networks, since the comparator circuit is not specific to fully connected layers, but the paper only demonstrates a 784-500-300-10 multilayer perceptron.
- Because thermal noise scales with temperature, inference accuracy could drift with operating temperature; a temperature-compensated read voltage or a calibration pass would be needed before deployment in uncontrolled environments.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes RACA, a ReRAM crossbar-based accelerator that replaces per-layer DACs, ADCs, and explicit activation-function units with stochastic binarization circuits driven by the intrinsic thermal noise of the ReRAM devices. It claims that comparator-thresholded currents yield sigmoid activation probabilities (Eq. 13) and that a winner-take-all (WTA) race among output neurons implements softmax (Eq. 14). Using a [784,500,300,10] fully connected network on MNIST, it reports 96.7% accuracy with repeated stochastic inference and majority voting, together with NeuroSim-based estimates of 58% energy and 38% area reduction relative to a 1-bit ADC baseline.
Significance. The idea of reusing device noise as a free stochastic source is attractive, and the reported energy/area gains would be important if the accuracy claim were supported. The paper presents a concrete architecture, qualitative circuit simulations, and a first-order NeuroSim comparison, and it explicitly acknowledges at least one missing component (the cumulative-probability counter). However, the central mathematical link between the stochastic comparator mechanism and softmax is not established, and the evaluation lacks a same-network baseline. The contribution is therefore not yet supported at the level required for publication.
major comments (5)
- [Section III-B, Eq. (14)] The WTA-to-softmax equivalence is asserted without derivation. If each neuron fires with the sigmoid probability of Eq. (13), then a WTA race among independent Bernoulli neurons selects neuron j with probability sigma(Z_j) / sum_k sigma(Z_k), not softmax. For two neurons with logits 10 and 0, this gives a winner probability of about 0.667 for the first neuron, whereas softmax gives about 0.99995. The paper provides no argument that the adaptive-threshold first-crossing mechanism produces Gumbel-distributed race times or any other distribution that would yield softmax. Because the network was trained with standard softmax cross-entropy, the deployed classifier may not implement the intended decision rule. The authors should either derive the race distribution from the circuit dynamics (including the noise statistics in Eq. (11)) or characterize it empirically and show that it matches softmax over the operating range; otherwise the 96.7% accuracy claim lacks a valid mechanism.
- [Section III-A, Eq. (13)] The approximation of the Gaussian CDF by a logistic sigmoid requires the noise standard deviation to be proportional to the scale parameter Vr*G0 in the sigmoid exponent. However, the variance in Eq. (11) is 4kT*Delta_f * sum_i(G_ij + G_ref), which depends on the conductance values and hence on the weights. No argument or bound is given for why this weight-dependent variance can be absorbed into the fixed scale Vr*G0. The paper should quantify the approximation error over the actual range of programmed conductances used in the MNIST experiments and state what assumptions about weight/conductance statistics are needed.
- [Section III-C] The text states that 'the hardware for computing the cumulative probability distribution of the classifier is not provided here, but it can be easily implemented at the output end with a simple counter.' This admitted omission directly contradicts the title and abstract's claim of a 'fully hardware implemented' accelerator. The counter is part of the classification readout and must be specified (including its area, energy, and timing overhead) before the architecture can be considered complete. The claim should be revised or the counter hardware should be included in the design and in Table I.
- [Section IV-C and Table I] No baseline accuracy for the same [784,500,300,10] FCNN with standard activations is reported, so the claim that RACA achieves 'without compromising inference accuracy' is unsubstantiated. The reader cannot tell whether 96.7% (with repeated trials and majority voting) matches the software-trained softmax accuracy for this architecture. In addition, Table I is based on a 'modified NeuroSim' but the modifications are not described; energy and area numbers for the missing counter hardware, the input DAC, and the WTA threshold circuitry are not itemized.
- [Abstract and Section III-C] The abstract and introduction claim that the design removes both DACs and ADCs, but Section III-C explicitly states that 'a DAC is used at the input stage to preserve the integrity of input data features.' The title says 'without ADCs,' which is consistent with keeping an input DAC, but the broader claim in the abstract overstates the contribution. The authors should state precisely which DACs/ADCs are eliminated and which remain.
minor comments (5)
- [Section III-A, Eq. (8)] The threshold variable p in Eq. (8) is not clearly connected to the comparator threshold Vth1 used later; aligning these notations would improve readability.
- [Section IV-C] In the sentence 'with set to 0.05V', the parameter name (presumably Vth0) is missing; please correct the missing symbol.
- [Section II-B, Eqs. (4)-(7)] The variable Ncol is mentioned in Section IV-A as an SNR-tuning knob but is not defined in the weight-mapping equations; please clarify how multiple devices per column enter the current summation.
- [Section IV-A] The text says the statistical probabilities in Fig. 4(c)-(f) are 'solely used for analysis and comparison.' This is ambiguous: it is not clear whether these curves come from SPICE-level circuit simulation, from a behavioral model, or from the analytic Eq. (13). Please specify the simulation methodology.
- [Section IV-C, Table I] The NeuroSim estimation should state the technology node, the ADC resolution and sampling rate assumptions for the baseline, and whether the baseline already includes the activation-function hardware whose area/energy is being removed.
Circularity Check
No input-equivalent circularity; Eq. 14 is an unsupported softmax assertion, not a circular reduction.
full rationale
The paper's derivation chain is self-contained with respect to the quantities it predicts. Eq. 13 derives the neuron activation probability from the physical Nyquist noise model (Eq. 11) and a Gaussian-CDF-to-logistic approximation; the target sigmoid is an external software activation, not a value fitted from the reported accuracy, and Vr, G0, Δf, and Ncol are described as SNR-tuning degrees of freedom rather than as parameters fitted to the final accuracy. The central efficiency claims (removal of DACs and ADCs) are structural circuit claims benchmarked in Table I via NeuroSim, not predictions that reduce to fitted inputs. No load-bearing argument is justified by a citation from the present authors; WTA is cited to Makhzani and Frey [12], an external source. The serious weakness is Eq. 14: the WTA-to-softmax relation is asserted, not derived from Eq. 13, and for two logits (10,0) a Bernoulli sigmoid race gives a winner probability of about 0.667 rather than softmax's 0.99995. That is an unsupported mathematical and empirical claim and a correctness risk, not a circular reduction: the paper never defines the hardware output in terms of the softmax result, and no fitted parameter is renamed as a prediction. The missing cumulative-probability counter hardware and absent baseline accuracy are completeness limitations rather than input-equivalent reasoning.
Assumptions & free parameters
free parameters (5)
- Read voltage Vr
- Readout bandwidth Δf
- Conductance scaling G0
- Columns per neuron Ncol
- Rest threshold voltage Vth0 =
0.05 V
assumptions (3)
- domain assumption Thermal noise current in ReRAM devices is Gaussian with variance 4kT G Δf and independent across devices
- standard math The Gaussian cumulative distribution function can be approximated by a logistic sigmoid
- ad hoc to paper Winner-take-all competition among stochastic neurons yields a softmax distribution
Cite this review
Pith. "Pith review of A Fully Hardware Implemented Accelerator Design in ReRAM Analog Computing without ADCs." pith.science (2026). https://pith.science/paper/QLBSLQ5M
@misc{pith2026241219869,
author = {Pith},
title = {Pith review of: A Fully Hardware Implemented Accelerator Design in ReRAM Analog Computing without ADCs},
year = {2026},
howpublished = {\url{https://pith.science/paper/QLBSLQ5M}},
note = {Machine review of arXiv:2412.19869}
}
read the original abstract
Emerging ReRAM-based accelerators process neural networks via analog Computing-in-Memory (CiM) for ultra-high energy efficiency. However, significant overhead in peripheral circuits and complex nonlinear activation modes constrain system energy efficiency improvements. This work explores the hardware implementation of the Sigmoid and SoftMax activation functions of neural networks with stochastically binarized neurons by utilizing sampled noise signals from ReRAM devices to achieve a stochastic effect. We propose a complete ReRAM-based Analog Computing Accelerator (RACA) that accelerates neural network computation by leveraging stochastically binarized neurons in combination with ReRAM crossbars. The novel circuit design removes significant sources of energy/area efficiency degradation, i.e., the Digital-to-Analog and Analog-to-Digital Converters (DACs and ADCs) as well as the components to explicitly calculate the activation functions. Experimental results show that our proposed design outperforms traditional architectures across all overall performance metrics without compromising inference accuracy.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
Equivalent-accuracy accelerated neural-network training using analogue memory,
S. Ambrogio, P. Narayanan, H. Tsai, R. M. Shelby, I. Boybat, C. Di Nolfo, S. Sidler, M. Giordano, M. Bodini, N. C. Farinha, et al. , “Equivalent-accuracy accelerated neural-network training using analogue memory,” Nature, vol. 558, no. 7708, pp. 60–67, 2018
work page 2018
-
[2]
A compute-in-memory chip based on resistive random-access memory,
W. Wan, R. Kubendran, C. Schaefer, S. B. Eryilmaz, W. Zhang, D. Wu, S. Deiss, P. Raina, H. Qian, B. Gao, et al., “A compute-in-memory chip based on resistive random-access memory,” Nature, vol. 608, no. 7923, pp. 504–512, 2022
2022
-
[3]
Hardware implementation of memristor-based artificial neural networks,
F. Aguirre, A. Sebastian, M. Le Gallo, W. Song, T. Wang, J. J. Yang, W. Lu, M.-F. Chang, D. Ielmini, Y . Yang, et al. , “Hardware implementation of memristor-based artificial neural networks,” Nature communications, vol. 15, no. 1, p. 1974, 2024
work page 1974
-
[4]
Towards a formally verified hardware root-of-trust for data-oblivious computing,
L. Deutschmann, J. M ¨uller, M. R. Fadiheh, D. Stoffel, and W. Kunz, “Towards a formally verified hardware root-of-trust for data-oblivious computing,” in Proceedings of the 59th ACM/IEEE Design Automation Conference, pp. 727–732, 2022
work page 2022
-
[5]
Design guidelines of rram based neural-processing-unit: A joint device- circuit-algorithm analysis,
W. Zhang, X. Peng, H. Wu, B. Gao, H. He, Y . Zhang, S. Yu, and H. Qian, “Design guidelines of rram based neural-processing-unit: A joint device- circuit-algorithm analysis,” in Proceedings of the 56th Annual Design Automation Conference 2019 , pp. 1–6, 2019
work page 2019
-
[6]
Fpcim: A fully- parallel robust reram cim processor for edge ai devices,
Y .-C. Guo, W.-T. Lin, T.-H. Hou, and T.-S. Chang, “Fpcim: A fully- parallel robust reram cim processor for edge ai devices,” in 2023 IEEE International Symposium on Circuits and Systems (ISCAS) , pp. 1–5, IEEE, 2023
work page 2023
-
[7]
P. Chi, S. Li, C. Xu, T. Zhang, J. Zhao, Y . Liu, Y . Wang, and Y . Xie, “Prime: A novel processing-in-memory architecture for neural network computation in reram-based main memory,” ACM SIGARCH Computer Architecture News, vol. 44, no. 3, pp. 27–39, 2016
work page 2016
-
[8]
P.-Y . Chen, X. Peng, and S. Yu, “Neurosim+: An integrated device- to-algorithm framework for benchmarking synaptic devices and array architectures,” in 2017 IEEE International Electron Devices Meeting (IEDM), pp. 6–1, IEEE, 2017
work page 2017
Show all 25 references
-
[9]
Compute-in-memory technologies and architectures for deep learning workloads,
M. Ali, S. Roy, U. Saxena, T. Sharma, A. Raghunathan, and K. Roy, “Compute-in-memory technologies and architectures for deep learning workloads,” IEEE Transactions on Very Large Scale Integration (VLSI) Systems, vol. 30, no. 11, pp. 1615–1630, 2022
2022
-
[10]
Fully hardware-implemented memristor convolutional neural network,
P. Yao, H. Wu, B. Gao, J. Tang, Q. Zhang, W. Zhang, J. J. Yang, and H. Qian, “Fully hardware-implemented memristor convolutional neural network,” Nature, vol. 577, no. 7792, pp. 641–646, 2020
2020
-
[11]
Enabling highly-efficient dna sequence mapping via reram-based tcam,
Y .-S. Lai, S.-H. Chen, and Y .-H. Chang, “Enabling highly-efficient dna sequence mapping via reram-based tcam,” in 2023 IEEE/ACM Inter- national Symposium on Low Power Electronics and Design (ISLPED) , pp. 1–6, IEEE, 2023
2023
-
[12]
Winner-take-all autoencoders,
A. Makhzani and B. J. Frey, “Winner-take-all autoencoders,” Advances in neural information processing systems , vol. 28, 2015
2015
-
[13]
Thermal agitation of electricity in conductors,
J. B. Johnson, “Thermal agitation of electricity in conductors,” Physical review, vol. 32, no. 1, p. 97, 1928
1928
-
[14]
Swipe: Enhancing robustness of reram crossbars for in-memory computing,
S. K. Gonugondla, A. D. Patil, and N. R. Shanbhag, “Swipe: Enhancing robustness of reram crossbars for in-memory computing,” inProceedings of the 39th International Conference on Computer-Aided Design , pp. 1– 9, 2020
2020
-
[15]
Mixed-signal computing for deep neural network in- ference,
B. Murmann, “Mixed-signal computing for deep neural network in- ference,” IEEE Transactions on Very Large Scale Integration (VLSI) Systems, vol. 29, no. 1, pp. 3–13, 2020
2020
-
[16]
Towards adc-less compute-in- memory accelerators for energy efficient deep learning,
U. Saxena, I. Chakraborty, and K. Roy, “Towards adc-less compute-in- memory accelerators for energy efficient deep learning,” in 2022 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 624– 627, IEEE, 2022
2022
-
[17]
Analog-to-digital conversion with reconfig- urable function mapping for neural networks activation function accel- eration,
M. Giordano, G. Cristiano, K. Ishibashi, S. Ambrogio, H. Tsai, G. W. Burr, and P. Narayanan, “Analog-to-digital conversion with reconfig- urable function mapping for neural networks activation function accel- eration,” IEEE Journal on Emerging and Selected Topics in Circuits a...
2019
-
[18]
X. Sun, X. Peng, P.-Y . Chen, R. Liu, J.-s. Seo, and S. Yu, “Fully parallel rram synaptic array for implementing binary neural network with (+ 1,-
-
[19]
574–579, IEEE, 2018
weights and (+ 1, 0) neurons,” in 2018 23rd Asia and South Pacific Design Automation Conference (ASP-DAC) , pp. 574–579, IEEE, 2018
2018
-
[20]
Improving robustness of reram-based spiking neural network accelerator with stochastic spike- timing-dependent-plasticity,
X. She, Y . Long, and S. Mukhopadhyay, “Improving robustness of reram-based spiking neural network accelerator with stochastic spike- timing-dependent-plasticity,” in 2019 International Joint Conference on Neural Networks (IJCNN) , pp. 1–8, IEEE, 2019
2019
-
[21]
Regularized binary network training,
S. Darabi, “Regularized binary network training,” arXiv:1812.11800, 2018
2018 arXiv
-
[22]
Rram-based spiking nonvolatile computing-in-memory processing engine with precision- configurable in situ nonlinear activation,
B. Yan, Q. Yang, W.-H. Chen, K.-T. Chang, J.-W. Su, C.-H. Hsu, S.- H. Li, H.-Y . Lee, S.-S. Sheu, M.-S. Ho, et al. , “Rram-based spiking nonvolatile computing-in-memory processing engine with precision- configurable in situ nonlinear activation,” in 2019 Symposium on VLSI Tech...
2019
-
[23]
Stochastic phase-change neurons,
T. Tuma, A. Pantazi, M. Le Gallo, A. Sebastian, and E. Eleftheriou, “Stochastic phase-change neurons,” Nature nanotechnology , vol. 11, no. 8, pp. 693–699, 2016
2016
-
[24]
Simple statistical gradient-following algorithms for con- nectionist reinforcement learning,
R. J. Williams, “Simple statistical gradient-following algorithms for con- nectionist reinforcement learning,” Machine learning , vol. 8, pp. 229– 256, 1992
1992
-
[25]
Versatile stochastic dot product circuits based on nonvolatile memories for high perfor- mance neurocomputing and neurooptimization,
M. Mahmoodi, M. Prezioso, and D. Strukov, “Versatile stochastic dot product circuits based on nonvolatile memories for high perfor- mance neurocomputing and neurooptimization,” Nature communica- tions, vol. 10, no. 1, p. 5113, 2019
2019
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.