REVIEW 4 major objections 5 minor 23 references
NIFA: Nonlinear IMC enhanced FPGA for efficient ML inference
T0 review · 4 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash
Pith's one-line read Replacing ADCs with analog content-addressable memories in FPGA in-memory blocks, plus a log-domain mapping for attention, yields up to 40x energy efficiency for CNNs and 1.9x for Transformers, with gains persisting as sequence length grows
desk verdict Clever FPGA/IMC architecture paper with a genuinely new log-domain attention mapping, but the 30–40x efficiency claims are inherited from unvalidated shared prior work and the Transformer accuracy is asserted, not shown. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The NL-DPE block: a ReRAM crossbar for vector-matrix multiplication, with each column ending in an analog content-addressable memory (ACAM) instead of an ADC. The ACAM is a small ReRAM array programmed with thresholds that encode a piecewise decision tree; it converts an analog input to a digital output while applying a nonlinear function such as ReLU or tanh, and can also be programmed as a pure analog-to-digital converter. The paper's attention mapping uses ACAMs configured for log and exp operations to perform dynamic matrix multiplications and softmax in the log domain, replacing multiplications with additions. A two-round design-space exploration sets the crossbar size and IMC area budg
What would settle it
Measure the NL-DPE block on real silicon at 22nm: program it for exp/log transforms, compute the per-transform MSE at INT8, and compare the energy per conversion and block area to an ADC-based IMC block. If the MSE is orders of magnitude above 1e-5 after chaining, or if the ACAM conversion energy is not ~27x lower than the ADC, the reported CNN and Transformer efficiency gains would not survive.
Extended reading notes
Core claim
The central claim is that an FPGA with embedded ADC-free, ACAM-based nonlinear in-memory computing blocks, combined with a log-domain mapping of attention's dynamic matrix multiplications and softmax, delivers up to 40x higher energy efficiency and 4.1x higher area efficiency on CNN benchmarks and 1.9x/2.5x on Transformer benchmarks compared to a prior ADC-based IMC-enhanced FPGA. The gain comes from two block-level effects: ACAMs replace ADCs and cut conversion energy by about 27x, and folding the nonlinear activation into the ACAM removes the separate soft-logic activation pass. On Transformers, the log-domain mapping lets the IMC blocks handle the dynamic matrix multiplications that previ
Load-bearing premise
The load-bearing premise is that the ACAM-based nonlinear dot product block is as accurate, energy-efficient, and area-efficient as modeled in prior work (per-transform MSE ~1e-8, INT8 accuracy within a few percent of FP32, ~27x lower conversion energy than an ADC); the paper does not measure or independently reproduce these block-level numbers.
Editorial extensions
If this is right
- FPGA-based Transformer inference becomes 1.7–1.9x more energy-efficient and up to 2.5x more area-efficient than the ADC-based IMC baseline, and the gain persists as sequence length grows.
- CNN inference gains 30–40x energy efficiency and 3–4x area efficiency, largely from removing ADC conversion and CLB activation passes.
- The log-domain mapping is a general recipe for accelerating dynamic matrix multiplications and softmax with analog IMC, not just for this architecture.
- The design-space exploration shows that roughly 20–25% of FPGA area devoted to IMC blocks is the sweet spot, balancing flexibility and throughput.
Reading between the lines
- The paper's quantitative results depend on block-level energy, area, and accuracy numbers for the ACAM taken from prior work and not measured here; a fair reading is that the architecture and mapping are the contribution, while the magnitude of the gains is inherited.
- A testable extension: fabricate or simulate the NL-DPE block at 22nm and measure the per-transform MSE and energy per operation directly; if the MSE is larger than ~1e-5 or conversion energy is within a small factor of an ADC, the log-domain chaining and the headline gains would need revision.
- The log-domain attention mapping could also be applied to other analog IMC platforms (e.g., standalone ReRAM accelerators) whenever the nonlinear transform is cheap and accurate, suggesting a broader design principle for attention acceleration in memory.
- If the FlexScore-based Pareto analysis is sound, the same methodology could guide integration of other hard blocks (tensor slices, BRAM compute) into FPGAs.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes NIFA, an FPGA architecture that integrates an ADC-free, ACAM-based nonlinear in-memory computing block (NL-DPE) as a first-class hard block. It performs a two-round FPGA-aware design-space exploration to select crossbar dimensions and IMC area budget, and maps Transformer Attention's dynamic matrix-matrix multiplications and softmax into the log domain using ACAM nonlinearities. The authors report up to 40x and 1.9x higher energy efficiency and 4.1x and 2.5x higher area efficiency versus the Azure-Lily analog-IMC FPGA baseline, based on an analytical simulator that combines VTR synthesis data with block-level energy/area parameters imported from co-authored prior work.
Significance. If the underlying NL-DPE block model is accurate, the architecture is a credible direction for making Transformer inference substantially more energy-efficient on FPGAs. The paper's strengths are its systematic DSE methodology, the use of standard VTR flows, the inclusion of non-DL flexibility evaluation, and a concrete mapping that extends IMC to Attention's dynamic matrix multiplications. However, the novelty is architectural rather than demonstrated at the device or system level: the quantitative claims are linear in block-level parameters borrowed from co-authored prior work, and the accuracy of the log-domain Attention mapping is asserted without an experiment. The contribution is publishable in principle, but the current evidence is not strong enough to support the headline numbers as stated.
major comments (4)
- [Sec. V-A, Table III, Fig. 9] The central energy and area claims are computed from block-level specifications imported from co-authored prior work: NL-DPE energy/area from [13] scaled from 32nm to 22nm, and the Azure-Lily baseline from [8]. The paper provides no silicon measurement, no independent reproduction, and no sensitivity analysis of these inputs. Because the reported inference/J and inference/s/mm^2 are essentially linear in these imported constants, a factor-of-2 error in ACAM energy or area would substantially shrink the headline 30-40x energy-efficiency gain. In addition, the ReRAM programming power-delivery network is explicitly out of scope (Sec. IV-C), leaving another unmodeled cost. The authors should at least provide a sensitivity sweep over the imported block parameters and clearly label the results as model-dependent.
- [Sec. IV-E, Fig. 5] The accuracy of the log-domain Attention mapping is load-bearing but never evaluated. The only evidence offered is a sentence citing [13] for 'no measurable accuracy loss on BERT-Tiny'; the paper contains no accuracy table, no comparison of log-domain versus full-precision Attention outputs, and no investigation of error accumulation across the chained QK^T -> softmax -> score*V pipeline. Since [13] apparently validates the NL-DPE block at the single-transform level, it cannot by itself validate the new compositional mapping proposed here. Even a simulation using the adopted noise model would substantially strengthen the claim.
- [Sec. IV-E] The claimed fusion of inverse exp/log stages is not formalized. The text states 'We therefore fuse these inverse stages so that they cancel rather than compound,' but no equation or algorithm shows how the cancellation is achieved; Fig. 5 appears to contain explicit exp and log operations in the softmax path. Without a formal description of the fused stages and an error-propagation analysis, the statement that the log-domain mapping 'closely track[s] their full-precision counterparts' is unsupported. This is especially important because the paper's own narrative acknowledges that chained transforms raise MSE from 1e-8 to 1e-5.
- [Sec. VI-B, Table III] The block-level comparison to Azure-Lily is not apples-to-apples. The proposed blocks use 4 ReRAM cells per weight versus 1 for Azure-Lily, larger crossbars (1024 rows versus 512), and a 40-bit data interface versus 16-bit. While Proposed-2 has comparable block area, the paper does not report a controlled comparison at equal crossbar size, cell encoding, and interface width. The claim that ACAM is more area- and energy-efficient than ADC would be more convincing if the 27x conversion-energy difference were shown to be due to ACAM itself rather than to the different crossbar organization and interface width.
minor comments (5)
- [Abstract / Sec. I] The headline numbers are inconsistent: the abstract says 1.9x Transformer energy efficiency, while the introduction bullet says 1.7x throughput efficiency, and the conclusion mentions 'up to 40x' energy and '1.7x' throughput. Please unify the metrics and ensure each reported ratio is clearly labeled (energy vs throughput vs area).
- [Fig. 5] The figure uses symbols such as Iexp, Ilog, and E& without definitions in the text. A step-by-step algorithm for the log-domain Attention mapping, including the exact role of the ACAM at each stage, would make the proposal reproducible.
- [Table III] The row 'ACAM Size / ADC Count' mixes units: for the proposed designs it reports an array dimension (130x128) while for Azure-Lily it reports a count (8 ADCs). Please use a consistent format.
- [Sec. IV-B] The sentence 'At INT8 precision, it restores accuracy to within a few percent of the FP32 baseline ... with essentially no loss on BERT-Tiny' is a claim from [13], not demonstrated here. It should be phrased as an assumption imported from prior work.
- [Sec. VI-D] The text says the DIMM share 'approaches 100% at sequence lengths >=4096,' but the largest evaluated sequence length is 2048. Either add an N=4096 point or explicitly mark this as a projection.
Circularity Check
Transformer accuracy and headline energy gains are inherited from co-authored prior work [13]; the log-domain mapping's accuracy is never independently shown.
-
self citation load bearing
[Section IV-E 'Attention-Head Mapping']
"As modeled in [13], a single transform is essentially exact at INT8, with a per-transform MSE on the order of 10^-8... We therefore fuse these inverse stages so that they cancel rather than compound. At INT8, the log-domain DIMM and Softmax closely track their full-precision counterparts, with no measurable accuracy loss on BERT-Tiny[13]."
The novel log-domain Attention mapping is the paper's distinctive Transformer contribution, and its INT8 accuracy is the premise that keeps the mapping from needing precision fallback. The only support offered is a citation to [13], co-authored by four of the present authors; the paper contains no BERT-Tiny accuracy table or experiment. Moreover [13] is invoked for the 'single transform' MSE, while the chained/fused DIMM+softmax composition is new; attributing the composition's accuracy to [13] assumes the very result that the mapping must establish.
-
other
[Section V-B 'FPGA Architecture' and Section VI-C 'CNN Evaluation']
"To model the IMC area in VTR, we estimate the ReRAM crossbar area and ACAM area from a 32 nm NL-DPE design evaluated in [13]... the CNN energy advantage stems from two block-level effects: replacing the ADC with the ACAM cuts the conversion energy by ∼27× (Fig. 9), and folding the activation into the ACAM removes the separate CLB activation pass... Together these account for the >30× end-to-end CNN energy efficiency over Azure-Lily."
The headline 30-40x CNN energy-efficiency gain is not observed or independently derived; by the paper's own decomposition it is the aggregate of block-level ACAM-vs-ADC conversion energy and activation-folding constants taken from the co-authored [13] design. The simulator's end-to-end numbers are therefore a weighted restatement of those input constants (plus the [21] FPGA energy model), making the 'prediction' inherit its magnitude from the self-cited block model. Because [13] is an unreproduced preprint, the key quantitative claim reduces to an unverified input rather than a testable result of this paper.
full rationale
The architectural ideas — embedding an ADC-free NL-DPE hard block, the two-round DSE, and the log-domain Attention mapping — are legitimate and not circular in themselves. However, the paper's quantitative claims are not self-contained. The energy/area parameters of the NL-DPE block and the NAF accuracy recovery are taken from [13] (and [17]), both with substantial author overlap, and the only evidence for the novel fused log-domain DIMM/softmax accuracy is the sentence 'no measurable accuracy loss on BERT-Tiny[13]'; no BERT-Tiny accuracy table is reported. Section VI-C explicitly attributes the >30x CNN energy advantage to block-level ACAM-vs-ADC conversion energy and activation folding, which are inputs from [13]. Thus the end-to-end 'predictions' reduce, through the paper's own simulation and decomposition, to constants supplied by co-authored prior work rather than to measurements or independent derivations in this paper. This is not a tautology — the DSE, mapping, and system integration add real content — but the central efficiency and accuracy results are load-bearing on self-citations that are not independently verified. Score 6 reflects partial circularity, not full equivalence by definition.
Assumptions & free parameters
free parameters (5)
- Crossbar dimensions (R,C) =
1024x128 (Proposed-1), 1024x256 (Proposed-2)
- IMC area budget =
23% (Proposed-1), 21% (Proposed-2)
- ReRAM cells per weight =
4
- ACAM size =
130x128 / 130x256
- Input/output connection flexibility =
FC_in=15%, FC_out=10%
assumptions (6)
- domain assumption The NL-DPE block from [13] has the claimed ACAM nonlinearity, per-transform MSE of ~1e-8, and INT8 accuracy after NAF.
- domain assumption The ADC-vs-ACAM area/energy comparison from [13] transfers unchanged to a 22 nm FPGA tile with 4 ReRAM cells per weight.
- domain assumption The Azure-Lily noise model and NAF method [8,13,17] restore accuracy under realistic device variation at scale.
- domain assumption The analytical energy model of [21] accurately estimates FPGA CLB/DSP/routing energy for these designs.
- domain assumption The log-domain DIMM mapping with fused exp/log stages preserves INT8 accuracy on the full Attention pipeline.
- domain assumption The VTR model of the IMC tile, with 15%/10% connection flexibility and registered I/O, accurately represents integrating BEOL ReRAM into a leading-edge FPGA fabric.
Cite this review
Pith. "Pith review of NIFA: Nonlinear IMC enhanced FPGA for efficient ML inference." pith.science (2026). https://pith.science/paper/AYGQDRQP
@misc{pith2026260715123,
author = {Pith},
title = {Pith review of: NIFA: Nonlinear IMC enhanced FPGA for efficient ML inference},
year = {2026},
howpublished = {\url{https://pith.science/paper/AYGQDRQP}},
note = {Machine review of arXiv:2607.15123}
}
read the original abstract
Recent FPGAs have improved deep learning (DL) inference efficiency through dedicated tensor blocks and in-BRAM computation. ReRAM-based analog in-memory computing (IMC) pushes efficiency further, offering an order-of-magnitude improvement in compute density and energy efficiency over conventional digital logic by performing vector-matrix multiplication (VMM) directly within the ReRAM crossbar; prior work has integrated such IMC blocks into FPGAs for DL inference. However, conventional IMC designs support only static-weight VMM, leaving nonlinear operations and dynamic matrix-matrix multiplication (DIMM) to the FPGA fabric. As a result, the benefits of IMC are largely confined to static-weight models, whereas Transformer-based models, which rely on frequent nonlinear and DIMM operations, gain only limited improvement. Moreover, the ADCs within each IMC block consume more than 70% of its area and power, further limiting system efficiency and scalability. To address these limitations, we propose a novel FPGA architecture that integrates an ADC-free IMC block, replacing the conventional ADC with analog content-addressable memories (ACAMs) that natively perform nonlinear operations inside the block. To fully exploit this block, we conduct an FPGA-aware design-space exploration that determines optimal crossbar dimensions while balancing FPGA area, flexibility, and DL performance, and we develop an efficient mapping that leverages ACAMs to carry out DIMM operations, extending the applicability of IMC to attention computation. On CNN and Transformer-based benchmarks, the proposed architecture achieves up to 40x and 1.9x higher energy efficiency and 4.1x and 2.5x higher area efficiency, respectively. Overall, it significantly improves FPGA DL inference efficiency and sustains robust gains on Transformer-based workloads across long input sequences, advancing domain-specialized FPGA design.
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[13]
NL-DPE: An analog in-memory non- linear dot product engine for efficient CNN and LLM inference,
L. Zhao, G. Pedretti, Z. Mao, R. R. Sunketa, A. Arora, L. Buonanno, J. Ignowski, and P. Faraboschi, “NL-DPE: An analog in-memory non- linear dot product engine for efficient CNN and LLM inference,”arXiv preprint arXiv:2511.13950, 2025
arXiv 2025
-
[8]
Azure-lily: An FPGA architecture with analog IMC engines for efficient AI,
A. Gajjar, R. Sunketa, L. Zhao, O. Eldash, A. Natarajan, G. Pedretti, A. Arora, P. Faraboschi, J. Ignowski, and L. Buonanno, “Azure-lily: An FPGA architecture with analog IMC engines for efficient AI,”ACM Trans. Archit. Code Optim., Feb. 2026
2026
-
[1]
Tensor slices to the rescue: Supercharging ML acceleration on FPGAs,
A. Arora, S. Mehta, V . Betz, and L. K. John, “Tensor slices to the rescue: Supercharging ML acceleration on FPGAs,” inProc. ACM/SIGDA Intl. Symp. on Field-Programmable Gate Arrays (FPGA), 2021
2021
-
[2]
Systolic sparse tensor slices: FPGA building blocks for sparse and dense AI acceleration,
E. Taka, N.-C. Huang, C.-C. Chang, K.-C. Wu, A. Arora, and D. Mar- culescu, “Systolic sparse tensor slices: FPGA building blocks for sparse and dense AI acceleration,” inProceedings of the 2025 ACM/SIGDA International Symposium on Field Programmable Gate Arrays, FPGA ’25, (New York, NY , USA), p. 159–171, Association for Computing Machinery, 2025
2025
-
[3]
Stratix 10 NX architecture,
M. Langhammer, E. Nurvitadhi, S. Gribok, and B. Pasca, “Stratix 10 NX architecture,”ACM Trans. Reconfigurable Technol. Syst., vol. 15, Aug. 2022
2022
-
[4]
CoMeFa: Compute-in-memory blocks for FPGAs,
A. Arora, T. Anand, A. Borda, R. Sehgal, B. Hanindhito, J. Kulkarni, and L. K. John, “CoMeFa: Compute-in-memory blocks for FPGAs,” inProc. IEEE Intl. Symp. on Field-Programmable Custom Computing Machines (FCCM), 2022
2022
-
[5]
BRAMAC: Compute-in-BRAM archi- tectures for multiply-accumulate on FPGAs,
Y . Chen and M. S. Abdelfattah, “BRAMAC: Compute-in-BRAM archi- tectures for multiply-accumulate on FPGAs,” inProc. IEEE Intl. Symp. on Field-Programmable Custom Computing Machines (FCCM), 2023
2023
-
[6]
M4BRAM: Mixed-precision matrix-matrix multiplication in FPGA block RAMs,
Y . Chen, J. Dotzel, and M. S. Abdelfattah, “M4BRAM: Mixed-precision matrix-matrix multiplication in FPGA block RAMs,” in2023 Interna- tional Conference on Field Programmable Technology (ICFPT), pp. 69– 78, 2023
2023
Show all 23 references
-
[7]
Compute-capable block RAMs for efficient deep learning acceleration on FPGAs,
X. Wang, V . Goyal, J. Yu, V . Bertacco, A. Boutros, E. Nurvitadhi, C. Augustine, R. Iyer, and R. Das, “Compute-capable block RAMs for efficient deep learning acceleration on FPGAs,” in2021 IEEE 29th Annual International Symposium on Field-Programmable Custom Computing Machine...
2021
-
[9]
Analog in- memory computing enhanced FPGA for high-throughput and energy- efficient acceleration,
A. Gajjar, L. Zhao, O. Eldash, A. Natarajan, X. Sheng, G. Pedretti, A. Arora, P. Faraboschi, J. Ignowski, and L. Buonanno, “Analog in- memory computing enhanced FPGA for high-throughput and energy- efficient acceleration,” in2025 IEEE 33rd Annual International Sym- posium on F...
2025
-
[10]
Analog computing: from fundamentals to applications,
L. Buonanno and M. Carminati, “Analog computing: from fundamentals to applications,” in2025 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1–6, 2025
2025
-
[11]
PRIME: A novel processing-in-memory architecture for neural network computation in ReRAM-based main memory,
P. Chi, S. Li, C. Xu, T. Zhang, J. Zhao, Y . Liu, Y . Wang, and Y . Xie, “PRIME: A novel processing-in-memory architecture for neural network computation in ReRAM-based main memory,” in2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA), pp. 27–39, 2016
2016
-
[12]
Attention is all you need,
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” inAdvances in Neural Information Processing Systems (NeurIPS), 2017
2017
-
[14]
RACE-IT: A reconfigurable analog computing engine for in-memory transformer acceleration,
L. Zhao, A. Natarajan, L. Buonanno, A. Gajjar, R. Roth, S. Serebryakov, J. Moon, O. Eldash, J. Ignowski, and G. Pedretti, “RACE-IT: A reconfigurable analog computing engine for in-memory transformer acceleration,” in2025 IEEE 43rd International Conference on Computer Design (I...
2025
-
[15]
Hamamu: Specializing FPGAs for ML applications by adding hard matrix multiplier blocks,
A. Arora, Z. Wei, and L. K. John, “Hamamu: Specializing FPGAs for ML applications by adding hard matrix multiplier blocks,” in2020 IEEE 31st International Conference on Application-specific Systems, Architectures and Processors (ASAP), pp. 53–60, 2020
2020
-
[16]
Experimentally-validated crossbar model for defect-aware training of neural networks,
R. Mao, B. Wen, M. Jiang, J. Chen, and C. Li, “Experimentally-validated crossbar model for defect-aware training of neural networks,”IEEE Transactions on Circuits and Systems II: Express Briefs, vol. 69, no. 5, pp. 2468–2472, 2022
2022
-
[17]
Noise aware finetuning for analog non-linear dot product engine,
L. Zhao, L. Buonanno, A. Natarajan, J. Ignowski, and G. Pedretti, “Noise aware finetuning for analog non-linear dot product engine,” in Proceedings of the NeurIPS 2024 Workshop on Machine Learning with New Compute Paradigms, 2024
2024
-
[18]
FlexScore: Quantifying flexibility,
T. Tan, E. Nurvitadhi, A. Dasu, M. Langhammer, and D. Chiou, “FlexScore: Quantifying flexibility,”IEEE Computer Architecture Let- ters, vol. 20, no. 1, pp. 58–61, 2021
2021
-
[19]
A configurable cloud-scale DNN processor for real-time AI,
J. Fowers, K. Ovtcharov, M. Papamichael, T. Massengill, M. Liu, D. Lo, S. Alkalay, M. Haselman, L. Adams, M. Ghandi, S. Heil, P. Patel, A. Sapek, G. Weisz, L. Woods, S. Lanka, S. K. Reinhardt, A. M. Caulfield, E. S. Chung, and D. Burger, “A configurable cloud-scale DNN process...
2018
-
[20]
VTR 9: Open-source CAD for fabric and beyond FPGA architecture exploration,
M. A. Elgammal, A. Mohaghegh, S. G. Shahrouz, F. Mahmoudi, F. Kos ¸ar, K. Talaei, J. Fife, D. Khadivi, K. Murray, A. Boutros, K. B. Kent, J. Goeders, and V . Betz, “VTR 9: Open-source CAD for fabric and beyond FPGA architecture exploration,”ACM Trans. Reconfigurable Technol. S...
2025
-
[21]
CoMeFa: Deploy- ing Compute-in-Memory on FPGAs for Deep Learning Acceleration,
A. Arora, A. Bhamburkar, A. Borda, T. Anand, R. Sehgal, B. Hanindhito, P.-E. Gaillardon, J. Kulkarni, and L. K. John, “CoMeFa: Deploy- ing Compute-in-Memory on FPGAs for Deep Learning Acceleration,” ACM Transactions on Reconfigurable Technology and Systems, vol. 16, pp. 50:1–5...
2023
-
[22]
Koios: A deep learning benchmark suite for FPGA architecture and CAD re- search,
A. Arora, A. Boutros, D. Rauch, A. Rajen, A. Borda, S. A. Damghani, S. Mehta, S. Kate, P. Patel, K. B. Kent, V . Betz, and L. K. John, “Koios: A deep learning benchmark suite for FPGA architecture and CAD re- search,” in2021 31st International Conference on Field-Programmable ...
2021
-
[23]
Scaling equations for the accurate prediction of CMOS device performance from 180nm to 7nm,
A. Stillmaker and B. Baas, “Scaling equations for the accurate prediction of CMOS device performance from 180nm to 7nm,”Integration, vol. 58, pp. 74–81, 2017
2017
Reviewed August 2, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.