Pith. sign in

REVIEW 4 major objections 5 minor 23 references

Topkima-Former: Low-energy, Low-Latency Inference for Transformers using top-k In-memory ADC

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Topkima-Former claims that embedding top-k selection into the analog-to-digital converters inside an SRAM compute-in-memory array removes sorting latency and cuts softmax macro latency about 15x versus conventional and 8x versus digital…

desk verdict A genuinely new IMA circuit trick for top-k softmax; the accuracy-reporting gap is real but fixable, and the core contribution deserves review. read the letter →

arxiv 2411.13050 v1 pith:BBB77I2H submitted 2024-11-20 cs.AR

Shuai Dong , Junyi Yang , Xiaoqi Peng , Hongyang Shang , Ye Ke , Xiaofeng Yang , Hongjie Liu , Arindam Basu This is my paper · ORCID
classification cs.AR
keywords in-memorycomputingtop-ksoftmaxtransformeraccelerationSRAMcompute-in-memoryrampADCquantization-awaretrainingscale-freeattentionwinner-take-all
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Topkima-Former sets out to show that the softmax bottleneck in transformer attention, which can consume up to 40% of inference latency, can be attacked at the circuit level by selecting only the $k$ largest attention scores inside the analog-to-digital converter of an SRAM compute-in-memory array. The proposed topkima macro uses a decreasing ramp so that larger MAC voltages trip the comparator first, and a counter stops conversion once $k$ columns have fired, eliminating the separate sorting step that slows digital top-k designs and shrinking the softmax workload from $d$ terms to $k$. A modified training scheme, top-k forward with complete backward propagation, keeps the accuracy cost of $k=5$ to a reported 0.4-1.2% across ViT, DistilBERT, and BERT-base on CIFAR-10, CIFAR-100, and SQuAD. The paper reports that the resulting softmax macro is about 15x faster and 30x more energy-efficient than a conventional softmax macro, about 8x faster and 3x more energy-efficient than a digital top-k softmax macro, and that the full Topkima-Former system is 1.8x-84x faster and 1.3x-35x more energy-efficient than earlier in-memory accelerators. This matters because softmax overhead grows with sequence length, so the payoff should increase for long-context models.

What carries the argument

The load-bearing mechanism is the topkima macro: a dual-10T SRAM compute-in-memory array for $Q \cdot K^T$ whose column ADC is a decreasing-ramp converter that doubles as a winner-take-all selector. A common ramp starts higher than every MAC voltage and falls one step per cycle; the column with the largest value crosses first, its sense amplifier latches, and the crossing time becomes the digitized value while the order of crossings gives the rank. An arbiter-encoder, using the same request/acknowledge handshake as event-based vision sensors, reports which columns fired, and a counter halts the conversion once $k$ have fired, so no sorting network is needed. Two supporting pieces carry the accuracy and architecture claims: TFCBP training, which uses only top-k activations in the forward pass but full gradients in the backward pass, and a scale-free rewrite that folds the $1/\sqrt{d_k}$ normalization of attention into $W_Q$, removing per-element scaling hardware.

What would settle it

Feed attention-score vectors into a circuit-level model of the decreasing-ramp IMA with 5-bit conversion, deliberately including ties and cases where the $k$th and $(k{+}1)$th largest scores differ by less than one LSB, and count how often the indices reported by the arbiter-encoder differ from the exact top-k; a non-negligible mismatch rate would break the claim that top-k is obtained without sorting latency in hardware.

Watch

Extended reading notes

Core claim

At its core, the paper claims that top-k selection, normally a sorting problem, can be merged into the analog-to-digital conversion step of a compute-in-memory macro at almost no extra cost. The macro computes $Q \cdot K^T$ in the analog domain, then uses a decreasing ramp so the column with the largest MAC voltage crosses the sense-amplifier threshold first; an arbiter-encoder records the fired columns and a counter stops the ramp after $k$ firings. The values and identities of the top-k scores emerge from the conversion itself, and only those $k$ values proceed to a digital softmax core. The paper's experiments at $k=5$ show a 0.4-1.2% accuracy drop for ViT, DistilBERT, and BERT-base on the tested datasets, macro-level speedups of about 15x over a conventional softmax macro and 8x over a digital top-k softmax macro, and energy savings of about 30x and 3x respectively. The paper also reports 1.8x-84x speedup and 1.3x-35x energy-efficiency gains for the full system when the scale-free attention trick is included.

Load-bearing premise

Everything rests on the analog comparator and arbiter-encoder reliably picking out the true top-k MAC results on every input under process, voltage, and temperature variation and ties, while the split crossbars preserve enough global top-k structure for the headlined accuracy to hold.

Editorial extensions

If this is right

  • Softmax cost scales with $k$ instead of the full sequence dimension $d$, so the speedup from this design grows as transformer sequence lengths increase.
  • Digital top-k softmax is shown to be unattractive because sorting dominates its latency; embedding selection in the ADC is what removes that overhead.
  • The attention scaling factor can be absorbed into the trained $W_Q$ weights, eliminating a per-element division from hardware.
  • At $k=5$ with TFCBP training, accuracy loss stays within 0.4-1.2% on the tested ViT, DistilBERT, and BERT-base workloads.
  • System-level comparisons with earlier in-memory accelerators report 1.8x-84x speedup and 1.3x-35x energy-efficiency improvements.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported early-stopping factor $\alpha$ of about 0.31 is an average; an input with many near-top scores would stop the ramp later and shrink the 15x macro speedup, so the distribution of stopping cycles is worth measuring.
  • Because physical crossbars force $K^T$ to be split, global top-k becomes sub-top-k per array, and the paper's own data show an accuracy cost from that split; the real hardware accuracy is therefore worse than the headline 0.4-1.2% algorithm-level number.
  • The same winner-take-all-in-ADC idea could plausibly extend to other cheap-attention schemes such as ReLU attention or sparsemax, or to any layer where only the largest activations matter.
  • TFCBP has only been exercised at $k=5$ on three models; whether smaller $k$ becomes viable with more training is an open question, though the CIFAR-10 result at $k=1$ hints that the ceiling may be higher.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents Topkima-Former, a hardware-software co-design for transformer inference that accelerates the softmax operation by combining top-k activation selection with in-memory ADC (IMA). The circuit-level contribution is a decreasing-ramp IMA with an arbiter-encoder that identifies the k largest dot-product results in Q·K^T without an explicit sorting network. The algorithm-level contribution is a top-k forward, complete-backward training scheme (TFCBP) that keeps the accuracy loss at k=5 between 0.4% and 1.2% on ViT, DistilBERT, and BERT-base. The architecture-level contribution is a scale-free attention that folds the 1/sqrt(dk) scaling into WQ. The paper reports macro-level speedups of 15x and 8x versus a conventional softmax macro and a digital top-k softmax macro, and system-level speedups of 1.8x-84x and energy-efficiency improvements of 1.3x-35x over prior IMC accelerators.

Significance. If the claims hold, the work offers a practical way to remove a known bottleneck in transformer inference. The decreasing-ramp IMA idea is clean, and the combination of top-k selection with the ADC ramp eliminates explicit sorting, which is a genuine contribution. The paper honestly reports the accuracy degradation of sub-top-k fragmentation in Fig. 4(c) but does not numerically report the final hardware accuracy. The use of SPICE simulation for the circuit and NeuroSim for the system is commendable, and the accuracy evaluation is performed on held-out data (Fig. 3). The TFCBP training is simple and appears to be effective in reducing the accuracy drop compared to prior top-k approaches. The scale-free attention is a known but useful trick. The overall co-design story is coherent and of interest to the IMC and transformer-acceleration community.

major comments (4)
  1. [Section IV-B, Fig. 4(c)] The headline accuracy claim ('0.4% to 1.2% reduction' at k=5) is obtained from a software experiment with global top-k (Fig. 3). The proposed hardware, however, selects per-sub-crossbar top-k with no crossbar-global arbitration: the 256x256 configuration uses sub-top-k 3 and 2 for the two halves of K^T. Fig. 4(c) shows that sub-top-k differs from global top-k, and the text only states that the 256x256 accuracy is 'comparable' to global top-k without reporting a numerical value. Since the final hardware accuracy is the conjunction of the software top-k approximation, sub-top-k fragmentation, 4-bit K^T precision, 5-bit quantization, and IMA errors, the reported 85.1% accuracy after IMA error injection does not establish the end-to-end accuracy. Please report the numerical accuracy for the actual 256x256 sub-top-k configuration for all four datasets, or explicitly state the combined accuracy drop.
  2. [Section IV-B and Table I] The macro-level SPICE simulations are performed in 65 nm CMOS, while the system-level NeuroSim evaluation and the comparison in Table I are reported at 32 nm. The paper does not describe how the SPICE-derived parameters (Tarb, Tima, α, and energy values) are scaled from 65 nm to 32 nm, or why this scaling does not change the speedup and EE conclusions. Without a scaling methodology, the consistency between the macro-level and system-level numbers is unclear. Please either present the SPICE macro at the target 32 nm technology or provide a transparent scaling model.
  3. [Section IV-B, Eq. (4)] The early-stopping factor α is given as 0.31 averaged over the dataset, and the latency model uses this average in Tima,arb. The claimed 15x and 8x speedups of topkima-SM depend critically on this value. If the distribution of α has a heavy tail (e.g., inputs with many scores near the top-k threshold), the average may not be representative. Please report the distribution or the worst-case α across the dataset and show the sensitivity of the speedup claim to α.
  4. [Section IV-B] The paper reports that after injecting IMA errors in the SW simulation, the BERT-base SQuAD accuracy drops from 86.7% to 85.1% (a 1.6% absolute drop). This exceeds the 'less than 1.2%' top-k-only drop stated in Section IV-A. The relationship between these numbers and the headline 0.4-1.2% claim should be clarified: how much of the 1.6% drop is due to top-k approximation, quantization, and IMA errors, respectively?
minor comments (5)
  1. [Abstract] There is a typo: 'tokima softmax macro' should be 'topkima softmax macro'.
  2. [Section III-A, Fig. 2] The description of the arbiter-encoder timing is terse; please clarify what happens if two SAs fire in the same cycle and whether the ranking among them is arbitrary.
  3. [Section IV-B, Fig. 4(a)] The latency numbers in Fig. 4(a) appear to be in microseconds, while the text uses nanoseconds for Tima; please ensure the units are consistent and clearly labeled.
  4. [Abstract and Introduction] The abstract says softmax 'comprises up to 40% of the total latency' while the introduction says 'up to 40% inference time'; please unify the phrasing.
  5. [Section II-B] The claim that the sorting operation consumes at least 75% of the latency is not backed by a specific reference or a quantitative example; please provide the basis for this estimate.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the top-k accuracy, macro latency/energy, and scale-free results are measured or derived from independent simulations and benchmarks rather than reduced from their own conclusions.

full rationale

The paper's central accuracy claim (0.4% to 1.2% drop at k=5) is obtained from held-out software evaluations on CIFAR-10, CIFAR-100, and SQuAD; it is an empirical measurement, not a prediction derived from the hardware model. The TFCBP training scheme uses top-k only in the forward pass, but the reported accuracy is the outcome of that experiment, not an input defined to produce the claim. The macro latency and energy benefits are computed from an analytic model whose parameters come from SPICE simulations, NeuroSim, and literature values, with the early-stopping factor alpha measured from the data; this is performance characterization rather than a fitted-input-called-prediction circularity. The scale-free technique is an algebraic identity (Qs = X*WQ/sqrt(dk)) and does not smuggle the conclusion into the premise. The paper does cite prior work for components such as NeuroSim, ReTransformer, and softmax approximations, but these citations are standard external support rather than self-citations that carry the load of the central claim. The most serious concern is the sub-top-k versus global-top-k accuracy gap: the hardware selects per-sub-crossbar top-k, and the paper's own Fig. 4(c) shows accuracy degradation, while no numerical result is given for the 256x256 hardware configuration. That is a completeness or overclaim issue about the final hardware accuracy, not a circularity in the derivation chain. Therefore no circular step satisfies the requirement of exhibiting a specific equation or fitted parameter that reduces the output to the input.

Assumptions & free parameters 3 free parameters · 5 assumptions · 1 invented entities

The central claims depend on tuning the top-k count and quantization to the target datasets, an averaged early-stopping factor that conditions the speedup numbers, and three empirical or domain assumptions: top-k sufficiency for softmax, correct analog ordering, and the efficacy of the TFCBP training scheme. No new physical entities are introduced beyond the proposed circuit block, which lacks independent silicon validation.

free parameters (3)
  • k (top-k count) = 5
    Chosen from the accuracy-versus-k curve in Fig. 3; the headline accuracy drop (0.4-1.2%) and the softmax speedup (15x/8x) are conditioned on k=5.
  • alpha (early-stopping factor) = 0.31 average over the dataset
    Extracted from the SPICE macro simulation and used in the topkima-SM latency model (Eq. 4). Only the average is reported; the worst case is not given, so the claimed speedup may not be representative.
  • Quantization bit-widths = Q: 5 bits, K: 4 bits, X/A: 5 bits, W: 8 bits
    Selected to balance accuracy and hardware cost; the 15x/8x speedup and the accuracy drop are both specific to these precisions, which are determined by the ADC resolution and ternary-cell scheme.
assumptions (5)
  • domain assumption Softmax outputs are dominated by a small number of large scores, so top-k pruning preserves accuracy.
    Motivated by winner-take-all competition (Maass 2000) and empirically validated only on the three model families and three datasets tested; not proven for all transformers.
  • domain assumption The decreasing ramp makes comparator trigger order match descending MAC magnitude, and the arbiter-encoder resolves all triggers within one clock cycle across corners.
    Required for correct top-k selection without sorting; supported by SPICE simulations at a few corners and voltages, but no silicon results.
  • domain assumption TFCBP training (top-k forward, full backward) yields model weights that are robust to top-k pruning at inference.
    The central accuracy claim rests on this empirical result; reported without error bars and without a direct ablation against baseline top-k training.
  • standard math Matrix multiplication linearity permits absorbing 1/sqrt(dk) into W_Q.
    Used in Section III-C; algebraically exact.
  • domain assumption NeuroSim and SPICE simulations faithfully model the RRAM/SRAM arrays and the proposed dual-10T SRAM cell.
    All system-level numbers come from NeuroSim with peripheral overheads estimated rather than measured; the macro-level SPICE is for one configuration.
invented entities (1)
  • topkima-M macro with decreasing-ramp IMA, arbiter-encoder early stopping
    purpose: Performs top-k selection on the Q*K^T MAC results in the analog domain, replacing digital sorting and reducing softmax input count.
    New hardware block; correctness and performance are supported only by in-paper SPICE simulation, not by a fabricated chip or external benchmark.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Topkima-Former: Low-energy, Low-Latency Inference for Transformers using top-k In-memory ADC." pith.science (2026). https://pith.science/paper/BBB77I2H

@misc{pith2026241113050,
  author       = {Pith},
  title        = {Pith review of: Topkima-Former: Low-energy, Low-Latency Inference for Transformers using top-k In-memory ADC},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BBB77I2H}},
  note         = {Machine review of arXiv:2411.13050}
}
read the original abstract

Transformer model has gained prominence as a popular deep neural network architecture for neural language processing (NLP) and computer vision (CV) applications. However, the extensive use of nonlinear operations, like softmax, poses a performance bottleneck during transformer inference and comprises up to 40% of the total latency. Hence, we propose innovations at the circuit, architecture, and algorithm levels to accelerate the transformer. At the circuit level, we propose topkima-combining top-k activation selection with in-memory ADC (IMA) to implement a low-energy and low-latency softmax without any sorting latency. Only the k largest activations are sent to the softmax calculation block, reducing the huge computational cost of softmax. Using a modified training scheme with top-k only in the forward pass, experimental results demonstrate only a 0.4% to 1.2% reduction in accuracy across ViT, distilBERT, and BERT-base models when evaluated on CIFAR-10, CIFAR-100, and SQuAD datasets with k=5. At the architecture level, an improved scale-free technique is introduced to reduce the computational cost of attention. The combined system, dubbed Topkima-Former, enhances 1.8x-84x speedup and 1.3x-35x energy efficiency (EE) over prior In-memory computing (IMC) accelerators. Compared to a conventional softmax macro and a digital top-k (Dtopk) softmax macro, our proposed tokima softmax macro achieves about 15x and 8x faster speed respectively.

Figures

Figures reproduced from arXiv: 2411.13050 by the authors.

Figure 1
Figure 1. The attention module Topkima-Former achieves 1.8×−84× faster and 1.3×−35× more EE than prior IMC-based accelerators. Compared to a conventional softmax macro and a digital top-k (Dtopk) softmax macro, our proposed topkima softmax macro (topkima￾SM) achieves about 15× and 8× faster speed. II. PRELIMINARIES AND RELATED WORKS A. Transformer models Transformer models are built up with multiple encoders and/or decoders. … view at source ↗
Figure 2
Figure 2. Topkima-M hardware: (a) Block diagram. (b) Concept of early stopping in topkima. (c) Circuit diagram detail of one column. (d) Basic multiplication [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Accuracy evaluation of top-k include the operations of Q · KT and the following softmax in the complexity comparisons of topkima-SM done later. Topkima-SM benefits. The benefit of our proposed method can be understood by considering the latency of a conventional softmax macro, Tconv−SM, given by: Tconv−SM = Twr + d · (Tpwm,inp + Tima + d · TNL,dig) where Twr is the time to write the KT values in SRAM, Tpwm,inp is th… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Hardware evaluation results. (a) Latency breakdown across Conv-SM, Dtopk-SM and topkima-SM. (b) Theoretical and simulated MAC value. (c) Impact [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 23 canonical work pages

  1. [1]

    ReTransformer: ReRAM-based processing-in-memory architecture for transformer acceleration,

    X. Yang, B. Yan, H. Li and Y . Chen, “ReTransformer: ReRAM-based processing-in-memory architecture for transformer acceleration,” ICCAD, 2020, pp. 1-9

    work page 2020

  2. [2]

    Spatten: Efficient sparse attention architecture with cascade token and head pruning,

    H. Wang, Z. Zhang and S. Han, “Spatten: Efficient sparse attention architecture with cascade token and head pruning,” in HPCA, 2021, pp. 97-110

    work page 2021

  3. [3]

    A length adaptive algorithm-hardware co-design of transformer on fpga through sparse attention and dynamic pipelining,

    H. Peng, S. Huang, S. Chen, B. Li, T. Geng, et al, “A length adaptive algorithm-hardware co-design of transformer on fpga through sparse attention and dynamic pipelining,” in DAC, 2022, pp. 1135-1140

    work page 2022

  4. [4]

    X-former: In- memory acceleration of transformers,

    S. Sridharan, J. R. Stevens, K. Roy and A. Raghunathan, “X-former: In- memory acceleration of transformers,” IEEE T VLSI SYST, vol. 31, no.8, 2023, pp. 1223-1233

    work page 2023

  5. [5]

    DNN+ NeuroSim V2. 0: An end-to-end benchmarking framework for compute-in-memory accelerators for on-chip training,

    X. Peng, S. Huang, H. Jiang, A. Lu and S. Yu, “DNN+ NeuroSim V2. 0: An end-to-end benchmarking framework for compute-in-memory accelerators for on-chip training,” IEEE T COMPUT AID D, vol. 40, no.11, 2020, pp. 2306-2319

    work page 2020

  6. [6]

    A 16K current- based 8T SRAM compute-in-memory macro with decoupled read/write and 1-5bit column ADC,

    C. Yu, T. Yoo, T. T. H. Kim, K. C. T. Chuan and B. Kim, “A 16K current- based 8T SRAM compute-in-memory macro with decoupled read/write and 1-5bit column ADC,” in CICC, 2020, pp. 1-4

    work page 2020

  7. [7]

    Softermax: Hardware/software co-design of an efficient softmax for transformers,

    J. R. Stevens, R. Venkatesan, S. Dai, B. Khailany and A. Raghunathan, “Softermax: Hardware/software co-design of an efficient softmax for transformers,” in DAC, 2021, pp. 469-474

    work page 2021

  8. [8]

    Base-2 softmax function: Suitability for training and efficient hardware implementation,

    Y . Zhang, Y . Zhang, L. Peng, L. Quan, S. Zheng, et al, “Base-2 softmax function: Suitability for training and efficient hardware implementation,” TCAS-I, vol. 69, no.9, 2022, pp. 3605-3618

    work page 2022

Show all 23 references
  1. [9]

    Hardware implementation of the exponential function using Taylor series,

    P. Nilsson, A. U. R. Shaik, R. Gangarajaiah R and E. Hertz, “Hardware implementation of the exponential function using Taylor series,” in NORCHIP, 2014, pp. 1-4

    2014

  2. [10]

    On the Computational Power of Winner-Take-All,

    W. Maass, “On the Computational Power of Winner-Take-All,” Neural Computation, vol. 12, no. 11, pp. 2519-35, 2000

    2000

  3. [11]

    Vitality: Unifying low-rank and sparse approximation for vision transformer acceleration with a linear taylor attention,

    J. Dass, S. Wu, H. Shi, C. Li, Z. Ye, et al, “Vitality: Unifying low-rank and sparse approximation for vision transformer acceleration with a linear taylor attention,” in HPCA, 2023, pp. 415-428

    2023

  4. [12]

    I-bert: Integer- only bert quantization,

    S. Kim, A. Gholami, Z. Yao, M. W. Mahoney, K. Keutzer, “I-bert: Integer- only bert quantization,” in ICML, 2021, pp. 5506-5518

    2021

  5. [13]

    Efficient softmax hardware architecture for deep neural networks,

    G. Du, C. Tian, Z. Li, D. Zhang, Y . Yin and Y . Ouyang, “Efficient softmax hardware architecture for deep neural networks,” in GLSVLSI, 2019, pp. 75-80

    2019

  6. [14]

    TranCIM: Full- digital bitline-transpose CIM-based sparse transformer accelerator with pipeline/parallel reconfigurable modes,

    F. Tu, Z. Wu, Y . Wang, L. Liang, L Liu, Y . Ding, et al, “TranCIM: Full- digital bitline-transpose CIM-based sparse transformer accelerator with pipeline/parallel reconfigurable modes,” JSSC, vol. 58, no. 6, 2022, pp. 1798-1809

    2022

  7. [15]

    Challenges and trends of SRAM-based computing-in-memory for AI edge devices,

    C. Jhang, C. Xue, J. Hung, F. Chang and M. Chang, “Challenges and trends of SRAM-based computing-in-memory for AI edge devices,” TCAS-I, vol. 68, no. 5, 2021, pp. 1773-1786

    2021

  8. [16]

    A 240×180 130 db 3 µs latency global shutter spatiotemporal vision sensor,

    C. Brandli, R. Berner, M. Yang, S. Liu and T. Delbruck, “A 240×180 130 db 3 µs latency global shutter spatiotemporal vision sensor,” JSSC, vol. 49, no. 10, 2014, pp. 2333-2341

    2014

  9. [17]

    Hardware-aware softmax approximation for deep neural networks,

    X. Geng, J. Lin, B. Zhao, A. Kong, M. M. S. Aly and V . Chandrasekhar, “Hardware-aware softmax approximation for deep neural networks,” in ACCV , Perth, 2019, pp. 107-122

    2019

  10. [18]

    Memristor-based edge computing of blaze block for image recognition,

    H. Ran, S. Wen, Q. Li, Y . Yang and K. Shi, et al, “Memristor-based edge computing of blaze block for image recognition,” IEEE T NEUR NET LEAR, vol. 33, no. 5, 2020, pp. 2121-2131

    2020

  11. [19]

    19.7 A 16Gb ReRAM with 200MB/s write and 1GB/s read in 27nm technology,

    R. Fackenthal, M. Kitagawa, W. Otsuka, K. Prall, D. Mills, et al, “19.7 A 16Gb ReRAM with 200MB/s write and 1GB/s read in 27nm technology,” in ISSCC, 2014, pp. 338-339

    2014

  12. [20]

    90 nm 32×32 bit Tunneling SRAM Memory Array With 0.5 ns Write Access Time, 1 ns Read Access Time and 0.5 V Operation,

    A. Ramesh, S. Y . Park, P. R. Berger, “90 nm 32×32 bit Tunneling SRAM Memory Array With 0.5 ns Write Access Time, 1 ns Read Access Time and 0.5 V Operation,” TCAS-I, vol.58, no. 10, 2011, pp. 2432-2445

    2011

  13. [21]

    Tron: Transformer neural network acceleration with non-coherent silicon photonics,

    S. Afifi, F. Sunny, M. Nikdast and S. Pasricha, “Tron: Transformer neural network acceleration with non-coherent silicon photonics,” in GLSVLSI, June 2023, pp. 15-21

    2023

  14. [22]

    ELSA: Hardware-software co-design for efficient, lightweight self-attention mechanism in neural networks,

    T. J. Ham, Y . Lee, S. H. Seo, S. Kim, and H. Choi, et al, “ELSA: Hardware-software co-design for efficient, lightweight self-attention mechanism in neural networks,” in ISCA, 2021, pp. 692-705

    2021

  15. [23]

    Hardsea: Hybrid analog-reram clustering and digital-sram in-memory computing accelera- tor for dynamic sparse self-attention in transformer,

    S. Liu, C. Mu, H. Jiang, Y . Wang, J. Zhang, et al, “Hardsea: Hybrid analog-reram clustering and digital-sram in-memory computing accelera- tor for dynamic sparse self-attention in transformer,” IEEE T VLSI SYST, 2023

    2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.