Pith. sign in

REVIEW 4 major objections 6 minor 50 references

RED-PIM cuts transformer attention data movement from O(N²) to O(N) by never building the full attention map.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

RED-PIM restructures attention to use a compact d×d intermediate matrix, reducing PIM inter-bank data movement from O(N²) to O(N) and claiming up to 99.99% latency reduction in simulation.

T0 review reviewed 2026-08-01 challenge →

load-bearing objection A plausible PIM mapping for linear attention, but the headline speedups rest on an unverified single-bank fit of the d×d intermediate, and the evaluation never goes beyond d=256. the 4 major comments →

arxiv 2607.21731 v1 pith:YQCVZOB2 submitted 2026-07-23 cs.LG cs.AR

RED-PIM: Reducing Data Movement for Transformers using Processing-in-Memory

Zahra Yousefijamarani , Alaa Alameldeen This is my paper
classification cs.LG cs.AR
keywords transformersself-attentionprocessing-in-memoryHBMdata movementlinear attentioninference accelerationinter-bank communication
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the dominant cost of transformer attention on processing-in-memory hardware is inter-bank data movement, which grows quadratically with sequence length because the N×N attention matrix must be built and passed between memory banks. RED-PIM is an algorithm-architecture co-design that replaces that construction with a linear-attention reformulation: it applies softmax to keys and queries separately, computes a compact d×d intermediate matrix L once, then multiplies by queries. That shrinks the attention intermediate from N×N to d×d and, with hierarchical in-bank aggregation, cuts inter-bank traffic from O(N²) to O(N). If right, it would make PIM practical for long-context transformer inference, where bank capacity is currently the barrier. On a simulated HBM-PIM system, attention inference time falls by 16.05% to 99.99% (geometric mean 66.42%), with the largest gains on long sequences and near-equivalent accuracy.

Core claim

RED-PIM claims that the standard scaled dot-product attention can be reorganized in PIM so that no full attention map is ever materialized. Instead of computing softmax(QK^T)V, it computes σ(K^T)V first — a d×d matrix the paper calls L — and then multiplies by σ(Q). Because L is tiny relative to N×N, it can live in a single memory bank; because each bank already holds the matching slices of σ(K^T) and V, the first multiplication is local, and only d×d partial products need to be aggregated across banks. With hierarchical accumulation and a bit-shift/LUT softmax, inter-bank data movement drops from O(N²) to O(N). The paper reports that this yields 16.05%–99.99% inference-time reductions over

What carries the argument

The load-bearing object is the d×d matrix L = σ(K^T)V. In standard attention the intermediate is the N×N score matrix; RED-PIM never builds it. L is computed from locally stored key/value slices, then aggregated hierarchically, and finally multiplied by σ(Q). The argument's force is that d (embedding/head dimension) is far smaller than N (sequence length), so all attention state shrinks to a per-head d×d matrix that fits in one bank; hierarchical aggregation keeps the cost of combining partial results logarithmic in bank count rather than linear in sequence length. A hardware softmax based on 2^(x log2 e) with bit-shift for integer part and a small interpolated LUT for the fractional part su

Load-bearing premise

The compact d×d intermediate matrix L must fit entirely in one memory bank; for modern models with d=4096 or more, L alone is about 32 MB in FP16, the full capacity of an HBM bank, and the paper does not analyze the case where L is split across banks.

What would settle it

A reader could check the bank-fit calculation: for d=4096, L = d^2 × 2 bytes = 32 MiB, equal to a standard 32 MB HBM bank capacity. Since weights, operands, and partial sums also occupy the bank, L cannot actually fit in one bank for such models; measuring inter-bank traffic for d=4096 at long sequence lengths and seeing it grow with N rather than stay O(N) would settle the claim.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If correct, attention intermediates no longer scale with sequence length, so PIM banks can serve sequences far beyond the roughly 8,000-token FP16 limit of an N×N map in a 32 MB bank.
  • Inter-bank traffic for attention decreases from quadratic to linear in N, so long-context inference benefits disproportionately; the paper's measured gains grow with sequence length.
  • The compact L matrix must be multiplied by σ(Q), and the cost of moving σ(Q) grows with N but with vector width d — embedding size, not sequence length, now dominates per-token transfer cost.
  • The softmax approximation keeps GLUE accuracy within 0.58% average drop and improves accuracy on some long-document datasets, so the speedup is not bought with large accuracy loss.
  • Each attention head has its own d×d L, so the reorganization applies independently across heads and should integrate with multi-head attention as currently built.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The O(N) claim depends on L fitting a single bank. For d=4096, L is 32 MiB in FP16, equal to a standard HBM bank's capacity; a reader should expect the scheme to need a split-bank variant before it applies to modern large models.
  • The reformulation changes attention semantics (softmax on queries and keys separately rather than on dot products), so models trained with standard attention may need fine-tuning; the paper's accuracy results come from BERT-base, and transfer to larger pretrained models is an open question.
  • A natural testable extension is to run the same algorithm with d=512/1024/4096 heads and measure whether inter-bank traffic actually stays O(N) as sequence length grows; the paper's sensitivity analysis only varies d up to 256.
  • The hierarchical bank-group/channel/die aggregation pattern could be applied to other memory-bound reductions in transformers, such as layer normalization or softmax over very long sequences.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. RED-PIM proposes an algorithm-architecture co-design for transformer inference on PIM-enabled HBM. It replaces standard softmax attention with a linear-attention formulation from Shen et al. [36], computing a compact d×d intermediate matrix L = σ(K^T)V and then multiplying by σ(Q), thereby avoiding the N×N attention map. The paper also introduces near-bank PCUs, a hardware-friendly LUT softmax, and a hierarchical inter-bank aggregation scheme. Using a DAMOV-based simulator with FIMDRAM-style HBM, it reports inference-time reductions of 16.05%–99.99% (geometric mean 66.42%) over a baseline PIM implementation, accuracy changes within ±4.3% on real datasets, and sensitivity studies over sequence length, embedding size, and bank count.

Significance. If the reported results hold, RED-PIM would be a useful contribution: it applies an established linear-attention algorithm to a concrete PIM memory system, addresses inter-bank data movement rather than only FLOPs, and evaluates on long sequences up to 1M tokens, which is directly relevant to current LLM inference. The paper's strengths include the use of a physical memory simulator (DAMOV/Ramulator/ZSim), evaluation on multiple real-world long- and short-context datasets, and a quantitative comparison of softmax approximation methods. However, the central quantitative claims—especially the O(N) data-movement reduction and the geomean 66.42% speedup—depend on assumptions about the d×d intermediate fitting in a single bank and on a baseline that is explicitly unoptimized. The asymptotic idea is plausible, but the paper does not yet establish the large-d regime that matters for modern LLMs, and its comparison to prior accelerators is not backed by a described methodology.

major comments (4)
  1. [Section III-B, IV-F, VI] The O(N) data-movement argument and the reported speedups rely on the d×d intermediate L being storable in a single bank. The paper states in III-B that L is aggregated and then, for the final multiplication, assumes 'L is stored in a single bank without replication.' For d=4096, L is 32 MB in FP16—the entire nominal capacity of an HBM bank—and Section VI concedes that FIMDRAM PCUs reduce usable bank capacity. Section IV-F only evaluates d up to 256 (Figure 12), so the large-d regime is not tested. If L spans b banks, the final σ(Q)·L step requires bN row transfers rather than N, and L aggregation itself becomes more expensive. The asymptotic O(N) may survive for fixed d, but the constant-factor degradation and the 16.05–99.99% speedup numbers are not established for modern LLM embedding dimensions. Please either extend the sensitivity analysis to d=1024/4096 (with L tiling or replicatio
  2. [Section III-B] The complexity accounting switches units without warning. The baseline is counted in 'N²/m vector movements' (per-row transfers), while the RED-PIM final multiplication is said to require 'N/m data movements.' Since each of the N rows of σ(Q) must be transferred to the bank(s) holding L, the row-vector cost is N, not N/m; N/m is only the number of bank-to-bank messages if each source bank sends its local m rows as one batch. Similarly, the L-accumulation step moves d×d partial matrices across N/m banks, which is d row-vectors per bank, not one. This conflation matters because the claimed gap between O(N²/m) and O(N/m) is partly an artifact of using message counts for RED-PIM and row-vector counts for the baseline. Please define one metric (e.g., number of d-element row transfers or total elements) and recompute the asymptotic terms consistently.
  3. [Section IV-E, Figures 10-11] The comparison to TransPIM, HAIMA, and PACT-3D is not reproducible as presented. There is no description of whether these accelerators were reimplemented in the same DAMOV-based simulator, what configurations were used, which baseline all improvements are relative to, or whether the numbers were taken from the original papers. Without this, 'RED-PIM consistently outperforms all other methods' is unsupported. Please provide the methodology: same simulator, same workloads, same baseline, and the source of each prior result, or present the comparison as indicative rather than a head-to-head measurement.
  4. [Section IV-C, Abstract] The headline speedups (16.05–99.99%, geomean 66.42%) are measured against a deliberately naive baseline: Taylor-series softmax and naive broadcast, as stated in Section IV-C. This is a legitimate reference point, but the abstract and introduction present the numbers as a general 'PIM implementation' improvement. Given that prior PIM attention works use ring broadcast and more efficient softmax, the practical significance of the 66.42% geomean is overstated unless the comparison is also made against an optimized baseline or the naive-baseline condition is prominently disclosed wherever results are summarized.
minor comments (6)
  1. [Introduction] The example says an 8000-token attention map is '64 million values, requiring approximately 32MB' in FP16. 64M×2 bytes is 128 MB, not 32 MB. Please correct the calculation.
  2. [Section II, III-B] The notation σ(K^T) is ambiguous: it should state explicitly that softmax is applied row-wise to K^T (equivalently, column-wise to K), and whether the normalization is over the N dimension or the d dimension. This affects the validity of the d×d L matrix.
  3. [Section III-B] The phrase 'In the worst-case scenario, if L is stored in a single bank without replication' is confusing: a single copy of L minimizes query-to-L movement; a distributed L is the more costly case. Please reword.
  4. [Section IV-B] The default embedding size d, number of tokens per bank m, and the exact DRAM bank capacity used in the main experiments are not reported. These are needed to connect Figures 7–9 to the analysis in Section III-B.
  5. [Section III-C3] The reported Taylor-series average relative error of 2.6×10^77 is likely a numerical artifact for values near zero; please clarify how the error is computed and whether the comparison is meaningful.
  6. [Section IV-D] Accuracy results are reported as single numbers without confidence intervals, number of runs/seeds, or fine-tuning details. Since some datasets show improvement and others show a drop, a brief statement on statistical significance would strengthen the 'maintaining accuracy' claim.

Circularity Check

0 steps flagged

No significant circularity: RED-PIM's speedups are simulated against an independent baseline, and its linear-attention formulation is attributed to external prior work rather than derived from its own assumptions.

full rationale

The central derivation chain is not circular. Section III-B explicitly adopts the compact d×d formulation from Shen et al. [36] as an external algorithmic prior, then derives the O(N/m) inter-bank movement count algebraically from the placement of tokens across banks and the stored L matrix. The reported inference-time improvements (Section IV-D) come from a DAMOV/FIMDRAM simulation compared against a separately described naive PIM baseline (Section IV-C); no fitted parameter is renamed as a prediction. The single-bank-fit assumption for L is a stated modeling condition that affects the constants of the data-movement estimate, and Section VI candidly lists reduced usable bank capacity and simulation-only evaluation as limitations. These are correctness or generality concerns, not evidence that the result is equivalent to its inputs. There is no load-bearing self-citation, no imported uniqueness theorem by the authors, and no ansatz smuggled in via the authors' own prior work. The algorithmic reduction from N×N to d×d is a property of the adopted efficient-attention definition, but that is an attribution, not a circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 4 axioms · 2 invented entities

The paper introduces no fitted parameters. Its central claims rest on adopting linear attention from prior work, on the bank-capacity assumption for L, on the fidelity of the DAMOV simulation, and on the accuracy of the hardware softmax approximation.

axioms (4)
  • domain assumption The linear attention reformulation σ(Q)σ(K^T)V approximates standard softmax attention with acceptable accuracy.
    Adopted from Zhuoran et al. [36]; used throughout Section III-B. The accuracy evaluation in Section IV-D shows small drops, but this approximation is not derived in the paper.
  • domain assumption Each HBM bank can hold m tokens and the d×d intermediate matrix L fits in a single bank.
    Assumed in Section III-B; not validated for large d.
  • domain assumption The DAMOV simulator, ZSim, and Ramulator accurately model FIMDRAM and the proposed network.
    Evaluation methodology in Section IV-B; no validation against real hardware.
  • standard math The softmax exponentiation identity e^x = 2^{x log2 e} with LUT interpolation for the fractional part is accurate enough.
    Section III-C3; they evaluate it experimentally.
invented entities (2)
  • Hierarchical aggregation tree across banks/channels/dies no independent evidence
    purpose: To reduce inter-bank data movement for partial sums in softmax and matrix multiplication.
    Architectural design in Section III-C5; simulated only, no physical prototype.
  • Near-bank PCU modifications and network topology (horizontal/diagonal links) no independent evidence
    purpose: To enable local computation and efficient inter-bank communication.
    Based on FIMDRAM and NoM [33]; the specific modifications are not fabricated.

reviewed 2026-08-01 · how reviews work

0 comments
Cite this review

Pith. "Pith review of RED-PIM: Reducing Data Movement for Transformers using Processing-in-Memory." pith.science (2026). https://pith.science/paper/YQCVZOB2

@misc{pith2026260721731,
  author       = {Pith},
  title        = {Pith review of: RED-PIM: Reducing Data Movement for Transformers using Processing-in-Memory},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YQCVZOB2}},
  note         = {Machine review of arXiv:2607.21731}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Transformers are widely used across many domains, including natural language processing, computer vision, web search, and DNA sequence analysis. Given their broad applicability, improving the performance of transformer models is critical. However, the high volume of data movement between processing units and memory during attention operations significantly limits their efficiency. Processing-In-Memory (PIM) mitigates this issue by performing computations directly inside memory. While prior work has proposed PIM-based transformer implementations, they suffer from costly inter-bank communication, and struggle to scale due to the limited capacity of memory banks. As a result, attention-related data must be split across banks, diminishing the potential benefits of PIM. In this work, we propose RED-PIM, an algorithm-architecture co-design that reduces attention latency by minimizing inter-bank data movement from O(N^2) to O(N) and shrinking intermediate attention matrices from N x N to d x d. By reorganizing matrix operations, performing computations locally, and employing an optimized data transfer strategy, RED-PIM significantly reduces computation cost and interconnect traffic. Compared to baseline PIM implementation, RED-PIM achieves inference time reductions ranging from 16.05% to 99.99% (geometric mean of 66.42%), with the largest gains on longer sequences. On real-world datasets, RED-PIM improves performance by 99.60% for long documents and 13.44% for shorter ones, while maintaining or improving accuracy. These results demonstrate RED-PIM's effectiveness for scalable and efficient transformer inference.

Figures

Figures reproduced from arXiv: 2607.21731 by Alaa Alameldeen, Zahra Yousefijamarani.

Figure 1
Figure 1. Figure 1: The Transformer model architecture [42]. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: d illustrates the internal architecture of a PCU. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 2
Figure 2. Figure 2: a) HBM architecture; b) Conventional HBM2; c) FIMDRAM architecture; d) Computing unit block. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Token and weight matrix mapping across banks. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Regular Scaled Dot-Product Attention mechanism. Here, N denotes the sequence length, d is the embedding dimension, and m represents the maximum [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: RED-PIM Scaled Dot-Product Attention. Parts highlighted in red indicate portions stored locally within the same bank. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Hierarchical data transfer architecture. Each colored arrow represents [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 8
Figure 8. Figure 8: Execution time improvement vs. baseline across different datasets [PITH_FULL_IMAGE:figures/full_fig_p009_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Execution time improvement on the GLUE benchmark for our method [PITH_FULL_IMAGE:figures/full_fig_p009_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Performance improvement analysis of RED-PIM, TransPIM [49], [PITH_FULL_IMAGE:figures/full_fig_p010_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Performance improvement comparison of RED-PIM, TransPIM [49], [PITH_FULL_IMAGE:figures/full_fig_p010_11.png] view at source ↗
Figure 13
Figure 13. Figure 13: Sensitivity of model performance to varying numbers of memory [PITH_FULL_IMAGE:figures/full_fig_p011_13.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

50 extracted references · 10 linked inside Pith

  1. [1]

    A scalable processing- in-memory accelerator for parallel graph processing,

    J. Ahn, S. Hong, S. Yoo, O. Mutlu, and K. Choi, “A scalable processing- in-memory accelerator for parallel graph processing,” inProceedings of the 42nd Annual International Symposium on Computer Architecture, 2015, pp. 105–117

    2015

  2. [2]

    The fifth pascal recognizing textual entailment challenge

    L. Bentivogli, P. Clark, I. Dagan, and D. Giampiccolo, “The fifth pascal recognizing textual entailment challenge.”TAC, vol. 7, no. 8, p. 1, 2009

    2009

  3. [3]

    Google workloads for consumer devices: Mitigating data movement bottle- necks,

    A. Boroumand, S. Ghose, Y . Kim, R. Ausavarungnirun, E. Shiu, R. Thakur, D. Kim, A. Kuusela, A. Knies, P. Ranganathanet al., “Google workloads for consumer devices: Mitigating data movement bottle- necks,” inProceedings of the Twenty-Third International Conference on Architectural Support for Programming Languages and Operating Systems, 2018, pp. 316–331

    2018

  4. [4]

    In-memory transformer self-attention mechanism using passive mem- ristor crossbar,

    J. Cai, M. A. Kaleem, R. Genov, M. R. Azghadi, and A. Amirsoleimani, “In-memory transformer self-attention mechanism using passive mem- ristor crossbar,” in2024 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE, 2024, pp. 1–5

    2024

  5. [5]

    Semeval- 2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation,

    D. Cer, M. Diab, E. Agirre, I. Lopez-Gazpio, and L. Specia, “Semeval- 2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation,”arXiv preprint arXiv:1708.00055, 2017

    Pith/arXiv arXiv 2017

  6. [6]

    A discourse-aware attention model for abstractive summarization of long documents,

    A. Cohan, F. Dernoncourt, D. S. Kim, T. Bui, S. Kim, W. Chang, and N. Goharian, “A discourse-aware attention model for abstractive summarization of long documents,”arXiv preprint arXiv:1804.05685, 2018

    Pith/arXiv arXiv 2018

  7. [7]

    Gemini 2 models,

    G. DeepMind, “Gemini 2 models,” 2024, available at https://deepmind. google/technologies/gemini

    2024

  8. [8]

    Longnet: Scaling transformers to 1,000,000,000 tokens,

    J. Ding, S. Ma, L. Dong, X. Zhang, S. Huang, W. Wang, N. Zheng, and F. Wei, “Longnet: Scaling transformers to 1,000,000,000 tokens,”arXiv preprint arXiv:2307.02486, 2023

    Pith/arXiv arXiv 2023

  9. [9]

    Haima: A hybrid sram and dram accelerator-in-memory architecture for transformer,

    Y . Ding, C. Liu, M. Duan, W. Chang, K. Li, and K. Li, “Haima: A hybrid sram and dram accelerator-in-memory architecture for transformer,” in 2023 60th ACM/IEEE Design Automation Conference (DAC). IEEE, 2023, pp. 1–6

    2023

  10. [10]

    Automatically constructing a corpus of sen- tential paraphrases,

    B. Dolan and C. Brockett, “Automatically constructing a corpus of sen- tential paraphrases,” inThird international workshop on paraphrasing (IWP2005), 2005

    2005

  11. [11]

    The llama 3 herd of models,

    A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Yang, A. Fanet al., “The llama 3 herd of models,”arXiv preprint arXiv:2407.21783, 2024

    Pith/arXiv arXiv 2024

  12. [12]

    Processing-in-memory: A workload-driven perspective,

    S. Ghose, A. Boroumand, J. S. Kim, J. G ´omez-Luna, and O. Mutlu, “Processing-in-memory: A workload-driven perspective,”IBM Journal of Research and Development, vol. 63, no. 6, pp. 3–1, 2019

    2019

  13. [13]

    Sal-pim: A subarray-level processing-in-memory architecture with lut-based linear interpolation for transformer-based text generation,

    W. Han, H. Cho, D. Kim, and J.-Y . Kim, “Sal-pim: A subarray-level processing-in-memory architecture with lut-based linear interpolation for transformer-based text generation,”arXiv preprint arXiv:2401.17005, 2024

    Pith/arXiv arXiv 2024

  14. [14]

    Neupims: Npu-pim heterogeneous acceleration for batched llm inferencing,

    G. Heo, S. Lee, J. Cho, H. Choi, S. Lee, H. Ham, G. Kim, D. Ma- hajan, and J. Park, “Neupims: Npu-pim heterogeneous acceleration for batched llm inferencing,” inProceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, V olume 3, 2024, pp. 722–737

    2024

  15. [15]

    Efficient attentions for long document summarization,

    L. Huang, S. Cao, N. Parulian, H. Ji, and L. Wang, “Efficient attentions for long document summarization,”arXiv preprint arXiv:2104.02112, 2021

    Pith/arXiv arXiv 2021

  16. [16]

    Hbm (high bandwidth memory) dram technology and architecture,

    H. Jun, J. Cho, K. Lee, H.-Y . Son, K. Kim, H. Jin, and K. Kim, “Hbm (high bandwidth memory) dram technology and architecture,” in2017 IEEE International Memory Workshop (IMW). IEEE, 2017, pp. 1–4

    2017

  17. [17]

    Performance evaluation of high bandwidth memory for hpc workloads,

    A. K. Kabat, S. Pandey, and V . T. Gopalakrishnan, “Performance evaluation of high bandwidth memory for hpc workloads,” in2022 IEEE 35th International System-on-Chip Conference (SOCC), 2022, pp. 1–6

    2022

  18. [18]

    A framework for accelerating transformer-based language model on reram-based architecture,

    M. Kang, H. Shin, and L.-S. Kim, “A framework for accelerating transformer-based language model on reram-based architecture,”IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 41, no. 9, pp. 3026–3039, 2021

    2021

  19. [19]

    Introduction to pytorch,

    N. Ketkar, J. Moolayil, N. Ketkar, and J. Moolayil, “Introduction to pytorch,”Deep learning with python: learn best practices of deep learning models with PyTorch, pp. 27–91, 2021

    2021

  20. [20]

    Ramulator: A fast and extensible dram simulator,

    Y . Kim, W. Yang, and O. Mutlu, “Ramulator: A fast and extensible dram simulator,”IEEE Computer Architecture Letters, vol. 15, no. 1, pp. 45–49, 2016

    2016

  21. [21]

    25.4 a 20nm 6gb function-in-memory dram, based on hbm2 with a 1.2 tflops pro- grammable computing unit using bank-level parallelism, for machine learning applications,

    Y .-C. Kwon, S. H. Lee, J. Lee, S.-H. Kwon, J. M. Ryu, J.-P. Son, O. Seongil, H.-S. Yu, H. Lee, S. Y . Kimet al., “25.4 a 20nm 6gb function-in-memory dram, based on hbm2 with a 1.2 tflops pro- grammable computing unit using bank-level parallelism, for machine learning applications,” in2021 IEEE International Solid-State Circuits Conference (ISSCC), vol. 6...

    2021

  22. [22]

    In-memory com- puting based accelerator for transformer networks for long sequences,

    A. F. Laguna, A. Kazemi, M. Niemier, and X. S. Hu, “In-memory com- puting based accelerator for transformer networks for long sequences,” in2021 Design, Automation & Test in Europe Conference & Exhibition (DATE). IEEE, 2021, pp. 1839–1844

    2021

  23. [23]

    Hardware-software co-design of an in-memory transformer network accelerator,

    A. F. Laguna, M. M. Sharifi, A. Kazemi, X. Yin, M. Niemier, and X. S. Hu, “Hardware-software co-design of an in-memory transformer network accelerator,”Frontiers in Electronics, vol. 3, p. 847069, 2022

    2022

  24. [24]

    The winograd schema challenge,

    H. Levesque, E. Davis, and L. Morgenstern, “The winograd schema challenge,” inThirteenth international conference on the principles of knowledge representation and reasoning, 2012

    2012

  25. [25]

    Specpim: Ac- celerating speculative inference on pim-enabled system via architecture- dataflow co-exploration,

    C. Li, Z. Zhou, S. Zheng, J. Zhang, Y . Liang, and G. Sun, “Specpim: Ac- celerating speculative inference on pim-enabled system via architecture- dataflow co-exploration,” inProceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, V olume 3, 2024, pp. 950–965

    2024

  26. [26]

    H3datten: Heterogeneous 3-d integrated hybrid analog and digital compute-in- memory accelerator for vision transformer self-attention,

    W. Li, M. Manley, J. Read, A. Kaul, M. S. Bakir, and S. Yu, “H3datten: Heterogeneous 3-d integrated hybrid analog and digital compute-in- memory accelerator for vision transformer self-attention,”IEEE Trans- actions on V ery Large Scale Integration (VLSI) Systems, 2023

    2023

  27. [27]

    An rram-based computing-in-memory architecture and its application in accelerating transformer inference,

    Z. Lu, X. Wang, M. T. Arafin, H. Yang, Z. Liu, J. Zhang, and G. Qu, “An rram-based computing-in-memory architecture and its application in accelerating transformer inference,”IEEE Transactions on V ery Large Scale Integration (VLSI) Systems, 2023

    2023

  28. [28]

    Learning word vectors for sentiment analysis,

    A. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y . Ng, and C. Potts, “Learning word vectors for sentiment analysis,” inProceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, 2011, pp. 142–150

    2011

  29. [29]

    A modern primer on processing in memory,

    O. Mutlu, S. Ghose, J. G ´omez-Luna, and R. Ausavarungnirun, “A modern primer on processing in memory,” inEmerging computing: from devices to systems: looking beyond Moore and V on Neumann. Springer, 2022, pp. 171–243

    2022

  30. [30]

    DAMOV: A new method- ology and benchmark suite for evaluating data movement bottlenecks,

    G. F. Oliveira, J. G ´omez-Luna, L. Orosa, S. Ghose, N. Vijaykumar, I. Fernandez, M. Sadrosadati, and O. Mutlu, “DAMOV: A new method- ology and benchmark suite for evaluating data movement bottlenecks,” IEEE Access, vol. 9, pp. 134 457–134 502, 2021

    2021

  31. [31]

    Introducing gpt-4.1 in the api,

    OpenAI, “Introducing gpt-4.1 in the api,” 2025, openAI blog. [Online]. Available: https://openai.com

    2025

  32. [32]

    Squad: 100,000+ questions for machine comprehension of text,

    P. Rajpurkar, “Squad: 100,000+ questions for machine comprehension of text,”arXiv preprint arXiv:1606.05250, 2016

    Pith/arXiv arXiv 2016

  33. [33]

    Nom: Network-on-memory for inter- bank data transfer in highly-banked memories,

    S. H. S. Rezaei, M. Modarressi, R. Ausavarungnirun, M. Sadrosadati, O. Mutlu, and M. Daneshtalab, “Nom: Network-on-memory for inter- bank data transfer in highly-banked memories,”IEEE Computer Archi- tecture Letters, vol. 19, no. 1, pp. 80–83, 2020

    2020

  34. [34]

    Zsim: Fast and accurate microarchitec- tural simulation of thousand-core systems,

    D. Sanchez and C. Kozyrakis, “Zsim: Fast and accurate microarchitec- tural simulation of thousand-core systems,” inProceedings of the 40th Annual International Symposium on Computer Architecture. New York, NY , USA: Association for Computing Machinery, 2013, pp. 475–486

    2013

  35. [35]

    Ambit: In- memory accelerator for bulk bitwise operations using commodity dram technology,

    V . Seshadri, D. Lee, T. Mullins, H. Hassan, A. Boroumand, J. Kim, M. A. Kozuch, O. Mutlu, P. B. Gibbons, and T. C. Mowry, “Ambit: In- memory accelerator for bulk bitwise operations using commodity dram technology,” inProceedings of the 50th Annual IEEE/ACM International Symposium on Microarchitecture, 2017, pp. 273–287

    2017

  36. [36]

    Efficient attention: Attention with linear complexities,

    Z. Shen, M. Zhang, H. Zhao, S. Yi, and H. Li, “Efficient attention: Attention with linear complexities,” inProceedings of the IEEE/CVF winter conference on applications of computer vision, 2021, pp. 3531– 3539

    2021

  37. [37]

    A dram-based near-memory architecture for accelerated and energy-efficient execution of transformers,

    G. Singh and S. Vrudhula, “A dram-based near-memory architecture for accelerated and energy-efficient execution of transformers,” inProceed- ings of the Great Lakes Symposium on VLSI 2024, 2024, pp. 57–62

    2024

  38. [38]

    Recursive deep models for semantic compositionality over a sentiment treebank,

    R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Y . Ng, and C. Potts, “Recursive deep models for semantic compositionality over a sentiment treebank,” inProceedings of the 2013 conference on empirical methods in natural language processing, 2013, pp. 1631–1642

    2013

  39. [39]

    A 1.2 v 20 nm 307 gb/s hbm dram with at-speed wafer-level io test scheme and adaptive refresh considering temperature distribution,

    K. Sohn, W.-J. Yun, R. Oh, C.-S. Oh, S.-Y . Seo, M.-S. Park, D.-H. Shin, W.-C. Jung, S.-H. Shin, J.-M. Ryuet al., “A 1.2 v 20 nm 307 gb/s hbm dram with at-speed wafer-level io test scheme and adaptive refresh considering temperature distribution,”IEEE Journal of Solid- State Circuits, vol. 52, no. 1, pp. 250–260, 2016

    2016

  40. [40]

    X-former: In-memory acceleration of transformers,

    S. Sridharan, J. R. Stevens, K. Roy, and A. Raghunathan, “X-former: In-memory acceleration of transformers,”IEEE Transactions on V ery Large Scale Integration (VLSI) Systems, vol. 31, no. 8, pp. 1223–1233, 2023

    2023

  41. [41]

    Multi-hop reading comprehension across multiple documents by reasoning over heterogeneous graphs,

    M. Tu, G. Wang, J. Huang, Y . Tang, X. He, and B. Zhou, “Multi-hop reading comprehension across multiple documents by reasoning over heterogeneous graphs,”arXiv preprint arXiv:1905.07374, 2019

    Pith/arXiv arXiv 1905

  42. [42]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” inProceedings of the 31st International Conference on Neural Information Processing Systems, ser. NIPS’17. Red Hook, NY , USA: Curran Associates Inc., 2017, p. 6000–6010

    2017

  43. [43]

    Glue: A multi-task benchmark and analysis platform for natural lan- guage understanding,

    A. Wang, A. Singh, J. Michael, F. Hill, O. Levy, and S. R. Bowman, “Glue: A multi-task benchmark and analysis platform for natural lan- guage understanding,”arXiv preprint arXiv:1804.07461, 2018

    Pith/arXiv arXiv 2018

  44. [44]

    Deepnet: Scaling transformers to 1,000 layers,

    H. Wang, S. Ma, L. Dong, S. Huang, D. Zhang, and F. Wei, “Deepnet: Scaling transformers to 1,000 layers,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  45. [45]

    Neural network accept- ability judgments,

    A. Warstadt, A. Singh, and S. R. Bowman, “Neural network accept- ability judgments,”Transactions of the Association for Computational Linguistics, vol. 7, pp. 625–641, 2019

    2019

  46. [46]

    A broad-coverage challenge corpus for sentence understanding through inference,

    A. Williams, N. Nangia, and S. R. Bowman, “A broad-coverage challenge corpus for sentence understanding through inference,”arXiv preprint arXiv:1704.05426, 2017

    Pith/arXiv arXiv 2017

  47. [47]

    Grok by xai,

    xAI, “Grok by xai,” 2024, available at https://x.ai

    2024

  48. [48]

    Retransformer: Reram-based processing-in-memory architecture for transformer acceleration,

    X. Yang, B. Yan, H. Li, and Y . Chen, “Retransformer: Reram-based processing-in-memory architecture for transformer acceleration,” in Proceedings of the 39th International Conference on Computer-Aided Design, 2020, pp. 1–9

    2020

  49. [49]

    Transpim: A memory- based acceleration via software-hardware co-design for transformer,

    M. Zhou, W. Xu, J. Kang, and T. Rosing, “Transpim: A memory- based acceleration via software-hardware co-design for transformer,” in 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 2022, pp. 1071–1085

    2022

  50. [50]

    Performance evaluation and optimization of hbm-enabled gpu for data-intensive applications,

    M. Zhu, Y . Zhuo, C. Wang, W. Chen, and Y . Xie, “Performance evaluation and optimization of hbm-enabled gpu for data-intensive applications,”IEEE Transactions on V ery Large Scale Integration (VLSI) Systems, vol. 26, no. 5, pp. 831–840, 2018

    2018

This paper was first reviewed by deepseek-v4-flash on August 1, 2026.