Pith. sign in

REVIEW 3 major objections 6 minor 58 references

Atleus: Accelerating Transformers on the Edge Enabled by 3D Heterogeneous Manycore Architectures

T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Atleus claims a 3D heterogeneous chip accelerates transformer fine-tuning and inference by up to 56x and cuts energy by 64.5x.

desk verdict Novel heterogeneous 3D accelerator for transformer fine-tuning, but the ReRAM capacity assumption doesn't fit the claimed benchmark models. read the letter →

arxiv 2501.09588 v1 pith:FX6OYQBI submitted 2025-01-16 cs.AR cs.LG

Pratyush Dhingra , Janardhan Rao Doppa , Partha Pratim Pande This is my paper · ORCID
classification cs.ARcs.LG
keywords transformeracceleration3DheterogeneousmanycoreReRAMprocessing-in-memorysystolicarrayLoRAfine-tuningedgeAIcrossbar-wisequantizationnetwork-on-chip
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes Atleus, a 3D heterogeneous manycore accelerator designed so that a single edge device can both fine-tune and run transformer models. It claims that by sending static weight matrix multiplications to ReRAM crossbars and dynamic attention plus LoRA computations to systolic arrays, the hybrid design avoids ReRAM write-endurance limits while keeping most computation in memory. Atleus adds an intra-layer pipeline, a 3D NoC with skip links, and crossbar-wise quantization that dequantizes only after each crossbar's matrix-vector product. The reported result is up to 56x speedup and 64.5x energy efficiency over state-of-the-art transformer accelerators including GPU, 3D-TPU, and HAIMA. If these numbers hold, transformer personalization on resource-constrained edge devices becomes practical.

What carries the argument

The load-bearing object is the 3D heterogeneous manycore system: three ReRAM tiers for weight-stationary matrix multiplications and one systolic-array tier for dynamic operands and LoRA, connected by TSV links and a NoC that uses a space-filling curve on ReRAM tiers and a mesh on the systolic tier. Its work is to keep pretrained weights resident in memory so most MACs happen in situ, while routing every write-heavy, dynamic computation away from NVM cells, which protects endurance and enables pipelined fine-tuning. Supporting this is a deterministic intra-layer pipeline and a crossbar-wise quantization scheme with per-crossbar scale registers and shift-and-add dequantization units.

What would settle it

With the stated configuration, on-chip ReRAM capacity is about 302 million 8-bit weights, while BERT-Large, GPT-2 Medium, and BLOOM-560m have more parameters at M8F8. A cycle-accurate simulation that includes HBM weight fetches for those models, or a hardware measurement of end-to-end fine-tuning time with no preloaded weights, would settle whether the 56x and 64.5x numbers survive when the residency assumption is relaxed.

Watch

Extended reading notes

Core claim

Atleus's central claim is that transformer fine-tuning and inference can be accelerated together on one edge chip by partitioning matrix multiplications according to whether operands are static or dynamic. The pretrained weight multiplications of MHA and feed-forward networks are mapped to ReRAM crossbars, which perform in-situ MACs in O(1) time and account for more than 90% of matrix operations; the dynamic Q.K^T score, softmax, layer norm, and LoRA low-rank updates are mapped to a 128x32 systolic array using output-stationary dataflow. A pipelined execution across four stages lets encoder layers run concurrently, and a crossbar-wise quantization scheme dequantizes each crossbar's output after the MVM, unlike GPUs that dequantize before computation. The paper reports that Atleus outperforms HAIMA, 3D-TPU, and GPU baselines by up to 56x in execution time and improves energy efficiency by up to 64.5x.

Load-bearing premise

The paper assumes the full pretrained model is already mapped into the on-chip ReRAM crossbars before inference or fine-tuning begins; if the model exceeds that capacity, the modeled speedup and energy numbers would not cover the off-chip traffic needed to fetch weights.

Editorial extensions

If this is right

  • Transformer fine-tuning and inference can run on the same edge accelerator without server-class GPUs, since LoRA updates never write to ReRAM.
  • Quantized models become more energy-efficient on Atleus as bit-width shrinks, while GPU, TPU, and HBM-based baselines consume more energy under the same quantization because they dequantize before computing.
  • The rectangular 128x32 systolic array, chosen over square arrays, makes LoRA's low-rank matrices execute with higher utilization, which matters for fine-tuning workloads.
  • The paper's analytical ratio $MM_{ReRAM}/MM_{systolic} \propto O(12 d_{model}/n)$ implies the architecture's compute split is tailored to long-sequence transformers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported speedups assume the entire pretrained model already resides in the on-chip ReRAM crossbars before execution; extending Atleus to models larger than the on-chip capacity would require modeling off-chip weight streaming, which is not included in the current evaluation.
  • Because the ReRAM-to-systolic ratio grows with $d_{model}/n$, Atleus should favor models with large hidden dimensions and moderate sequence lengths; a testable extension would sweep sequence length to find where the systolic tier becomes the bottleneck.
  • One could co-design the crossbar-wise quantization scales with the LoRA training loss rather than choosing scales per crossbar independently, potentially recovering some of the M4F4 perplexity loss.
  • Loading different LoRA adapters from DRAM to handle multiple tasks on the same chip is an implicit benefit; a concrete extension is to benchmark task-switching latency when only the adapter changes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes Atleus, a four-tier 3D heterogeneous manycore accelerator for transformer fine-tuning and inference on edge devices. Static-weight matrix multiplications (MHA-1, MHA-4, FF-1, FF-2) are mapped to ReRAM crossbars, while dynamic attention computations, normalization, softmax, and LoRA adapters are mapped to rectangular systolic arrays. The paper contributes an intra-layer pipeline across ReRAM and systolic stages, a hybrid SFC/mesh NoC with TSV skip links, crossbar-wise quantization with on-tile dequantization, and a noise-aware fine-tuning procedure. Using cycle-accurate simulations (SCALE-Sim, NeuroSim, BookSim2), the authors report up to 56x speedup and 64.5x energy efficiency over HAIMA, 3D-TPU, and GPU baselines on Roberta-Base, BERT-Large, GPT-2 Medium, and BLOOM-560m.

Significance. The central idea of using 3D heterogeneity to exploit the static/dynamic computational split in transformers is timely, and, if the simulation assumptions hold, the architecture could be a meaningful step toward on-device personalization of foundation models. The paper has concrete strengths: it gives a cycle-accurate component-level evaluation, a cost model for 3D integration, a detailed NoC design with topology comparison, a quantization scheme with crossbar-level dequantization, and a noise-aware fine-tuning evaluation. However, the headline results currently rest on an unverified storage-capacity premise: the on-chip ReRAM capacity implied by Table IV is smaller than the parameter counts of three of the four evaluated models. Because the speedup and energy claims depend on static weights being resident on crossbars, the published numbers are not yet convincing as stated.

major comments (3)
  1. [Section V-A and Table IV] The assumption that 'the pre-trained model parameters are mapped to ReRAM crossbars prior to inferencing or fine-tuning' is not satisfied for three of the four benchmarks. Table IV gives 3 ReRAM tiers x 16 cores x 16 tiles x 96 crossbars x 128 x 128 cells x 2 bits/cell = 2,415,919,104 bits = 301,989,888 8-bit weights under the paper's M8F8 quantization. BERT-Large (~340M), GPT-2 Medium (~355M), and BLOOM-560m (~560M) all exceed this capacity. If only the MHA-1/MHA-4/FF-1/FF-2 matrices of a 24-layer d_model=1024 model are stored, their count is exactly 301,989,888, leaving zero slack for embeddings, biases, LayerNorm parameters, or crossbar-wise quantization scale metadata; embeddings alone exceed the residual capacity for GPT-2 and BLOOM. The latency and energy model in Section V-A accounts for DRAM weight loading for systolic arrays but not for ReRAM, so the reported speedups and energy gains implicitly assume zero off-chip weight traffic for these models. This capacity premise is load-bearing for the central claim and must be either satisfied by the modeled workloads or explicitly modeled as off-chip traffic in the cycle and energy estimates.
  2. [Section V-C, Eq. (5), and Fig. 7] The blanket claim that ReRAM performs between 90.08% and 94.7% of the computations needs to be reconciled with the paper's own asymptotic ratio. Equation (5) gives MM_ReRAM / MM_systolic proportional to 12 * d_model / n, so the ReRAM fraction depends directly on the sequence length n. For a model with d_model=1024 evaluated at n=2048, the asymptotic fraction would be 12/13, or about 92.3%? No: the fraction of ReRAM operations is 12*d/n divided by (12*d/n + 1), which for d=1024 and n=2048 is 6/7, about 85.7%. The manuscript should state the sequence length used for each model and report the per-model ReRAM fraction; the reported range 90.08-94.7% is not derivable from Eq. (5) for all listed models unless all are evaluated at sufficiently short sequences.
  3. [Section V-F and Figs. 10-11] The end-to-end comparison does not report raw latency and energy numbers, the pipeline fill and drain overhead, or the exact quantization configuration used for the main performance results. Figure 10 reports only stage delays, and Figs. 11 and 15 report only normalized values. Without these details, the reader cannot reconstruct the claimed 56x speedup and 64.5x energy efficiency, verify that the Atleus and baseline configurations are workload-equivalent, or determine whether the maximum values come from the same configuration and model. Please report the underlying data and specify the quantization setting (16-bit baseline or M8F8) used for each claimed maximum.
minor comments (6)
  1. [Section V-D] The cost model states 'phi_wafer = 300 nm'; this should be '300 mm', since a 300 nm wafer diameter is physically impossible and would change the die cost calculations.
  2. [Section V] There are two subsections labeled 'F' ('Performance Analysis' and 'Quantization Evaluation'); the second should be renumbered to avoid confusion.
  3. [Section V-F] In the paragraph describing Fig. 10, 'Alteus' is a typo for 'Atleus'.
  4. [Abstract and Conclusion] The abstract attributes the gains to 'state-of-the-art' while the conclusion specifies 'compared to GPU'; please state the exact baseline and configuration that produce the 56x and 64.5x maxima in one place.
  5. [Overall reproducibility] No simulation scripts, configuration files, or traffic traces are released. Since all results are simulator-generated, releasing the BookSim2 connectivity and traffic inputs, the SCALE-Sim and NeuroSim configurations, and the model workload descriptions would materially help reproducibility.
  6. [Section V-E] The noise-aware fine-tuning evaluation reports accuracy only for Roberta-Base and BERT-Large; the robustness of GPT-2 and BLOOM under ReRAM noise is not demonstrated, so the claim of general fine-tuning support is only partially supported.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the claimed speedups and energy gains are produced by cycle-accurate simulation against external baselines, and the ReRAM/systolic computational split is an asymptotic-complexity consistency check rather than a fitted parameter renamed as a prediction.

full rationale

The paper's central results are not definitionally forced. The end-to-end latency and energy numbers come from cycle-accurate simulators (NeuroSim for ReRAM, SCALE-Sim for systolic arrays, BookSim2 for the NoC) and are compared against independently configured baselines: HAIMA, a 3D-TPU-style systolic array, and an NVIDIA V100 GPU. The ReRAM-to-systolic workload ratio is derived analytically in Eq. 5 from the asymptotic complexities of Eqs. 2 and 3, then confirmed in simulation as 91.9% for GPT-2; this is a consistency check, not a fitting procedure. The systolic array size and shape are selected by design-space exploration and then evaluated, not used as a fitted input to produce the headline speedups. The crossbar-wise quantization scheme is an architectural extension of QLoRA's block-wise quantization, implemented with added shift-and-add hardware and simulated, so it is not a renaming of a known result. The main caveat is the explicit modeling assumption in Section V-A that 'the pre-trained model parameters are mapped to ReRAM crossbars prior to inferencing or fine-tuning'; this is a capacity and correctness concern for the larger models listed, but it is an assumption about the input configuration, not a circular derivation. References to prior work, including possibly same-group work on SFC-based NoCs and noise-aware training, are used as design precedents that are re-implemented and re-simulated in this paper, so they are not load-bearing self-citations that force the conclusions. Overall, no circular step can be identified from the paper's own equations or construction.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The central design rests on standard architectural assumptions (LoRA freezing weights, pre-mapped weights, deterministic pipeline balance) rather than on fitted statistical parameters. The main free parameters are architectural sizing choices tuned to the workload, so they are not independent evidence. The capacity assumption is the least supported.

free parameters (5)
  • Systolic array grid size = 128x32 (4096 PEs)
    Chosen experimentally to balance the intra-layer pipeline; directly determines achievable throughput and all latency/energy numbers.
  • Core allocation ratio = 3:1 ReRAM to systolic (48 ReRAM cores, 16 systolic cores)
    Derived from the asymptotic MM ratio, but enforced as a design choice; the 4-stage pipeline and speedup depend on it.
  • Pipeline stage count = 4 stages (S1-S4)
    Partition of a transformer layer into ReRAM, systolic, ReRAM, ReRAM stages; changing this changes load balancing and results.
  • ReRAM crossbar size and cell resolution = 128x128, 2-bit/cell
    Adopted from prior ISAAC-style tiles; determines quantization block size and on-chip capacity, and is load-bearing for the model-fit assumption.
  • Quantization configuration = M8F8 and M8F4 for reported energy/perplexity
    The paper selects configurations that keep perplexity stable based on the experimental outcomes; M4F4 is dropped from the final recommendation.
assumptions (6)
  • standard math Transformer kernel complexity expressions in Table II (O(d_model^2 n) for linear layers, O(d_model n^2) for attention) are correct and standard.
    Used to derive the ReRAM vs systolic workload split in Eqs. 2-5.
  • domain assumption LoRA freezes the pretrained weights, so ReRAM crossbars need not be rewritten during fine-tuning.
    Basis for the claim that ReRAM endurance is no longer a bottleneck; stated in Section IV-A.
  • domain assumption The full pretrained model is mapped to ReRAM crossbars before execution and stays there.
    Stated in Section V-A; necessary for the energy/latency results, but capacity is never checked.
  • ad hoc to paper The intra-layer pipeline can be balanced with deterministic latencies and no synchronization stalls across heterogeneous cores.
    The design uses in-order execution and a deterministic model; no full-system co-simulation with all components is shown.
  • ad hoc to paper ReRAM peripheral modifications (extra S&A units, scale registers) add only 2.15% area and 1.5% power overhead.
    The overhead figures come from a modified NeuroSim model and are used to argue quantization energy gains.
  • domain assumption Crossbar-wise dequantization after MVM preserves accuracy for the recommended M8F8/M8F4 configurations.
    Supported by perplexity experiments for two models, but M4F4 fails, showing the approach degrades at 4-bit.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Atleus: Accelerating Transformers on the Edge Enabled by 3D Heterogeneous Manycore Architectures." pith.science (2026). https://pith.science/paper/FX6OYQBI

@misc{pith2026250109588,
  author       = {Pith},
  title        = {Pith review of: Atleus: Accelerating Transformers on the Edge Enabled by 3D Heterogeneous Manycore Architectures},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FX6OYQBI}},
  note         = {Machine review of arXiv:2501.09588}
}
read the original abstract

Transformer architectures have become the standard neural network model for various machine learning applications including natural language processing and computer vision. However, the compute and memory requirements introduced by transformer models make them challenging to adopt for edge applications. Furthermore, fine-tuning pre-trained transformers (e.g., foundation models) is a common task to enhance the model's predictive performance on specific tasks/applications. Existing transformer accelerators are oblivious to complexities introduced by fine-tuning. In this paper, we propose the design of a three-dimensional (3D) heterogeneous architecture referred to as Atleus that incorporates heterogeneous computing resources specifically optimized to accelerate transformer models for the dual purposes of fine-tuning and inference. Specifically, Atleus utilizes non-volatile memory and systolic array for accelerating transformer computational kernels using an integrated 3D platform. Moreover, we design a suitable NoC to achieve high performance and energy efficiency. Finally, Atleus adopts an effective quantization scheme to support model compression. Experimental results demonstrate that Atleus outperforms existing state-of-the-art by up to 56x and 64.5x in terms of performance and energy efficiency respectively

Figures

Figures reproduced from arXiv: 2501.09588 by the authors.

Figure 1
Figure 1. High-level illustration of the transformer encoder with low-rank adapters added to the MHA module for fine-tuning. w A B X d d d d d r r d Back Propagation based on Error Weight Update Frozen Feed Forward Add & Norm Add & Norm Multi-Head Attention LoRA Encoder 1 X Loss function [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 42 canonical work pages

  1. [1]

    A Survey of Transformers,

    T. Lin, Y. Wang, X. Liu and X. Qiu, "A Survey of Transformers," in ArXiv preprint ArXiv:2106.04554, 2021

    arXiv 2021

  2. [2]

    PaLM: Scaling Language Modeling with Pathways,

    A. Chowdhery et al., "PaLM: Scaling Language Modeling with Pathways," arXiv preprint arXiv:2204.02311, 2022

    arXiv 2022

  3. [3]

    Language Models are Few -Shot Learners,

    T. B. Brown et al., "Language Models are Few -Shot Learners," arXiv preprint arXiv:2005.14165, 2020

    arXiv 2005

  4. [4]

    Finetuned Language Models Are Zero -Shot Learners,

    J. Wei et al., "Finetuned Language Models Are Zero -Shot Learners," arXiv preprint arXiv:2109.01652, 2022

    arXiv 2022

  5. [5]

    QLoRA: Efficient Finetuning of Quantized LLMs,

    T. Dettmers, A. Pagnoni, A. Holtzman and L. Zettlemoyer, "QLoRA: Efficient Finetuning of Quantized LLMs," in arXiv preprint arXiv:2305.14314, 2023

    arXiv 2023

  6. [6]

    LoRA: Low -Rank Adaptation of Large Language Models,

    E. J. Hu et al., "LoRA: Low -Rank Adaptation of Large Language Models," in arXiv preprint arXiv:2106.09685, 2021

    arXiv 2021

  7. [7]

    Few -Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In -Context Learning,

    H. Liu et al., "Few -Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In -Context Learning," arXiv preprint arXiv:2205.05638, 2022

    arXiv 2022

  8. [8]

    Parameter -Efficient Transfer Learning for NLP,

    N. Houlsby et al., "Parameter -Efficient Transfer Learning for NLP," arXiv preprint arXiv:1902.00751, 2019

    arXiv 1902

Show all 58 references
  1. [9]

    PANTHER: A Programmable Architecture for Neural Network Training Harnessing Energy -Efficient ReRAM,

    A. Ankit et al., "PANTHER: A Programmable Architecture for Neural Network Training Harnessing Energy -Efficient ReRAM," IEEE TC, 2020

    2020

  2. [10]

    Architectures and Circuits for Analog -memory-based Hardware Accelerators for Deep Neural Networks,

    H. Tsai et al., "Architectures and Circuits for Analog -memory-based Hardware Accelerators for Deep Neural Networks," ISCAS, 2023

    2023

  3. [11]

    On Endurance of Processing in (Nonvolatile) Memory,

    S. Resch et al., "On Endurance of Processing in (Nonvolatile) Memory," in ISCA, 2023

    2023

  4. [12]

    X -Former: In- Memory Acceleration of Transformers,

    S. Sridharan, J. Stevens, K. Roy and A. Raghunathan, "X -Former: In- Memory Acceleration of Transformers," IEEE TVLSI, 2023

    2023

  5. [13]

    Noise Injection Adaption: End -to-End ReRAM Crossbar Non-ideal Effect Adaption for Neural Network Mapping,

    Z. He et al., "Noise Injection Adaption: End -to-End ReRAM Crossbar Non-ideal Effect Adaption for Neural Network Mapping," in DAC, 2019

    2019

  6. [14]

    Temperature -Resilient RRAM -Based In -Memory Computing for DNN Inference,

    J. Meng et al., "Temperature -Resilient RRAM -Based In -Memory Computing for DNN Inference," in IEEE Micro, 2022

    2022

  7. [15]

    Fault -Free: A Framework for Analysis and Mitigation of Stuck -at-Fault on Realistic ReRAM -Based DNN Accelerators,

    H. Shin, M. Kang and L. -S. Kim, "Fault -Free: A Framework for Analysis and Mitigation of Stuck -at-Fault on Realistic ReRAM -Based DNN Accelerators," IEEE TC, vol. 72, pp. 2011-2024, 2023

    2011

  8. [16]

    Long live TIME: Improving lifetime for training -in- memory engines by structured gradient sparsification,

    Y. Cai et al., "Long live TIME: Improving lifetime for training -in- memory engines by structured gradient sparsification," in IEEE DAC, 2018

    2018

  9. [17]

    In -datacenter performance analysis of a tensor processing unit.,

    N. P. Jouppi et al., "In -datacenter performance analysis of a tensor processing unit.," ISCA, 2017

    2017

  10. [18]

    TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings,

    N. P. Jouppi et al., "TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings," arXiv prevprint arXiv:2304.01433, 2023. Figure 1 5: Normalized inference execution time when compared to the baseline Atleus for different transf...

    2023 arXiv

  11. [19]

    A survey of architectural approaches for improving GPGPU performance, programmability and heterogeneity,

    M. Khairy, A. G. Wassal and M. Zahran, "A survey of architectural approaches for improving GPGPU performance, programmability and heterogeneity," J. Parallel Distrib. Comput., vol. 127, p. 65–88, 2019

    2019

  12. [20]

    ISAAC: a convolutional neural network accelerator with in -situ analog arithmetic in crossbars.,

    A. Shafiee et al, "ISAAC: a convolutional neural network accelerator with in -situ analog arithmetic in crossbars.," in ISCA, Seoul, Korea, 2016

    2016

  13. [21]

    ReTransformer: ReRAM -based Processing-in-Memory Architecture for Transformer Acceleration,

    X. Yang, B. Yan, H. Li and Y. Chen, "ReTransformer: ReRAM -based Processing-in-Memory Architecture for Transformer Acceleration," in ICCAD, San Diego, 2020

    2020

  14. [22]

    AccelTran: A Sparsity -Aware Accelerator for Dynamic Inference with Transformers,

    S. Tuli and N. K. Jha, "AccelTran: A Sparsity -Aware Accelerator for Dynamic Inference with Transformers," IEEE TCAD, 2023

    2023

  15. [23]

    A Framework for Accelerating Transformer-Based Language Model on ReRAM -Based Architecture,

    M. Kang, H. Shin and L. -S. Kim, "A Framework for Accelerating Transformer-Based Language Model on ReRAM -Based Architecture," TCAD, 2022

    2022

  16. [24]

    FTRANS: Energy -Efficient Acceleration of Transformers Using FPGA,

    L. Bingbing et al., "FTRANS: Energy -Efficient Acceleration of Transformers Using FPGA," ACM, 2020

    2020

  17. [25]

    TransPIM: A Memory-based Acceleration via Software -Hardware Co -Design for Transformer,

    M. Zhou, W. Xu, J. Kang and T. Rosing, "TransPIM: A Memory-based Acceleration via Software -Hardware Co -Design for Transformer," in HPCA, Korea, 2022

    2022

  18. [26]

    HAIMA: A Hybrid SRAM and DRAM Accelerator -in- Memory Architecture for Transformer,

    Y. Ding et al., "HAIMA: A Hybrid SRAM and DRAM Accelerator -in- Memory Architecture for Transformer," in DAC, 2023

    2023

  19. [27]

    H3D -Transformer: A Heterogeneous 3D (H3D) Computing Platform for Transformer Model Acceleration on Edge Devices,

    Y. Luo and S. Yu, "H3D -Transformer: A Heterogeneous 3D (H3D) Computing Platform for Transformer Model Acceleration on Edge Devices," in ACM TOADES, 2024

    2024

  20. [28]

    SwiftTron: An efficient hardware accelerator for quantized transformers,

    A. Marchisio et al., "SwiftTron: An efficient hardware accelerator for quantized transformers," IJCNN, 2023

    2023

  21. [29]

    Prefix -Tuning: Optimizing Continuous Prompts for Generation,

    X. Lisa Li and P. Liang, "Prefix -Tuning: Optimizing Continuous Prompts for Generation," arXiv preprint aRxIv:2101.00190, 2021

    2021 arXiv

  22. [30]

    Exploring Versatile Generative Language Model Via Parameter -Efficient Transfer Learning,

    Z. Lin, A. Madotto and P. Fung, "Exploring Versatile Generative Language Model Via Parameter -Efficient Transfer Learning," arXiv preprint arXiv:2004.03829

    2004 arXiv

  23. [31]

    BitFit: Simple Parameter- efficient Fine-tuning for Transformer-based Masked Language-models,

    E. Ben Zaken, S. Ravfogel and Y. Goldberg, "BitFit: Simple Parameter- efficient Fine-tuning for Transformer-based Masked Language-models," arXiv preprint arXiv:2106.10199, 2022

    2022

  24. [32]

    A systematic methodology for characterizing scalability of DNN accelerators using SCALE-sim,

    A. Samajdar et al., "A systematic methodology for characterizing scalability of DNN accelerators using SCALE-sim," in ISPASS, 2020

    2020

  25. [33]

    Small Pitch, High Aspect Ratio Via -Last TSV Module,

    S. Van Huylenbroeck et al., "Small Pitch, High Aspect Ratio Via -Last TSV Module," IEEE ECTC, pp. 43-49, 2016

    2016

  26. [34]

    System on integrated chips (SoIC(TM) for 3D heterogeneous integration,

    M. F. Chen, F. C. Chen, W. C. Chiou and D. C. H. Yu, " System on integrated chips (SoIC(TM) for 3D heterogeneous integration," ECTC, 2019

    2019

  27. [35]

    3D Packaging for Heterogeneous Integration,

    R. Agarwal et al., "3D Packaging for Heterogeneous Integration," ECTC, 2022

    2022

  28. [36]

    An Overview of the Development of a GPU with Integrated HBM on Silicon Interposer,

    C. -C. Lee et al., "An Overview of the Development of a GPU with Integrated HBM on Silicon Interposer," ECTC, 2016

    2016

  29. [37]

    https://ddr-phy.org/,

    "https://ddr-phy.org/," [Online]

  30. [38]

    PipeLayer: A Pipelined ReRAM-Based Accelerator for Deep Learning,

    L. Song, Q. Xuehai, H. Li and Y. Chen, "PipeLayer: A Pipelined ReRAM-Based Accelerator for Deep Learning," in HPCA, 2017

    2017

  31. [39]

    FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning,

    T. Dao, "FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning," in arXiv preprint arXiv:2307.08691, 2023

    2023 arXiv

  32. [40]

    Demystifying 3D ICs: the pros and cons of going vertical,

    W. R. Davis et al., "Demystifying 3D ICs: the pros and cons of going vertical," IEEE Design & Test of Computers, pp. 498-510, 2005

    2005

  33. [41]

    An overview of through-silicon-via technology and manufacturing challenges,

    J. P. Gambino, S. A. Adderly and J. U. Knickerbocker, "An overview of through-silicon-via technology and manufacturing challenges," Microelectronic Engineering, 2015

    2015

  34. [42]

    Florets for Chiplets: Data Flow -aware High - Performance and Energy -efficient Network -on-Interposer for CNN Inference Tasks,

    H. Sharma et al., "Florets for Chiplets: Data Flow -aware High - Performance and Energy -efficient Network -on-Interposer for CNN Inference Tasks," ACM Transactions on Embedded Computing Systems, 2023

    2023

  35. [43]

    A Survey of Design and Optimization for Systolic Array - based DNN Accelerators,

    R. Xu et al., "A Survey of Design and Optimization for Systolic Array - based DNN Accelerators," ACM Computing Surveys, 2023

    2023

  36. [44]

    8-bit Optimizers via Block-wise Quantization,

    T. Dettmers et al., "8-bit Optimizers via Block-wise Quantization," arXiv prefix arXiv:2110.02861, 2022

    2022 arXiv

  37. [45]

    HuggingFace’s Transformers: State -of-the-Art Natural Language Processing,

    T. Wolf et al., "HuggingFace’s Transformers: State -of-the-Art Natural Language Processing," arXiv preprint arXiv:1910.03771, 2019, 2019

    1910 arXiv

  38. [46]

    SQUAD: 100,000+Questions for Machine Comprehension of Text,

    P. Rajpurkar, J. Zhang, K. Lopyrev and P. Liang, "SQUAD: 100,000+Questions for Machine Comprehension of Text," arXiv preprint arXiv:1606.05250, 2016

    2016 arXiv

  39. [47]

    DNN+NeuroSim V2.0: An end -to-end benchmarking framework for compute -in-memory accelerators for on -chip training,

    X. Peng et al., "DNN+NeuroSim V2.0: An end -to-end benchmarking framework for compute -in-memory accelerators for on -chip training," IEEE TCAD, vol. 40, pp. 2306-2319, 2021

    2021

  40. [48]

    Detailed and Flexible Cycle-Accurate Network-on-Chip Simulator,

    N. Jiang et al., "Detailed and Flexible Cycle-Accurate Network-on-Chip Simulator," in ISPASS, 2013

    2013

  41. [49]

    https://huggingface.co/docs/peft/en/index,

    "https://huggingface.co/docs/peft/en/index," [Online]

  42. [50]

    Cost analysis and cost-driven IP reuse methodology for SoC design based on 2.5D/3D integration,

    D. Stow et al., "Cost analysis and cost-driven IP reuse methodology for SoC design based on 2.5D/3D integration," in ICCAD, 2016

    2016

  43. [51]

    Temperature - and Cost-Aware Design of 3D Multiprocessor Architectures,

    A. Coskun, A. Kahng and T. Rosing, "Temperature - and Cost-Aware Design of 3D Multiprocessor Architectures," in Euromicro Conference on Digital System Design, 2009

    2009

  44. [52]

    Cost Modeling and Analysis of TSV and Contactless 3D-ICs,

    M. Jiang, I. A. Papistas and V. F. Pavlidis, "Cost Modeling and Analysis of TSV and Contactless 3D-ICs," in GLSVLSI, 2020

    2020

  45. [53]

    Cost and Thermal Analysis of High -Performance 2.5D and 3D Integrated Circuit Design Space,

    D. Stow et al., "Cost and Thermal Analysis of High -Performance 2.5D and 3D Integrated Circuit Design Space," in IEEE Computer Society Annual Symposium on VLSI (ISVLSI), 2016

    2016

  46. [54]

    Negative Feedback Training: A Novel Concept to Improve Robustness of NVCiM DNN Accelerators,

    Y. Qin et al., "Negative Feedback Training: A Novel Concept to Improve Robustness of NVCiM DNN Accelerators," arXiv preprint arXiv:2305.14561, 2023

    2023 arXiv

  47. [55]

    Multi -objective optimization of ReRAM crossbars for robust DNN inferencing under stochastic noise,

    X. Yang et al., "Multi -objective optimization of ReRAM crossbars for robust DNN inferencing under stochastic noise," in IEEE/ACM ICCAD, 2021

    2021

  48. [56]

    Improving the robustness of analog deep neural networks through a Bayes -optimized noise injection approach,

    N. Ye et al., "Improving the robustness of analog deep neural networks through a Bayes -optimized noise injection approach," in Communications Engineering, 2023

    2023

  49. [57]

    AccuReD: High Accuracy Training of CNNs on ReRAM/GPU Heterogeneous 3D Architecture,

    B. K. Joardar et al., "AccuReD: High Accuracy Training of CNNs on ReRAM/GPU Heterogeneous 3D Architecture," IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2020

    2020

  50. [58]

    Newton: A DRAM -maker’s Accelerator -in-Memory (AiM) Architecture for Machine Learning,

    M. He et al., "Newton: A DRAM -maker’s Accelerator -in-Memory (AiM) Architecture for Machine Learning," in IEEE MICRO, 2020. Pratyush Dhingra (Graduate Student Member, IEEE) received the B.E degree in electronics and computer engineering from NSIT, Delhi University, Delhi, Ind...

    2020

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.