Pith. sign in

REVIEW 2 major objections 6 minor 74 references

LightNobel: Improving Sequence Length Limitation in Protein Structure Prediction Model via Adaptive Activation Quantization

T0 review · 2 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Token-wise quantization lets protein models fold sequences up to 9,945 residues long.

desk verdict A credible, well-executed PPM accelerator with one real methodological soft spot: the quantization scheme is chosen and evaluated on the same accuracy datasets. read the letter →

arxiv 2505.05893 v1 pith:76V6ICWB submitted 2025-05-09 cs.AR cs.AIcs.ETcs.LGq-bio.BM

Seunghee Han , Soongyu Choi , Joo-Young Kim This is my paper · ORCID
classification cs.ARcs.AIcs.ETcs.LGq-bio.BM
keywords proteinstructurepredictionactivationquantizationtoken-wiseoutlierhandlinghardware-softwareco-designsequencelengthscalabilitypairrepresentationattention-basedmodels
topics Protein Folding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that the real obstacle to scaling protein-structure prediction to long amino-acid sequences is not model weights but the activation tensors of the pair representation, which grow quadratically with sequence length and dominate both memory and latency. Its proposed fix, Token-wise Adaptive Activation Quantization (AAQ), quantizes each token of these activations separately, using 4- or 8-bit inliers plus 16-bit outliers whose count is chosen at runtime, and pairs the scheme with accelerator hardware that can execute that mixed-precision dataflow. If the paper is right, a single 80 GB system can fold proteins up to 9,945 residues long, roughly 1.45 times longer than the longest CASP16 target, while keeping TM-Score loss below 0.001 and cutting peak memory by up to 120.05 times. That matters because long proteins and multimers are precisely the cases where current models run out of memory or pay heavy chunking overhead.

What carries the argument

The load-bearing object is Token-wise Adaptive Activation Quantization (AAQ): uniform symmetric quantization applied per token, where a token is the 128-element vector in the pair representation associated with one residue pair position, and each token has its own runtime-computed scaling factor. A dynamic top-k selection finds the outliers in each token, so the number of 16-bit outliers varies at runtime; inliers are 4- or 8-bit depending on which of three activation groups the tensor belongs to. On the hardware side, the Reconfigurable Matrix Processing Unit (RMPU) uses bit-level reconfiguration so that 4-bit inliers and 16-bit outliers feed the same multiplier/adder tree without per-value dequantization, and the Versatile Vector Processing Unit (VVPU) performs layer norms, softmax, runtime quantization, and bitonic top-k sorting. The argument works because token-wise grouping matches the distogram structure of the activations, and the small hidden dimension (128) makes per-token top-k affordable.

What would settle it

Take a protein of roughly 9,945 residues with a known structure, run the unquantized baseline and the AAQ-quantized model, and compare TM-Scores; if the difference exceeds 0.001, the central length-extension claim fails. A cheaper proxy is to instrument the pair-representation activations at that length and count outliers per token with the 3-sigma rule: if Group B tokens regularly have more than four outliers, the selected outlier budget is already insufficient.

Watch

Extended reading notes

Core claim

The central claim is that PPM activations have token-wise structure: the pair representation stores a distogram-like interaction pattern, so value ranges and outliers are similar across channels but differ strongly from token to token. AAQ exploits this by giving each token its own scaling factor, isolating a dynamically chosen number of extreme values (outliers) with a top-k operation, and quantizing the remaining inliers to INT4 or INT8 while weights stay at 16-bit fixed point. Based on measured activation statistics, the paper assigns three quantization schemes: Group A (pre-LayerNorm, residual-connected activations with large values) uses 8-bit inliers with 4 outliers; Group B (post-LayerNorm activations) uses 4-bit inliers with 4 outliers; Group C (multiplication outputs with few outliers) uses 4-bit inliers with no outlier handling. On CAMEO, CASP14, and CASP15 the averaged TM-Score changes by less than 0.001 relative to the unquantized baseline, while the accelerator claims up to 8.44x lower latency and 37.29x higher power efficiency than the compared GPUs, and 120.05x lower peak memory. The paper also shows that token-wise multi-head attention avoids materializing the full score matrix, which is part of the same memory reduction.

Load-bearing premise

The quantization budgets chosen by design-space exploration were validated for accuracy only up to 3,364 residues; the claim that 9,945-residue proteins run with TM-Score loss under 0.001 assumes the token-wise activation statistics and outlier counts stay the same at that length.

Editorial extensions

If this is right

  • Proteins up to 9,945 residues fit in an 80 GB memory budget, versus 6,879 for the longest CASP16 target, removing the current length ceiling for a single-node fold.
  • Peak memory for the PPM drops by up to 120.05x without chunking, and by 1.26-5.05x even against chunked execution, so chunking's kernel overhead and recomputation can be avoided.
  • Latency improves up to 8.44x over the A100 and 8.41x over the H100, with power efficiency up to 37.29x and 43.35x, under the paper's simulation methodology.
  • The average computational cost falls 43.38% and memory footprint 74.10% vs the baseline, because scaling factors are applied once per token instead of repeatedly during matrix multiplication.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Accuracy is verified only on proteins up to 3,364 residues; if a 9,945-residue protein produces more than 4 outliers in Group B tokens or wider value ranges, the chosen outlier budgets might need to grow, reducing the advertised memory savings.
  • A testable extension is to apply the same token-wise outlier budget idea to other pairwise or grid-structured attention tensors, such as 2D geometry or molecular interaction maps, where a distogram-like regularity may also hold.
  • The accelerator's claimed speedups cover the Protein Folding Block; for short proteins the input-embedding and structure-module time, which run on the CPU/GPU baseline, may dilute end-to-end gains, so the headline speedups are a best case for full inference.
  • A direct check of the memory claims would run the quantized folding block on a real 80 GB system at 9,945 residues and record whether peak usage indeed stays under budget without chunking.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper proposes LightNobel, a hardware-software co-designed accelerator for protein structure prediction models (PPMs), targeting the memory and latency bottleneck caused by the quadratic/cubic growth of pair-representation activations. At the software level, it introduces Token-wise Adaptive Activation Quantization (AAQ), which groups PPM activations into three classes with different inlier precisions and outlier counts, using token-wise scaling factors and dynamic top-k outlier selection. At the hardware level, LightNobel combines reconfigurable matrix processing units (RMPUs), versatile vector processing units (VVPUs), crossbar networks, and a token-wise multi-head attention dataflow to execute the multi-precision, dynamically quantized computations. The paper reports a TM-Score change below 0.001 relative to baseline ESMFold on CAMEO/CASP14/CASP15, speedups up to 8.44x over A100 and 8.41x over H100, peak memory reduction up to 120.05x, and support for sequence lengths up to 9,945 within 80 GB, based on a cycle-accurate simulator cross-validated against RTL synthesis at 28nm.

Significance. If the claims hold after revision, this is a strong ISCA-class systems contribution. It addresses a real and growing bottleneck in protein structure inference, proposes a quantization scheme grounded in the token-wise distogram characteristics of PPM activations, and provides substantial hardware backing: RTL synthesis at 28nm, a cycle-accurate simulator cross-validated against RTL with an average discrepancy of 3.30%, detailed area/power analysis, and evaluation on standard CAMEO/CASP benchmarks. The accuracy benchmark uses external PDB ground truth, so the core TM-Score comparison is not circular in the sense of being defined by the proposed method. The strongest elements are the memory analysis, the token-wise MHA dataflow that avoids materializing full score matrices, and the explicit reporting of simulator cross-validation. The main risks are the selection-evaluation overlap in the AAQ configuration and the extrapolation of accuracy and memory claims to sequence lengths beyond those directly measured.

major comments (2)
  1. [Section 7.1 and Section 8.1] The AAQ quantization schemes for Groups A, B, and C are selected by design-space exploration in Section 7.1 using TM-Score on CAMEO, CASP14, and CASP15, and the same three datasets are then used in Figure 13 to support the central claim that AAQ preserves TM-Score within 0.001. This is a selection-on-evaluation-set protocol: the reported accuracy loss is not an independent estimate of generalization. Please re-run the evaluation on a held-out split (for example, CASP16 once ground truth is available, or a per-protein cross-validation), and define the efficiency objective in Figure 11 as an explicit equation so the selection rule is transparent.
  2. [Section 8.3 and Figure 15(b)] The claim that LightNobel supports sequence lengths up to 9,945 within 80 GB is based on extrapolation of peak-memory estimates from shorter proteins, and no TM-Score accuracy is reported beyond the 3,364-residue CASP15 target. The accuracy-preservation claim is therefore not demonstrated in the advertised long-sequence regime, where outlier statistics could differ. Please either add accuracy measurements on long proteins or explicitly limit the accuracy claim to the tested length range and present the 9,945 figure as a memory-capacity projection.
minor comments (6)
  1. [Section 8.1 and Figure 13] The sentence attributing significant TM-Score degradation to both Tender and MeFold appears inconsistent with the values in Figure 13, where only one method is clearly below baseline; please verify the label-to-bar mapping.
  2. [Section 7.1] The efficiency metric in Figure 11 is described only verbally; please define it as an equation so the design-space trade-off is reproducible.
  3. [Section 6] Please report the simulator discrepancy as a range (minimum and maximum) rather than only the average, and state how the 3.30% average discrepancy translates into uncertainty on the reported speedup factors.
  4. [Sections 4.2 and 7.1] The text says the number of outliers k is set adaptively at runtime, but the design-space exploration fixes k per group (4, 4, 0); please clarify whether k is a per-activation-type constant or varies per token at runtime.
  5. [Section 8.3 and Table 1] The relationship between the 120.05x memory reduction in Figure 15(a) and the much smaller reduction implied by Table 1 (which excludes token-wise MHA) should be stated explicitly to avoid confusion.
  6. [Throughout] There are minor consistency issues: 'archives' should be 'achieves' in Section 8.2, and 'MEFold' and 'MeFold' are used interchangeably.

Circularity Check

1 steps flagged · score 6.0 of 10

AAQ schemes are picked using TM-Score on CAMEO/CASP14/CASP15 and then 'validated' on those same datasets, making the <0.001 accuracy-loss claim selection-dependent.

  1. fitted input called prediction [Section 7.1 (AAQ Quantization Scheme, Fig. 11) and Section 8.1 (Accuracy Evaluation, Fig. 13)]
    "Since accuracy evaluation requires ground truth data, we use CAMEO [9], CASP14 [11], and CASP15 [12] datasets for the experiment."

    Section 7.1 selects the deployed per-group AAQ schemes (Group A: 8-bit with 4 outliers; Group B: 4-bit with 4 outliers; Group C: 4-bit with 0 outliers) on CAMEO/CASP14/CASP15, using TM-Score as part of the efficiency objective: 'Efficiency is calculated by considering the memory size of the quantized tokens and the resulting TM-Score for each configuration.' Section 8.1 then evaluates the resulting AAQ on the same three datasets and reports 'a negligible TM-Score change of less than 0.001'. The configuration was chosen precisely because it preserved TM-Score on those datasets, so the final accuracy number is a restatement of the selection criterion rather than an out-of-sample prediction.

full rationale

The hardware and performance claims are not circular: the cycle-accurate simulator is cross-validated against RTL simulation, area/power come from synthesis and memory tools, and the baselines are standard GPUs evaluated on public datasets. No load-bearing self-citation chain, imported uniqueness theorem, or ansatz-smuggling citation was found. However, the central 'negligible accuracy loss' claim for AAQ is partially circular: the quantization schemes are selected by design-space exploration whose objective includes TM-Score on exactly the datasets later used for the accuracy evaluation. Reporting a TM-Score drop of less than 0.001 on those same datasets is therefore an expected consequence of the selection procedure, not an independent verification. The accuracy metric itself is grounded in external PDB/TM-Score ground truth, so this is not a fully definitional circularity, but the quantitative claim is not validated on held-out proteins.

Assumptions & free parameters 8 free parameters · 4 assumptions · 0 invented entities

The accuracy and performance claims depend on empirically chosen quantization hyperparameters (bit-widths and outlier counts per activation group), on the heuristic 3-sigma outlier rule, and on the representativeness of ESMFold for PPM. The long-sequence scalability claim (9,945) additionally assumes these settings transfer to proteins much longer than those used in accuracy evaluation. No new physical entities are introduced.

free parameters (8)
  • Group A inlier bit-width = 8 bits
    Selected by DSE on CAMEO/CASP14/CASP15 (Section 7.1, Figure 11a) as best efficiency for activations before LayerNorm with residual connections.
  • Group A outlier count = 4 outliers
    DSE-selected number of top-k outliers handled per token for Group A (Figure 11a).
  • Group B inlier bit-width = 4 bits
    DSE-selected for activations after LayerNorm (Figure 11b).
  • Group B outlier count = 4 outliers
    DSE-selected for Group B (Figure 11b).
  • Group C inlier bit-width = 4 bits
    DSE-selected for activations with small range and few outliers (Figure 11c).
  • Group C outlier count = 0 outliers
    DSE-selected; no outlier handling for Group C (Figure 11c).
  • Number of RMPUs = 32
    Selected by hardware DSE as the saturation point for average latency (Section 7.2, Figure 12b).
  • Number of VVPUs per RMPU = 4
    Selected by hardware DSE (Section 7.2, Figure 12a).
assumptions (4)
  • domain assumption PPM Pair Representation activations have small variance across channels and large variance across tokens, with outliers concentrated in specific tokens (distogram pattern)
    Motivates token-wise quantization. Based on sampled proteins in Section 3.3, Figure 5; not proven to hold for all proteins or all sequence lengths.
  • domain assumption Outliers are identified by the 3-sigma rule
    Used throughout Sections 4.1 and 4.2 to classify outlier values; a heuristic threshold, not a derived property.
  • domain assumption ESMFold is representative of the PPM Pair Representation dataflow shared with AlphaFold2
    Section 6 states AlphaFold2 and ESMFold share the same Pair Representation dataflow, which is the target of AAQ; the paper evaluates only on ESMFold.
  • domain assumption The cycle-accurate simulator and RTL faithfully model the real hardware
    Section 6: the Python simulator is cross-validated against RTL within 5%, but no fabricated chip measurement is provided; energy and area come from 28nm synthesis and Cacti 7.0 with scaling.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LightNobel: Improving Sequence Length Limitation in Protein Structure Prediction Model via Adaptive Activation Quantization." pith.science (2026). https://pith.science/paper/76V6ICWB

@misc{pith2026250505893,
  author       = {Pith},
  title        = {Pith review of: LightNobel: Improving Sequence Length Limitation in Protein Structure Prediction Model via Adaptive Activation Quantization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/76V6ICWB}},
  note         = {Machine review of arXiv:2505.05893}
}
read the original abstract

Recent advances in Protein Structure Prediction Models (PPMs), such as AlphaFold2 and ESMFold, have revolutionized computational biology by achieving unprecedented accuracy in predicting three-dimensional protein folding structures. However, these models face significant scalability challenges, particularly when processing proteins with long amino acid sequences (e.g., sequence length > 1,000). The primary bottleneck that arises from the exponential growth in activation sizes is driven by the unique data structure in PPM, which introduces an additional dimension that leads to substantial memory and computational demands. These limitations have hindered the effective scaling of PPM for real-world applications, such as analyzing large proteins or complex multimers with critical biological and pharmaceutical relevance. In this paper, we present LightNobel, the first hardware-software co-designed accelerator developed to overcome scalability limitations on the sequence length in PPM. At the software level, we propose Token-wise Adaptive Activation Quantization (AAQ), which leverages unique token-wise characteristics, such as distogram patterns in PPM activations, to enable fine-grained quantization techniques without compromising accuracy. At the hardware level, LightNobel integrates the multi-precision reconfigurable matrix processing unit (RMPU) and versatile vector processing unit (VVPU) to enable the efficient execution of AAQ. Through these innovations, LightNobel achieves up to 8.44x, 8.41x speedup and 37.29x, 43.35x higher power efficiency over the latest NVIDIA A100 and H100 GPUs, respectively, while maintaining negligible accuracy loss. It also reduces the peak memory requirement up to 120.05x in PPM, enabling scalable processing for proteins with long sequences.

Figures

Figures reproduced from arXiv: 2505.05893 by the authors.

Figure 1
Figure 1. Visualization of Protein Structure Prediction Model [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of PPM. (a) Block diagram of the PPM. (b) Dataflow of Protein Folding Block (1 Block). [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Latency Breakdown of PPM with (a) protein R0271 [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: Analysis of total weight size and peak activation [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Analysis of activation value distribution in PPM. [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]
Figure 6
Figure 6. Figure 6: Dataflow of (a) Triangular Multiplication Block and (b) Triangular Attention Block. Red, blue, and green lines [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Memory layout of the quantized tokens. precision can be assigned to inliers without compromising quan￾tization accuracy. Finally, Group C consists of activations that do not belong to the previous two groups. These activations undergo multiplication with relatively sma…
Figure 8
Figure 8. Figure 8: Overall block diagram of LightNobel architecture. [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: Microarchitecture of (a) Processing Element (PE), (b) PE Lane, (c) PE Cluster, (d) RMPU Engine, and (e) Dynamic [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: shows the microarchitecture of VVPU. The VVPU comprises multiple SIMD Lanes, a Scalar Support Unit (SSU), and a Local Crossbar Network (LCN). Each SIMD Lane includes a SIMD Core with an ALU that can process operations between two 16-bit operands for weights, scratchpa…
Figure 11
Figure 11. Figure 11: Design space exploration on quantization scheme for (a) Group A, (b) Group B, and (c) Group C. [PITH_FULL_IMAGE:figures/full_fig_p010_11.png]
Figure 12
Figure 12. Figure 12: Design space exploration on hardware configura [PITH_FULL_IMAGE:figures/full_fig_p010_12.png]
Figure 13
Figure 13. Figure 13: Accuracy evaluation result across datasets. [PITH_FULL_IMAGE:figures/full_fig_p011_13.png]
Figure 14
Figure 14. Figure 14: (a) End-to-end performance evaluation result across various recent PPMs, and hardware performance evaluation [PITH_FULL_IMAGE:figures/full_fig_p012_14.png]
Figure 15
Figure 15. Figure 15: Peak memory requirement of PPM across (a) [PITH_FULL_IMAGE:figures/full_fig_p012_15.png]
Figure 16
Figure 16. Figure 16: (a) Computational cost of PPM and (b) memory [PITH_FULL_IMAGE:figures/full_fig_p012_16.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

74 extracted references · 56 canonical work pages

  1. [1]

    IEEE Standard for SystemVerilog–Unified Hardware Design, Specification, and Verification Language.IEEE Std 1800-2023 (Revision of IEEE Std 1800-2017) (2024), 1–1354

    2024. IEEE Standard for SystemVerilog–Unified Hardware Design, Specification, and Verification Language.IEEE Std 1800-2023 (Revision of IEEE Std 1800-2017) (2024), 1–1354

    work page 2024

  2. [2]

    Ballard, Joshua Bambrick, Sebastian W

    Josh Abramson, Jonas Adler, Jack Dunger, Richard Evans, Tim Green, Alexander Pritzel, Olaf Ronneberger, Lindsay Willmore, Andrew J. Ballard, Joshua Bambrick, Sebastian W. Bodenstein, David A. Evans, Chia-Chun Hung, Michael O’Neill, David Reiman, Kathryn Tunyasuvunakool, Zachary Wu, Akvil˙e Žemgulyt˙e, Eirini Arvaniti, Charles Beattie, Ottavia Bertolli, Al...

    work page 2024

  3. [3]

    O’Donnell, Daniel Berenberg, Ian Fisk, Niccolò Zanichelli, Bo Zhang, Arkadiusz Nowaczynski, Bei Wang, Marta M

    Gustaf Ahdritz, Nazim Bouatta, Christina Floristean, Sachin Kadyan, Qinghui Xia, William Gerecke, Timothy J. O’Donnell, Daniel Berenberg, Ian Fisk, Niccolò Zanichelli, Bo Zhang, Arkadiusz Nowaczynski, Bei Wang, Marta M. Stepniewska- Dziubinska, Shang Zhang, Adegoke Ojewole, Murat Efe Guney, Stella Biderman, Andrew M. Watkins, Stephen Ra, Pablo Ribalta Lor...

    work page 2024

  4. [4]

    Croci, Bo Li, Pash- mina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman

    Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L. Croci, Bo Li, Pash- mina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman. 2024. QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs. https://openreview.net/forum?id=dfqsW38v1X. InThe Thirty-eighth Annual Con- ference on Neural Information Processing Systems

    work page 2024

  5. [5]

    Kinch, R

    Minkyung Baek, Frank DiMaio, Ivan Anishchenko, Justas Dauparas, Sergey Ovchinnikov, Gyu Rie Lee, Jue Wang, Qian Cong, Lisa N. Kinch, R. Dustin Schaeffer, Claudia Millán, Hahnbeom Park, Carson Adams, Caleb R. Glassman, Andy DeGiovanni, Jose H. Pereira, Andria V. Rodrigues, Alberdina A. van Dijk, Ana C. Ebrecht, Diederik J. Opperman, Theo Sagmeister, Christ...

    work page 2021

  6. [6]

    Rajeev Balasubramonian, Andrew B Kahng, Naveen Muralimanohar, Ali Shafiee, and Vaishnav Srinivas. 2017. CACTI 7: New tools for interconnect exploration in innovative off-chip memories.ACM Transactions on Architecture and Code Optimization (TACO)14, 2 (2017), 1–25. LightNobel: Improving Sequence Length Limitation in Protein Structure Prediction Model via A...

    work page 2017

  7. [7]

    Mark Bohr. 2012. Silicon technology leadership for the mobility era. InIntel developer forum, Vol. 2012. Intel Senior Fellow

    work page 2012

  8. [8]

    Ewen Callaway. 2024. Chemistry Nobel goes to developers of AlphaFold AI that predicts protein structures. https://www.nature.com/articles/d41586-024-03214- 7

    work page 2024

Show all 74 references
  1. [9]

    CAMEO. 2024. CAMEO: Continuous Automated Model EvaluatiOn. https://www. cameo3d.org/

    2024

  2. [10]

    Protein Structure Prediction Center. 2007. CASP. https://predictioncenter.org/

    2007

  3. [11]

    Protein Structure Prediction Center. 2020. CASP14. https://predictioncenter.org/ casp14/

    2020

  4. [12]

    Protein Structure Prediction Center. 2022. CASP15. https://predictioncenter.org/ casp15/

    2022

  5. [13]

    Protein Structure Prediction Center. 2024. CASP16. https://predictioncenter.org/ casp16/

    2024

  6. [14]

    Shenggan Cheng, Xuanlei Zhao, Guangyang Lu, Jiarui Fang, Tian Zheng, Ruidong Wu, Xiwen Zhang, Jian Peng, and Yang You. 2024. FastFold: Optimizing AlphaFold Training and Inference on GPU Clusters. https://doi.org/10.1145/3627535.3638465. InProceedings of the 29th ACM SIGPLAN An...

    2024

  7. [15]

    NVIDIA Corporation. 2007. NVIDIA Nsight Systems. https://developer.nvidia. com/nsight-systems

    2007

  8. [16]

    Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. 2022. Flashat- tention: Fast and memory-efficient exact attention with io-awareness.Advances in Neural Information Processing Systems35 (2022), 16344–16359

    2022

  9. [17]

    Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer

  10. [18]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognit...

    2021 arXiv

  11. [19]

    Jacques Dubochet, Marc Adrian, Jiin-Ju Chang, Jean-Claude Homo, Jean Lepault, Alasdair W McDowall, and Patrick Schultz. 1988. Cryo-electron microscopy of vitrified specimens.Quarterly reviews of biophysics21, 2 (1988), 129–228

    1988

  12. [20]

    facebookresearch. 2007. esm. https://github.com/facebookresearch/esm

    2007

  13. [21]

    Fallon, Vikram V

    Timothy R. Fallon, Vikram V. Shende, Igor H. Wierzbicki, Amanda L. Pendlet on, Nathan F. Watervoort, Robert P. Auber, David J. Gonzalez, Jenni fer H. Wisecaver, and Bradley S. Moore. 2024. Giant polyketide syn- thase enzymes in the biosynthesis of giant marine polyether toxins...

    2024 doi

  14. [22]

    Python Software Foundation. 2007. python. https://www.python.org/

    2007

  15. [23]

    Walter Friedrich, Paul Knipping, and Max Laue. 1913. Interferenzerscheinungen bei roentgenstrahlen.Annalen der Physik346, 10 (1913), 971–988

    1913

  16. [24]

    Mu Gao and Jeffrey Skolnick. 2021. A general framework to learn tertiary structure for protein sequence characterization.Frontiers in bioinformatics1 (2021), 689960

    2021

  17. [25]

    Cong Guo, Jiaming Tang, Weiming Hu, Jingwen Leng, Chen Zhang, Fan Yang, Yunxin Liu, Minyi Guo, and Yuhao Zhu. 2023. Olive: Accelerating large language models via hardware-friendly outlier-victim pair quantization. InProceedings of the 50th Annual International Symposium on Com...

    2023

  18. [26]

    Oh, Yeonhong Park, Yoonho Song, Jung-Hun Park, Sanghee Lee, Kyoung Park, Jae W

    Tae Jun Ham, Sung Jun Jung, Seonghak Kim, Young H. Oh, Yeonhong Park, Yoonho Song, Jung-Hun Park, Sanghee Lee, Kyoung Park, Jae W. Lee, and Deog- Kyoon Jeong. 2020. Aˆ 3: Accelerating attention mechanisms in neural networks with approximation. In2020 IEEE International Symposi...

    2020

  19. [27]

    Seunghee Han, Seungjae Moon, Teokkyu Suh, JaeHoon Heo, and Joo-Young Kim

  20. [28]

    Wei Huang, Yangdong Liu, Haotong Qin, Ying Li, Shiming Zhang, Xianglong Liu, Michele Magno, and Xiaojuan Qi. 2024. Billm: Pushing the limit of post-training quantization for llms.arXiv preprint arXiv:2402.04291(2024)

    2024 arXiv

  21. [29]

    Wei Huang, Karthick Rajamani, Mircea R Stan, and Kevin Skadron. 2011. Scaling with design constraints: Predicting the future of big chips.IEEE Micro31, 4 (2011), 16–29

    2011

  22. [30]

    Intel INC. 2007. Intel Xeon Platinum 8452Y Processor. https://www.intel.com /content/www/us/en/products/sku/231761/intel-xeon-platinum-8452y- processor-67-5m-cache-2-00-ghz/specifications.html

    2007

  23. [31]

    Synopsys Inc. 2007. Design Compiler. https://www.synopsys.com/implemet ation-and-signoff/rtl-synthesis-test/design-compiler-graphical.html

    2007

  24. [32]

    Yanfeng Jiang, Ning Sun, Zhengxian Lu, Shuang Peng, Yi Zhang, Fei Yang, and Tao Li. 2024. MEFold: Memory-Efficient Optimization for Protein Language Models via Chunk and Quantization. In2024 International Joint Conference on Neural Networks (IJCNN). IEEE, Piscataway, NJ, USA, 1–8

    2024

  25. [33]

    John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, Alex Bridgland, Clemens Meyer, Simon A. A. Kohl, Andrew J. Ballard, Andrew Cowie, Bernardino Romera-Paredes, Stani...

    2021

  26. [34]

    Yoongu Kim, Weikun Yang, and Onur Mutlu. 2015. Ramulator: A fast and exten- sible DRAM simulator.IEEE Computer architecture letters15, 1 (2015), 45–49

    2015

  27. [35]

    Jungi Lee, Wonbeom Lee, and Jaewoong Sim. 2024. Tender: Accelerating Large Language Models via Tensor Decomposition and Runtime Requantization.arXiv preprint arXiv:2406.12930(2024), 1048–1062

    2024 arXiv

  28. [36]

    Hongbin Li, Wolfgang A Linke, Andres F Oberhauser, Mariano Carrion-Vazquez, Jason G Kerkvliet, Hui Lu, Piotr E Marszalek, and Julio M Fernandez. 2002. Reverse engineering of the giant muscle protein titin.Nature418, 6901 (2002), 998–1002

    2002

  29. [37]

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. 2024. AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration.Proceedings of Machine Learning and Systems6 (202...

    2024

  30. [38]

    Yujun Lin, Haotian Tang, Shang Yang, Zhekai Zhang, Guangxuan Xiao, Chuang Gan, and Song Han. 2024. Qserve: W4a8kv4 quantization and system co-design for efficient llm serving.arXiv preprint arXiv:2405.04532(2024)

    2024 arXiv

  31. [39]

    Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, Allan dos Santos Costa, Maryam Fazel-Zarandi, Tom Sercu, Salvatore Candido, and Alexander Rives. 2023. Evolutionary-scale prediction of atomic-l...

    2023

  32. [40]

    Yuexiao Ma, Huixia Li, Xiawu Zheng, Feng Ling, Xuefeng Xiao, Rui Wang, Shilei Wen, Fei Chao, and Rongrong Ji. 2024. Affinequant: Affine transformation quan- tization for large language models.arXiv preprint arXiv:2403.12544(2024)

    2024 arXiv

  33. [41]

    Micron. 2007. HBM2E memory. https://www.micron.com/products/memory/ hbm/hbm2e

    2007

  34. [42]

    Shervin Minaee, Tomas Mikolov, Narjes Nikzad, Meysam Chenaghlu, Richard Socher, Xavier Amatriain, and Jianfeng Gao. 2024. Large language models: A survey.arXiv preprint arXiv:2402.06196(2024)

    2024 arXiv

  35. [43]

    Milot Mirdita, Konstantin Schutze, Yoshitaka Moriwaki, Lim Heo, Sergey Ovchin- nikov, and Martin Steinegger. 2022. Easy and accurate protein structure prediction using ColabFold.Nature Methods19 (2022), 679–682

    2022

  36. [44]

    Markus Nagel, Marios Fournarakis, Rana Ali Amjad, Yelysei Bondarenko, Mart Van Baalen, and Tijmen Blankevoort. 2021. A white paper on neural network quantization.arXiv preprint arXiv:2106.08295(2021)

    2021 arXiv

  37. [45]

    NVIDIA. 2024. NVIDIA A100 Tensor Core GPU. https://www.nvidia.com/content /dam/en-zz/Solutions/Data-Center/a100/pdf/nvidia-a100-datasheet-nvidia-us- 2188504-web.pdf

    2024

  38. [46]

    NVIDIA. 2024. NVIDIA H100 Tensor Core GPU. https://resources.nvidia.com/en- us-tensor-core/nvidia-tensor-core-gpu-datasheet

    2024

  39. [47]

    NVIDIA. 2024. NVIDIA H200 Tensor Core GPU. https://resources.nvidia.com/en- us-data-center-overview-mc/en-us-data-center-overview/hpc-datasheet-sc23- h200

    2024

  40. [48]

    RCSB PDB Core Operations. 2007. RCSB PDB. https://www.rcsb.org/

    2007

  41. [49]

    Keiron O’Shea and Ryan Nash. 2015. An Introduction to Convolutional Neural Networks. https://arxiv.org/abs/1511.08458

    2015 arXiv

  42. [50]

    Corey, and H

    Linus Pauling, Robert B. Corey, and H. R. Branson. 1951. The structure of proteins: Two hydrogen-bonded helical configurations of the polypeptide chain. https://www.pnas.org/doi/abs/10.1073/pnas.37.4.205.Proceedings of the National Academy of Sciences37, 4 (1951), 205–211

    1951 doi

  43. [51]

    Shuang Peng, Fei Yang, Ning Sun, Sheng Chen, Yanfeng Jiang, and Aimin Pan. 2023. Exploring Post-Training Quantization of Protein Language Models. https://arxiv.org/abs/2310.19624

    2023 arXiv

  44. [52]

    Markus N Rabe and Charles Staats. 2021. Self-attention does not need 𝑂(𝑛 2) memory.arXiv preprint arXiv:2112.05682(2021)

    2021 arXiv

  45. [53]

    Senior, Richard Evans, John Jumper, James Kirkpatrick, Laurent Sifre, Tim Green, Chongli Qin, Augustin Žídek, Alexander W

    Andrew W. Senior, Richard Evans, John Jumper, James Kirkpatrick, Laurent Sifre, Tim Green, Chongli Qin, Augustin Žídek, Alexander W. R. Nelson, Alex Bridgland, Hugo Penedones, Stig Petersen, Karen Simonyan, Steve Crossan, Pushmeet Kohli, David T. Jones, David Silver, Koray Kav...

    2020

  46. [54]

    Robert F. Service. 2020. The game has changed. AI triumphs at solving pro- tein structures. https://www.science.org/content/article/game-has-changed-ai- triumphs-solving-protein-structures

    2020

  47. [55]

    Dreslinski, Thomas Manville, Sudhir Satpathy, Nathaniel Pinckney, Geoffrey Blake, Michael Cieslak, Reetuparna Das, Thomas F

    Korey Sewell, Ronald G. Dreslinski, Thomas Manville, Sudhir Satpathy, Nathaniel Pinckney, Geoffrey Blake, Michael Cieslak, Reetuparna Das, Thomas F. Wenisch, Dennis Sylvester, David Blaauw, and Trevor Mudge. 2012. Swizzle-switch net- works for many-core systems.IEEE Journal on...

    2012

  48. [56]

    Anil Shanbhag, Holger Pirk, and Samuel Madden. 2018. Efficient top-k query processing on massively parallel hardware. InProceedings of the 2018 International Conference on Management of Data. Association for Computing Machinery, New York, NY, USA, 1557–1570

    2018

  49. [57]

    Hardik Sharma, Jongse Park, Naveen Suda, Liangzhen Lai, Benson Chau, Joon Kyung Kim, Vikas Chandra, and Hadi Esmaeilzadeh. 2018. Bit fusion: Bit- level dynamically composable architecture for accelerating deep neural network. In2018 ACM/IEEE 45th Annual International Symposium...

    2018

  50. [58]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guil- laume Lample. 2023. LLaMA: Open and Efficient Foundation ...

    2023 arXiv

  51. [59]

    A Vaswani. 2017. Attention is all you need.Advances in Neural Information Processing Systems(2017)

    2017

  52. [60]

    Johnson, Mike Oconnor, Evgeny Bolotin, David Nellans, Justin Luitjens, Nikolai Sakharnykh, Peng Wang, Paulius Micikevicius, Anthony Scudiero, Stephen W

    Oreste Villa, Daniel R. Johnson, Mike Oconnor, Evgeny Bolotin, David Nellans, Justin Luitjens, Nikolai Sakharnykh, Peng Wang, Paulius Micikevicius, Anthony Scudiero, Stephen W. Keckler, and William J. Dally. 2014. Scaling the power wall: a path to exascale. InSC’14: Proceeding...

    2014

  53. [61]

    Huizheng Wang, Jiahao Fang, Xinru Tang, Zhiheng Yue, Jinxi Li, Yubin Qin, Sihan Guan, Qize Yang, Yang Wang, Chao Li, Yang Hu, and Shouyi Yin. 2024. SOFA: A Compute-Memory Optimized Sparsity Accelerator via Cross-Stage Coordinated Tiling. https://arxiv.org/abs/2407.10416

    2024 arXiv

  54. [62]

    Wikipedia. 2024. 68–95–99.7 rule. https://en.wikipedia.org/wiki/68-95-99.7_rule

    2024

  55. [63]

    Wikipedia. 2024. C (programming language). https://en.wikipedia.org/wiki/C_(p rogramming_language)

    2024

  56. [64]

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. 2023. Smoothquant: Accurate and efficient post-training quantization for large language models. InInternational Conference on Machine Learning. PMLR, PMLR, 38087–38099

    2023

  57. [65]

    Jinbo Xu. 2019. Distance-based protein folding powered by deep learning. http://dx.doi.org/10.1073/pnas.1821309116.Proceedings of the National Acad- emy of Sciences116, 34 (Aug. 2019), 16856–16865

    2019 doi

  58. [66]

    Ali Hadi Zadeh, Isak Edo, Omar Mohamed Awad, and Andreas Moshovos. 2020. Gobo: Quantizing attention-based nlp models for low latency and energy ef- ficient inference. In2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). IEEE, IEEE, Piscataway, NJ, ...

    2020

  59. [67]

    Ali Hadi Zadeh, Mostafa Mahmoud, Ameer Abdelhadi, and Andreas Moshovos

  60. [68]

    Jinnian Zhang, Houwen Peng, Kan Wu, Mengchen Liu, Bin Xiao, Jianlong Fu, and Lu Yuan. 2022. Minivit: Compressing vision transformers with weight multiplexing. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Piscataway, NJ, USA, 12145–12154

    2022

  61. [69]

    Yang Zhang and Jeffrey Skolnick. 2005. TM-align: a protein structure alignment algorithm based on the TM-score.Nucleic acids research33, 7 (2005), 2302–2309

    2005

  62. [70]

    InProceedings of the 49th Annual International Sym- posium on Computer Architecture

    Mokey: Enabling narrow fixed-point inference for out-of-the-box floating- point transformer models. InProceedings of the 49th Annual International Sym- posium on Computer Architecture. Association for Computing Machinery, New York, NY, USA, 888–901

  63. [73]

    Feiwen Zhu, Arkadiusz Nowaczynski, Rundong Li, Jie Xin, Yifei Song, Michal Marcinkiewicz, Sukru Burc Eryilmaz, Jun Yang, and Michael Andersch

  64. [74]

    https://doi.org/10.1145/3649329.3657326

    ScaleFold: Reducing AlphaFold Initial Training Time to 10 Hours. https://doi.org/10.1145/3649329.3657326. InProceedings of the 61st ACM/IEEE Design Automation Conference(San Francisco, CA, USA)(DAC ’24). Association for Computing Machinery, New York, NY, USA, Article 265, 6 pages

  65. [2022]

    https://arxiv.org/abs/2208.07339

    LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale. https://arxiv.org/abs/2208.07339

    arXiv

  66. [2024]

    In2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA)

    BLESS: Bandwidth and Locality Enhanced SMEM Seeding Acceleration for DNA Sequencing. In2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA). IEEE, 582–596

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.