Pith. sign in

REVIEW 4 major objections 4 minor 32 references

FlashSVD: Memory-Efficient Inference with Streaming for Low-Rank Models

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that fusing low-rank SVD projections into attention and feed-forward kernels cuts peak activation memory from $O(BM(D_F+D_A+M))$ to $O(r(BM+D_F+D_A))$ without changing model accuracy.

desk verdict Worth reviewing for the fused low-rank kernel idea, but the headline memory/latency numbers don't match the paper's own tables. read the letter →

arxiv 2508.01506 v1 pith:4KWMTYS5 submitted 2025-08-02 cs.LG cs.AIcs.PF

Zishan Shao , Yixiao Wang , Qinsi Wang , Ting Jiang , Zhixu Du , Hancheng Ye , Danyang Zhuo , Yiran Chen
show 1 more author
Hai Li
This is my paper
classification cs.LGcs.AIcs.PF
keywords singularvaluedecompositiontransformercompressionactivationmemorykernelfusionstreaminginferencelow-rankprojectionsmulti-headattentionmemory-efficient
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SVD compression shrinks transformer weights but, run with ordinary matrix kernels, still creates full-size query, key, value, and feed-forward activation tensors, so total inference memory barely drops. FlashSVD claims to fix this by fusing the low-rank projections into the attention and feed-forward kernels: tiles of the SVD factors are read into on-chip SRAM, the needed activation tile is multiplied and reduced on the fly, then discarded before the next tile arrives. The paper proves rank-aware memory bounds that replace the dense activation footprint with one that scales with the compression rank instead of hidden width, and it reports measured transient-memory reductions of about 69 to 75 percent on BERT and RoBERTa with accuracy unchanged from the upstream SVD compressor. A reader should care because this turns weight-only SVD compression from a file-size win into a genuine deployment win for memory-constrained inference.

What carries the argument

The carrying object is a fused, tiled low-rank kernel: small blocks of the truncated SVD factors $U$ and $V$ are loaded into on-chip SRAM, multiplied to rebuild only the activation tile currently needed, reduced with softmax or the nonlinearity, accumulated into the output, and evicted before the next tile arrives. This is FlashAttention-style tiling applied to the factor matrices themselves, so asymptotic peak HBM usage tracks the rank $r$ rather than the hidden widths $D_A$, $D_F$, or the sequence length $M$. Two FFN variants trade I/O against parallelism: V1 keeps one rank-sized intermediate on HBM and uses vendor GEMMs for the outer projections, while V2 streams everything on-chip at the cost of finer tiling and lower occupancy. The multi-head SVD threshold also carries part of the argument: per-head factorization yields a gentler rank-loss requirement for the same parameter count.

What would settle it

Run a memory profiler that includes every allocation, including output activations $\mathbb{R}^{BM \times D_A}$ and persistent low-rank projection factors, at the paper's own batch and sequence settings; if any setting with $r \ll D_A$ shows FlashSVD's measured total peak memory equal to the dense baseline, then the variable-buffer bound is not the term that actually decides peak memory in that regime.

Watch

Extended reading notes

Core claim

On its own terms, the paper's claim is that low-rank structure, if consumed in a rank-aware streaming kernel instead of being expanded back into dense tensors, changes where transformer inference spends its memory. FlashSVDAttention reconstructs per-head query, key, and value tiles inside SRAM from their factors $P_a \in \mathbb{R}^{B \times M \times r}$ and $V_a \in \mathbb{R}^{r \times d_{A,h}}$, applies scores and softmax in tiles, and never writes the full score or projection tensors to HBM. FlashSVDFFN V1 writes only the rank-sized factor-space intermediate and streams the feed-forward width in blocks through a fused GEMM-activation-GEMM, while V2 fuses the entire feed-forward pass on-chip. Together these kernels reduce per-layer peak activation memory from $O(BM(D_F+D_A+M))$ to $O(r(BM+D_F+D_A))$, with FLOP reductions that can turn extreme low-rank compression into a speedup, up to $1.9\times$ on the FFN at rank 96. A second claim is structural: applying truncated SVD per attention head lowers the rank needed to hit a parameter budget from $r < D_A/2$ for a single full matrix to $r < D_A/(H+1)$, so multi-head models can be compressed more gently.

Load-bearing premise

The central premise is that the activation buffers FlashSVD removes are the ones that dominate peak memory; if output activations and stored projection factors dominate instead, the headline savings shrink to a smaller fraction of total memory.

Editorial extensions

If this is right

  • Weight-only SVD compression finally pays off at inference: for a fixed parameter budget, a compressed encoder can run with less peak HBM than the dense original rather than merely fewer stored weights.
  • Peak activation memory scales with rank, so rank-aware fine-tuning compounds the benefit; the paper reports fine-tuned low-rank BERT on STS-B with about 48 percent lower transient memory and about 29 percent lower peak memory than dense.
  • Low-rank kernels can switch from memory win to speed win as rank falls; FlashSVDAttention reaches parity or speedup over dense at ranks around 16 to 32, and FlashSVDFFN V1 reaches up to $1.9\times$ over the dense FFN at rank 96.
  • Because per-head multi-head SVD reaches a parameter budget with a smaller rank cut, compressors can choose moderate ranks that preserve accuracy instead of the drastic single-matrix cuts that hurt quality.
  • The framework is agnostic to the upstream SVD method, so existing training-free and fine-tuned SVD compressors inherit the rank-aware memory behavior without further accuracy loss.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same on-chip factor streaming should apply to decoder KV-cache compression, where keys and values are already stored as low-rank factors; a fused decode kernel could attend against those factors directly, extending the memory win to the cache-dominated regime the paper only sketches.
  • Editorial inference: the headline 70.2 percent figure and the $O(r(BM+D_F+D_A))$ bound count only the variable activation buffers FlashSVD removes; measured total peaks in the tables are 20 to 40 percent below dense once output activations and persistent projection factors are included, so the correct reading is that the activation overhead canceling SVD's gains is gone, not that every deployment
  • Editorial inference: the rank-sensitivity propositions imply a practical per-layer rank allocation rule, since each unit of rank in a head saves $O(BM+d_{A,h})$ elements and each unit in the FFN saves $O(BM+D_F)$ elements; ranking layers by this marginal saving could let a deployment hit a target memory budget with minimal accuracy cost.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This paper proposes FlashSVD, a streaming inference scheme for SVD-compressed transformers. Instead of reconstructing full dense query/key/value activations and FFN intermediates in HBM, FlashSVD tiles the low-rank factors and performs the projections on-chip, with two FFN variants (V1, V2) trading off GEMM friendliness against memory. The paper derives asymptotic memory and complexity bounds, claims an end-to-end memory reduction from O(BM(D_A + D_F + M)) to O(r(BM + D_A + D_F)), and reports experiments on BERT-Base and RoBERTa-Base across GLUE tasks at 25% and 50% parameter retention, measuring peak/transient memory, latency, and accuracy versus dense, vanilla SVD, and FWSVD baselines.

Significance. The problem addressed is real and timely: SVD weight compression alone does not reduce peak inference memory when activations are materialized densely, and the proposed tiled low-rank kernels are a plausible way to close that gap. The paper's strengths include the two FFN fusion variants, the multi-head SVD rank-loss observation, and a fairly extensive ablation matrix over batch sizes, sequence lengths, and ranks. The memory-counting analysis is transparent enough to be checked, and Remark 1 honestly identifies the scope of the asymptotic claims. However, the headline claims as currently stated are not supported by the paper's own end-to-end measurements: the 'no extra latency' claim is contradicted by Table 2, and the 70.2% peak memory reduction is not reproduced by the full-path Peak columns. The technical core is likely salvageable, but the claims must be re-scoped and the measurements reconciled before the paper can be accepted.

major comments (4)
  1. [Abstract; §5.2, Table 2] The abstract states that FlashSVD adds 'no extra latency,' but Table 2 shows FlashSVD v1/v2 are slower than Vanilla SVD in nearly every configuration. For example, on BERT SST-2 at 25% parameters, FlashSVD v1 takes 188.8 ms versus 161.1 ms for Vanilla SVD; on QQP, 264.8 ms versus 200.2 ms; on MNLI, 490.5 ms versus 359.7 ms. RoBERTa MNLI at 25% shows 393.1 ms versus 225.0 ms. The text in §5.2 itself concedes 'substantial latency overhead, with up to 60% in some cases.' This direct contradiction of a headline claim must be fixed, for example by stating the latency claim only for specific rank/context regimes or by reporting the overhead prominently in the abstract.
  2. [Remark 1; Theorem 13; Table 2] The memory analysis explicitly excludes persistent low-rank factors and final output activations from the off-chip bounds, and Theorem 13's O(r(BM + D_A + D_F)) bound therefore does not include the always-present O(BM D_A) output activation. The abstract's 70.2% 'peak activation memory' reduction is not what Table 2's end-to-end Peak columns show: those columns report reductions of about 17% (BERT SST-2 25%: 576.7 vs 695.0 MiB dense) to about 28% (BERT MNLI 50%: 1120.7 vs 1547.4 MiB dense). The 70.2% figure appears to come from the isolated module ablations in Tables S4/S5 rather than the full inference path. Since the paper motivates FlashSVD with the claim that current SVD gives 'no reduction in peak inference memory,' the headline result should be expressed in terms of total measured peak HBM, or the scope of the claimed reduction should be stated precisely as 'transient activation buffers excluding output activations and persistent factors.'
  3. [Theorem 10; Algorithm S1; Table S5] Theorem 10 states that FlashSVDFFN V2 achieves M_flash-svd-ffn-V2 = 0, but Table S5 reports nonzero memory for V2 in every measured configuration, e.g., 550.8 MB for B=64, M=1024, rank 384. If the theorem is intended to count only intermediate activation buffers while excluding output buffers and the persistent factors that Algorithm S1 streams, that scope must be stated inside the theorem and its proof, not only in a later remark. As written, the 'zero memory' claim is misleading and should be reconciled with the measured values and with the abstract's more modest '75% intermediate transient memory' claim.
  4. [Theorem 4] The final sentence of Theorem 4 claims that setting r=1 yields an H-fold memory reduction, but this does not follow from the formula in the theorem. With r=1, the FlashSVD bound is HBM + D_A, while the dense baseline is BM D_A, so the reduction factor is BM D_A / (HBM + D_A), which for large BM is approximately D_A/H, not H. This overstates the multi-head benefit and should be corrected or removed.
minor comments (4)
  1. [Table S2] Several speedup entries in Table S2 are internally inconsistent: for M=512 and rank 96, the reported time is 1.67 ms versus a Dense time of 0.68 ms, yet the speedup column shows 1.60x. The speedup values should be recomputed from the reported times.
  2. [References] Reference [31] labels xFormers but cites 'Xformer: Hybrid X-shaped Transformer for image denoising,' which appears to be a different work; the correct xFormers library reference should be used.
  3. [Table 2] The 'Ratio' column in Table 2 is unclear: values such as 0.60, 0.55, and 0.54 do not match the stated 25% or 50% parameter-retention ratios, so the caption should define what this ratio measures.
  4. [Abstract; §6] There are several grammatical slips, including 'incur no accuracy loss' and 'incuring no additional arithmetic cost'; these should be corrected throughout.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the memory bounds are direct buffer-size counts, accuracy is externally measured on GLUE, and the central claims do not reduce to fitted parameters or to a self-citation chain.

full rationale

The derivation chain is self-contained. Theorems 2, 4, 9, and 10 establish peak variable off-chip memory by summing the sizes of the tensors that the proposed algorithms actually allocate (e.g., P_a in R^{BM x r}, Z_tile in R^{BM x r}, and the tiled on-chip buffers), with no fitted constants and no parameter inferred from the memory results; the rank r is an input supplied by the upstream SVD compressor rather than a quantity derived from FlashSVD's own outputs. Table 2 accuracy is measured on GLUE rather than predicted from the model, so there is no fitted-input-called-prediction pattern. The attention- and FFN-memory comparisons (Theorems 3, 5, 12, 13) are algebraic counts of buffer sizes under a stated common scope, and the multi-head-versus-single-head compression analysis is a parameter-count calculation, not a result imported from the authors' prior work. No load-bearing self-citation appears: FlashAttention [2, 3], Palu [1], ASVD [30], FWSVD, and other cited works are external sources used for context or baseline comparison, not to define the memory bounds into existence. The one caveat that deserves explicit weighing is Remark 1, which states that the off-chip analyses omit persistent terms such as the HBM-resident matrices V* in R^{r x D_A} and the final output activations R^{BM x D_A}. This is a clearly disclosed scope choice, not a hidden circular step, because the omitted output activations and persistent low-rank factors are common to both the dense and FlashSVD sides of the comparison, so the asymptotic reduction is not manufactured by dropping a term that only affects one implementation. The gap between the abstract's 70.2% headline and the 17-28% end-to-end peak reductions in Table 2 is a measurement-consistency and scope-communication concern, not evidence that the central result is equivalent to its inputs by construction. Therefore the circularity score is 0.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

There are no fitted physical constants and no new postulated entities. The memory gains are buffer-size counts at a given rank r, where r comes from upstream compression. The only hand-set knobs are ranks and kernel tile sizes, which affect the reported latency and memory numbers.

free parameters (2)
  • SVD rank rTheta* = attention ranks 16-64; FFN ranks 96-768
    All memory and latency results depend on the chosen rank. The rank is inherited from the compression ratio targets (25 percent and 50 percent) and set by the upstream SVD method, not fitted to memory targets.
  • Kernel tile sizes B_M, B_Df, G = not reported
    Autotuned streaming block sizes control SRAM occupancy and latency. Exact values are not given, so latency numbers are not portable or exactly reproducible.
assumptions (5)
  • standard math Eckart-Young-Mirsky theorem gives the optimal rank-r Frobenius approximation
    Invoked in Section 3 to justify truncated SVD as the best rank-r weight approximation. This is a standard result and not rederived.
  • domain assumption FlashAttention-style stream-softmax is exact enough for accuracy comparisons
    Algorithm 1 relies on FlashAttention tiling and online softmax assumed to be numerically equivalent to dense attention. The paper uses this to claim no accuracy loss from the streaming computation.
  • domain assumption GPU HBM/SRAM roofline model lower-bounds latency
    Theorem 15 assumes latency is at least max(FLOPs / peak FLOP rate, bytes / bandwidth). This is standard but idealized and ignores occupancy, launch overhead, and kernel scheduling effects.
  • domain assumption Typical transformer FFN width is at least 4 times the hidden dimension
    Theorem 11 uses D_F = 4 D_A for BERT, RoBERTa, LLaMA, and OPT to show the FFN dominates after attention compression. This is architecture-specific and not universal.
  • ad hoc to paper Variable-memory scope excludes output activations and persistent factors from peak memory
    Remark 1 and Theorem 2 explicitly exclude R^{BM*D_A}, R^{BM*D_F}, and the HBM-resident low-rank bases from the complexity bounds. This scope choice is load-bearing for the headline memory reduction numbers.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FlashSVD: Memory-Efficient Inference with Streaming for Low-Rank Models." pith.science (2026). https://pith.science/paper/4KWMTYS5

@misc{pith2026250801506,
  author       = {Pith},
  title        = {Pith review of: FlashSVD: Memory-Efficient Inference with Streaming for Low-Rank Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4KWMTYS5}},
  note         = {Machine review of arXiv:2508.01506}
}
read the original abstract

Singular Value Decomposition (SVD) has recently seen a surge of interest as a simple yet powerful tool for large language models (LLMs) compression, with a growing number of works demonstrating 20-80% parameter reductions at minimal accuracy loss. Previous SVD-based approaches have focused primarily on reducing the memory footprint of model weights, largely overlooking the additional activation memory overhead incurred during inference when applying truncated factors via standard dense CUDA kernels. Our experiments demonstrate that this activation overhead, scaling with sequence length and hidden dimension, prevents current SVD compression techniques from achieving any reduction in peak inference memory, thereby limiting their viability for real-world, on-device deployments. We introduce FlashSVD, a novel, end-to-end rank-aware streaming inference framework specifically designed for SVD-compressed large language models. FlashSVD can be seamlessly integrated with any model that employs SVD-based methods for parameter reduction. By fusing low-rank projection kernels directly into both the self-attention and feed-forward network (FFN) pipelines, FlashSVD avoid materializing full-size activation buffers. Instead, small tiles of the truncated factors are loaded into on-chip SRAM, multiplied and reduced on the fly, and immediately evicted, preserving high GPU occupancy and adding no extra latency. On standard encoder benchmarks (e.g., BERT-Base), FlashSVD cuts peak activation memory by up to 70.2% and intermediate transient memory by 75%, all while incur no accuracy loss with upstreaming compression methods, offering a practical path toward memory-constrained deployment of low-rank LLMs.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

32 extracted references · 12 canonical work pages

  1. [1]

    Chang, W.-C

    C.-C. Chang, W.-C. Lin, C.-Y. Lin, C.-Y. Chen, Y.-F. Hu, P.-S. Wang, N.-C. Huang, L. Ceze, M. S. Abdelfattah, and K.-C. Wu. Palu: KV-cache compression with low-rank projection. InThe Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id= LWMS4pk2vK

    work page 2025

  2. [2]

    Flashattention:Fastandmemory-efficientexactattention with io-awareness.arXiv preprint, 2022

    T.Dao,D.Y.Fu,S.Ermon,A.Rudra,andC.Ré. Flashattention:Fastandmemory-efficientexactattention with io-awareness.arXiv preprint, 2022. arXiv:2205.14135

    arXiv 2022

  3. [3]

    Flashattention-2:Fasterattentionwithbetterparallelism and work partitioning.arXiv preprint, 2023

    T.Dao,D.Y.Fu,S.Ermon,A.Rudra,andC.Ré. Flashattention-2:Fasterattentionwithbetterparallelism and work partitioning.arXiv preprint, 2023. arXiv:2307.08691

    arXiv 2023

  4. [4]

    Dettmers, A

    T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer. Qlora: Efficient finetuning of quantized llms. arXiv preprint, 2023. arXiv:2305.14314

    arXiv 2023

  5. [5]

    Devlin, M.-W

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. InProceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171–4186, 2019

    work page 2019

  6. [6]

    Theapproximationofonematrixbyanotheroflowerrank

    C.EckartandG.Young. Theapproximationofonematrixbyanotheroflowerrank. Psychometrika,1(3): 211–218, 1936

    work page 1936

  7. [7]

    MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks

    A. Gordon, E. Eban, O. Nachum, B. Chen, H. Wu, T.-J. Yang, and E. Choi. Morphnet: Fast & simple resource-constrained structure learning of deep networks.arXiv preprint arXiv:1711.06798, 2017

    work page Pith review arXiv 2017

  8. [8]

    S. Han, H. Mao, and W. J. Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. In International Conference on Learning Representations (ICLR), 2016. arXiv:1510.00149

    arXiv 2016

Show all 32 references
  1. [9]

    Y. He, X. Zhang, and J. Sun. Channel pruning for accelerating very deep neural networks. InIEEE International Conference on Computer Vision (ICCV), Oct 2017

    2017

  2. [10]

    Y.-C. Hsu, T. Hua, S. Chang, Q. Lou, Y. Shen, and H. Jin. Language model compression with weighted low-rank factorization.arXiv preprint arXiv:2207.00112, 2022

    2022 arXiv

  3. [11]

    E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen. Lora: Low-rank adaptation of large language models. InICLR, 2021. arXiv:2106.09685

    2021 arXiv

  4. [12]

    Quantization and training of neural networks for efficient integer-arithmetic-only inference

    B.Jacob,S.Kligys,B.Chen,M.Zhu,M.Tang,A.Howard,H.Adam,andD.Kalenichenko. Quantization and training of neural networks for efficient integer-arithmetic-only inference. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2704–2713, 20...

    2018 arXiv

  5. [13]

    H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf. Pruning filters for efficient convnets. In International Conference on Learning Representations (ICLR), 2017. arXiv:1608.08710

    2017 arXiv

  6. [14]

    Lin and Colleagues

    B. Lin and Colleagues. Modegpt: Modular decomposition for large language model compression. In ICML, 2024. OpenReview: 8EfxjTCg2k. 12 FlashSVD: Memory-Efficient Inference with Streaming for Low-Rank Models

    2024

  7. [15]

    Towardscompactconvnetsviastructure-sparsityregularizedfilter pruning

    S.Lin,R.Ji,Y.Li,C.Deng,andX.Li. Towardscompactconvnetsviastructure-sparsityregularizedfilter pruning. arXiv preprint arXiv:1901.07827, 2019

    1901 arXiv

  8. [16]

    Liu and N

    F. Liu and N. Team. Star attention: Efficient llm inference over long sequences. arXiv preprint, 2024. arXiv:2411.17116

    2024 arXiv

  9. [17]

    Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov. Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692, 2019

    1907 arXiv

  10. [18]

    Luo and J

    J.-H. Luo and J. Wu. An entropy-based pruning method for cnn compression. arXiv preprint arXiv:1706.05791, 2017

    2017 arXiv

  11. [19]

    J.-H. Luo, J. Wu, and W. Lin. Thinet: A filter level pruning method for deep neural network compression. InIEEE International Conference on Computer Vision (ICCV), 2017. arXiv:1707.06342

    2017 arXiv

  12. [20]

    R. Pope, S. Douglas, A. Chowdhery, J. Devlin, J. Bradbury, J. Heek, K. Xiao, S. Agrawal, and J. Dean. Efficiently scaling transformer inference. Proceedings of Machine Learning and Systems, 5:606–624, 2023

    2023

  13. [21]

    J. Shah, G. Bikshandi, Y. Zhang, V. Thakkar, P. Ramani, and T. Dao. Flashattention-3: Fast and accurate attention with asynchrony and low-precision.Advances in Neural Information Processing Systems, 37: 68658–68685, 2024

    2024

  14. [22]

    Shi and Team

    E. Shi and Team. Blockllm: Multi-tenant finer-grained serving for large language models. arXiv preprint,

  15. [23]

    Shoeybi, M

    M. Shoeybi, M. Patwary, R. Puri, P. LeGresley, J. Casper, and B. Catanzaro. Megatron-lm: Training multi-billion parameter language models using model parallelism. arXiv preprint arXiv:1909.08053, 2019

    1909 arXiv

  16. [24]

    Llama:Openandefficientfoundationlanguagemodels

    H.Touvron,T.Lavril,G.Izacard,X.Martinet,M.-A.Lachaux,T.Lacroix,B.Rozière,N.Goyal,E.Hambro, F.Azhar,etal. Llama:Openandefficientfoundationlanguagemodels. arXivpreprintarXiv:2302.13971 , 2023

    2023 arXiv

  17. [25]

    A. Wang, A. Singh, J. Michael, F. Hill, O. Levy, and S. R. Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding.arXiv preprint arXiv:1804.07461, 2018

    2018 arXiv

  18. [26]

    Q. Wang, J. Ke, M. Tomizuka, K. Keutzer, and C. Xu. Dobi-svd: Differentiable svd for llm compression and some new perspectives. InICLR, 2025. OpenReview: kws76i5XB8

    2025

  19. [27]

    X. Wang, Y. Zheng, Z. Wan, and M. Zhang. Svd-llm: Truncation-aware singular value decomposition for large language model compression.arXiv preprint arXiv:2403.07378, 2024

    2024 arXiv

  20. [28]

    Svd-llmv2:Optimizingsingularvaluetruncationfor large language model compression.arXiv preprint arXiv:2503.12340, 2025

    X.Wang,S.Alam,Z.Wan,H.Shen,andM.Zhang. Svd-llmv2:Optimizingsingularvaluetruncationfor large language model compression.arXiv preprint arXiv:2503.12340, 2025

    2025 arXiv

  21. [29]

    Integerquantizationfordeeplearninginference: Principles and empirical evaluation.arXiv preprint arXiv:2004.09602, 2020

    H.Wu,P.Judd,X.Zhang,M.Isaev,andP.Micikevicius. Integerquantizationfordeeplearninginference: Principles and empirical evaluation.arXiv preprint arXiv:2004.09602, 2020

    2004 arXiv

  22. [30]

    Yuan and Others

    A. Yuan and Others. Activation-aware singular value decomposition for large language models. OpenRe- view preprint, 2023. arXiv:2502.01403. 13 FlashSVD: Memory-Efficient Inference with Streaming for Low-Rank Models

    2023

  23. [31]

    Zhang, Y

    J. Zhang, Y. Zhang, J. Gu, J. Dong, L. Kong, and X. Yang. Xformer: Hybrid x-shaped transformer for image denoising. InICLR, 2024

    2024

  24. [32]

    Zhang, S

    S. Zhang, S. Roller, N. Goyal, M. Artetxe, M. Chen, S. Chen, C. Dewan, M. Diab, X. Li, X. V. Lin, et al. Opt: Open pre-trained transformer language models.arXiv preprint arXiv:2205.01068, 2022. 14 FlashSVD: Memory-Efficient Inference with Streaming for Low-Rank Models Algorith...

    2022 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.