Pith. sign in

REVIEW 4 major objections 7 minor 36 references

Parameter-Efficient Fine-Tuning with Circulant and Diagonal Vectors

T0 review · 4 major / 7 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fine-tuning with circulant and diagonal vectors matches existing adapters while sharply reducing FLOPs and trainable parameters.

desk verdict A solid incremental PEFT method with correct math and real FLOP savings, but the fixed m=2 factorization is asserted rather than validated. read the letter →

arxiv 2505.00580 v2 pith:F4PRYRSO submitted 2025-05-01 cs.LG

Xinyu Ding , Lexuan Chen , Siyu Liao , Zhongfeng Wang This is my paper · ORCID
classification cs.LG
keywords parameter-efficientfine-tuningcirculantmatricesdiagonalfastFouriertransformlow-rankadaptationdomainmethodsblockpartitionfoundationmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a parameter-efficient fine-tuning method, CDVFT, that represents the weight-change matrix as a product of interleaved circulant and diagonal matrices instead of a low-rank or dense Fourier transform. Because a circulant matrix is defined by a single vector and its product with a vector can be computed with one-dimensional FFTs, the method never needs to build the dense update matrix. On GLUE with RoBERTa-base it reports a 51.81× reduction in FLOPs versus FourierFT and 5.33× fewer trainable parameters than LoRA, with an average score of 85.1 versus LoRA's 85.2. The same recipe transfers to ViT image classification and LLaMA2-7B instruction tuning, using block partitioning to handle non-square weight matrices.

What carries the argument

The central identity is that a circulant matrix-vector product equals $\mathrm{IFFT}(\mathrm{FFT}(\mathbf{c}) \odot \mathrm{FFT}(\mathbf{x}))$, where $\mathbf{c}$ is the vector defining the circulant matrix. CDVFT iterates this identity with diagonal (elementwise) multiplications, so the whole adapter is a sequence of vector operations; the backward pass reuses forward FFTs via conjugation, $\mathrm{IFFT}(\mathrm{conj}(\mathbf{F}) \odot \mathbf{F}_y)$, avoiding extra transforms. The block-partition scheme extends the square-only circulant construction to rectangular weight matrices by splitting them into square circulant blocks.

What would settle it

Fine-tune RoBERTa-base on the six GLUE tasks with CDVFT at $m=2$ and at $m=3$ (adding one more circulant-diagonal pair), holding all other settings fixed; if $m=3$ improves any task's score by more than the reported standard deviation, the sufficiency claim for $m=2$ is refuted on that task.

Watch

Extended reading notes

Core claim

Building on the theorem that any complex square matrix can be expressed as a product of at most $2n-1$ interleaved circulant and diagonal matrices, CDVFT factorizes the weight update $\Delta W$ as $\mathrm{diag}(\mathbf{a}_{2m-1}) \times \mathrm{circ}(\mathbf{a}_{2m-2}) \times \cdots \times \mathrm{diag}(\mathbf{a}_1)$. In practice it sets $m=2$, i.e., two diagonal matrices and one circulant matrix, and computes the adapter output by an alternating chain of elementwise multiplications and 1D FFTs, so the quadratic cost of constructing and applying $\Delta W$ becomes loglinear in the dimension. For non-square weights, the circulant factor is partitioned into square blocks, each block itself a circulant matrix. The paper claims this yields accuracy comparable to LoRA and FourierFT while using far fewer trainable parameters than LoRA and far fewer FLOPs than FourierFT across natural language understanding, image classification, and instruction tuning.

Load-bearing premise

The method's accuracy rests on the untested assumption that a single circulant matrix sandwiched between two diagonal matrices ($m=2$) is expressive enough to capture the useful weight update on any given task, with no ablation over $m$ provided.

Editorial extensions

If this is right

  • Fine-tuning memory and compute scale with vector length $d$ and factor count $m$ rather than with the full weight-matrix dimension, making adaptation of larger models more feasible on limited hardware.
  • The $m=2$ configuration (diagonal-circulant-diagonal) is claimed to be sufficient for task-level accuracy on the evaluated benchmarks, so practitioners can use CDVFT without tuning the number of factors.
  • Block partitioning with larger block sizes $p$ reduces parameter count and FLOPs, giving a direct knob to trade cost against accuracy on non-square layers such as those in LLaMA2-7B.
  • Because the trained factors are vectors, the final update $\Delta W$ can be reconstructed and merged into the frozen weights after training, so inference latency is unchanged.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The factorization family suggests a natural expressivity-cost ladder: increasing $m$ from 1 to larger values approaches arbitrary dense updates, so CDVFT could serve as a bridge between extremely cheap structured adapters and full fine-tuning.
  • One testable extension is to apply CDVFT to other structured-matrix problems, such as compressing attention projections or convolution kernels, where circulant structure is a known inductive bias.
  • The paper's observation that larger block sizes sometimes do not hurt accuracy hints that the circulant-diagonal structure itself may act as a regularizer; a systematic study of this effect would separate regularization from capacity in the accuracy numbers.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes CDVFT, a parameter-efficient fine-tuning method that represents the weight update ΔW as a product of interleaved diagonal and circulant matrices and computes the adapter output with 1D FFT operations, without materializing ΔW during training. Non-square weight matrices are handled by partitioning the circulant factor into blocks. The method is evaluated on GLUE with RoBERTa-base, on CIFAR-100 and RESISC45 with ViT-base, and on Alpaca and GSM8K with LLaMA2-7B, using m=2 in all experiments. The reported results show accuracy comparable to LoRA and FourierFT while using fewer trainable parameters than LoRA and substantially fewer FLOPs than FourierFT.

Significance. If the fixed m=2 factorization is indeed sufficient across tasks, CDVFT is a useful contribution to the PEFT efficiency frontier: it achieves parameter counts close to FourierFT with much lower FLOPs and no accuracy loss on the tested benchmarks. The paper's strengths include the clean 1D-FFT formulation, the backward-pass rules in Eq. (11) that reuse forward FFTs via conjugation, and the breadth of evaluation across NLU, vision, and instruction tuning. The main open question is whether the small number of factors is an expressivity bottleneck; the paper currently provides no systematic evidence on this point, which is load-bearing for the central efficiency claim.

major comments (4)
  1. [Section 3.1/Eq. (3) and Section 4.1] The efficiency claims in the abstract and Fig. 2 rest on fixing m=2, but the paper provides no ablation over m or over the block size p. The parameter count is (2m−1)d per square adapter and the FLOPs scale as O(m d log d), so the advertised 55K-parameter/27M-FLOP operating point is only meaningful if m=2 is expressive enough on all tested tasks. The cited Huhtanen–Perämäki theorem guarantees factorization with up to 2n−1 factors, not that two diagonal and one circulant factor suffice, and no approximation-error bound for m=2 is given. Please add an ablation on at least one GLUE task and one image task sweeping m (e.g., 1, 2, 3, 4) together with the corresponding parameter and FLOP counts, and sweep p in the same ablation. Without this, the Pareto comparison to LoRA/FourierFT is incomplete.
  2. [Fig. 1 caption and Section 4.1] The caption states 'In practice, we find m = 1 (no loops required) can effectively fine-tune the model, where there are two diagonal matrices and one circulant matrix.' This contradicts Section 4.1, which sets 'a total of 3 factor matrices, i.e., m = 2,' and also contradicts Eq. (3), under which m=1 gives a single diagonal factor and no circulant matrix. The definition of m must be made consistent across the figure, the equations, and the experimental sections.
  3. [Section 3.4 and Figs. 2/Table 3] The FLOPs values that support the '51.81× FLOPs reduction' claim are presented without a counting convention. It is not specified whether the numbers are per sample, per optimizer step, forward-only or forward+backward, or whether FFT operations are counted as complex or real operations. Please provide the exact formulas used for each method so the headline complexity comparison can be reproduced.
  4. [Section 2 and Section 3.1] The factorization theorem of Huhtanen and Perämäki is quoted for complex matrices, while CDVFT trains real diagonal and circulant vectors. The paper does not explain whether the same 2n−1 factor bound holds over the reals or whether the theorem's factors can be chosen real for a real target matrix. Please either supply a real-matrix statement of the theorem or weaken the claim that the construction 'theoretically can approximate any dense matrix.'
minor comments (7)
  1. [Section 4.3, Implementation details] The sentence 'we apply block circulant fine-tuning on query and value weight matrices inside the attention layer of two RoBERTa models and the LLaMA2-7B model fine-tuned on the alpaca dataset' appears to be a copy-paste error; the experiments use one RoBERTa model for GLUE and LLaMA2-7B for instruction tuning, and the sentence should be rewritten.
  2. [Eq. (2)] Eq. (2) defines the circulant matrix as circ(a_{2j}) while the surrounding text and Eq. (3) use c_{2j} for circulant vectors; please unify the notation.
  3. [Section 4.2] The text says 'we fine-tune on several challenging image classification datasets, only two are listed here for observation,' but Table 2 reports only two datasets; if additional datasets were evaluated, they should be reported, and if not, the sentence should be corrected.
  4. [Table 3 caption] The phrase 'LaMDA++ is lack of rank information' should be 'LaMDA++ lacks rank information.'
  5. [Section 3.4] The sentence containing '(Priority is primarily given to cases where the weight matrix is square.)' is incomplete or misplaced; please clean up this sentence.
  6. [Section 3.3] The block-partition description says matrices are 'automatically padded through replication'; please specify exactly how rows and columns are padded when d1 or d2 is not divisible by p, and whether padding affects the eventual merge into the pretrained weights.
  7. [Introduction] There is a typo on the first page: 'fullfill' should be 'fulfill.'

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: CDVFT's efficiency claims are benchmarked externally and the factorization theorem is cited from independent prior work.

full rationale

The central efficiency claims (55K trainable parameters, 27M FLOPs on RoBERTa-base, 51.81x FLOP reduction vs FourierFT, 5.33x parameter savings vs LoRA) are not derived from fitted inputs. The parameter count (2m-1)dL_t and the O(md log d) FLOP estimate follow analytically from the definition of the interleaved diagonal/circulant factors in Section 3.4, and the performance numbers are measured on public benchmarks (GLUE, CIFAR-100, RESISC45, Alpaca, GSM8K) against external baselines (LoRA, VeRA, FourierFT, LaMDA). The representability guarantee is cited from Huhtanen and Peramaki (2015), an external mathematical theorem, and is not used to fit any parameter; the paper instead fixes m=2 as an empirical design choice. The block-circulant partitioning cites Ding et al. (2017), which shares an author with the present paper, but the partition is described self-containedly in Eq. (12) and serves as an implementation component, not as evidence for a prediction, so this self-citation is not load-bearing. The only substantive weakness is that m=2 is asserted without an ablation or approximation bound, and Fig. 1's caption says m=1 while Section 4.1 says m=2; this is an unsupported expressivity assumption and an internal inconsistency, but not a circular reduction, because the method's claims stand or fall on the public benchmark numbers rather than on the assumption itself.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the expressiveness of the m=2 parameterization, which is an empirical assumption rather than a derivation. There are no fitted constants that are then used as predictions; all listed numbers are standard hyperparameters. No new entities are introduced.

free parameters (3)
  • m (number of diagonal matrices, total factors 2m-1) = 2 for all experiments
    Chosen by hand; paper states 'only a few matrices and a small number of blocks is sufficient' without sensitivity analysis.
  • p (block size for circulant partition) = 768 for RoBERTa/ViT, 4096 or 2048 for LLaMA2-7B
    Set 'as large as possible' to minimize FLOPs and parameters; the value trades expressiveness against efficiency.
  • alpha (scaling hyperparameter) = not specified in the paper
    Standard LoRA-style scalar scaling; exact value not reported, affecting reproducibility.
assumptions (5)
  • standard math Any complex n x n matrix can be factorized into a product of interleaved diagonal and circulant matrices with at most 2n-1 factors (Huhtanen and Peramaki 2015).
    Invoked in Section 2 and 3.1 to justify the parameterization of the weight change matrix.
  • domain assumption A single circulant matrix between two diagonal matrices (m=2) is sufficient to approximate fine-tuning updates.
    Empirical claim stated in Sections 3 and 4; no theoretical bound or ablation over m is provided.
  • standard math FFT of a shifted real vector is the conjugate of the FFT of the original vector.
    Used to simplify the backward pass in Eq. (11); relies on trainable parameters being real.
  • domain assumption Block-circulant partitioning with padding by replication preserves accuracy for non-square weight matrices.
    Assumed in Section 3.3 for LLaMA experiments; no theoretical guarantee is given.
  • domain assumption Fine-tuning only query and value projection matrices is sufficient for the evaluated tasks.
    Follows the experimental setup of FourierFT and LoRA; the paper does not compare with fine-tuning other projections.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Parameter-Efficient Fine-Tuning with Circulant and Diagonal Vectors." pith.science (2026). https://pith.science/paper/F4PRYRSO

@misc{pith2026250500580,
  author       = {Pith},
  title        = {Pith review of: Parameter-Efficient Fine-Tuning with Circulant and Diagonal Vectors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/F4PRYRSO}},
  note         = {Machine review of arXiv:2505.00580}
}
read the original abstract

Foundation models have achieved tremendous success in different domains. However, their huge computation and storage complexity make these models difficult to fine-tune and also less applicable in practice. Recent study shows training in Fourier domain can be an effective fine-tuning method in terms of both model performance and number of training parameters. In this work, we propose to further reduce the complexity by the factorization through the product of interleaved circulant and diagonal matrices. In addition, we address the case of non-square fine-tuning weights by partitioning the circulant matrix into blocks. Our method avoids the construction of weight change matrix and utilizes 1D fast Fourier transform (FFT) instead of 2D FFT. Experimental results show that our method achieves similar or better performance across various tasks with much less floating-point operations (FLOPs) and the number of trainable parameters.

Figures

Figures reproduced from arXiv: 2505.00580 by the authors.

Figure 1
Figure 1. Overview of FourierFT (left) and our CDVFT (right). In FourierFT, one coefficient vector c ∈ R n is trained, and it is used to construct the weight change ∆W through 2D FFT operation. In contrast, our CDVFT avoids the construction of ∆W, where matrix vector products are transformed into vector operations, i.e., element-wise product and 1D FFT, significantly reducing computation complexity and memory requirement. In … view at source ↗
Figure 2
Figure 2. Complexity Analysis of different adapters. FF method is [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 13 canonical work pages

  1. [1]

    Lamda: Large model fine-tuning via spectrally decomposed low-dimensional adaptation,

    [Azizi et al., 2024] Seyedarmin Azizi, Souvik Kundu, and Massoud Pedram. Lamda: Large model fine-tuning via spectrally decomposed low-dimensional adaptation,

    2024

  2. [5]

    An exploration of parameter redundancy in deep networks with circulant projections

    [Cheng et al., 2015] Yu Cheng, Felix X Yu, Rogerio S Feris, Sanjiv Kumar, Alok Choudhary, and Shi-Fu Chang. An exploration of parameter redundancy in deep networks with circulant projections. In Proceedings of the IEEE international conference on computer vision, pages 2857– 2865,

    2015

  3. [7]

    The pascal recognising textual entailment challenge

    [Dagan et al., 2005] Ido Dagan, Oren Glickman, and Bernardo Magnini. The pascal recognising textual entailment challenge. In Machine learning challenges workshop, pages 177–190. Springer,

    2005

  4. [10]

    Automatically constructing a corpus of sentential para- phrases

    [Dolan and Brockett, 2005] Bill Dolan and Chris Brockett. Automatically constructing a corpus of sentential para- phrases. In Third international workshop on paraphrasing (IWP2005),

    2005

  5. [11]

    An image is worth 16x16 words: Trans- formers for image recognition at scale,

    [Dosovitskiy et al., 2021] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Min- derer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Trans- formers for image recognition at scale,

    2021

  6. [12]

    Parameter-efficient fine-tuning with discrete fourier trans- form

    [Gao et al., 2024b] Ziqi Gao, Qichao Wang, Aochuan Chen, Zijing Liu, Bingzhe Wu, Liang Chen, and Jia Li. Parameter-efficient fine-tuning with discrete fourier trans- form. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024 ,

    work page 2024

  7. [13]

    [gpt, 2023] Gpt-4 technical report

    2023

  8. [17]

    Kopiczko, Tijmen Blankevoort, and Yuki M

    [Kopiczko et al., 2024] Dawid J. Kopiczko, Tijmen Blankevoort, and Yuki M. Asano. Vera: Vector-based random matrix adaptation,

    2024

Show all 36 references
  1. [18]

    Learning multiple layers of features from tiny im- ages

    [Krizhevsky et al., 2009] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny im- ages

    2009

  2. [20]

    Prefix- tuning: Optimizing continuous prompts for generation,

    [Li and Liang, 2021] Xiang Lisa Li and Percy Liang. Prefix- tuning: Optimizing continuous prompts for generation,

    2021

  3. [21]

    Open challenges and opportunities in fed- erated foundation models towards biomedical healthcare

    [Li et al., 2024b] Xingyu Li, Lu Peng, Yuping Wang, and Weihua Zhang. Open challenges and opportunities in fed- erated foundation models towards biomedical healthcare. arXiv preprint arXiv:2405.06784,

    arXiv

  4. [22]

    Roberta: A robustly optimized bert pretraining approach,

    [Liu et al., 2019] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach,

    2019

  5. [23]

    Decoupled weight decay regularization

    [Loshchilov, 2017] I Loshchilov. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101,

    2017 arXiv

  6. [24]

    Parameter-efficient multi-task fine-tuning for transformers via shared hypernetworks

    [Mahabadi et al., 2021] Rabeeh Karimi Mahabadi, Sebas- tian Ruder, Mostafa Dehghani, and James Henderson. Parameter-efficient multi-task fine-tuning for transformers via shared hypernetworks. In Proceedings of the 59th An- nual Meeting of the Association for Computational Lin-...

    2021

  7. [25]

    Knowledge acquired by foundation models

    [Paaß and Giesselbach, 2023] Gerhard Paaß and Sven Gies- selbach. Knowledge acquired by foundation models. In Foundation Models for Natural Language Processing: Pre-trained Language Models Integrating Media , pages 161–185. Springer,

    2023

  8. [26]

    Pytorch: An imperative style, high- performance deep learning library

    [Paszke et al., 2019] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. Advances in neural in- formatio...

    2019

  9. [27]

    Improving language understanding by gen- erative pre-training

    [Radford and Narasimhan, 2018] Alec Radford and Karthik Narasimhan. Improving language understanding by gen- erative pre-training

    2018

  10. [28]

    Squad: 100,000+ ques- tions for machine comprehension of text

    [Rajpurkar, 2016] P Rajpurkar. Squad: 100,000+ ques- tions for machine comprehension of text. arXiv preprint arXiv:1606.05250,

    2016 arXiv

  11. [32]

    C-lstm: Enabling efficient lstm using structured com- pression techniques on fpgas

    [Wang et al., 2018] Shuo Wang, Zhe Li, Caiwen Ding, Bo Yuan, Qinru Qiu, Yanzhi Wang, and Yun Liang. C-lstm: Enabling efficient lstm using structured com- pression techniques on fpgas. In Proceedings of the 2018 ACM/SIGDA International Symposium on Field- Programmable Gate Arra...

    2018

  12. [33]

    [Wang et al., 2019] Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding,

    2019

  13. [34]

    [Warstadt et al., 2019] Alex Warstadt, Amanpreet Singh, and Samuel R. Bowman. Neural network acceptability judgments,

    2019

  14. [35]

    Adaptive budget allocation for parameter- efficient fine-tuning

    [Zhang et al., 2023] Qingru Zhang, Minshuo Chen, Alexan- der Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. Adaptive budget allocation for parameter- efficient fine-tuning. In The Eleventh International Con- ference on Learning Representations,

    2023

  15. [36]

    Xing, Hao Zhang, Joseph E

    [Zheng et al., 2023] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging llm-as-a- judge with mt-bench and chatbot arena, 2023

    2023

  16. [2005]

    Bert: Pre-training of deep bidi- rectional transformers for language understanding

    [Devlin, 2018] Jacob Devlin. Bert: Pre-training of deep bidi- rectional transformers for language understanding. arXiv preprint arXiv:1810.04805,

    2018 arXiv

  17. [2009]

    Conditional adapters: Parameter-efficient transfer learning with fast in- ference

    [Lei et al., 2023] Tao Lei, Junwen Bai, Siddhartha Brahma, Joshua Ainslie, Kenton Lee, Yanqi Zhou, Nan Du, Vin- cent Zhao, Yuexin Wu, Bo Li, et al. Conditional adapters: Parameter-efficient transfer learning with fast in- ference. Advances in Neural Information Processing Sys-...

    2023

  18. [2013]

    Lst: Ladder side-tuning for parameter and mem- ory efficient transfer learning

    [Sung et al., 2022] Yi-Lin Sung, Jaemin Cho, and Mohit Bansal. Lst: Ladder side-tuning for parameter and mem- ory efficient transfer learning. Advances in Neural Infor- mation Processing Systems, 35:12991–13005,

    2022

  19. [2015]

    Remote sensing image scene classification: Benchmark and state of the art

    [Cheng et al., 2017] Gong Cheng, Junwei Han, and Xiao- qiang Lu. Remote sensing image scene classification: Benchmark and state of the art. Proceedings of the IEEE, 105(10):1865–1883,

    2017

  20. [2016]

    Recursive deep models for semantic compositionality over a sentiment treebank

    [Socher et al., 2013] Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in ...

    2013

  21. [2017]

    End- to-end autonomous driving: Challenges and frontiers

    [Chen et al., 2024] Li Chen, Penghao Wu, Kashyap Chitta, Bernhard Jaeger, Andreas Geiger, and Hongyang Li. End- to-end autonomous driving: Challenges and frontiers. IEEE Transactions on Pattern Analysis and Machine In- telligence,

    2024

  22. [2018]

    Circnn: accelerat- ing and compressing deep neural networks using block- circulant weight matrices

    [Ding et al., 2017] Caiwen Ding, Siyu Liao, Yanzhi Wang, Zhe Li, Ning Liu, Youwei Zhuo, Chao Wang, Xuehai Qian, Yu Bai, Geng Yuan, et al. Circnn: accelerat- ing and compressing deep neural networks using block- circulant weight matrices. In Proceedings of the 50th An- nual IEE...

    2017

  23. [2019]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    [Hu et al., 2021] Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models,

    2021

  24. [2020]

    Semeval- 2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation

    [Cer et al., 2017] Daniel Cer, Mona Diab, Eneko Agirre, Inigo Lopez-Gazpio, and Lucia Specia. Semeval- 2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation. arXiv preprint arXiv:1708.00055,

    2017 arXiv

  25. [2021]

    Factoring matrices into the product of circulant and diagonal matrices

    [Huhtanen and Per¨am¨aki, 2015] Marko Huhtanen and Allan Per¨am¨aki. Factoring matrices into the product of circulant and diagonal matrices. Journal of Fourier Analysis and Applications, 21:1018 – 1033,

    2015

  26. [2022]

    Llama: Open and efficient foundation language models,

    [Touvron et al., 2023] Hugo Touvron, Thibaut Lavril, Gau- tier Izacard, Xavier Martinet, Marie-Anne Lachaux, Tim- oth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Ham- bro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open an...

    2023

  27. [2023]

    Parameter-efficient transfer learning for nlp

    [Houlsby et al., 2019] Neil Houlsby, Andrei Giurgiu, Stanis- law Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. In Interna- tional conference on machine learning, pages 2790...

    2019

  28. [2024]

    Language models are few-shot learn- ers

    [Brown et al., 2020] Tom Brown, Benjamin Mann, Nick Ry- der, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh,...

    2020

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.