Pith. sign in

REVIEW 4 major objections 5 minor 41 references

Block Circulant Adapter for Large Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A block circulant adapter can fine-tune large language models with far fewer trainable parameters and FLOPs than LoRA or FourierFT while matching their accuracy.

desk verdict A reasonable PEFT idea whose headline efficiency ratios are contradicted by its own figures; needs a corrected counting methodology before the central claim can be believed. read the letter →

arxiv 2505.00582 v2 pith:QHAQY5GR submitted 2025-05-01 cs.CL cs.LG

Xinyu Ding , Meiqi Wang , Siyu Liao , Zhongfeng Wang This is my paper · ORCID
classification cs.CLcs.LG
keywords blockcirculantmatrixparameter-efficientfine-tuninglargelanguagemodelsFFTFourierdomainLoRAadaptergradientexplosion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes the Block Circulant Adapter (BCA), a parameter-efficient fine-tuning method in which each weight-change matrix is a block circulant matrix: a block matrix whose blocks are circulant matrices, each determined by a single vector. Because a circulant matrix multiplies a vector through one-dimensional FFTs, the adapter stores $O(n^2/p)$ parameters and computes a forward pass in $O((n^2/p)\log p)$ operations. The paper claims that this makes BCA match LoRA, VeRA, and FourierFT on GLUE, Alpaca, and GSM8K while using about $16\times$ fewer trainable parameters than LoRA and about $32\times$ fewer FLOPs than FourierFT. The practical obstacle is that gradients of the block circulant parameters are amplified by block size $p$; the paper's fix is to divide the learning rate by $p$, which it shows restores convergence. If the efficiency ratios survive a consistent counting rule, BCA is a new, cheaper operating point between the parameter-sparse and FLOP-sparse corners of the PEFT design space.

What carries the argument

The central object is the block circulant matrix $B \in \mathbb{R}^{n \times n}$ with partition size $p$ and $n/p = q$ blocks per row and column, where each block $B_{i,j}$ is the circulant matrix generated by a vector $c_{i,j} \in \mathbb{R}^p$. The load-bearing identity is the FFT diagonalization of each block, $\mathrm{circ}(c)x = \mathrm{IFFT}(\mathrm{FFT}(c) \circ \mathrm{FFT}(x))$, extended in Eq. (4) so that each output block $h_i$ is computed by a single IFFT of the sum of element-wise FFT products. This structure is what turns the weight-change matrix into $n^2/p$ trainable parameters and $O((n^2/p)\log p)$ FLOPs per forward pass. The complementary mechanism is the training heuristic $\alpha \leftarrow \alpha/p$, which scales the learning rate by the block size to counteract the $p$-fold gradient amplification proved in Propositions 1–3 and observed in Fig. 2.

What would settle it

Recompute the FLOPs and parameter counts for BCA, LoRA, VeRA, and FourierFT on RoBERTa-large under one explicit counting rule (for example, forward plus backward, charging each complex FFT butterfly at its arithmetic cost) and check whether BCA's FLOPs ratio to FourierFT is the advertised $32\times$; Fig. 4's plotted values appear to imply a ratio closer to $578\times$, so a mismatch would falsify the cost claim independently of accuracy.

Watch

Extended reading notes

Core claim

The paper's central discovery is that a block circulant matrix can work as a mergeable adapter for fine-tuning large language models: after training, the learned block circulant weight change $B$ is added directly to the pretrained weight $W$, so the deployed model costs no extra inference overhead. The adapter is built from $q \times q$ vectors $c_{i,j} \in \mathbb{R}^p$, each generating a $p \times p$ circulant block, and the forward map is computed with the identity $h_i = \mathrm{IFFT}\left(\sum_j \mathrm{FFT}(c_{i,j}) \circ \mathrm{FFT}(x_j)\right)$, so storage is linear in the block vectors and computation stays log-linear. The paper also proves (Propositions 1–3) that first-order gradients of this parameterization are amplified by a factor of $p$ relative to a dense matrix, gives a single-layer simulation showing the gradient magnitude grows with $p$, and demonstrates that the heuristic $\alpha \leftarrow \alpha/p$ turns a diverging training run into a convergent one. On RoBERTa-base, RoBERTa-large, and LLaMA2-7B, BCA reaches average scores within run-to-run noise of the compared adapters while reporting substantially lower parameter counts than LoRA/VeRA and lower FLOPs than FourierFT.

Load-bearing premise

The headline efficiency claim rests on an unstated rule for counting FLOPs and trainable parameters: the paper never says whether counts are forward-only or forward-plus-backward, or how FFT primitives are charged, so a consistent counting convention must reproduce the advertised $32\times$ FLOPs reduction before the cost advantage is established.

Editorial extensions

If this is right

  • Because BCA is mergeable, once fine-tuning finishes the block circulant matrix can be summed into the pretrained weights, leaving inference latency and memory exactly the same as the base model.
  • Setting the partition size to the full hidden dimension ($p = n$) gives a plain circulant adapter with $O(n \log n)$ computation and the smallest possible parameter count, while smaller $p$ gives more capacity at higher cost, making $p$ a direct efficiency-accuracy dial.
  • On the reported benchmarks, BCA's task scores sit within one standard deviation of LoRA, VeRA, and FourierFT, so the efficiency gain does not come at a visible accuracy cost.
  • BCA fills the gap between LoRA's low-FLOP/high-parameter design and FourierFT's high-FLOP/low-parameter design, offering a middle point where neither budget dominates.
  • The learning-rate heuristic is coupled to block size, so any deployment that changes $p$ must rescale $\alpha$ accordingly to retain the convergence guarantee.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the $p$-scaled learning rate is not specific to circulant matrices; any FFT-parameterized structured adapter with gradients proportional to a transform length might use the same $\alpha/p$ trick to stabilize training.
  • Editorial inference: the paper reports asymptotic FLOPs rather than wall-clock time; an end-to-end GPU profile of BCA versus FourierFT, including the backward pass and batched FFT kernels, would tell whether the asymptotic advantage survives real implementations.
  • Editorial inference: treating $p$ as a per-layer hyperparameter, with small blocks in attention and large blocks in feed-forward layers, is a natural extension the paper leaves implicit, and the method's own complexity formulas predict where such a split would pay off.
  • Editorial inference: since block circulant matrices generalize from circulant ($p=n$) to dense ($p=1$), BCA occupies a spectrum of PEFT designs; probing intermediate $p$ values outside the reported set would map the full parameter-FLOP-performance frontier.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes the Block Circulant Adapter (BCA), a parameter-efficient fine-tuning method in which the weight update matrix is a block circulant matrix computed via 1D FFT. To prevent divergence, the authors divide the base learning rate by the block size p (Eq. 13). They report GLUE results for RoBERTa-base/large and MT-Bench/GSM8K results for LLaMA2-7B, and claim in the abstract that BCA uses 14x fewer parameters than VeRA, 16x fewer than LoRA, and 32x fewer FLOPs than FourierFT while matching task performance. The manuscript also gives propositions intended to show that block circulant matrices have gradients proportional to p and that the learning-rate heuristic mitigates this.

Significance. If the efficiency claims held, BCA would be a valuable point in the PEFT design space: linear parameter storage and loglinear FFT-based computation with mergeable adapters. The experimental design is reasonable: multiple baselines (LoRA, VeRA, FourierFT), multiple seeds on GLUE, and results on a 7B model. The stability heuristic and the attempt to explain a known training instability are also useful. However, I cannot endorse the central efficiency claim in its present form, because the abstract and Section 5.3 cite ratios that are inconsistent with the paper's own Figures and Tables, and no FLOPs-counting protocol is given.

major comments (4)
  1. [Abstract and Section 5.3] The claimed efficiency ratios are inconsistent with the paper's own data. Figure 4 shows FourierFT at 5.2G FLOPs on RoBERTa-large, while BCA is 7.5M (p=1024) or 9.0M (p=512), a factor of roughly 693x or 578x, not 32x; on RoBERTa-base the factor is about 368-518x, not 32x. Table 2 gives FourierFT 133.14G vs BCA 0.08G-0.32G on Alpaca (roughly 400-1600x). The 14x vs VeRA claim is also unsupported: RoBERTa-base VeRA 43K vs BCA 18-24K is 1.8-2.4x; RoBERTa-large VeRA 61K vs BCA 49K (p=1024) is 1.2x, and BCA p=512 (98K) has more parameters; Table 2 gives VeRA 1.65M vs BCA 1.05M (1.6x). The 16x vs LoRA claim holds only for selected configurations, and for RoBERTa-large it holds for p=1024, the configuration with the worst average GLUE score (87.2 vs 87.8 for LoRA), whereas the best BCA (p=512, 88.1) is only about 8x smaller in parameters. The headline ratios must be corrected to match the measurements, or the measurements must be recomputed under a clearly stated counting rule.
  2. [Section 5 (Metrics) and Section 5.3] No counting methodology is defined for FLOPs or parameters. The manuscript never states whether FLOPs count forward pass only or forward+backward, how FFT operations are counted (complex vs real, number of butterflies, etc.), whether the classification head and biases are included, or how the LLaMA FLOPs in Table 2 are derived. Without this, the efficiency comparisons cannot be reproduced or falsified. Please add an explicit complexity-analysis section or appendix with formulas for each baseline and for BCA, and state exactly which tensors are counted.
  3. [Section 4.1, Proposition 2] The proposition as stated does not prove the claimed gradient explosion. The inequality min{∇f(c)} ≥ n × min{∇f(A)} is a statement about the algebraic minimum of the gradient entries, not about their magnitude or norm. If the entries are negative, multiplying the minimum by n makes it more negative, so the inequality can hold while the gradient entries of the circulant matrix are smaller in magnitude than n times the dense gradient. The empirical 'Gradient Mean' curves in Figure 2 are supportive evidence of a p-scaling effect, but the theoretical claim in Corollary 1 should be either restated as a statement about mean absolute gradient (with proof) or downgraded to an empirical observation.
  4. [Section 5.1 and Section 4.3] The stability heuristic α←α/p is only demonstrated on RoBERTa-large MRPC (Figure 3). The paper claims this 'can effectively ensure a stable training process' generally, but no ablation varying p and learning rate jointly, no task-by-task divergence statistics, and no analysis of interaction with the optimizer are provided. Since stable convergence is a central component of the method's contribution, the generality of the heuristic should be tested or the claim should be narrowed.
minor comments (5)
  1. [Throughout] The text contains several typos: 'noval' in the Conclusion, 'frourier' in Section 3.1, 'empircal' and 'finetuing' in Section 4.3, and 'RoBERTA-base' in Section 5.3.
  2. [Figure 4] The axis labels are inconsistent: the parameter axis reads 'Param. (M)' but the bar labels are in K (e.g., 49K). Please use consistent units.
  3. [Table 2] Table 2 reports no standard deviations or number of runs for MT-Bench and GSM8K, whereas Table 1 does; please add this information or explain why it is omitted.
  4. [References] Reference [gpt, 2023] lacks author and title details; please expand it into a full citation.
  5. [Equation (13)] Equation (13) is stated without specifying whether the learning-rate division applies only to the circulant adapter parameters or to all trainable parameters including the classification head; please clarify.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: BCA's construction, gradient heuristic, and efficiency claims are self-contained; the inconsistent headline ratios are a reproducibility issue, not circularity.

full rationale

The paper's derivation chain does not reduce any output to its inputs. The block circulant adapter is constructed directly from the definition of block circulant matrices built from vectors c_{i,j} and the textbook FFT identity circ(c)x = IFFT(FFT(c) * FFT(x)) (Eq. 2). The claimed efficiency (linear parameter storage O(n^2/p), log-linear FLOPs O((n^2/p) log p)) follows algebraically from this construction and is evaluated against baseline counts in Fig. 4, not fitted to performance. No hyperparameter is fitted to a subset and then relabeled as a prediction: the learning-rate heuristic alpha <- alpha/p (Eq. 13) is derived from the in-paper gradient bound (Prop. 3, Cor. 1) and verified by convergence curves in Fig. 3; it does not encode benchmark accuracies. Prior block-circulant work by the authors is cited for the matrix structure and its FFT multiplication, but those facts are independently re-derived here and are textbook results (Oppenheim 1999), so the self-citations are not load-bearing. The inconsistency between the abstract's '14x/16x/32x' ratios and the values in Fig. 4/Table 2 is an internal arithmetic/counting-protocol problem and a correctness risk rather than circularity: nothing in those ratios is defined in terms of the performance target. The central claim is tested on external benchmarks (GLUE, MT-Bench, GSM8K), so the derivation and evaluation are self-contained.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical entities. The key free parameters are the block size p (tuned per model/task) and the base learning rate (inherited from FourierFT). The axioms include standard FFT mathematics plus two ad hoc assumptions tied to the stability heuristic and the undefined FLOPs accounting.

free parameters (2)
  • block size p = 256/768 for RoBERTa-base; 512/1024 for RoBERTa-large; 128/256/512/1024 for LLaMA2-7B
    Chosen per model and task; the paper reports the best-performing p (e.g., p=512 for RoBERTa-large), so p is effectively tuned.
  • base learning rate = 0.06 (from FourierFT)
    Adopted from FourierFT and then scaled by 1/p; no learning rate sweep for BCA is reported.
assumptions (4)
  • standard math Block circulant matrix-vector products are exactly computed via 1D FFT with O(n^2/p log p) cost.
    Section 3.2 Eq. (4); standard property of circulant matrices.
  • ad hoc to paper Proposition 2's minimum-entry bound implies a p-fold gradient magnitude explosion.
    The proof uses min of entries, not a norm or magnitude; a sum of p products can be small due to cancellation, so the conclusion is not established. This is a load-bearing step for the heuristic justification.
  • ad hoc to paper Dividing the learning rate by p makes update steps comparable to dense-matrix updates and yields stable training.
    Section 4.3 Eq. (13); the heuristic is motivated by the flawed gradient-explosion claim, not by an independent derivation.
  • domain assumption FLOPs counts across LoRA, VeRA, FourierFT, and BCA use the same accounting rules.
    Section 5.3 and Fig. 4 present cross-method FLOPs comparisons without defining the counting method; the claimed ratios contradict the figure.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Block Circulant Adapter for Large Language Models." pith.science (2026). https://pith.science/paper/QHAQY5GR

@misc{pith2026250500582,
  author       = {Pith},
  title        = {Pith review of: Block Circulant Adapter for Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QHAQY5GR}},
  note         = {Machine review of arXiv:2505.00582}
}
abstract

Fine-tuning large language models (LLMs) is difficult due to their huge model size. Recent Fourier domain-based methods show potential for reducing fine-tuning costs. We propose a block circulant matrix-based fine-tuning method with a stable training heuristic to leverage the properties of circulant matrices and one-dimensional Fourier transforms to reduce storage and computation costs. Experiments show that our method uses $14\times$ less number of parameters than VeRA, $16\times$ smaller than LoRA and $32\times$ less FLOPs than FourierFT, while maintaining close or better task performance. Our approach presents a promising way in frequency domain to fine-tune large models on downstream tasks.

Figures

Figures reproduced from arXiv: 2505.00582 by the authors.

Figure 1
Figure 1. Illustration of block circulant adapter. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Gradient of single layer neural network with [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Training loss curve of the RoBERTa-large model on the [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Complexity comparison of different adapters. The larger [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 26 canonical work pages

  1. [1]

    Lamda: Large model fine-tuning via spectrally decomposed low-dimensional adaptation,

    [Azizi et al., 2024] Seyedarmin Azizi, Souvik Kundu, and Massoud Pedram. Lamda: Large model fine-tuning via spectrally decomposed low-dimensional adaptation,

    work page 2024

  2. [7]

    Training verifiers to solve math word prob- lems

    [Cobbe et al., 2021] Karl Cobbe, Vineet Kosaraju, Moham- mad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word prob- lems. arXiv preprint arXiv:2110.14168,

    arXiv 2021

  3. [11]

    BERT: pre-training of deep bidirectional transformers for language understand- ing

    [Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understand- ing. In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human ...

    work page 2019

  4. [13]

    Automatically constructing a corpus of sentential para- phrases

    [Dolan and Brockett, 2005] Bill Dolan and Chris Brockett. Automatically constructing a corpus of sentential para- phrases. In Third international workshop on paraphrasing (IWP2005),

    work page 2005

  5. [14]

    Parameter-efficient fine-tuning with discrete fourier trans- form

    [Gao et al., 2024] Ziqi Gao, Qichao Wang, Aochuan Chen, Zijing Liu, Bingzhe Wu, Liang Chen, and Jia Li. Parameter-efficient fine-tuning with discrete fourier trans- form. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024 ,

    work page 2024

  6. [15]

    [gpt, 2023] Gpt-4 technical report

    work page 2023

  7. [17]

    Lora: Low-rank adaptation of large language models

    [Hu et al., 2021a] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685,

    arXiv

  8. [18]

    Kopiczko, Tijmen Blankevoort, and Yuki M

    [Kopiczko et al., 2024] Dawid J. Kopiczko, Tijmen Blankevoort, and Yuki M. Asano. Vera: Vector-based random matrix adaptation,

    work page 2024

Show all 41 references
  1. [19]

    A theoretical framework for back- propagation

    [LeCun et al., 1988] Yann LeCun, D Touresky, G Hinton, and T Sejnowski. A theoretical framework for back- propagation. In Proceedings of the 1988 connectionist models summer school, volume 1, pages 21–28,

    1988

  2. [21]

    Roberta: A robustly optimized bert pretraining approach

    [Liu et al., 2019] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. ArXiv, abs/1907.11692,

    2019 arXiv

  3. [22]

    Fantastically or- dered prompts and where to find them: Overcoming few- shot prompt order sensitivity

    [Lu et al., 2022] Yao Lu, Max Bartolo, Alastair Moore, Se- bastian Riedel, and Pontus Stenetorp. Fantastically or- dered prompts and where to find them: Overcoming few- shot prompt order sensitivity. In Proceedings of the 60th Annual Meeting of the Association for Computationa...

    2022

  4. [23]

    ACDC: A struc- tured efficient linear layer

    [Moczulski et al., 2016] Marcin Moczulski, Misha Denil, Jeremy Appleyard, and Nando de Freitas. ACDC: A struc- tured efficient linear layer. In Yoshua Bengio and Yann LeCun, editors, 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, Ma...

    2016

  5. [26]

    Pytorch: An imperative style, high- performance deep learning library

    [Paszke et al., 2019] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. Advances in neural in- formatio...

    2019

  6. [27]

    Sftc: Machine unlearning via selective fine-tuning and targeted confusion

    [Perifanis et al., 2024] Vasileios Perifanis, Efstathios Kary- pidis, Nikos Komodakis, and Pavlos Efraimidis. Sftc: Machine unlearning via selective fine-tuning and targeted confusion. In European Interdisciplinary Cybersecurity Conference, pages 29–36,

    2024

  7. [28]

    Squad: 100,000+ ques- tions for machine comprehension of text

    [Rajpurkar, 2016] P Rajpurkar. Squad: 100,000+ ques- tions for machine comprehension of text. arXiv preprint arXiv:1606.05250,

    2016 arXiv

  8. [29]

    Recursive deep models for semantic compositionality over a sentiment treebank

    [Socher et al., 2013] Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in ...

    2013

  9. [31]

    Hashimoto

    [Taori et al., 2023] Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction-following llama model. https://github.com/ tatsu-lab/stanford alpaca,

    2023

  10. [32]

    Learning compressed transforms with low displacement rank

    [Thomas et al., 2018] Anna Thomas, Albert Gu, Tri Dao, Atri Rudra, and Christopher R ´e. Learning compressed transforms with low displacement rank. Advances in neu- ral information processing systems, 31,

    2018

  11. [34]

    Bow- man

    [Wang et al., 2018] Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bow- man. Glue: A multi-task benchmark and analysis plat- form for natural language understanding. In Black- boxNLP@EMNLP,

    2018

  12. [35]

    Multitask prompt tuning enables parameter-efficient transfer learning

    [Wang et al., 2023] Zhen Wang, Rameswar Panda, Leonid Karlinsky, Rog ´erio Schmidt Feris, Huan Sun, and Yoon Kim. Multitask prompt tuning enables parameter-efficient transfer learning. ArXiv, abs/2303.02861,

    2023 arXiv

  13. [36]

    [Warstadt et al., 2019] Alex Warstadt, Amanpreet Singh, and Samuel R. Bowman. Neural network acceptability judgments,

    2019

  14. [37]

    Joe Qin, Xiao- hui Tao, and Fu Lee Wang

    [Xu et al., 2023] Lingling Xu, Haoran Xie, S. Joe Qin, Xiao- hui Tao, and Fu Lee Wang. Parameter-efficient fine-tuning methods for pretrained language models: A critical review and assessment. ArXiv, abs/2312.12148,

    2023 arXiv

  15. [38]

    Adadelta: an adaptive learning rate method

    [Zeiler, 2012] Matthew D Zeiler. Adadelta: an adaptive learning rate method. arXiv preprint arXiv:1212.5701 ,

    2012 arXiv

  16. [40]

    Xing, Hao Zhang, Joseph E

    [Zheng et al., 2023] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging llm-as-a- judge with mt-bench and chatbot arena,

    2023

  17. [41]

    Pre-trained large language models use fourier features to compute addition

    [Zhou et al., 2024b] Tianyi Zhou, Deqing Fu, Vatsal Sha- ran, and Robin Jia. Pre-trained large language models use fourier features to compute addition. arXiv preprint arXiv:2406.03445, 2024

    2024 arXiv

  18. [1988]

    Circconv: A structured convolution with low complexity

    [Liao and Yuan, 2019] Siyu Liao and Bo Yuan. Circconv: A structured convolution with low complexity. In Proceed- ings of the AAAI Conference on Artificial Intelligence, vol- ume 33, pages 4287–4294,

    2019

  19. [1994]

    [Brown et al., 2020] Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, ...

    2020

  20. [1999]

    Automatic differentiation in pytorch

    [Paszke et al., 2017] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary De- Vito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch

    2017

  21. [2005]

    Monarch: Expressive structured matrices for efficient and accurate training

    [Dao et al., 2022] Tri Dao, Beidi Chen, Nimit S Sohoni, Ar- jun Desai, Michael Poli, Jessica Grogan, Alexander Liu, Aniruddh Rao, Atri Rudra, and Christopher R´e. Monarch: Expressive structured matrices for efficient and accurate training. In International Conference on Machin...

    2022

  22. [2012]

    Theoretical properties for neural networks with weight matrices of low displacement rank

    [Zhao et al., 2017] Liang Zhao, Siyu Liao, Yanzhi Wang, Zhe Li, Jian Tang, and Bo Yuan. Theoretical properties for neural networks with weight matrices of low displacement rank. In international conference on machine learning , pages 4082–4090. PMLR,

    2017

  23. [2013]

    Lst: Ladder side-tuning for parameter and mem- ory efficient transfer learning

    [Sung et al., 2022] Yi-Lin Sung, Jaemin Cho, and Mohit Bansal. Lst: Ladder side-tuning for parameter and mem- ory efficient transfer learning. Advances in Neural Infor- mation Processing Systems, 35:12991–13005,

    2022

  24. [2015]

    Prompt sapper: a llm-empowered production tool for building ai chains

    [Cheng et al., 2024] Yu Cheng, Jieshan Chen, Qing Huang, Zhenchang Xing, Xiwei Xu, and Qinghua Lu. Prompt sapper: a llm-empowered production tool for building ai chains. ACM Transactions on Software Engineering and Methodology, 33(5):1–24,

    2024

  25. [2016]

    Discrete-time sig- nal processing

    [Oppenheim, 1999] Alan V Oppenheim. Discrete-time sig- nal processing. Pearson Education India,

    1999

  26. [2017]

    An exploration of parameter redundancy in deep networks with circulant projections

    [Cheng et al., 2015] Yu Cheng, Felix X Yu, Rogerio S Feris, Sanjiv Kumar, Alok Choudhary, and Shi-Fu Chang. An exploration of parameter redundancy in deep networks with circulant projections. In Proceedings of the IEEE international conference on computer vision, pages 2857– 2865,

    2015

  27. [2018]

    Llama: Open and efficient foundation language models

    [Touvron et al., 2023] Hugo Touvron, Thibaut Lavril, Gau- tier Izacard, Xavier Martinet, Marie-Anne Lachaux, Tim- oth´ee Lacroix, Baptiste Rozi `ere, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and...

    2023 arXiv

  28. [2019]

    Circnn: accelerat- ing and compressing deep neural networks using block- circulant weight matrices

    [Ding et al., 2017] Caiwen Ding, Siyu Liao, Yanzhi Wang, Zhe Li, Ning Liu, Youwei Zhuo, Chao Wang, Xuehai Qian, Yu Bai, Geng Yuan, et al. Circnn: accelerat- ing and compressing deep neural networks using block- circulant weight matrices. In Proceedings of the 50th An- nual IEE...

    2017

  29. [2020]

    [Cer et al., 2017] Daniel Cer, Mona Diab, Eneko Agirre, Inigo Lopez-Gazpio, and Lucia Specia

    Curran Associates Inc. [Cer et al., 2017] Daniel Cer, Mona Diab, Eneko Agirre, Inigo Lopez-Gazpio, and Lucia Specia. Semeval- 2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation. arXiv preprint arXiv:1708.00055,

    2017 arXiv

  30. [2021]

    The pascal recognising textual entailment challenge

    [Dagan et al., 2005] Ido Dagan, Oren Glickman, and Bernardo Magnini. The pascal recognising textual entailment challenge. In Machine learning challenges workshop, pages 177–190. Springer,

    2005

  31. [2022]

    Qlora: Efficient fine- tuning of quantized llms

    [Dettmers et al., 2024] Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient fine- tuning of quantized llms. Advances in Neural Information Processing Systems, 36,

    2024

  32. [2023]

    Parameter-efficient transfer learning for nlp

    [Houlsby et al., 2019] Neil Houlsby, Andrei Giurgiu, Stanis- law Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. In Interna- tional conference on machine learning, pages 2790...

    2019

  33. [2024]

    Learning long-term dependencies with gradient descent is difficult

    [Bengio et al., 1994] Yoshua Bengio, Patrice Simard, and Paolo Frasconi. Learning long-term dependencies with gradient descent is difficult. IEEE transactions on neural networks, 5(2):157–166,

    1994

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.