{"id":"0d4c68ed-97b4-4c8b-8b7a-aa1fcaf6dc35","arxiv_id":"2505.00582","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"Block Circulant Adapter (BCA) fine-tunes LLMs with block circulant weight matrices, matching LoRA/FourierFT task performance while using fewer parameters than LoRA and fewer FLOPs than FourierFT.","lead":"The paper proposes a new way to fine-tune large language models by using block circulant matrices as trainable adapters, aiming to cut storage and compute. It reports competitive task scores with far fewer trainable parameters than LoRA and much lower FLOPs than FourierFT, though several claimed ratios do not match its own tables.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline efficiency ratios (14x vs VeRA, 16x vs LoRA, 32x vs FourierFT) are inconsistent with the paper's own Fig. 4/Table 2, and the counting method is undefined; the central efficiency claim is not presently established.","rationale":"I am not disputing that BCA is efficient in an absolute structural sense: the O(n^2/p) parameter count and O((n^2/p) log p) FLOP count are real design features, and the task results in Table 1 and the LLaMA2 MT-Bench numbers are competitive. The load-bearing problem is that the paper's central quantified claims, especially the abstract's 14x/16x/32x ratios, are not reproducible from the paper's own reported data. An efficiency paper must have internally consistent accounting; otherwise the central claim cannot be evaluated. The reader's weakest_assumption already identified the undefined FLOPs methodology and the Fig. 4 mismatch; I agree with that and extend it: the parameter ratios are also inconsistent, and the configurations that realize the headline parameter ratios are not the configurations with the best accuracy. The gradient-explosion proofs in Propositions 1-3 are also informal and rely on comparing minima of possibly sign-indefinite entries, but that is secondary to the efficiency accounting. The issue is concrete and fixable, so conditional acceptance is appropriate; the authors should correct the numbers or state the counting protocol that makes them true.","tokens_in":13456,"tokens_out":7353,"duration_ms":68486,"concrete_test":"Write a one-page counting protocol (forward-only FLOPs, FFT length-p operation cost, per-module parameter attribution) and recompute every row of Fig. 4 and Table 2 using that protocol, including all BCA partition sizes. Then verify the abstract's three ratios: 14x vs VeRA, 16x vs LoRA, 32x vs FourierFT. A minimal version: recompute RoBERTa-large p=512 and p=1024 FLOPs with the same FFT cost model used for FourierFT; if the ratio to FourierFT is about 580-690x rather than 32x, the headline number is wrong and the abstract and Section 5.3 must be revised before the efficiency claim can be evaluated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5.3 and the abstract assert BCA uses 32x fewer FLOPs than FourierFT and 14x fewer parameters than VeRA. Figure 4 does not support either ratio: on RoBERTa-large, FourierFT is 5.2G FLOPs while BCA is 7.5-9.0M FLOPs (about 580-690x fewer), and VeRA is 61K parameters while BCA is 49K (p=1024) or 98K (p=512) parameters (1.2-1.6x, not 14x). On RoBERTa-base, FourierFT 1.4G vs BCA 2.7-3.8M is about 368-518x, and VeRA 43K vs BCA 18-24K is about 1.8-2.4x. Table 2 gives VeRA 1.65M vs BCA 1.05M at p=1024 (1.6x) and 2.10M at p=512 (more than VeRA). The 16x-vs-LoRA parameter ratio holds only for RoBERTa-base p=768 (0.3M/18K), RoBERTa-large p=1024 (0.8M/49K), or LLaMA2-Alpaca p=512 (33.55M/2.10M); the RoBERTa-large p=1024 configuration is the one with the worst average GLUE score (87.2 vs 87.8 for LoRA), while the best BCA config (p=512, 88.1) has only about 8x fewer parameters than LoRA and more parameters than VeRA. No counting protocol is defined (forward vs forward+backward, FFT operation count, parameter attribution of biases/heads), so these ratios cannot be reproduced or falsified from the paper.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes the Block Circulant Adapter (BCA), a parameter-efficient fine-tuning method in which the weight update matrix is a block circulant matrix computed via 1D FFT. To prevent divergence, the authors divide the base learning rate by the block size p (Eq. 13). They report GLUE results for RoBERTa-base/large and MT-Bench/GSM8K results for LLaMA2-7B, and claim in the abstract that BCA uses 14x fewer parameters than VeRA, 16x fewer than LoRA, and 32x fewer FLOPs than FourierFT while matching task performance. The manuscript also gives propositions intended to show that block circulant matrices have gradients proportional to p and that the learning-rate heuristic mitigates this.","tokens_in":13892,"tokens_out":6666,"duration_ms":60698,"significance":"If the efficiency claims held, BCA would be a valuable point in the PEFT design space: linear parameter storage and loglinear FFT-based computation with mergeable adapters. The experimental design is reasonable: multiple baselines (LoRA, VeRA, FourierFT), multiple seeds on GLUE, and results on a 7B model. The stability heuristic and the attempt to explain a known training instability are also useful. However, I cannot endorse the central efficiency claim in its present form, because the abstract and Section 5.3 cite ratios that are inconsistent with the paper's own Figures and Tables, and no FLOPs-counting protocol is given.","major_comments":[{"comment":"The claimed efficiency ratios are inconsistent with the paper's own data. Figure 4 shows FourierFT at 5.2G FLOPs on RoBERTa-large, while BCA is 7.5M (p=1024) or 9.0M (p=512), a factor of roughly 693x or 578x, not 32x; on RoBERTa-base the factor is about 368-518x, not 32x. Table 2 gives FourierFT 133.14G vs BCA 0.08G-0.32G on Alpaca (roughly 400-1600x). The 14x vs VeRA claim is also unsupported: RoBERTa-base VeRA 43K vs BCA 18-24K is 1.8-2.4x; RoBERTa-large VeRA 61K vs BCA 49K (p=1024) is 1.2x, and BCA p=512 (98K) has more parameters; Table 2 gives VeRA 1.65M vs BCA 1.05M (1.6x). The 16x vs LoRA claim holds only for selected configurations, and for RoBERTa-large it holds for p=1024, the configuration with the worst average GLUE score (87.2 vs 87.8 for LoRA), whereas the best BCA (p=512, 88.1) is only about 8x smaller in parameters. The headline ratios must be corrected to match the measurements, or the measurements must be recomputed under a clearly stated counting rule.","section":"Abstract and Section 5.3"},{"comment":"No counting methodology is defined for FLOPs or parameters. The manuscript never states whether FLOPs count forward pass only or forward+backward, how FFT operations are counted (complex vs real, number of butterflies, etc.), whether the classification head and biases are included, or how the LLaMA FLOPs in Table 2 are derived. Without this, the efficiency comparisons cannot be reproduced or falsified. Please add an explicit complexity-analysis section or appendix with formulas for each baseline and for BCA, and state exactly which tensors are counted.","section":"Section 5 (Metrics) and Section 5.3"},{"comment":"The proposition as stated does not prove the claimed gradient explosion. The inequality min{∇f(c)} ≥ n × min{∇f(A)} is a statement about the algebraic minimum of the gradient entries, not about their magnitude or norm. If the entries are negative, multiplying the minimum by n makes it more negative, so the inequality can hold while the gradient entries of the circulant matrix are smaller in magnitude than n times the dense gradient. The empirical 'Gradient Mean' curves in Figure 2 are supportive evidence of a p-scaling effect, but the theoretical claim in Corollary 1 should be either restated as a statement about mean absolute gradient (with proof) or downgraded to an empirical observation.","section":"Section 4.1, Proposition 2"},{"comment":"The stability heuristic α←α/p is only demonstrated on RoBERTa-large MRPC (Figure 3). The paper claims this 'can effectively ensure a stable training process' generally, but no ablation varying p and learning rate jointly, no task-by-task divergence statistics, and no analysis of interaction with the optimizer are provided. Since stable convergence is a central component of the method's contribution, the generality of the heuristic should be tested or the claim should be narrowed.","section":"Section 5.1 and Section 4.3"}],"minor_comments":[{"comment":"The text contains several typos: 'noval' in the Conclusion, 'frourier' in Section 3.1, 'empircal' and 'finetuing' in Section 4.3, and 'RoBERTA-base' in Section 5.3.","section":"Throughout"},{"comment":"The axis labels are inconsistent: the parameter axis reads 'Param. (M)' but the bar labels are in K (e.g., 49K). Please use consistent units.","section":"Figure 4"},{"comment":"Table 2 reports no standard deviations or number of runs for MT-Bench and GSM8K, whereas Table 1 does; please add this information or explain why it is omitted.","section":"Table 2"},{"comment":"Reference [gpt, 2023] lacks author and title details; please expand it into a full citation.","section":"References"},{"comment":"Equation (13) is stated without specifying whether the learning-rate division applies only to the circulant adapter parameters or to all trainable parameters including the classification head; please clarify.","section":"Equation (13)"}],"recommendation":"major_revision","confidential_remarks":"The abstract's headline ratios are not supported by the paper's own figures; this is a credibility issue that must be resolved before publication. The authors should either recompute the ratios under a transparent counting protocol or revise the abstract. The FLOPs-counting ambiguity also affects comparisons with prior work. I see no sign of fabrication, but the current presentation would mislead readers."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The block circulant adapter (BCA) is a plausible addition to the PEFT toolbox, and the GLUE/LLaMA experiments suggest it works about as well as LoRA or FourierFT. But the abstract's efficiency claims—14× fewer parameters than VeRA, 32× fewer FLOPs than FourierFT—are not supported by the paper's own numbers. That is the load-bearing problem.\n\nWhat is new here: applying block circulant structure as a trainable adapter is a direct but reasonable extension of prior block-circulant compression (Cheng et al., Ding et al.) and FourierFT. The α/p learning-rate heuristic is a practical fix for the divergence you can see in the loss curves, and the single-layer simulation gives some empirical weight to the story. The experiments cover both RoBERTa and LLaMA2 with several partition sizes, and the task results are honestly reported with error bars on GLUE. Credit where it's due: the method is not a random hack; the authors clearly know the relevant literature.\n\nThe soft spots are larger. First, the counting methodology for FLOPs and parameters is never defined. Figure 4 shows FourierFT at 5.2G FLOPs on RoBERTa-large and BCA at 7.5–9.0M, which is about 600×, not 32×. VeRA is 61K parameters there versus BCA's 49K–98K, which is 1.2–1.6×, not 14×. Table 2 repeats the mismatch. The 16×-versus-LoRA parameter ratio holds only for the smallest partition sizes, and the best average GLUE score (p=512 on large) is only about 8× fewer parameters than LoRA. These inconsistencies suggest a different—likely looser—counting rule than the plots imply, and without a stated protocol the headline ratios are unfalsifiable. Second, the gradient-explosion proof in Propositions 2–3 is shaky: the step from a sum of products to a bound involving n times the minimum is not justified in general, and using minima of unsigned gradients is sloppy. The simulation shows a real phenomenon, but the theory overclaims. Third, the LLaMA2 results have no error bars, which weakens claims of parity on MT-Bench.\n\nNone of these are fatal to the core idea. They are fixable: define exactly what counts as a parameter and a FLOP, recompute the ratios, correct the abstract, tighten the proof to a norm-based statement, and add error bars for LLaMA2. As written, the efficiency claim is not established, and that is the main reason to demand a major revision.\n\nWho is this for? If you work on structured PEFT methods, BCA is worth a look as a data point. But don't cite the headline ratios until they are corrected. A serious editor should send this to peer review—the idea is worthwhile and the flaws are addressable—but my recommendation to the authors would be: fix the counting, tone down the abstract, and resubmit.","headline":"A reasonable PEFT idea whose headline efficiency ratios are contradicted by its own figures; needs a corrected counting methodology before the central claim can be believed.","tokens_in":14417,"tokens_out":2917,"would_cite":false,"duration_ms":29469,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A block circulant adapter can fine-tune large language models with far fewer trainable parameters and FLOPs than LoRA or FourierFT while matching their accuracy.","keywords":["block circulant matrix","parameter-efficient fine-tuning","large language models","FFT","Fourier domain","LoRA","adapter","gradient explosion"],"falsifier":"Recompute the FLOPs and parameter counts for BCA, LoRA, VeRA, and FourierFT on RoBERTa-large under one explicit counting rule (for example, forward plus backward, charging each complex FFT butterfly at its arithmetic cost) and check whether BCA's FLOPs ratio to FourierFT is the advertised $32\\times$; Fig. 4's plotted values appear to imply a ratio closer to $578\\times$, so a mismatch would falsify the cost claim independently of accuracy.","tokens_in":13250,"feed_emoji":"🌀","tokens_out":10653,"duration_ms":90013,"temperature":0.7,"pith_summary":"The paper proposes the Block Circulant Adapter (BCA), a parameter-efficient fine-tuning method in which each weight-change matrix is a block circulant matrix: a block matrix whose blocks are circulant matrices, each determined by a single vector. Because a circulant matrix multiplies a vector through one-dimensional FFTs, the adapter stores $O(n^2/p)$ parameters and computes a forward pass in $O((n^2/p)\\log p)$ operations. The paper claims that this makes BCA match LoRA, VeRA, and FourierFT on GLUE, Alpaca, and GSM8K while using about $16\\times$ fewer trainable parameters than LoRA and about $32\\times$ fewer FLOPs than FourierFT. The practical obstacle is that gradients of the block circulant parameters are amplified by block size $p$; the paper's fix is to divide the learning rate by $p$, which it shows restores convergence. If the efficiency ratios survive a consistent counting rule, BCA is a new, cheaper operating point between the parameter-sparse and FLOP-sparse corners of the PEFT design space.","feed_headline":"Block circulant adapter matches LoRA with 16× fewer parameters","feed_subtitle":"Using 1D FFTs instead of 2D, the adapter lowers FLOPs versus FourierFT while keeping GLUE and GSM8K accuracy.","key_machinery":"The central object is the block circulant matrix $B \\in \\mathbb{R}^{n \\times n}$ with partition size $p$ and $n/p = q$ blocks per row and column, where each block $B_{i,j}$ is the circulant matrix generated by a vector $c_{i,j} \\in \\mathbb{R}^p$. The load-bearing identity is the FFT diagonalization of each block, $\\mathrm{circ}(c)x = \\mathrm{IFFT}(\\mathrm{FFT}(c) \\circ \\mathrm{FFT}(x))$, extended in Eq. (4) so that each output block $h_i$ is computed by a single IFFT of the sum of element-wise FFT products. This structure is what turns the weight-change matrix into $n^2/p$ trainable parameters and $O((n^2/p)\\log p)$ FLOPs per forward pass. The complementary mechanism is the training heuristic $\\alpha \\leftarrow \\alpha/p$, which scales the learning rate by the block size to counteract the $p$-fold gradient amplification proved in Propositions 1–3 and observed in Fig. 2.","core_discovery":"The paper's central discovery is that a block circulant matrix can work as a mergeable adapter for fine-tuning large language models: after training, the learned block circulant weight change $B$ is added directly to the pretrained weight $W$, so the deployed model costs no extra inference overhead. The adapter is built from $q \\times q$ vectors $c_{i,j} \\in \\mathbb{R}^p$, each generating a $p \\times p$ circulant block, and the forward map is computed with the identity $h_i = \\mathrm{IFFT}\\left(\\sum_j \\mathrm{FFT}(c_{i,j}) \\circ \\mathrm{FFT}(x_j)\\right)$, so storage is linear in the block vectors and computation stays log-linear. The paper also proves (Propositions 1–3) that first-order gradients of this parameterization are amplified by a factor of $p$ relative to a dense matrix, gives a single-layer simulation showing the gradient magnitude grows with $p$, and demonstrates that the heuristic $\\alpha \\leftarrow \\alpha/p$ turns a diverging training run into a convergent one. On RoBERTa-base, RoBERTa-large, and LLaMA2-7B, BCA reaches average scores within run-to-run noise of the compared adapters while reporting substantially lower parameter counts than LoRA/VeRA and lower FLOPs than FourierFT.","pith_inferences":["Editorial inference: the $p$-scaled learning rate is not specific to circulant matrices; any FFT-parameterized structured adapter with gradients proportional to a transform length might use the same $\\alpha/p$ trick to stabilize training.","Editorial inference: the paper reports asymptotic FLOPs rather than wall-clock time; an end-to-end GPU profile of BCA versus FourierFT, including the backward pass and batched FFT kernels, would tell whether the asymptotic advantage survives real implementations.","Editorial inference: treating $p$ as a per-layer hyperparameter, with small blocks in attention and large blocks in feed-forward layers, is a natural extension the paper leaves implicit, and the method's own complexity formulas predict where such a split would pay off.","Editorial inference: since block circulant matrices generalize from circulant ($p=n$) to dense ($p=1$), BCA occupies a spectrum of PEFT designs; probing intermediate $p$ values outside the reported set would map the full parameter-FLOP-performance frontier."],"forward_implications":["Because BCA is mergeable, once fine-tuning finishes the block circulant matrix can be summed into the pretrained weights, leaving inference latency and memory exactly the same as the base model.","Setting the partition size to the full hidden dimension ($p = n$) gives a plain circulant adapter with $O(n \\log n)$ computation and the smallest possible parameter count, while smaller $p$ gives more capacity at higher cost, making $p$ a direct efficiency-accuracy dial.","On the reported benchmarks, BCA's task scores sit within one standard deviation of LoRA, VeRA, and FourierFT, so the efficiency gain does not come at a visible accuracy cost.","BCA fills the gap between LoRA's low-FLOP/high-parameter design and FourierFT's high-FLOP/low-parameter design, offering a middle point where neither budget dominates.","The learning-rate heuristic is coupled to block size, so any deployment that changes $p$ must rescale $\\alpha$ accordingly to retain the convergence guarantee."],"supporting_citations":[{"why":"Supplies the FourierFT baseline and the GLUE/LLaMA evaluation protocol BCA compares against.","marker":"[Gao et al., 2024]"},{"why":"Defines the LoRA low-rank adapter that is the primary parameter-count and accuracy baseline.","marker":"[Hu et al., 2021b]"},{"why":"Defines VeRA, the vector-based random matrix adapter that BCA claims to beat by 14 times in parameters.","marker":"[Kopiczko et al., 2024]"},{"why":"Introduces block-circulant weight matrices and their FFT-based multiplication, the structural ancestor of BCA.","marker":"[Ding et al., 2017]"},{"why":"Provides the combined-IFFT per-output-block computation used in Eq. (4) of the adapter's forward pass.","marker":"[Liao and Yuan, 2019]"},{"why":"Gives the low-displacement-rank theory that explains why circulant structures can be trained as neural weights.","marker":"[Zhao et al., 2017]"},{"why":"Supplies the FFT diagonalization identity $\\mathrm{circ}(c)x = \\mathrm{IFFT}(\\mathrm{FFT}(c) \\circ \\mathrm{FFT}(x))$ that is the computational engine of the method.","marker":"[Oppenheim, 1999]"},{"why":"Defines the RoBERTa models on which the GLUE experiments are run.","marker":"[Liu et al., 2019]"},{"why":"Defines the LLaMA2-7B model used for instruction tuning and GSM8K experiments.","marker":"[Touvron et al., 2023]"}],"fun_headline_variants":["Block circulant adapter: 16× fewer params, 32× fewer FLOPs","Merged block circulant adapter cuts LLM fine-tune cost 16×","FFT-based adapter: 16× param cut, no inference overhead","Block circulant matrix adapter: 16× less memory, 32× less compute","Circulant adapter beats LoRA on params, merges into model"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline efficiency claim rests on an unstated rule for counting FLOPs and trainable parameters: the paper never says whether counts are forward-only or forward-plus-backward, or how FFT primitives are charged, so a consistent counting convention must reproduce the advertised $32\\times$ FLOPs reduction before the cost advantage is established.","fun_headline_variants_meta":{"raw":{"variants":["Block circulant adapter: 16× fewer params, 32× fewer FLOPs","Merged block circulant adapter cuts LLM fine-tune cost 16×","FFT-based adapter: 16× param cut, no inference overhead","Block circulant matrix adapter: 16× less memory, 32× less compute","Circulant adapter beats LoRA on params, merges into model"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000335,"raw_usage":{"total_tokens":1856,"prompt_tokens":942,"completion_tokens":914,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":558,"completion_tokens_details":{"reasoning_tokens":808}},"tokens_in":558,"tokens_out":914,"duration_ms":8902,"temperature":1.0,"reasoning_tokens":808,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:40:05.213592+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the FLOPs and parameter counts for BCA, LoRA, VeRA, and FourierFT on RoBERTa-large under one explicit counting rule (for example, forward plus backward, charging each complex FFT butterfly at its arithmetic cost) and check whether BCA's FLOPs ratio to FourierFT is the advertised $32\\times$; Fig. 4's plotted values appear to imply a ratio closer to $578\\times$, so a mismatch would falsify the cost claim independently of accuracy.","supporting_citations":[{"cited_title":"Parameter-efficient fine-tuning with discrete fourier trans- form","cited_arxiv_id":null,"evidence_quote":"Supplies the FourierFT baseline and the GLUE/LLaMA evaluation protocol BCA compares against."},{"cited_title":"Kopiczko, Tijmen Blankevoort, and Yuki M","cited_arxiv_id":null,"evidence_quote":"Defines VeRA, the vector-based random matrix adapter that BCA claims to beat by 14 times in parameters."},{"cited_title":"Circnn: accelerat- ing and compressing deep neural networks using block- circulant weight matrices","cited_arxiv_id":null,"evidence_quote":"Introduces block-circulant weight matrices and their FFT-based multiplication, the structural ancestor of BCA."},{"cited_title":"Circconv: A structured convolution with low complexity","cited_arxiv_id":null,"evidence_quote":"Provides the combined-IFFT per-output-block computation used in Eq. (4) of the adapter's forward pass."},{"cited_title":"Theoretical properties for neural networks with weight matrices of low displacement rank","cited_arxiv_id":null,"evidence_quote":"Gives the low-displacement-rank theory that explains why circulant structures can be trained as neural weights."},{"cited_title":"Discrete-time sig- nal processing","cited_arxiv_id":null,"evidence_quote":"Supplies the FFT diagonalization identity $\\mathrm{circ}(c)x = \\mathrm{IFFT}(\\mathrm{FFT}(c) \\circ \\mathrm{FFT}(x))$ that is the computational engine of the method."}],"review_version":1}