{"id":"38aa3905-1a99-4888-a2a1-e78dc4a3bd1a","arxiv_id":"2501.14859","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"Dynamic LoRA, a layer-wise adaptive variant of LoRA, reportedly improves GLUE accuracy from 87.4% to 88.1% at only 0.1% more trainable parameters, but the write-up lacks reproducibility.","lead":"This paper proposes a version of LoRA fine-tuning that changes adapter sizes and weights per layer during training. It claims higher accuracy on GLUE benchmarks with almost no extra computation, but the method text is vague and key experimental details are missing.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The method as written is not computable: γ_l is a matrix fed into a scalar softmax, and Table 2's '0.1% more resources' is inconsistent with its own 7.4h vs 7.1h training times.","rationale":"The reader's weakest_assumption correctly targeted Section III's importance formula, and I agree that the vague, type-incoherent equations make the mechanism unimplementable as written. I additionally flag an internal arithmetic contradiction in Table 2: the reported '0.1% more resources' is not supported by the table's own 7.4h vs 7.1h training time or 0.9% vs 0.8% trainable-parameter ratio. The performance half is also unverifiable because no base model, per-task breakdown, or error bars are given, and GLUE does not have a single aggregate accuracy metric. These are internal correctness problems, not disagreements with field consensus. There are no machine-checked proofs, code releases, or other independent artifacts to offset the missing support. The reader's REJECT verdict remains appropriate, and no adjustment is needed.","tokens_in":7769,"tokens_out":7362,"duration_ms":85737,"concrete_test":"Ask the authors for executable code or detailed pseudocode plus a named base model and hyperparameters. First, check whether γ_l can be reduced to a scalar by any stated operation; if no scalarization is specified, Section III does not define the method. Then run dynamic LoRA vs standard LoRA on one GLUE task, e.g., QNLI, with identical budgets, and compare ACC, F1, and training time to Tables 1 and 2 to see whether 88.1/87.3 and the claimed 0.1% overhead reproduce.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract promises a drop-in improvement: 88.1% ACC / 87.3% F1 at only 0.1% more resources than LoRA. For this to be true, Section III must define an implementable algorithm and the table must support the cost claim. Section III fails the first test: γ_l = (∂L/∂W_l)·W_l has the shape of W_l, but α_l = exp(γ_l)/Σ_k exp(γ_k) is a softmax over scalars; no trace, norm, or other scalarization is given, so the formula cannot be evaluated. Likewise r_l = r_base(1+λ·Var(X_l)) is a per-layer scalar, but how it sets B_l A_l dimensions and how variable-rank adapters are combined is unspecified. The manuscript provides no algorithm box, no base model, no per-task GLUE results, and no error bars; GLUE has no single 'accuracy' metric, making the headline number undefined. Table 2 independently weakens the cost half: trainable parameters go 0.8%→0.9% and training time goes 7.1h→7.4h, a 4.2% increase, not '0.1%'. The central claim is therefore not falsifiable from the text as written.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes \"dynamic LoRA,\" a variant of Low-Rank Adaptation for fine-tuning large language models. The method augments static LoRA with two mechanisms: a layer-importance weight derived from the gradient of the loss with respect to frozen weight matrices, and an input-dependent rank adjustment based on the variance of layer inputs. The authors report that their method achieves 88.1% accuracy and 87.3% F1 on the GLUE benchmark, outperforming standard LoRA at a cost increase of only 0.1% more resources. Section IV presents tables comparing several fine-tuning baselines on accuracy, AUC, F1, recall, training time, inference time, and trainable parameter percentage. Section V concludes that dynamic LoRA is an effective and practical fine-tuning solution and proposes future extensions.","tokens_in":8032,"tokens_out":1743,"duration_ms":16257,"significance":"If the claims were substantiated, dynamic LoRA would be a practical drop-in improvement over LoRA for PEFT, with clear value for resource-constrained deployment. The motivation, that static rank allocation across layers ignores differences in layer importance and input complexity, is reasonable and matches a known limitation of fixed-rank LoRA. However, the manuscript as written does not establish the central claim: the method definition is not computable as stated, the experimental reporting is inconsistent with GLUE conventions, and the resource-cost claim is contradicted by the paper's own table. The result is not currently falsifiable from the text, so the significance is overshadowed by the lack of support.","major_comments":[{"comment":"The proposed importance measure γ_l = (∂L/∂W_l)·W_l has the shape of the frozen weight matrix W_l, but the softmax α_l = exp(γ_l)/Σ_k exp(γ_k) requires scalar inputs. No trace, norm, average, or other scalarization is defined, so the equations cannot be evaluated as written. Similarly, r_l = r_base(1+λ·Var(X_l)) is a per-layer scalar, but the manuscript never specifies how this rank is applied to the shapes of B_l and A_l, how variable-rank adapters are batched or trained, or how the parameter count in Table 2 is computed from these variable ranks. This is a load-bearing gap because the reported 88.1% accuracy is attributed to this mechanism.","section":"Section III, formulas for γ_l, α_l, and r_l"},{"comment":"The experiments are described as using the GLUE benchmark, but GLUE does not have a single 'accuracy' or 'F1' metric across all tasks; individual tasks use accuracy, F1, or Matthews correlation, and there is no standard 'AUC' aggregate. Table 1 reports one ACC, AUC, F1, and Recall value per method, but no base model, no per-task breakdown, no hyperparameters, and no task list are given. The headline '88.1% accuracy' is therefore undefined, and the comparison cannot be reproduced or interpreted.","section":"Section IV, Table 1 and dataset description"},{"comment":"The abstract asserts a cost increase of only 0.1% over standard LoRA. Table 2 shows trainable parameters rising from 0.8% to 0.9%, which is a 12.5% relative increase (0.1 percentage points), and training time rising from 7.1 hours to 7.4 hours, a 4.2% relative increase. The '0.1% more resources' claim is therefore inconsistent with the paper's own measured values, regardless of whether one reads it as a relative or absolute difference. The efficiency claim is a central part of the contribution and is not supported by the presented data.","section":"Table 2 and Abstract claim of '0.1% more resources'"},{"comment":"The manuscript provides no algorithm box, no pseudocode, no base model identifier, no optimizer or learning-rate settings, no batch size, no random seeds, no number of runs, and no code link. The loss curve in Figure 2 is said to stabilize at approximately 2 and shows an 'around the 150-epoch mark' transition, but no epoch or step budget is specified anywhere. Without these details, the central claim is not reproducible even if the equations were unambiguous.","section":"Section III and Section IV (overall reproducibility)"}],"minor_comments":[{"comment":"Reference [22] cites a paper on equestrian bridle and bit fit in animals as support for the BitFit baseline, and reference [23] cites an IoT farming paper that uses LoRa (long-range radio) as support for the LoRA method. These are not the correct sources; the BitFit method presumably refers to the bias-term fine-tuning paper, and LoRA should cite Hu et al. (2021).","section":"References [22] and [23]"},{"comment":"The loss-decline graph lacks axis labels, units, and a legend identifying which curve corresponds to which method, despite the text claiming it 'provides further validation for the performance and efficiency of the evaluated methods.'","section":"Figure 2"},{"comment":"The sentence 'By the end of the fine-tuning process, the loss stabilizes at approximately 2' is surprising for GLUE fine-tuning with a cross-entropy loss and likely indicates an error or a missing scale; a concrete loss value with the loss function used would clarify this.","section":"Section IV.B, paragraph on convergence"},{"comment":"Several equations are corrupted in the PDF (e.g., 'BAWWWW ·+=Δ+=', 'rdRA ×∈' and 'drRB ×∈' with garbled superscripts), making the method section difficult to read even aside from the substantive issues; the authors should resubmit with properly typeset mathematics.","section":"Throughout"}],"recommendation":"reject","confidential_remarks":"The paper has the appearance of a submission assembled from loosely related references, with the central method not defined sufficiently for implementation and the experimental section contradicting its own efficiency claim. The citation errors for foundational methods (LoRA and BitFit) suggest the authors are not familiar with the primary literature. Given that the load-bearing errors cannot be fixed within the scope of the current manuscript, I recommend rejection. If the authors were to substantially rewrite the method with a precise, computable algorithm, provide reproducible experiments with per-task GLUE results, and correct the cost accounting, a future submission could be considered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThis one is a clear reject, and not a close call. The headline claim—88.1% GLUE accuracy at only 0.1% more resources than LoRA—is unsupported by the text as written. The method section never defines a computable algorithm: γ_l = (∂L/∂W_l)·W_l is a matrix, but α_l is a softmax over scalars, so the key importance weight cannot be evaluated. The rank adjustment r_l = r_base(1+λ·Var(X_l)) is also a scalar with no specification of how it changes B_l A_l dimensions or how variable-rank adapters are combined. There is no algorithm box, no base model, no per-task GLUE breakdown, and no seeds or error bars. GLUE has no single 'accuracy' metric, so the 88.1% number is undefined.\n\nWhat credit is due: the general idea—per-layer importance weighting plus input-dependent rank—is reasonable and worth exploring. But it is not new; AdaLoRA and related adaptive LoRA variants already do this, and none are cited. The paper's own equations reduce to a softmax-weighted LoRA update with a variance-scaled rank. There is no formal result, no code, no reproducible experiment.\n\nThe citation pattern is a red flag: the LoRA reference is a LoRa radio paper, and BitFit is cited to a paper about equestrian bridles. That alone would make me doubt every baseline comparison. Table 2 also undercuts the cost claim: trainable parameters go 0.8%→0.9% and training time goes 7.1h→7.4h, a 4.2% increase, not \"0.1% more resources.\"\n\nThe reader's strongest claim and weakest assumption both land. The central claim is not falsifiable from the manuscript. I don't think this deserves a serious referee; desk rejection is appropriate. If the authors resubmit with a real implementation, AdaLoRA baseline, proper GLUE task breakdown, and corrected citations, that would be a different paper.","headline":"Garbled method and broken citations make this a clear desk reject; the dynamic LoRA idea is reasonable but neither new nor testable from the text.","tokens_in":8594,"tokens_out":1519,"would_cite":false,"duration_ms":12534,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a dynamic LoRA variant, which reallocates adapter rank and weight across layers based on gradient sensitivity and input-feature variance, improves GLUE accuracy to 88.1% at only 0.1% additional resources over…","keywords":["dynamic LoRA","parameter-efficient fine-tuning","low-rank adaptation","layer importance","input feature distribution","GLUE benchmark","large language models","adaptive rank allocation"],"falsifier":"An ablation that holds $\\alpha_\\ell$ fixed at $1/L$ while keeping the variance-based ranks, and another that holds $r_\\ell = r_\\text{base}$ while keeping $\\alpha_\\ell$, would settle it. If either variant reproduces the full method's 88.1% accuracy on GLUE, the corresponding mechanism is not doing the work; if both variants match plain LoRA, the reported gain is not attributable to dynamic adaptation.","tokens_in":7550,"feed_emoji":"⚙️","tokens_out":8748,"duration_ms":66645,"temperature":0.7,"pith_summary":"The paper argues that the fixed adapters of standard LoRA can be made dynamic by continuously measuring which layers matter for the current task and how variable their inputs are, and shifting low-rank capacity accordingly. That would make fine-tuning a large language model more task-specific without updating all parameters, and with almost no slowdown relative to plain LoRA. The reported experiments on GLUE put dynamic LoRA at 88.1% accuracy and 87.3% F1, with 0.9% trainable parameters versus 0.8% for LoRA. The paper's case rests on two formulas: a gradient-based layer importance weight and a variance-based rank adjustment.","feed_headline":"Dynamic LoRA: 88.1% accuracy, 0.1% extra resources","feed_subtitle":"Layer-importance and input-variance signals reallocate adapter capacity, beating standard LoRA on GLUE with almost no overhead.","key_machinery":"The central object is the pair of dynamic adapter controllers. The layer-importance signal $\\gamma_\\ell$ is the dot product of the loss gradient with the frozen layer weight $W_\\ell$; it is meant to measure how sensitive the task loss is to perturbations of that layer. The softmax weights $\\alpha_\\ell$ turn this into a normalized allocation of adapter capacity across layers. The rank controller $r_\\ell = r_\\text{base}(1+\\lambda\\,\\mathrm{Var}(X_\\ell))$ uses input-feature variance to give more low-rank capacity to layers whose inputs are more spread out. Together they replace LoRA's fixed $B_\\ell A_\\ell$ per layer with an adaptive $\\alpha_\\ell B_\\ell A_\\ell$ whose rank varies during fine-tuning.","core_discovery":"On its own terms, the paper claims that LoRA's static per-layer adapters are the bottleneck, and that making adapter allocation respond to the task fixes it. For each layer $\\ell$, it defines an importance $\\gamma_\\ell = (\\partial L/\\partial W_\\ell)\\cdot W_\\ell$ from the loss's gradient with respect to the frozen weight matrix, then softmax-normalizes these into $\\alpha_\\ell$ so that important layers get more of the adapter budget. It also sets the rank $r_\\ell = r_\\text{base}(1+\\lambda\\,\\mathrm{Var}(X_\\ell))$ from the variance of the layer's input features, so layers with more varied inputs get larger adapters. The update becomes $W'_\\ell = W_\\ell + \\alpha_\\ell B_\\ell A_\\ell$ with a regularization term on the adapter norms. The paper reports that this reaches 88.1% accuracy and 87.3% F1 on GLUE, beating standard LoRA's 87.4% and 86.5% while training 7.4 hours versus 7.1 and keeping trainable parameters at 0.9%.","pith_inferences":["Our inference: the importance signal $\\gamma_\\ell$ is a first-order sensitivity, so the paper implicitly assumes the loss landscape is smooth enough that this local gradient predicts the value of adding adapter capacity; an ablation replacing $\\gamma_\\ell$ with a leave-one-layer-out accuracy change would test that.","Our inference: using only variance of $X_\\ell$ is a coarse summary of input complexity; higher moments or entropy of the feature distribution might be a more faithful rank controller.","Our inference: the same dynamic allocation logic should transfer to adapters in vision or multimodal settings, although the paper only sketches that as future work."],"forward_implications":["If dynamic LoRA works as described, LoRA's uniform adapter budget is leaving accuracy on the table; shifting capacity to high-importance layers should improve tasks with uneven layer importance.","With 0.9% trainable parameters versus LoRA's 0.8%, the method would be a near drop-in replacement for LoRA in resource-constrained fine-tuning.","The variance-based rank rule implies adapter shape can track the input distribution during training, which should help on heterogeneous or streaming data.","The same gradient-importance measurement could guide decisions about which layers to keep fully frozen, further reducing memory cost."],"supporting_citations":[{"why":"Cited to establish LoRA as the parameter-efficient fine-tuning baseline that dynamic LoRA extends.","marker":"[2]"},{"why":"Supplies the dynamic adaptation mechanism the paper integrates with LoRA.","marker":"[3]"},{"why":"Supports dynamic scheduling of adapter parameters across layers.","marker":"[10]"},{"why":"Provides adaptive weight-masking precedent for the layer-wise parameter allocation.","marker":"[11]"},{"why":"Underlies the use of input feature statistics to drive adapter rank adjustment.","marker":"[19]"},{"why":"Grounds the GLUE train/validation/test procedure used for evaluation.","marker":"[20]"},{"why":"Cited as the widely adopted LoRA method against which dynamic LoRA is compared.","marker":"[23]"}],"fun_headline_variants":["Dynamic LoRA reallocates adapters by layer importance, hits 88.1% on GLUE with 0.1% overhe","Task-aware adapter allocation: dynamic LoRA achieves 88.1% accuracy, nearly zero extra cos","Dynamic LoRA: layer importance and input variance guide adapters, 88.1% on GLUE","LoRA gets dynamic: adaptive adapter placement yields 88.1% on GLUE with 0.1% overhead"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the gradient of the loss with respect to a frozen layer's weight matrix is a trustworthy measure of that layer's importance, so moving adapter capacity toward high-gradient layers is what produces the reported accuracy gain.","fun_headline_variants_meta":{"raw":{"variants":["Dynamic LoRA reallocates adapters by layer importance, hits 88.1% on GLUE with 0.1% overhead","Task-aware adapter allocation: dynamic LoRA achieves 88.1% accuracy, nearly zero extra cost","Dynamic LoRA: layer importance and input variance guide adapters, 88.1% on GLUE","LoRA gets dynamic: adaptive adapter placement yields 88.1% on GLUE with 0.1% overhead"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000588,"raw_usage":{"total_tokens":2834,"prompt_tokens":1094,"completion_tokens":1740,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":710,"completion_tokens_details":{"reasoning_tokens":1624}},"tokens_in":710,"tokens_out":1740,"duration_ms":11832,"temperature":1.0,"reasoning_tokens":1624,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T14:51:13.579798+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"An ablation that holds $\\alpha_\\ell$ fixed at $1/L$ while keeping the variance-based ranks, and another that holds $r_\\ell = r_\\text{base}$ while keeping $\\alpha_\\ell$, would settle it. If either variant reproduces the full method's 88.1% accuracy on GLUE, the corresponding mechanism is not doing the work; if both variants match plain LoRA, the reported gain is not attributable to dynamic adaptation.","supporting_citations":[{"cited_title":"LoRA-Mini : Adaptation Matrices Decomposition and Selective Training","cited_arxiv_id":"2411.15804","evidence_quote":"Cited to establish LoRA as the parameter-efficient fine-tuning baseline that dynamic LoRA extends."},{"cited_title":"Few-Shot Learning with Adaptive Weight Masking in Conditional GANs","cited_arxiv_id":"2412.03105","evidence_quote":"Provides adaptive weight-masking precedent for the layer-wise parameter allocation."},{"cited_title":"Comparison of Tree-Based Feature Selection Algorithms on Biological Omics Dataset,","cited_arxiv_id":null,"evidence_quote":"Underlies the use of input feature statistics to drive adapter rank adjustment."},{"cited_title":"Intelligent edge based smart farming with LoRa and IoT,","cited_arxiv_id":null,"evidence_quote":"Cited as the widely adopted LoRA method against which dynamic LoRA is compared."}],"review_version":1}