{"id":"69472b77-c7be-4c76-b00a-bdd81e90f84b","arxiv_id":"2607.09757","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"RSRA, a forward-only rank allocator combining spectral effective rank with Fréchet-distance sensitivity probing, reports higher average LoRA accuracy on three commonsense/GLUE settings with lower pre-allocation cost.","lead":"RSRA decides how many parameters each layer of a language model should receive during LoRA fine-tuning by measuring which layers change most on a small sample of task data. The authors report improved average accuracy on commonsense and GLUE benchmarks with lower allocation time than previous pre-allocation methods.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Parameter-count tables contradict the stated equal-budget assumption, so RSRA's reported gains may reflect a smaller rank budget rather than better allocation.","rationale":"The reader correctly identified that the empirical support is shaky, but focused on epsilon calibration as the weakest assumption. I find a more decisive, internally verifiable problem: the parameter counts reported in Table 8 and Appendix C contradict the paper's explicit equal-budget claim. The paper says average rank is fixed at 8 for fairness, yet RSRA uses ~40% fewer trainable parameters than LoRA/AdaLoRA. This means the central comparison is not apples-to-apples. The performance improvement could stem from reduced capacity, not from the sensitivity-based rank allocation. This concern is concrete, grounded in the paper's own numbers, and easily tested. It is more load-bearing than the epsilon choice because even if epsilon is perfectly calibrated, the reported empirical results would still be invalid under an unequal budget. I partially agree with the reader because they noted 'parameter-count tables contradict the stated fair-budget assumption' in their rationale, but their formal weakest_assumption was the epsilon calibration. My recommendation is REJECT (unchanged from the reader) because the central claim is not established until the budget discrepancy is resolved. The paper's other issues, such as implausible baselines and circular epsilon selection, reinforce the rejection but are secondary to this numerical inconsistency.","tokens_in":18028,"tokens_out":3924,"duration_ms":36818,"concrete_test":"Back-calculate the implied average rank from the reported parameter counts in Table 8 for each method on PIQA and CoLA (total trainable parameters divided by the sum over adapted modules of (dim_in + dim_out)). If RSRA's implied average rank is substantially below 8, rerun uniform LoRA with the same total parameter count (e.g., rank ≈4.8 on all modules). If matched-budget LoRA matches or exceeds RSRA's accuracy, the allocation signal is not the cause of the reported gains. Also recompute SR-LoRA and GoRA under a matched budget and recompare averages.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim—highest average performance under a fair parameter budget—is undermined by the paper's own efficiency tables. In the Experimental Setup, the authors state that 'the average rank across all layers is maintained at r=8 to ensure a fair parameter budget comparison with the LoRA baseline' (Appendix B, Table 6). However, Table 8 reports trainable parameters of 41.94M for LoRA and 25.35M for RSRA on PIQA (similarly on CoLA: 41.94M vs 25.61M). That is a ~40% reduction, implying an average RSRA rank of roughly 4.8, not 8. The appendix itself acknowledges 'RSRA reduces the trainable parameter count by approximately 40% compared with uniform LoRA,' directly contradicting the equal-budget claim. Because RSRA is compared against LoRA and AdaLoRA, which both use the full rank-8 budget, its higher average accuracy could plausibly result from implicit regularization or reduced overfitting rather than from sensitivity-based allocation. The comparison among pre-allocation methods (SR-LoRA, GoRA, RSRA) may be at similar budgets, but the headline 'highest average performance across all three model–benchmark settings' includes overpowered baselines LoRA and AdaLoRA, making the central claim unverified under matched conditions. This is an internal numerical inconsistency, not a matter of external consensus, and it directly challenges the core empirical assertion.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"RSRA is a forward-only, pre-fine-tuning rank allocator for LoRA. It scores layers by Spectral Effective Rank and modules by the Fréchet-distance displacement of hidden-state statistics under a standardized random low-rank probe, then combines these scores by hierarchical normalization into a rank configuration with a prescribed average rank. The paper reports experiments on Qwen3-4B and Mistral-7B across commonsense reasoning and GLUE, claiming the highest average accuracy in all three model–benchmark settings, a 1.48×–1.93× allocation-time speedup over the fastest competing pre-allocation method, and consistent gains when integrated with DoRA, LoRA-FA, and PiSSA.","tokens_in":18413,"tokens_out":5225,"duration_ms":43476,"significance":"If the empirical claims were reliable, RSRA would be a practically valuable pre-allocation method: it is simple, forward-only, potentially model-agnostic, and the paper includes useful ablations of calibration-set size and rank bounds. The motivation from heterogeneous module sensitivity is clearly presented. However, the central comparison is not currently trustworthy: the paper's own efficiency tables contradict the stated equal-parameter-budget condition, several published baselines are reported at implausibly low accuracy, and the probe magnitude is calibrated on tasks that appear in the evaluation suite. The core contribution is therefore not established by the present experiments.","major_comments":[{"comment":"The paper states that the average rank is maintained at r=8 to ensure a fair parameter-budget comparison with LoRA (Table 6), but Table 8 reports trainable parameters of 41.94M for LoRA versus 25.35M for RSRA on PIQA and 41.94M versus 25.61M on CoLA, a ~40% reduction. The appendix even acknowledges that RSRA reduces the parameter count by approximately 40% compared with uniform LoRA. Thus the headline accuracy comparisons against LoRA and AdaLoRA are not made under the claimed equal-budget condition, and the gains could arise from implicit regularization or reduced capacity rather than from the sensitivity-based allocation.","section":"Appendix B Table 6 vs Appendix C Table 8"},{"comment":"Several baseline results are implausible for published methods. GoRA drops to 52.77 average on Mistral-7B commonsense (e.g., BoolQ 47.49 vs LoRA 89.42); PiSSA baselines in Table 4 are 48.38 on QNLI, 40.33 on QQP, and 22.95 on ARC-C; AdaLoRA underperforms vanilla LoRA by large margins on many tasks. These numbers strongly suggest misconfigured hyperparameters or implementation errors for the baselines. Consequently the aggregate 'improves 15 of 18 combinations' claim and the claimed advantage over competing allocators are not credible until the baselines are verified.","section":"Tables 1, 2, and 4"},{"comment":"The probe magnitude epsilon=0.01 is set by measuring the relative Frobenius norm of trained LoRA updates on MRPC, RTE, and WNLI (Table 9). These three tasks are part of the GLUE evaluation in Table 2. The probe magnitude is therefore calibrated on a subset of the evaluation target, which undermines the claim that epsilon is a task-agnostic, conservative choice. The authors should calibrate epsilon on held-out tasks (or show insensitivity across a broader task set that excludes the GLUE test tasks) before claiming that the same epsilon applies universally.","section":"Appendix C, Eq. (6), Table 9"},{"comment":"The validation of the sensitivity signal is confounded by the rank allocation itself. Modules are ranked by RSRA sensitivity and the top/middle/bottom 10% groups are ablated after fine-tuning with RSRA's rank configuration. Because high-sensitivity modules are assigned larger ranks, their trained updates contain more parameters and hence more capacity; the larger loss increase after ablation could reflect rank/capacity rather than the intrinsic relevance of the sensitivity score. To support the claimed alignment, the ablation should be run under a uniform-rank configuration or the contribution should be normalized by capacity.","section":"Section 4, RQ4, Figure 5"}],"minor_comments":[{"comment":"The abstract says RSRA achieves the highest average performance in 'all three reported model–benchmark settings,' but the mathematical-reasoning experiments in Table 7 show AdaLoRA outperforming RSRA on average (40.04 vs 38.44). Please qualify the claim or explicitly exclude this setting.","section":"Appendix C, Table 7"},{"comment":"Unlike Tables 1 and 2, Table 4 reports no standard deviations or number of seeds. Given the very large PiSSA baseline swings, this table needs error bars and a description of how many seeds were used.","section":"Table 4"},{"comment":"RSRA has several free parameters (epsilon, r_probe, N, Rmin, Rmax). The paper ablates N and Rmin/Rmax, but not r_probe or epsilon (except for the GLUE-calibrated choice). Please report sensitivity to r_probe and epsilon on a held-out task.","section":"Section 3 and Appendix B"},{"comment":"The global importance I_{l,j} is a product of two normalized scores. A product is not the only way to combine inter-layer and intra-layer signals; the paper should justify why multiplication is preferable to addition or another aggregation, especially since the product's scale affects rounding in Eq. (11).","section":"Eq. (9)"},{"comment":"The term 'training-free' is potentially misleading: RSRA requires a task-specific calibration set and many forward passes. Please clarify that 'training-free' means no gradient-based optimization during allocation, not that the procedure is free of data or compute.","section":"General"},{"comment":"No code is provided. Given the sensitivity of the results to hyperparameter configuration and the implausible baselines, releasing code and exact baseline configurations is essential for reproducibility.","section":"General"}],"recommendation":"reject","confidential_remarks":"The parameter-budget contradiction in Tables 6 and 8 is an internal numerical inconsistency that directly invalidates the central equal-budget comparison, and the baseline numbers in Tables 1–4 are far outside the expected range for published methods. These are not cosmetic issues; they require re-running the entire experimental protocol. I would be willing to reconsider a substantially revised manuscript with matched-budget comparisons, verified baselines, and a non-leaked calibration procedure."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper has a genuinely new idea — forward-only rank allocation via standardized virtual low-rank probes and Fréchet-distance displacement — and the motivation study is clean. But the empirical section is not credible as reported. You should not cite its numbers until the baselines and budget are fixed, though the method deserves a serious referee.\n\nWhat's actually new: SR-LoRA uses stable rank of pretrained weights, GoRA uses gradients, AIRA uses activation statistics. None of them probe task-conditioned hidden-state sensitivity with virtual low-rank updates. RSRA's inter-layer effective rank plus intra-layer Fréchet displacement is a reasonable heuristic, and the ablation showing that top-ranked modules contribute more after training is a nice sanity check. The method is cheap (a few forward passes on 256 samples) and orthogonal to DoRA/LoRA-FA/PiSSA.\n\nThe soft spots are real. First, baselines: GoRA at 52.77 average on Mistral-7B commonsense (LoRA is 69.99) and PiSSA at 48.38 on QNLI and 40.33 on QQP — that's below random on QQP. These aren't plausible numbers for published methods; they look like misconfigured runs. AdaLoRA consistently underperforming LoRA also flies against the literature. Second, the equal-budget claim: the setup says average rank is kept at 8, but Table 8 shows RSRA using 25.35M trainable params vs LoRA's 41.94M — a ~40% cut — and the appendix explicitly says 'RSRA reduces the trainable parameter count by approximately 40%.' So the headline comparisons against LoRA and AdaLoRA are not at matched parameter budget, and the gains could simply be a smaller-model regularization effect. That's an internal inconsistency, not a matter of taste. Third, the probe magnitude epsilon=0.01 is calibrated on MRPC, RTE, and WNLI (Table 9), which then appear in the GLUE evaluation table — a mild circularity that needs addressing (e.g., calibration on held-out tasks or a sensitivity analysis). Fourth, the math reasoning appendix shows AdaLoRA beating RSRA on average, narrowing the 'wins everywhere' framing.\n\nOn the plus side, the calibration-size and rank-boundary ablations are reasonable, and the module-removal validation is a good idea. The method is clearly explained and the paper is honest about its limitations (e.g., AdaLoRA's advantage on some tasks). But without code, corrected baselines, and a matched-budget comparison, the central claim is unverified.\n\nMy take: this is a paper a good editor should send to review — the idea is worth refereeing — but any competent reviewer will ask for major revision. I wouldn't cite it yet; read it for the method, not for the numbers.","headline":"A genuinely new allocation heuristic with a plausible mechanism, but the empirical case as reported is undermined by implausible baselines and a ~40% parameter-budget mismatch.","tokens_in":18886,"tokens_out":2825,"would_cite":false,"duration_ms":26549,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A training-free, forward-only probe of hidden-state sensitivity can assign LoRA ranks before fine-tuning and beat uniform allocation on all three tested model–benchmark settings.","keywords":["LoRA rank allocation","representation sensitivity","Fréchet distance","spectral effective rank","training-free probing","parameter-efficient fine-tuning","calibration set","forward-only estimation"],"falsifier":"Run RSRA on a task whose trained LoRA update magnitudes lie well outside the 0.004–0.018 range used to set epsilon, and compare its allocation against a random permutation of the same rank budget; if the RSRA allocation does not beat the random one in downstream accuracy, the fixed-epsilon probe is not task-relevant.","tokens_in":17910,"feed_emoji":"🎯","tokens_out":9188,"duration_ms":78195,"temperature":0.7,"pith_summary":"The paper claims that where a Low-Rank Adaptation (LoRA) budget should be spent can be decided before any training, by probing how much each module's hidden-state distribution would move under a standardized random low-rank update. RSRA combines two forward-only signals — the spectral breadth of each layer's activations and the Fréchet-distance displacement caused by a virtual probe on each module — into a hierarchical rank allocation. If correct, this replaces training-time rank pruning and gradient-based pre-allocation with a cheap calibration pass and generalizes across adapter variants. The authors report the highest average accuracy in every model–benchmark setting they tested (Qwen3-4B and Mistral-7B on commonsense reasoning; Mistral-7B on GLUE), a 1.48–1.93x speedup in allocation time over the fastest competing pre-allocation method, and gains on 15 of 18 combinations when plugged into DoRA, LoRA-FA, and PiSSA.","feed_headline":"Probe picks LoRA ranks before training; beats uniform on 3 benchmarks","feed_subtitle":"Allocates rank budget on a 256-sample calibration set, boosting accuracy on commonsense and GLUE while cutting allocation time.","key_machinery":"The central object is the standardized virtual low-rank probe (Eq. 6): a random Gaussian low-rank matrix, normalized to a fixed relative magnitude epsilon=0.01 of the target weight's Frobenius norm, applied to one module at a time. The sensitivity score (Eq. 8) is the squared Fréchet distance between the baseline layer hidden-state distribution and the probed distribution, normalized by the baseline's mean and covariance energy. This intra-layer signal is combined with the layer's Spectral Effective Rank (Eq. 5) through hierarchical normalization (Eq. 9), and the resulting global importance scores are projected onto integer ranks in a bounded range (Eq. 11). The Fréchet distance matters beca","core_discovery":"RSRA asserts that representation sensitivity — the change in a layer's hidden-state distribution caused by a module-specific low-rank update — can be measured before fine-tuning with forward passes only. On a 256-sample calibration set, it computes each layer's Spectral Effective Rank from the singular values of its centered activations, then applies a standardized virtual low-rank update to each module (random Gaussian factors scaled to epsilon=0.01 of the weight's Frobenius norm). The squared Fréchet distance between baseline and probed layer statistics, normalized by baseline energy, gives an intra-layer sensitivity score; hierarchical normalization combines it with the spectral share to","pith_inferences":["The probe uses random directions, not task-optimized ones. A testable extension would compare random probes with gradient-informed or data-adaptive probe directions; if random directions suffice, the signal is carried by the geometry of the representation space itself, not by the specific update content.","The fixed epsilon=0.01 was calibrated on three GLUE tasks (MRPC, RTE, WNLI) that later appear in the evaluation. A robustness check would measure trained update magnitudes on a held-out task family and verify that re-running RSRA with those magnitudes keeps the rank configuration nearly unchanged.","The Fréchet-distance covariance term may be noisy on small calibration sets. A mean-only displacement variant (or a different distribution distance) would isolate which component of the sensitivity score carries the predictive signal.","Since the probe is forward-only, it could also be used to freeze low-sensitivity modules entirely (rank 0) rather than just giving them fewer dimensions, which would further reduce trainable parameters without harming performance."],"forward_implications":["Rank allocation can be fully decoupled from fine-tuning: the rank configuration is computed once, before training, so no iterative pruning or dynamic budget adjustment is needed during optimization.","The allocation signal transfers across adapter variants (DoRA, LoRA-FA, PiSSA) because it is independent of parameterization, initialization, and optimizer, making RSRA a drop-in allocator for other PEFT methods.","The sensitivity ranking is stable with small calibration sets — the paper reports high Spearman correlations even at 64 samples — so the probe's cost stays low for large models.","On harder tasks (e.g., ARC-Challenge), a non-zero minimum rank per module is important, meaning allocation should respect rank floors rather than aggressively sparsifying modules.","Allocation time grows only mildly with calibration size (a 16x increase in samples produces about a 1.73x increase in time), so practitioners can trade a little accuracy for speed by shrinking the calibration set."],"fun_headline_variants":["Training-free LoRA rank picker beats uniform on 3 tasks","RSRA probes layer sensitivity to allocate LoRA ranks pre-training","Pre-train LoRA ranks via sensitivity probe; 1.5-1.9x faster","LoRA rank allocation without training: RSRA tops 3 benchmarks","Measure hidden-state shifts to set LoRA ranks; wins in 3 tests"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The method assumes that one fixed probe magnitude (epsilon=0.01, chosen from trained LoRA update sizes on three GLUE tasks) and a 256-sample calibration set capture, before training, the same module importance that emerges after training.","fun_headline_variants_meta":{"raw":{"variants":["Training-free LoRA rank picker beats uniform on 3 tasks","RSRA probes layer sensitivity to allocate LoRA ranks pre-training","Pre-train LoRA ranks via sensitivity probe; 1.5-1.9x faster","LoRA rank allocation without training: RSRA tops 3 benchmarks","Measure hidden-state shifts to set LoRA ranks; wins in 3 tests"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00107,"raw_usage":{"total_tokens":4327,"prompt_tokens":760,"completion_tokens":3567,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":504,"completion_tokens_details":{"reasoning_tokens":3478}},"tokens_in":504,"tokens_out":3567,"duration_ms":21218,"temperature":1.0,"reasoning_tokens":3478,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T04:28:13.937699+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run RSRA on a task whose trained LoRA update magnitudes lie well outside the 0.004–0.018 range used to set epsilon, and compare its allocation against a random permutation of the same rank budget; if the RSRA allocation does not beat the random one in downstream accuracy, the fixed-epsilon probe is not task-relevant.","supporting_citations":[],"review_version":2}