Pith. sign in

REVIEW 3 major objections 4 minor 16 references

The paper claims that adaptive merging of publicly recycled LoRA adapters gives little benefit over simply training a new LoRA on the target task, and that when the target-task LoRA is included in the merge, the choice of other LoRAs barely

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 23:51 UTC pith:P5PGKPLV

load-bearing objection First in-the-wild LoRA-recycling study with a strong target-LoRA baseline; the random-init result is intriguing but hinges on underspecified LoRA rescaling. the 3 major comments →

arxiv 2602.12323 v2 pith:P5PGKPLV submitted 2026-02-12 cs.LG cs.SE

The Appeal and Reality of Recycling LoRAs with Adaptive Merging

Haokun Liu , Gyung Hyun Je , Marco Ciccone , Zhenlin Xu , Prasanth YSS , Colin Raffel This is my paper
classification cs.LG cs.SE
keywords LoRA recyclingadaptive mergingmodel mergingregularization effectcross-task transferparameter-efficient fine-tuningfew-shot adaptationpublic LoRA pool
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to test whether the rapidly growing supply of user-contributed LoRA adapters on open model hubs can be recycled through adaptive merging to improve performance on new tasks. It builds a pool of nearly 1,000 such adapters for one popular open language model and evaluates merging methods on 62 downstream tasks using only 100 labeled examples per task. The central finding is that adaptive merging does improve over the base model, but it barely beats the simple alternative of training a fresh LoRA on the same 100 samples, and once that target-task LoRA is included in the merge, the identity of the other recycled LoRAs stops mattering. Replacing real LoRAs with randomly initialized ones of matched scale yields similar performance, which the paper interprets as evidence that the gains are mostly regularization rather than positive knowledge transfer. Positive transfer shows up only when the pool contains highly relevant, consistently trained LoRAs — a situation common in controlled benchmarks but rare in the wild.

Core claim

The discovery is an empirical negative result about adaptive merging from realistic, heterogeneous LoRA pools. Using a unified design space covering LoRA selection, coefficient granularity, activation, and tuning, the authors find that evaluation-based selection and module-level coefficients help when the target-task LoRA is absent, but the advantage nearly disappears when it is present. With the target-task LoRA in the pool, random selection performs as well as informed selection, and randomly initialized LoRAs perform as well as real crowdsourced ones across k=5 to k=30 merged modules. The learned merging coefficients concentrate heavily on the target-task LoRA, while the recycled LoRAs ac

What carries the argument

The central mechanism is a unified design space for adaptive merging that isolates four choices: how LoRAs are selected from the pool (random, evaluation-ranked, parameter-similarity), the granularity of merging coefficients (model, layer, sublayer, module), the coefficient activation (softmax, leaky ReLU, linear), and the tuning procedure (gradient-based or gradient-free). The method the paper calls Ours combines module-level coefficients, evaluation-based selection, and leaky ReLU. The experimental probe that carries the argument is the random-initialization baseline: it replaces real recycled LoRAs with samples from a normal distribution of matched standard deviation while keeping the tar

Load-bearing premise

The claim that randomly initialized LoRAs match curated ones rests on the assumption that rescaling 958 heterogeneous LoRAs (with varying ranks and different target-module configurations) into a common form preserves enough of each LoRA's learned behavior; the paper says the LoRAs were 'rescaled' but does not specify how ranks are padded or sliced or how missing target modules are handled, so the equivalence could be an artifact of comparing already-degraded modules.

What would settle it

Take a heterogeneous pool but align all LoRAs in a rank-preserving way that exactly matches target modules (for example, full SVD expansion instead of padding or slicing), then rerun the random-vs-real comparison with the target-task LoRA included. If real LoRAs now outperform randomly initialized ones, the regularization-only conclusion is refuted. A cheaper check: compare the merged models produced from real and random LoRAs in parameter space — if they are nearly identical after coefficient tuning, the content is genuinely irrelevant; if they differ substantially yet perform the same, the m

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Practitioners should benchmark any adaptive-merging method against a target-task LoRA trained on the same few-shot data; on realistic pools, that simple baseline will often erase the apparent gain.
  • Including the target-task LoRA in the merging pool is almost necessary for adaptive merging to be competitive; without it, gains over the base model are modest and vary widely across methods.
  • Once the target-task LoRA is present, the choice of selection strategy (random, evaluation-based, or similarity-based) has little effect, so compute spent scoring and selecting recycled LoRAs may be wasted.
  • Previously reported successes of adaptive merging likely depend on close task relatedness that is easy to arrange in controlled experiments but rare in user-contributed pools; positive transfer decays quickly as relevant LoRAs are removed.
  • The recycled LoRAs' specific content may be replaceable by noise, implying a regularization-based explanation; future work should test this directly before assuming knowledge transfer.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: If the regularization effect is real, adaptive merging with randomly initialized modules could serve as a cheap, parameter-level noise regularizer for few-shot fine-tuning; this could be tested by comparing it against explicit weight-noise injection during standard LoRA training.
  • Editorial inference: The negative result is conditional on how heterogeneous LoRAs are aligned — the paper rescaled modules of varying rank and target-module coverage without specifying the alignment procedure in the main text, so the random-init equivalence may partly reflect information lost during rescaling; a rank-preserving alignment could restore an advantage for real LoRAs.
  • Editorial inference: The 62-task, 100-sample evaluation is one operating point; with larger target datasets or different base models, the balance between training a fresh LoRA and merging recycled ones could shift, so the conclusions may not transfer to higher-data regimes.
  • Editorial inference: The rapid decay of positive transfer with task dissimilarity reframes the problem as retrieval: the value of a large pool depends on finding rare, highly relevant adapters, and future work should focus on scalable search and ranking rather than on better merging coefficients.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper studies 'recycling' of nearly 1,000 user-contributed LoRAs from the Hugging Face Hub for Llama 3.1 8B-Instruct. It introduces a unified framework for adaptive merging methods (selection, granularity, activation, tuning), proposes a configuration called 'Ours', and evaluates it against LoraHub, AdaMerging, π-Tuning, and non-adaptive baselines on 62 downstream tasks. The main findings are: (1) adaptive merging improves over the base model but provides little consistent benefit over training a target-task LoRA on the same data; (2) once the target-task LoRA is included in the merging pool, the identity of the other recycled LoRAs matters little, and randomly initialized LoRAs perform comparably to real recycled LoRAs; (3) positive transfer reappears in a bespoke 'in-house' pool of target-task LoRAs, but degrades rapidly when the most relevant LoRAs are excluded. The paper releases code and checkpoints.

Significance. If the negative result holds, this is an important correction to the adaptive-merging literature: it suggests that reported gains from recycling LoRAs may often come from a regularization/optimization effect rather than from positive cross-task transfer, and that the practical value of in-the-wild LoRA recycling is currently limited. The paper's choice to compare against a target-task LoRA baseline and to test a random-initialization control is methodologically valuable. The design-space framework and the release of code/checkpoints are concrete assets. The in-house positive-transfer experiment usefully delineates when the negative result does and does not apply. However, the central claim depends on details of LoRA rescaling/alignment and on the statistical reliability of small performance gaps that are currently not fully reported.

major comments (3)
  1. [§4.1, §B.1, §D] The alignment/rescaling of heterogeneous LoRAs is underspecified. The main text only says the LoRAs are 'rescaled' (Fig. 1 caption; B.1), and Appendix D gives alignment details only for LoraHub (pad to max rank), TIES (convert to full B@A), and TSV (full B@A). The adaptive methods used for the central random-init experiment (Ours and AdaMerging) require a common rank/module structure, but the paper does not state how ranks are padded, truncated, or projected, nor how LoRAs with missing modules (e.g., the 91 LoRAs targeting only Attn(Q,V)) are treated. This is load-bearing: if alignment distorts real LoRAs, the finding that random reinitialization performs similarly may show only that the rescaled pool contributes little, not that original recycled LoRAs cannot transfer. Please specify the exact alignment procedure for Ours/AdaMerging and verify that alignment preserves a meaningful porti
  2. [Table 2, Fig. 3] The central equivalence claim is supported by small average differences: with target-task LoRA, at k=30, random selection gives 0.675 and random reinitialization gives 0.672, and Table 2 shows differences on the order of 0.005. No error bars, confidence intervals, or repeated-seed results are reported for Ours, AdaMerging, or LoraHub in these comparisons; non-adaptive methods average 3 seeds but adaptive methods appear to be single runs. Given the magnitude of the effects, please add at least 3 repeats of selection and coefficient tuning for the key methods, or report paired per-task confidence intervals for the Ours vs. random-reinit comparison. Without this, the claim that selection/reinit is negligible is not quantitatively supported.
  3. [§3, §E] The configuration of 'Ours' is selected on a 20-task ablation set (Section E), and those 20 tasks appear to be part of the 62-task testbed used for final evaluation in Table 2 and Fig. 3. This makes the reported 'Ours' performance partly optimized against the evaluation set. Since the random-init experiment in Fig. 3 also uses 'Ours', the circularity affects the central claim. Please state explicitly whether the 20 ablation tasks belong to the 62, and either hold them out from the final evaluation, or use a nested/non-overlapping selection procedure so that the reported numbers are unbiased.
minor comments (4)
  1. [Fig. 3] The legend terms 'reinit.', 'rand.', 'eval.' are easy to confuse. 'reinit.' means randomly reinitialized parameters, while 'rand.' means randomly selected real LoRAs. Please use more explicit labels, e.g., 'random init' vs. 'random selection', and define them in the caption.
  2. [Table 2] The column headers 'w/o LoRA' and 'w/ LoRA' should be 'w/o target-task LoRA' and 'w/ target-task LoRA' to avoid ambiguity, since 'LoRA' is also used for the target-task baseline.
  3. [Fig. 3, §5] The random-initialization control matches the marginal standard deviations of A and B separately, but the merged update is s·B@A. Coefficient tuning can absorb global scale, but the effective distribution of the product may differ. Please state the exact initialization (including whether the LoRA scaling s is applied) and, if available, include a control that matches the Frobenius norm of the merged update.
  4. [§C.1] For LoraHub, the text says coefficients are selected by lowest loss on the 'full 100 examples', while other methods use an 80/20 split. Please clarify whether the 20 validation examples are used for LoraHub or whether the optimization directly uses all 100 training examples.

Circularity Check

0 steps flagged

No significant circularity; the central claims are empirical and the random-init control is a genuine external benchmark.

full rationale

The paper's load-bearing claims are empirical: adaptive merging improves over the base model but gives limited benefit over a target-task LoRA; the choice of recycled LoRAs matters little once the target-task LoRA is in the pool; randomly initialized LoRAs perform similarly to real recycled LoRAs; and positive transfer appears only with highly relevant in-house LoRAs. None of these conclusions is derived by definition from a fitted parameter. The random-init experiment is a proper control: it keeps the same merging method and target-task LoRA while replacing the recycled LoRAs' learned content with random values, so it can falsify the positive-transfer explanation. The only mild concern is that the 'Ours' configuration was chosen via ablations on 20 tasks, and the paper does not explicitly state whether those 20 are disjoint from the 62-task evaluation; if they overlap, the reported Ours performance is partly selected on the testbed. However, the manuscript never states the overlap, and the random-versus-evaluation equivalence is shown across design choices in the ablations and replicated in the main experiment, so it is not an artifact of that configuration choice. Self-citations to Je & Raffel, Tam et al., and Li et al. supply benchmark tasks or design approximations, not load-bearing proofs of the empirical findings. Thus I find no step that reduces by construction to its own inputs; the appropriate score is low.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 0 invented entities

The paper introduces no new physical or model entities; 'Quasi-FIM' is an approximation of an existing metric, and 'Ours' is a configuration of existing design choices. The listed free parameters are experimental hyperparameters that the central comparisons depend on.

free parameters (5)
  • Selected LoRA count k = 30 (default; also 5, 10, 20)
    Main experiments merge 30 LoRAs; results vary with k and the no-LoRA vs with-LoRA gap narrows as k increases (Figure 3).
  • Target-task sample budget = 100 (80 train / 20 val)
    Central comparison uses 100 target examples for both coefficient tuning and LoRA training; changing the budget could change the relative benefit.
  • Target-task LoRA rank/steps/LR = r=64, 400 steps, LR=3e-4
    Defines the baseline strength; a stronger or weaker baseline would change the apparent benefit of merging.
  • Coefficient-tuning steps/LR = 100 steps, LR=5e-2
    Used for all adaptive methods; limited tuning steps may disadvantage coefficient optimization, affecting the comparison.
  • 'Ours' method configuration (module-level, leaky ReLU, evaluation selection) = Module granularity, leaky ReLU activation, eval-based selection, gradient-based tuning
    The best configuration is chosen via an ablation on 20 tasks (Section E) that are a subset of the 62 evaluation tasks; this is a design choice fitted to the testbed.
axioms (6)
  • domain assumption Llama 3.1 8B-Instruct is an adequate base; all pool LoRAs are assumed to target compatible base models (including quantized variants).
    The entire pool and downstream tasks are tied to this one base model (Section 4.1).
  • domain assumption The filtered pool of 958 LoRAs is representative of 'in-the-wild' modules.
    The authors exclude a large fraction of available LoRAs for license/format/feature reasons; the generalizability of the negative result depends on this representation (Section 4.1).
  • domain assumption Heterogeneous LoRAs (varying ranks and target modules) can be rescaled/aligned for merging without losing their functional content.
    The paper mentions 'rescale the target modules of 958 recycled LoRAs' (Section B.1) but does not fully specify the method; TIES/TSV use full task vectors and LoraHub uses rank padding (Section D). The adaptive methods' merging relies on this alignment.
  • domain assumption The 62-task benchmark from Je & Raffel (2025) and Tam et al. (2024b) is a fair testbed for measuring positive transfer.
    The chosen tasks span multiple domains but are all English/multiple-choice/generation benchmarks; hub LoRAs may target other skills (e.g., roleplay) that are not represented.
  • domain assumption Gradient-based coefficient tuning with 100 steps reaches a sufficiently good optimum.
    All adaptive methods are given 100 optimization steps; the paper notes gradient-free improves more slowly but does not establish convergence for the coefficient optimization (Sections C.1, E).
  • domain assumption Randomly reinitialized LoRAs preserve the same parameter scale as trained LoRAs, so the comparison isolates the effect of learned content.
    The random LoRAs are sampled from a normal distribution with std matched to the original A and B matrices (Section 5, Q3); the validity of the 'random content' control depends on this scale matching.

pith-pipeline@v1.3.0-alltime-deepseek · 31009 in / 12703 out tokens · 110770 ms · 2026-08-02T23:51:29.738274+00:00 · methodology

0 comments
read the original abstract

The widespread availability of fine-tuned LoRA modules for open pre-trained models has led to an interest in methods that can adaptively merge LoRAs to improve performance. These methods typically include some way of selecting LoRAs from a pool and tune merging coefficients based on a task-specific dataset. While adaptive merging methods have demonstrated improvements in some settings, no past work has attempted to recycle LoRAs found "in the wild" on model repositories like the Hugging Face Hub. To address this gap, we consider recycling from a pool of nearly 1,000 user-contributed LoRAs trained from the Llama 3.1 8B-Instruct language model. Our empirical study includes a range of adaptive and non-adaptive merging methods in addition to a new method designed via a wide search over the methodological design space. We demonstrate that adaptive merging methods can improve performance over the base model but provide limited benefit over training a new LoRA on the same data used to set merging coefficients. We additionally find not only that the specific choice of LoRAs to merge has little importance, but that using LoRAs with randomly initialized parameter values yields similar performance. This raises the possibility that adaptive merging from recycled LoRAs primarily works via some kind of regularization effect, rather than by enabling positive cross-task transfer. To better understand why past work has proven successful, we confirm that positive transfer is indeed possible when there are highly relevant LoRAs in the pool. We release the model checkpoints and code online.

Figures

Figures reproduced from arXiv: 2602.12323 by Colin Raffel, Gyung Hyun Je, Haokun Liu, Marco Ciccone, Prasanth YSS, Zhenlin Xu.

Figure 1
Figure 1. Figure 1: Rescaled LoRA performance across 62 downstream tasks, categorized into five buckets based on their relative % improvement over prompting or LoRA baselines. LoRAs are sorted along the x-axis by their performance. 3.1 8B-Instruct (Grattafiori et al., 2024) (or quantized vari￾ants thereof), which had the most LoRAs publicly available at the time of experimentation. We forgo all the LoRAs that do not have lice… view at source ↗
Figure 2
Figure 2. Figure 2: Avg. % acc. improvement from baselines across all 62 tasks, without and with the target-task LoRA in the merging pool. TSV, TIES, and Simple Averaging are non-adaptive methods, while AdaMerging, π-Tuning, LoraHub, and Ours are adaptive methods. than selecting 30 LoRAs at random, indicating that even though there is no computational obstacle to including all LoRAs, doing so is counterproductive. As another … view at source ↗
Figure 4
Figure 4. Figure 4: Distribution of merging coefficients across 20 evaluation￾based LoRAs for 62 downstream tasks, a) without and b) with the target-task LoRA in the merging pool. stem primarily from a regularization effect rather than from task-relevant knowledge transfer from the recycled LoRAs. To better understand why including the target LoRA reduces performance gap, we analyze how the distribution of learned merging coe… view at source ↗
Figure 3
Figure 3. Figure 3: Adding the target-task LoRA significantly reduces the impact of LoRA selection method. The average LoRA baseline performance across tasks is shown as a gray line. Once the target-task LoRA is included in the pool, differ￾ent adaptive merging methods exhibit similar performance trends despite employing vastly different selection mech￾anisms. To investigate, we conduct a more thorough ex￾periment varying the… view at source ↗
Figure 5
Figure 5. Figure 5: (a) In-house LoRAs outperform recycled LoRAs. (b) Excluding top-ranked in-house LoRAs degrades merging perfor￾mance, converging to the randomly reinitialized LoRA perfor￾mance. method in §3, but replace the recycled LoRA pool with all the target-task LoRAs trained on 100 examples (“in-house” LoRAs). We evaluate the accuracy of the 62 downstream task LoRAs on each other’s training and validation exam￾ples t… view at source ↗
Figure 6
Figure 6. Figure 6: Recycled LoRA summary statistics. B Downstream Task Details B.1 Recycled LoRA evaluated on downstream tasks We evaluate the recycled LoRAs on 100 samples from each of the 62 downstream task and plot the distribution of their evaluation scores ( [PITH_FULL_IMAGE:figures/full_fig_p015_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Distribution of recycled LoRA evaluation scores (100 data samples) across 62 tasks. glue_sst2 glue_qqp glue_mrpc super_glue_boolq glue_cola arc_easy arc_challenge formal_fallacies_syllogisms_negation sports_understanding glue_mnli twitter_complaints web_of_lies boolean_expressions typescript_chunks super_glue_wic overruling stackoverflow_questions sciq anli super_glue_rte xnli_hi ade_corpus_v2 toxic_chat t… view at source ↗
Figure 8
Figure 8. Figure 8: Distribution of relative % improvement from prompting by task. Most tasks have many LoRAs to gain relative % improvement from, some only with a few relevant LoRAs that can transfer positive knowledge, and a handful of tasks with little or worse performance gain after further optimization. B.2 Downstream task description In addition to the downstream tasks included in SuperGLUE (Wang et al., 2020), GLUE (Wa… view at source ↗
Figure 9
Figure 9. Figure 9: Average downstream task performance across 20 downstream tasks used in ablation, by the design choice in a) merging weight granularity, b) selection method, and c) activation. 05 10 20 30 k 0.54 0.56 0.58 0.60 0.62 0.64 0.66 0.68 0.70 Avg. task performance eval. rand. eval. w/ LoRA rand. w/ LoRA target-task LoRA (a) Model granularity 05 10 20 30 k 0.54 0.56 0.58 0.60 0.62 0.64 0.66 0.68 0.70 Avg. task perf… view at source ↗
Figure 10
Figure 10. Figure 10: Performance gap between random and evaluation-based selections, without and with target-task LoRA in the pool. Though there is a notable gap between random and evaluation-based selections, including the target-task LoRA in the pool closes this gap, across all granularity LoRAs have varying ranks, we pad each LoRA to the max rank present in the pool before merging. π-Tuning Since we do not have access to t… view at source ↗
Figure 11
Figure 11. Figure 11: Ablation studies varying merging coefficients granularity and selection strategy. k denotes the number of LoRAs in the pool, L the number of model layers, M the number of target modules. We bold our chosen configuration used in our method (“Ours”). 21 [PITH_FULL_IMAGE:figures/full_fig_p021_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: % improvement from (a) prompting and (b) LoRA baselines, across the 62 downstream tasks for each merging method. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Merging accuracy by number of merged LoRAs without (a) and with (b) for π-Tuning, LoraHub, and our method. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: % coefficients assigned to 20 LoRAs in the pool across the 62 downstream tasks, in a setting a) without and b) with the target-task LoRA. In a), 1st expert has the highest evaluation score for the given downstream task. In b), 1st expert is the target-task LoRA, and the 2nd LoRA has the highest evaluation score for the given downstream task. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_14.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

16 extracted references · 10 linked inside Pith

  1. [7]

    org/CorpusID:266053836

    URL https://api.semanticscholar. org/CorpusID:266053836. Kandpal, N., Deng, H., Roberts, A., Wallace, E., and Raffel, C. Large language models struggle to learn long-tail knowledge. InInternational Confer- ence on Machine Learning, 2022. URL https: //api.semanticscholar.org/CorpusID: 253522998. Khot, T., Clark, P., Guerquin, M., Jansen, P., and Sabhar- wa...

    Pith/arXiv arXiv 2022

  2. [8]

    org/CorpusID:280045706

    URL https://api.semanticscholar. org/CorpusID:280045706. Ling, W., Yogatama, D., Dyer, C., and Blunsom, P. Program induction by rationale generation : Learning to solve and explain algebraic word problems, 2017. URL https: //arxiv.org/abs/1705.04146. Liu, H., Tam, D., Muqeeth, M., Mohta, J., Huang, T., Bansal, M., and Raffel, C. A. Few-shot parameter-effi...

    Pith/arXiv arXiv 2017

  3. [11]

    org/CorpusID:218487733

    URL https://api.semanticscholar. org/CorpusID:218487733. Wahle, J. P., Ruas, T., Foltýnek, T., Meuschke, N., and Gipp, B.Identifying Machine-Paraphrased Pla- giarism, pp. 393–413. Springer International Pub- lishing, 2022. ISBN 9783030969578. doi: 10. 1007/978-3-030-96957-8_34. URL http://dx.doi. org/10.1007/978-3-030-96957-8_34. Wang, A., Singh, A., Mich...

    Pith/arXiv arXiv 2022

  4. [12]

    org/CorpusID:271859936

    URL https://api.semanticscholar. org/CorpusID:271859936. Yadav, P., Raffel, C., Muqeeth, M., Caccia, L., Liu, H., Chen, T., Bansal, M., Choshen, L., and Sordoni, A. A survey on model moerging: Recycling and routing among specialized experts for collaborative learning.Transac- tions on Machine Learning Research, 2025. ISSN 2835-

    2025

  5. [14]

    org/CorpusID:263620126

    URL https://api.semanticscholar. org/CorpusID:263620126. Zhu, L., Cong, P., Ji, A., Wu, W., Hou, J., Wu, C., Gao, X., Liu, J., Huan, Z., Sun, X., Yang, Y ., Jiao, J., Hu, L., Chen, X., Liu, J., Ding, J., Yang, T., Wang, Z., Zhang, G., and Huang, W. Lpfqa: A long-tail professional forum-based benchmark for llm evaluation.ArXiv, abs/2511.06346,

    arXiv

  6. [15]

    org/CorpusID:282912578

    URL https://api.semanticscholar. org/CorpusID:282912578. 14 The Appeal and Reality of Recycling LoRAs with Adaptive Merging A Recycled LoRA Analysis We analyze the distribution of the recycled LoRA’s ranks, target modules, and sources and include the result in Figure 6. (a)# of recycled LoRAs uploaded per contributor. Percentile # recycled LoRAs Avg. 3.25...

    2020

  7. [153]

    findings-eacl.153/

    URL https://aclanthology.org/2023. findings-eacl.153/. Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., and Tafjord, O. Think you have solved question answering? try arc, the ai2 reasoning challenge, 2018. URL https://arxiv.org/abs/ 1803.05457. Cobbe, K., Kosaraju, V ., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., ...

    Pith/arXiv arXiv 2023

  8. [2017]

    cryptonews_articles

    contains crowdsourced science exam questions spanning topics such as physics, chemistry, and biology. Document understanding.unarxive_imrad_clf (Saier et al., 2023) involves classifying arXiv paper sections into IMRAD categories (Introduction, Methods, Results, and Discussion). machine_paraphrase_dataset (Wahle et al., 2022) focuses on detecting whether a...

    2023

  9. [2020]

    org/CorpusID:220045835

    URL https://api.semanticscholar. org/CorpusID:220045835. Rebuffi, S.-A., Bilen, H., and Vedaldi, A. Learning multiple visual domains with residual adapters.Advances in neural information processing systems, 30, 2017. Ruder, S. Neural transfer learning for natural language processing. 2019. URL https: //api.semanticscholar.org/CorpusID: 208121296. Saier, T...

    arXiv 2017

  10. [2021]

    eacl-main.39

    URL https://aclanthology.org/2021. eacl-main.39. Preotiuc-Pietro, D., Gaman, M., and Aletras, N. Automati- cally identifying complaints in social media, 2019. URL https://arxiv.org/abs/1906.03890. Pruksachatkun, Y ., Phang, J., Liu, H., Htut, P. M., Zhang, X., Pang, R. Y ., Vania, C., Kann, K., and Bowman, S. R. Intermediate-task transfer learning with pr...

    Pith/arXiv arXiv 2021

  11. [2022]

    org/CorpusID:254408495

    URL https://api.semanticscholar. org/CorpusID:254408495. Ilharco, G., Ribeiro, M. T., Wortsman, M., Schmidt, L., Hajishirzi, H., and Farhadi, A. Editing models with task arithmetic. InICLR, 2023. URL https:// openreview.net/forum?id=6t0Kwf8-jrj. Je, G. H. and Raffel, C. Efficiently estimating data effi- ciency for language model fine-tuning.arXiv preprint...

    arXiv 2023

  12. [2023]

    org/CorpusID:256697368

    URL https://api.semanticscholar. org/CorpusID:256697368. Han, Z., Gao, C., Liu, J., Zhang, J., and Zhang, S. Q. Parameter-efficient fine-tuning for large models: A com- prehensive survey.Transactions on Machine Learn- ing Research, 2024. ISSN 2835-8856. URL https: //openreview.net/forum?id=lIsCS8b6zj. Horwitz, E., Kurer, N., Kahana, J., Amar, L., and Hosh...

    Pith/arXiv arXiv 2024

  13. [2024]

    org/CorpusID:274436302

    URL https://api.semanticscholar. org/CorpusID:274436302. Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., Yang, A., Fan, A., Goyal, A., Hartshorn, A., Yang, A., Mitra, A., Sravankumar, A., Korenev, A., Hinsvark, A., Rao, A., Zhang, A., Rodriguez, A., Gregerson, A., Spataru, A....

    Pith/arXiv arXiv 2024

  14. [2025]

    Antypas, D., Ushio, A., Camacho-Collados, J., Neves, L., Silva, V ., and Barbieri, F

    URL https://openreview.net/forum? id=L1Bm396P0X. Antypas, D., Ushio, A., Camacho-Collados, J., Neves, L., Silva, V ., and Barbieri, F. Twitter topic classifica- tion, 2022. URL https://arxiv.org/abs/2209. 09824. Biderman, D., Portes, J., Gonzalez Ortiz, J. J., Paul, M., Greengard, P., Jennings, C., King, D., Havens, S., Chiley, V ., Frankle, J., et al. Lo...

    Pith/arXiv arXiv 2022

  15. [2799]

    Hsu, P.-L., Dai, Y ., Kothapalli, V ., Song, Q., Tang, S., Zhu, S., Shimizu, S., Sahni, S., Ning, H., and Chen, Y

    PMLR, 2019. Hsu, P.-L., Dai, Y ., Kothapalli, V ., Song, Q., Tang, S., Zhu, S., Shimizu, S., Sahni, S., Ning, H., and Chen, Y . Liger kernel: Efficient triton kernels for llm training, 2025. URLhttps://arxiv.org/abs/2410.10989. Hu, E. J., yelong shen, Wallis, P., Allen-Zhu, Z., Li, Y ., Wang, S., Wang, L., and Chen, W. LoRA: Low-rank adap- tation of large...

    Pith/arXiv arXiv 2019

  16. [8856]

    Survey Certification

    URL https://openreview.net/forum? id=u0azVc9Y0y. Survey Certification. Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025. Yang, E., Wang, Z., Shen, L., Liu, S., Guo, G., Wang, X., and Tao, D. Adamerging: Adaptive model merg- ing for multi-task ...

    Pith/arXiv arXiv 2025