Pith. sign in

REVIEW 5 major objections 5 minor 38 references

Low-Rank Adaptation with Task-Relevant Feature Enhancement for Fine-tuning Language Models

T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Adding a task-aware filter to each layer lets low-rank fine-tuning match or beat stronger baselines with fewer trainable parameters.

desk verdict A plausible LoRA extension undercut by an unreconciled HellaSwag number: the text says 85.6 while Table 2 says 82.4, and the claimed reasoning advantage disappears on inspection. read the letter →

arxiv 2412.09827 v1 pith:PON54SUF submitted 2024-12-13 cs.CL cs.CV

Changqun Li , Chaofan Ding , Kexin Luan , Xinhan Di This is my paper · ORCID
classification cs.CLcs.CV
keywords parameter-efficientfine-tuninglow-rankadaptationtask-awarefiltertaskvectorrepresentationeditingLoRAGLUELLaMA
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes LoRATRF, a parameter-efficient fine-tuning method that augments LoRA with a per-layer task-aware filter. The filter learns a task vector, reweights hidden tokens by cosine similarity to that vector, and adds the refined representation back into the network. The authors claim this narrows the gap between LoRA and full fine-tuning, reducing trainable parameters by 33.71% while matching or exceeding state-of-the-art low-rank methods on GLUE, commonsense reasoning, and math reasoning tasks.

What carries the argument

The task-aware filter consists of a learnable task vector $t_\xi$ and a low-rank transformation matrix $T$. Each token $h_i^l$ is reweighted by $\mathrm{sim}(h_i^l, t_\xi)$ clamped to $[0,1]$, then multiplied by $T$, and the refined vector is added to the original hidden state. The filter's role is to selectively amplify task-relevant features in every layer, with parameter efficiency coming from approximating $T$ as a product of two low-rank matrices. This mechanism is the only added component beyond standard LoRA, so it carries the paper's entire performance claim.

What would settle it

Re-run the HellaSwag experiment and check whether the accuracy is 85.6 or 82.4, since the paper's text claims 85.6 but Table 2 reports 82.4; if the true value is 82.4, the claimed gain over DoRA disappears. Also freeze the task vector $t_\xi$ to a random vector and repeat the GLUE runs: if accuracy stays the same, the learned direction is not the cause of the gains.

Watch

Extended reading notes

Core claim

The central claim is that editing hidden representations with a task-aware filter makes low-rank adaptation more effective. In each Transformer layer, a learnable task vector $t_\xi$ computes a soft mask via cosine similarity clamped to $[0,1]$, the masked token is linearly transformed by a low-rank matrix $T$, and the result is added back to the original representation. On GLUE with DeBERTaV3-base, LoRATRF uses 0.88M trainable parameters versus 1.27M for AdaLoRA and reaches an average score of 89.52, the best among the compared low-rank methods. On LLaMA-7B reasoning benchmarks, the paper reports the best GSM8K result and claims an accuracy gain over DoRA on HellaSwag.

Load-bearing premise

The method works only if a single learned task vector, combined with cosine-similarity reweighting and a low-rank transformation, reliably identifies which features matter for each task in every layer of the network.

Editorial extensions

If this is right

  • LoRATRF can be dropped into existing LoRA pipelines without changing the frozen backbone, offering a direct upgrade for current fine-tuning code.
  • If the gains hold across ranks, the method makes low-rank fine-tuning viable at very small ranks such as $r=4$ and $r=8$, where LoRA degrades sharply.
  • The 33.71% parameter reduction on GLUE suggests that feature reweighting can substitute for a substantial amount of adapter capacity.
  • Improvements on math and commonsense reasoning benchmarks imply the filter's benefit is not confined to classification-style NLU tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not ablate the task vector against a fixed random vector, so a natural test is whether the learned direction encodes genuine task identity or merely acts as a learnable per-layer scaling.
  • If the filter works by amplifying task-relevant tokens, the learned $t_\xi$ directions should be consistent across similar tasks and should shift systematically when the task changes, which could be inspected by probing trained filters.
  • The method might transfer to multi-task or continual learning settings, where a shared task filter could either help or interfere; that extension is untested.
  • Resolving the paper's internal HellaSwag discrepancy (85.6 in the text versus 82.4 in Table 2) would clarify whether the claimed gain over DoRA is real.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes LoRATRF, a parameter-efficient fine-tuning method that augments LoRA with a learned task vector tξ and a low-rank transformation matrix. In each Transformer layer, hidden representations are reweighted by cosine similarity with tξ, linearly transformed, and added back to the original representation, with the goal of enhancing task-relevant features. The method is evaluated on GLUE with DeBERTaV3-base and on mathematical/commonsense reasoning benchmarks with LLaMA-7B. The authors claim that LoRATRF reduces trainable parameters by 33.71% and outperforms SOTA low-rank methods (LoRA, AdaLoRA, DoRA), and they include analyses of module choice and rank robustness.

Significance. If the claims were established, LoRATRF would be a simple and computationally cheap way to improve low-rank adaptation through explicit task-dependent feature reweighting, and the representation-editing perspective could be of interest to the PEFT community. The paper gives a clear algorithmic sketch and reports a concrete parameter reduction in the GLUE setting. However, the current empirical support is not adequate: the central LLaMA-7B comparison contains a direct numeric contradiction, the 33.71% parameter-reduction claim is not attached to the LLaMA-7B experiments, baselines are borrowed without matched conditions, and no run-to-run variability is reported. The contribution cannot be accepted on the present evidence.

major comments (5)
  1. [Main Results: Commonsense and Mathematical Reasoning (Table 2)] The text states that 'LoRATRF attains an accuracy of 85.6% on HellaSwag, surpassing DoRA baseline by 0.8%,' but Table 2 reports LoRATRF at 82.4 and DoRA at 84.8. These numbers are irreconcilable: if Table 2 is correct, LoRATRF is 2.4 points below DoRA, reversing the claimed superiority on HellaSwag. The authors must identify which value is correct, correct the other, and re-derive the conclusions that depend on this comparison.
  2. [Abstract and Tables 1-2] The abstract's claim of a 33.71% parameter reduction is supported only by Table 1, where LoRATRF uses 0.88M parameters versus 1.33M for LoRA r=8 on GLUE. In the LLaMA-7B experiments (Table 2), no parameter counts are reported, and the text states that LoRATRF uses the same rank (r=32) as the baseline while additionally introducing a task vector tξ and a low-rank transformation matrix T. The added components make it unlikely that LoRATRF has fewer trainable parameters than DoRA in that setting, so the paper should report parameter counts for every method in every table and restrict parameter-reduction claims to the configurations where they are actually measured.
  3. [Experimental Setup: Baselines and Evaluation] Baseline numbers in Table 1 and Table 2 are taken directly from earlier papers (Zhang et al. 2023a; Liu et al. 2024; Hu et al. 2023) without matching hyperparameters, training budgets, or evaluation protocols. For Table 1 the authors report a mean over 5 random seeds but give no standard deviation or significance test, and for Table 2 no seeding information is provided. Differences such as +0.2% on CoLA and +0.8% on HellaSwag are within typical run-to-run variance, so the claimed improvements over AdaLoRA and DoRA are not statistically supported.
  4. [Methodology: Task-Aware Filters (Eqs. 4-5)] The contribution of the proposed filter is not isolated in any experiment. There is no ablation comparing LoRATRF against LoRA with an equivalent number of additional parameters (e.g., a learned per-dimension scaling or an unstructured low-rank residual), and no analysis of what the learned mask or transformation does in different layers or across tasks. As a result, the paper does not demonstrate that 'task-relevant feature enhancement' is the cause of any observed gain rather than simply added capacity.
  5. [Quantitative Analysis: Robustness towards different rank settings (Figure 3)] The robustness experiment reports LoRA average accuracies of 59.3% at r=8 and 51.2% at r=4 on HellaSwag, which are far below the LoRA accuracy of 78.1% at r=32 in Table 2. The text does not explain the protocol for Figure 3 (e.g., whether the same data, training steps, and evaluation are used as in Table 2), so the large reported gaps (+19.3% at r=4, +14.6% at r=8) may be artifacts of different settings rather than evidence of robustness. The authors should align the setups or report the missing details.
minor comments (5)
  1. [Abstract] The word 'vareity' should be 'variety'.
  2. [Motivation] 'Featur Enhancement' should be 'Feature Enhancement'.
  3. [Task-Aware Filters (Eq. 4)] The similarity function 'sim' is not formally defined; the text later says cosine similarity, but the definition should appear together with the equation.
  4. [Task-Aware Filters (after Eq. 5)] The final residual addition is only described in words; an explicit equation such as h_out = h + \hat{h} would clarify the forward pass.
  5. [Table 2] The column header 'LLM Method' is ambiguous; the GPT-3.5 row should be labeled as a reference model rather than a method under comparison, and 'SV AMP' should be 'SVAMP'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: LoRATRF is a standard supervised adaptation method whose reported gains are empirical outcomes, not consequences of its definitions.

full rationale

The paper's derivation chain consists of defining a learnable task vector t_xi in Eq. (4), a cosine-similarity reweighting, and a low-rank transformation T in Eq. (5), then training these parameters on the target task. This is ordinary supervised fitting: the task vector and transformation are estimated from training data and evaluated on held-out test sets. Nothing in the method's definition forces the claimed improvements; whether Eq. (4)-(5) help is an empirical question answered by the experiments. The parameter reduction claim is arithmetic from the reported parameter counts (0.88M vs 1.33M), not a renamed input. The paper relies on baselines from other groups (LoRA, AdaLoRA, DoRA) and does not invoke a self-citation chain or a uniqueness theorem. The only notable defect is the internal inconsistency between the text ('LoRATRF attains an accuracy of 85.6% on HellaSwag, surpassing DoRA baseline by 0.8%') and Table 2 (82.4 for LoRATRF and 84.8 for DoRA); that is a correctness or reproducibility problem, not circular reasoning. Consequently, no step of the claimed derivation reduces to its own inputs by construction.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The method introduces two learned components (task vector and low-rank transformation) and several hand-chosen hyperparameters (rank, alpha, adapted modules). All are trained or tuned on the target tasks, so the performance claim rests on these choices rather than on a derivation. No new physical or formal entities are introduced.

free parameters (5)
  • Learnable task vector tξ = trained per task, dimension = hidden size
    Introduced in Eq. (4); its values are fit to the target task, and the method's behavior depends entirely on it.
  • Low-rank transformation matrices approximating T = rank not reported
    Footnote 1 says T is approximated by two low-rank matrices, but dimensions and rank are not given.
  • LoRA rank r = 8 for GLUE, 32 for LLaMA-7B
    Hand-chosen; Figure 3 shows performance varies strongly with r.
  • LoRA scaling factor alpha = not reported
    LoRA uses alpha/r scaling; the paper never states alpha for either model.
  • Selection of adapted modules = text says Wv, Wf1, Wf2; Figure 2 and LLaMA setup use Wf1, Wf2, Wo
    The module set is chosen based on a single SVAMP experiment and then reused across tasks; the text and figure are inconsistent.
assumptions (3)
  • domain assumption Cosine similarity between a hidden vector and a learned task vector is a meaningful measure of token-level task relevance.
    Assumed in Eq. (4) without justification; the whole method relies on this similarity.
  • domain assumption A single task vector tξ can be shared across all Transformer layers and still identify task-relevant features.
    The paper uses one tξ for each layer (Eq. 4) but does not discuss layerwise differences.
  • ad hoc to paper Adding the filtered and linearly transformed representation back to the original hidden state improves task performance without disrupting the pretrained model.
    The additive enhancement in Eqs. (5)-(6) is the core novel design choice and is not derived from any principle.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Low-Rank Adaptation with Task-Relevant Feature Enhancement for Fine-tuning Language Models." pith.science (2026). https://pith.science/paper/PON54SUF

@misc{pith2026241209827,
  author       = {Pith},
  title        = {Pith review of: Low-Rank Adaptation with Task-Relevant Feature Enhancement for Fine-tuning Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PON54SUF}},
  note         = {Machine review of arXiv:2412.09827}
}
read the original abstract

Fine-tuning pre-trained large language models in a parameter-efficient manner is widely studied for its effectiveness and efficiency. LoRA is one of the most widely used methods, which assumes that the optimization process is essentially low dimensional. Although LoRA has demonstrated commendable performance, there remains a significant performance gap between LoRA and full fine-tuning when learning new tasks. In this work, we propose Low-Rank Adaptation with Task-Relevant Feature Enhancement(LoRATRF) for enhancing task-relevant features from the perspective of editing neural network representations. To prioritize task-relevant features, a task-aware filter that selectively extracts valuable knowledge from hidden representations for the target or current task is designed. As the experiments on a vareity of datasets including NLU, commonsense reasoning and mathematical reasoning tasks demonstrates, our method reduces 33.71% parameters and achieves better performance on a variety of datasets in comparison with SOTA low-rank methods.

Figures

Figures reproduced from arXiv: 2412.09827 by the authors.

Figure 1
Figure 1. The overview of our approach. Task-aware fil [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. Testing performance of LLaMA-7B on HellaSwag [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 2
Figure 2. Testing performance of LLaMA-7B on SVAMP [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 17 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Aghajanyan, A.; Gupta, S.; and Zettlemoyer, L. 2021. Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning. In Zong, C.; Xia, F.; Li, W.; and Navigli, R., eds., Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volu...

    work page 2021

  4. [4]

    Ben Zaken, E.; Goldberg, Y.; and Ravfogel, S. 2022. B it F it: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models. In Muresan, S.; Nakov, P.; and Villavicencio, A., eds., Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 1--9. Dublin, Ireland: Association for ...

    work page 2022

  5. [5]

    Bisk, Y.; Zellers, R.; Gao, J.; Choi, Y.; et al. 2020. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence, volume 34, 7432--7439

    2020

  6. [6]

    B.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al

    Brown, T. B.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. 2020. Language models are few-shot learners. In Proceedings of the 34th International Conference on Neural Information Processing Systems, 1877--1901

    work page 2020

  7. [7]

    Clark, C.; Lee, K.; Chang, M.-W.; Kwiatkowski, T.; Collins, M.; and Toutanova, K. 2019. B ool Q : Exploring the Surprising Difficulty of Natural Yes/No Questions. In Burstein, J.; Doran, C.; and Solorio, T., eds., Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies...

    2019

  8. [8]

    Cobbe, K.; Kosaraju, V.; Bavarian, M.; Chen, M.; Jun, H.; Kaiser, L.; Plappert, M.; Tworek, J.; Hilton, J.; Nakano, R.; et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

    arXiv 2021

Show all 38 references
  1. [9]

    Dettmers, T.; Pagnoni, A.; Holtzman, A.; and Zettlemoyer, L. 2023. QLoRA: Efficient Finetuning of Quantized LLMs. arXiv preprint arXiv:2305.14314

    2023 arXiv

  2. [10]

    Ding, N.; Lv, X.; Wang, Q.; Chen, Y.; Zhou, B.; Liu, Z.; and Sun, M. 2023. Sparse Low-rank Adaptation of Pre-trained Language Models. In Bouamor, H.; Pino, J.; and Bali, K., eds., Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 4133--414...

    2023

  3. [11]

    Q.; et al

    Han, Z.; Gao, C.; Liu, J.; Zhang, S. Q.; et al. 2024. Parameter-efficient fine-tuning for large models: A comprehensive survey. arXiv preprint arXiv:2403.14608

    2024 arXiv

  4. [12]

    He, P.; Gao, J.; and Chen, W. 2022. DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing. In The Eleventh International Conference on Learning Representations

    2022

  5. [13]

    J.; Hajishirzi, H.; Etzioni, O.; and Kushman, N

    Hosseini, M. J.; Hajishirzi, H.; Etzioni, O.; and Kushman, N. 2014. Learning to Solve Arithmetic Word Problems with Verb Categorization. In Moschitti, A.; Pang, B.; and Daelemans, W., eds., Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing ...

    2014

  6. [14]

    Houlsby, N.; Giurgiu, A.; Jastrzebski, S.; Morrone, B.; De Laroussilhe, Q.; Gesmundo, A.; Attariyan, M.; and Gelly, S. 2019. Parameter-efficient transfer learning for NLP. In International Conference on Machine Learning, 2790--2799. PMLR

    2019

  7. [15]

    J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; and Chen, W

    Hu, E. J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; and Chen, W. 2022. Lo RA : Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations

    2022

  8. [16]

    Hu, Z.; Wang, L.; Lan, Y.; Xu, W.; Lim, E.-P.; Bing, L.; Xu, X.; Poria, S.; and Lee, R. 2023. LLM -Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models. In Bouamor, H.; Pino, J.; and Bali, K., eds., Proceedings of the 2023 Conference on Empi...

    2023

  9. [17]

    Y.; Pang, T.; Du, C.; and Lin, M

    Huang, C.; Liu, Q.; Lin, B. Y.; Pang, T.; Du, C.; and Lin, M. 2023. Lorahub: Efficient cross-task generalization via dynamic lora composition. arXiv preprint arXiv:2307.13269

    2023 arXiv

  10. [18]

    Kenton, J. D. M.-W. C.; and Toutanova, L. K. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of NAACL-HLT, 4171--4186

    2019

  11. [19]

    Lester, B.; Al-Rfou, R.; and Constant, N. 2021. The Power of Scale for Parameter-Efficient Prompt Tuning. In Moens, M.-F.; Huang, X.; Specia, L.; and Yih, S. W.-t., eds., Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 3045--3059. Online...

    2021

  12. [20]

    Lewis, M.; Liu, Y.; Goyal, N.; Ghazvininejad, M.; Mohamed, A.; Levy, O.; Stoyanov, V.; and Zettlemoyer, L. 2020. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. In Proceedings of the 58th Annual Meeting of the ...

    2020

  13. [21]

    Li, C.; Farkhoor, H.; Liu, R.; and Yosinski, J. 2018. Measuring the Intrinsic Dimension of Objective Landscapes. In International Conference on Learning Representations

    2018

  14. [22]

    L.; and Liang, P

    Li, X. L.; and Liang, P. 2021 a . Prefix-Tuning: Optimizing Continuous Prompts for Generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long ...

    2021

  15. [23]

    L.; and Liang, P

    Li, X. L.; and Liang, P. 2021 b . Prefix-Tuning: Optimizing Continuous Prompts for Generation. In Zong, C.; Xia, F.; Li, W.; and Navigli, R., eds., Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conferen...

    2021

  16. [24]

    F.; Cheng, K.-T.; and Chen, M.-H

    Liu, S.-Y.; Wang, C.-Y.; Yin, H.; Molchanov, P.; Wang, Y.-C. F.; Cheng, K.-T.; and Chen, M.-H. 2024. Dora: Weight-decomposed low-rank adaptation. arXiv preprint arXiv:2402.09353

    2024 arXiv

  17. [25]

    Liu, X.; Ji, K.; Fu, Y.; Tam, W.; Du, Z.; Yang, Z.; and Tang, J. 2022. P -Tuning: Prompt Tuning Can Be Comparable to Fine-tuning Across Scales and Tasks. In Muresan, S.; Nakov, P.; and Villavicencio, A., eds., Proceedings of the 60th Annual Meeting of the Association for Compu...

    2022

  18. [26]

    Liu, Y.; Ott, M.; Goyal, N.; Du, J.; Joshi, M.; Chen, D.; Levy, O.; Lewis, M.; Zettlemoyer, L.; and Stoyanov, V. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692

    2019 arXiv

  19. [27]

    Patel, A.; Bhattamishra, S.; and Goyal, N. 2021. Are NLP Models really able to Solve Simple Math Word Problems? In Toutanova, K.; Rumshisky, A.; Zettlemoyer, L.; Hakkani-Tur, D.; Beltagy, I.; Bethard, S.; Cotterell, R.; Chakraborty, T.; and Zhou, Y., eds., Proceedings of the 2...

    2021

  20. [28]

    Pfeiffer, J.; Kamath, A.; R \"u ckl \'e , A.; Cho, K.; and Gurevych, I. 2021. A dapter F usion: Non-Destructive Task Composition for Transfer Learning. In Merlo, P.; Tiedemann, J.; and Tsarfaty, R., eds., Proceedings of the 16th Conference of the European Chapter of the Associ...

    2021

  21. [29]

    Sap, M.; Rashkin, H.; Chen, D.; Le Bras, R.; and Choi, Y. 2019. Social IQ a: Commonsense Reasoning about Social Interactions. In Inui, K.; Jiang, J.; Ng, V.; and Wan, X., eds., Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th I...

    2019

  22. [30]

    Touvron, H.; Martin, L.; Stone, K.; Albert, P.; Almahairi, A.; Babaei, Y.; Bashlykov, N.; Batra, S.; Bhargava, P.; Bhosale, S.; et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  23. [31]

    Valipour, M.; Rezagholizadeh, M.; Kobyzev, I.; and Ghodsi, A. 2023. D y L o RA : Parameter-Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptation. In Vlachos, A.; and Augenstein, I., eds., Proceedings of the 17th Conference of the European Chapter...

    2023

  24. [32]

    Wang, A.; Singh, A.; Michael, J.; Hill, F.; Levy, O.; and Bowman, S. R. 2019. Glue: A multi-task benchmark and analysis platform for natural language understanding. In 7th International Conference on Learning Representations, ICLR 2019

    2019

  25. [33]

    H.; and Gao, J

    Wang, Y.; Agarwal, S.; Mukherjee, S.; Liu, X.; Gao, J.; Awadallah, A. H.; and Gao, J. 2022. A da M ix: Mixture-of-Adaptations for Parameter-efficient Model Tuning. In Goldberg, Y.; Kozareva, Z.; and Zhang, Y., eds., Proceedings of the 2022 Conference on Empirical Methods in Na...

    2022

  26. [34]

    Wu, X.; Huang, S.; and Wei, F. 2024. MoLE: Mixture of LoRA Experts. In The Twelfth International Conference on Learning Representations

    2024

  27. [35]

    Zellers, R.; Holtzman, A.; Bisk, Y.; Farhadi, A.; and Choi, Y. 2019. H ella S wag: Can a Machine Really Finish Your Sentence? In Korhonen, A.; Traum, D.; and M \`a rquez, L., eds., Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 4791--4...

    2019

  28. [36]

    Zhang, Q.; Chen, M.; Bukharin, A.; He, P.; Cheng, Y.; Chen, W.; and Zhao, T. 2023 a . Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning. In The Eleventh International Conference on Learning Representations

    2023

  29. [37]

    Zhang, Y.; Wang, P.; Tan, M.; and Zhu, W. 2023 b . Learned Adapters Are Better Than Manually Designed Adapters. In Rogers, A.; Boyd-Graber, J.; and Okazaki, N., eds., Findings of the Association for Computational Linguistics: ACL 2023, 7420--7437. Toronto, Canada: Association ...

    2023

  30. [38]

    Zou, A.; Phan, L.; Chen, S.; Campbell, J.; Guo, P.; Ren, R.; Pan, A.; Yin, X.; Mazeika, M.; Dombrowski, A.-K.; et al. 2023. Representation engineering: A top-down approach to ai transparency. arXiv preprint arXiv:2310.01405

    2023 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.