Pith. sign in

REVIEW 3 major objections 4 minor 43 references

GenKnowSub: Improving Modularity and Reusability of LLMs through General Knowledge Subtraction

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Subtracting a Wikipedia-trained general-knowledge LoRA from each task LoRA leaves residual modules that sharpen Arrow routing and lift zero-shot accuracy by up to 3.9 points.

desk verdict A cheap and plausible preprocessing step for modular LLMs, but the knowledge-disentanglement story needs a control before I'd believe it. read the letter →

arxiv 2505.10939 v2 pith:CAA5K3D2 submitted 2025-05-16 cs.CL cs.AI

Mohammadtaha Bagherifard , Sahar Rajabi , Ali Edalat , Yadollah Yaghoobzadeh This is my paper · ORCID
classification cs.CLcs.AI
keywords modularLLMsLoRAsubtractionzero-shotgeneralizationcross-lingualtransfertaskarithmeticparameter-efficientfine-tuningArrowroutinggeneralknowledgedisentanglement
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that modular zero-shot transfer is held back because task-specific LoRA adapters still carry redundant general knowledge inherited from the base model. It proposes GenKnowSub: train a LoRA on a small Wikipedia-like corpus, then subtract that general-domain LoRA from each task LoRA to leave a residual module focused on task-specific information. These residual modules are then combined by the Arrow routing algorithm, which selects and mixes the most relevant modules for each input token without extra training. In experiments on Phi-3, this subtraction improves average accuracy over Arrow by 1.6 percentage points on English benchmarks, 3.9 points on German, and 3.6 points on French, and it also lifts Rouge-L on the open-ended SNI benchmark. The broader point is that if general and task knowledge can be separated in parameter space, modular LLM libraries become more reusable and more transferable across languages.

What carries the argument

The central object is the residual LoRA, defined as $\operatorname{LoRA}^i_{\text{res}} = \operatorname{LoRA}^i_{\text{ts}} - \operatorname{LoRA}_g$: a task-specific low-rank adapter minus a general-knowledge LoRA trained on a small Wikipedia corpus with a causal language-modeling objective. The general LoRA acts as a proxy for the broad linguistic and factual knowledge already present in the base model; subtracting it is meant to remove redundant content and force each module to carry only task-specific signal. The other load-bearing component is the Arrow routing algorithm, which computes the SVD of each LoRA, uses the top right singular vector as a prototype, projects input tokens onto these prototypes, and forms a softmax-weighted combination of the top $k$ modules per token. Together, the subtraction step makes modules more distinctive and the router more selective.

What would settle it

Train the general LoRA on scrambled or randomly shuffled Wikipedia text, or on a corpus from a completely different domain, while keeping every other detail identical; if zero-shot routing gains persist, the effect is not due to general-knowledge subtraction but to a generic parameter perturbation. A second check is to measure pairwise similarity or overlap between task LoRAs before and after subtraction; if the residual modules are not more distinct, the proposed mechanism of disentanglement is not what drives the improvement.

Watch

Extended reading notes

Core claim

On its own terms, the discovery is that a single subtraction operation in LoRA parameter space can disentangle general linguistic knowledge from task-specific adaptation. Formally, the paper writes each residual module as $\operatorname{LoRA}^i_{\text{res}} = \operatorname{LoRA}^i_{\text{ts}} - \operatorname{LoRA}_g$, where $\operatorname{LoRA}_g$ is trained on 5,000 Wikipedia segments per language with a causal language-modeling objective. The resulting residual modules are more distinct from one another, and the Arrow router, which computes a per-token weighted combination of the top-$k$ modules, can then select and compose them more effectively. The paper reports consistent gains across English, German, and French benchmarks with Phi-3, and on SNI GenKnowSub reaches a Rouge-L of 46.91 versus 45.44 for Arrow. On Phi-2, the benefit appears only when the base model has enough multilingual competence; in weak non-English settings GenKnowSub does not beat Arrow.

Load-bearing premise

The load-bearing premise is that fine-tuning a LoRA on 5,000 Wikipedia-like segments with a causal language-modeling objective actually captures the general knowledge that is redundant across task LoRAs; if that proxy fails, subtraction is just removing arbitrary parameter directions and the reported gains could come from a generic regularization effect.

Editorial extensions

If this is right

  • Existing LoRA libraries can be upgraded without retraining: subtracting one shared general LoRA from every task module is a single arithmetic step before routing.
  • Multilingual zero-shot transfer is improved when the base model has cross-lingual ability; the same residual modules route better on German and French benchmarks even when the subtracted general LoRA is English.
  • The method stays parameter-efficient: routing remains post-hoc and training-free, so the gains do not require joint training of experts and router.
  • On weaker or less multilingual base models, the expected benefit is limited to languages the base model already knows; GenKnowSub does not create multilinguality that is not there.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the Wikipedia-trained LoRA truly recalls pretraining knowledge, then the strength of subtraction should be tunable: smaller corpora or domain-matched corpora should produce weaker or stronger disentangling. This is a testable prediction not reported in the paper.
  • The method could be combined with any training-free router, not just Arrow; a router that uses distances in parameter space might benefit even more from the increased distinctiveness of residual modules.
  • Subtracting a general module is a form of centering in LoRA space; viewed that way, GenKnowSub suggests that the shared knowledge of a model can be approximated by a single centroid, which could be learned or updated as new tasks enter the library.
  • An important untested boundary is low-resource languages: the paper uses only English, French, and German general LoRAs, so whether subtraction helps when the base model has little exposure to the target language remains open.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes GenKnowSub, a method that subtracts a general-domain LoRA (trained on small Wikipedia corpora with a causal language-modeling objective) from each task-specific LoRA in a library, then applies the zero-shot Arrow routing algorithm to select and combine the resulting residual modules. The method is evaluated on Phi-3 and Phi-2 across English, German, and French benchmarks, including multiple-choice reasoning tasks and open-ended SNI generation. The main reported results are average accuracy gains over Arrow: 1.6% on English, 3.9% on German, and 3.6% on French, plus a Rouge-L gain on SNI. The paper frames these gains as evidence that subtracting redundant general knowledge improves modularity and reusability of LLMs, and it releases code and data.

Significance. If the average-gain claim is robust, GenKnowSub is a simple, training-free post-processing step that could improve the reusability of existing LoRA libraries in modular LLM systems, with no changes to the routing algorithm. The paper's strengths include evaluation on multiple languages and base models, a held-out zero-shot setup with no circularity, and public code and data. However, the empirical support is currently incomplete: no variance or significance testing is reported, the 'consistent gains' claim is contradicted by several per-dataset drops, and the central interpretation that a Wikipedia-trained LoRA captures 'general knowledge' is not tested against content-free alternatives. These issues are fixable and do not invalidate the method's average-gain observation, but they need to be addressed before the paper's stronger claims can be accepted.

major comments (3)
  1. [§3.3, Tables 1 and 2] The core claim that GenKnowSub improves over Arrow rests on average gains of 1.6%, 3.9%, and 3.6%, but no variance, standard error, or significance test is reported. Many per-dataset differences are small; for example, PIQA-French drops from 80.20 (Arrow) to 78.78 (GenKnowSub-Fr), and ARC-Easy is lower for every GenKnowSub variant than for Arrow. Without multiple seeds, confidence intervals, or at least a paired test across datasets, it is not possible to tell whether the average improvements are systematic or noise. Please report variance or explicitly state that the results are single runs and temper the significance accordingly.
  2. [§2.1, §3.2] The explanatory claim that subtracting a Wikipedia-trained LoRA removes redundant general knowledge is not supported by the current experiments. The Mean Normalization baseline subtracts the average task LoRA and fails, but it does not control for subtracting an arbitrary LoRA of comparable norm, a LoRA trained on shuffled or non-Wikipedia text, or a domain LoRA trained on news or other text. If any of these substitutes produces the same gains, the results would be explained by a generic centering or regularization effect rather than by general-knowledge subtraction. Please add controls with random LoRA, shuffled-text LoRA, and a non-Wikipedia domain LoRA to establish that the content of the general module is what matters.
  3. [Abstract, §3.3] The abstract and Section 3.3 state that GenKnowSub yields 'consistent performance gains,' but this is contradicted by the per-dataset numbers in Table 1: all GenKnowSub variants underperform Arrow on ARC-Easy and WinoGrande, and GenKnowSub-Fr underperforms on PIQA. Table 3 also shows Phi-2 BoolQ dropping from 81.13 (Arrow) to 80.12 (GenKnowSub-En). Please replace 'consistent' with an accurate description (e.g., average gains) and report per-dataset win/loss counts, or provide a statistical justification for calling the pattern consistent.
minor comments (4)
  1. [§3.2] The construction of LoRA_avg as the simple average of the English, French, and German general LoRAs is not justified; please explain why this average is a better approximation of 'general knowledge' than any individual language module, or present it as a convenience baseline.
  2. [Appendix B.2] There is a typo in the heading: 'Resutls' should be 'Results.'
  3. [Appendix B, Tables 3 and 4] The Phi-2 experiments use the task LoRAs from Ostapenko et al. trained on full cluster data, whereas the Phi-3 task LoRAs are trained on a 20% subsample; please make this difference explicit in the main text when comparing cross-model conclusions, since it complicates the interpretation of the weaker Phi-2 gains.
  4. [Table 3 and SNI paragraph] The SNI Rouge-L scores and the Phi-2 Table 3 results are reported without any error bars or sensitivity analysis; adding at least one repeated-run or bootstrap estimate would strengthen confidence in these numbers.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: GenKnowSub is an empirical transformation evaluated on held-out benchmarks, with the general LoRA trained independently on unlabeled Wikipedia data.

full rationale

The paper's central operation, LoRA_res = LoRA_ts - LoRA_g (Eq. 1), is a defined arithmetic transformation rather than a derivation whose output is presupposed by its input. The general knowledge LoRA is obtained by fine-tuning on 5,000 Wikipedia segments per language with a causal language modeling objective (Appendix A.4); it is not fitted to any evaluation target, and the task-specific LoRAs come from the clustered Flan dataset or, for Phi-2, from Ostapenko et al.'s public library. Evaluations are performed on held-out English, German, and French benchmarks and SNI, with no hyperparameter tuned against those results. The claim that a Wikipedia-trained LoRA captures redundant general knowledge is explicitly a hypothesis (Section 2.1: 'We hypothesize that fine-tuning the model with LoRA on even a small Wikipedia-like dataset...'), not a consequence of the definition of subtraction; if the hypothesis is false, the method would simply fail to improve or would improve for a different reason, which is an empirical weakness rather than circular reasoning. Citations to task arithmetic (Ilharco et al., 2023) and module arithmetic (Zhang et al., 2023) are external prior work and are not used as a uniqueness theorem or as a substitute for evidence. The Mean Normalization baseline is an ablation, not a fitted input. The absence of a control with an arbitrary or non-Wikipedia LoRA is a possible confound worth a correctness-risk discussion, but it does not make the paper circular. Therefore the circularity score is 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on untested assumptions about the content of LoRA modules trained on Wikipedia and about the effect of subtraction on routing. The hyperparameters are standard and reported. No new physical entities are introduced.

free parameters (5)
  • LoRA rank r = 4
    Rank of all PEFT modules; not tuned.
  • Arrow top-k = 3
    Number of modules combined per token per layer.
  • Wikipedia segments per language = 5000
    Training size for each general knowledge LoRA; chosen for uniformity and hardware limits.
  • Task data subsample = 20% (about 2000 samples per cluster)
    Used for Phi-3 task LoRA training to reduce compute.
  • General LoRA composition choice = En, De, Fr, or Avg
    The paper reports all variants; Avg tends to work best. This is a design choice evaluated on all options.
assumptions (4)
  • domain assumption Fine-tuning a LoRA on small Wikipedia-like data with a causal LM objective surfaces the model's pretrained general knowledge
    Section 2.1; the method depends on this proxy.
  • domain assumption Task-specific LoRA modules contain redundant general knowledge that subtraction can remove without harming task skills
    Section 2.1; if false, subtraction would degrade modules.
  • domain assumption The Flan cluster structure learned on one model transfers to other base models (Phi-3, Phi-2)
    Section 3.1; the authors explicitly assume this.
  • domain assumption Arrow's SVD-based token routing performs better when modules are more distinct after subtraction
    Section 2.2; no analysis shows why residual modules improve routing.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GenKnowSub: Improving Modularity and Reusability of LLMs through General Knowledge Subtraction." pith.science (2026). https://pith.science/paper/CAA5K3D2

@misc{pith2026250510939,
  author       = {Pith},
  title        = {Pith review of: GenKnowSub: Improving Modularity and Reusability of LLMs through General Knowledge Subtraction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CAA5K3D2}},
  note         = {Machine review of arXiv:2505.10939}
}
read the original abstract

Large language models often struggle with zero-shot generalization, and several modular approaches have been proposed to address this challenge. Yet, we hypothesize that a key limitation remains: the entanglement of general knowledge and task-specific adaptations. To overcome this, we propose a modular framework that disentangles these components by constructing a library of task-specific LoRA modules alongside a general-domain LoRA. By subtracting this general knowledge component from each task-specific module, we obtain residual modules that focus more exclusively on task-relevant information, a method we call general knowledge subtraction (GenKnowSub). Leveraging the refined task-specific modules and the Arrow routing algorithm \citep{ostapenko2024towards}, we dynamically select and combine modules for new inputs without additional training. Our studies on the Phi-3 model and standard Arrow as baselines reveal that using general knowledge LoRAs derived from diverse languages, including English, French, and German, yields consistent performance gains in both monolingual and cross-lingual settings across a wide set of benchmarks. Further experiments on Phi-2 demonstrate how GenKnowSub generalizes to weaker LLMs. The complete code and data are available at https://github.com/saharsamr/Modular-LLM.

Figures

Figures reproduced from arXiv: 2505.10939 by the authors.

Figure 1
Figure 1. Overview of our proposed approach. (a) illustrates the process of training task-specific and general [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 9 canonical work pages

  1. [1]

    Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Martin Cai, Qin Cai, Vishrav Chaudhary, Dong Chen, Dongdong Chen, Weizhu Chen, Yen-Chun Chen, Yi-Ling Chen, Hao Cheng, Parul Chopra, Xiyang Dai, Matt...

    arXiv 2024

  2. [2]

    Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. 2019. https://api.semanticscholar.org/CorpusID:208290939 Piqa: Reasoning about physical commonsense in natural language . In AAAI Conference on Artificial Intelligence

    2019

  3. [3]

    Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S

    Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri Chatterji, Annie Chen, Kathleen Creel, Jared Quincy Davis, Dora Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stef...

    arXiv 2022

  4. [4]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gr...

    2020

  5. [5]

    Lucas Caccia, Edoardo Ponti, Zhan Su, Matheus Pereira, Nicolas Le Roux, and Alessandro Sordoni. 2023. https://openreview.net/forum?id=qcQhBli5Ho Multi-head adapter routing for cross-task generalization . In Thirty-seventh Conference on Neural Information Processing Systems

    work page 2023

  6. [6]

    Alexandra Chronopoulou, Matthew Peters, Alexander Fraser, and Jesse Dodge. 2023. https://doi.org/10.18653/v1/2023.findings-eacl.153 A dapter S oup: Weight averaging to improve generalization of pretrained language models . In Findings of the Association for Computational Linguistics: EACL 2023, pages 2054--2063, Dubrovnik, Croatia. Association for Computa...

    doi:10.18653/v1/2023.findings-eacl.153 2023

  7. [7]

    Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1300 B ool Q : Exploring the surprising difficulty of natural yes/no questions . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language...

    doi:10.18653/v1/n19-1300 2019

  8. [8]

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. https://api.semanticscholar.org/CorpusID:3922816 Think you have solved question answering? try arc, the ai2 reasoning challenge . ArXiv, abs/1803.05457

    arXiv 2018

Show all 43 references
  1. [9]

    Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. https://doi.org/10.18653/v1/D18-1269 XNLI : Evaluating cross-lingual sentence representations . In Proceedings of the 2018 Conference on Empirical Methods ...

    2018 doi

  2. [10]

    Tim Dettmers, Mike Lewis, Sam Shleifer, and Luke Zettlemoyer. 2022. https://openreview.net/forum?id=shpkpVXzo3h 8-bit optimizers via block-wise quantization . In International Conference on Learning Representations

    2022

  3. [11]

    William Fedus, Barret Zoph, and Noam Shazeer. 2022. https://arxiv.org/abs/2101.03961 Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity . Preprint, arXiv:2101.03961

    2022 arXiv

  4. [12]

    Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. 2024. https://openreview.net/forum?id=lIsCS8b6zj Parameter-efficient fine-tuning for large models: A comprehensive survey . Transactions on Machine Learning Research

    2024

  5. [13]

    Junheng He, Nankai Lin, Menglan Shen, Dong Zhou, and Aimin Yang. 2023. https://doi.org/10.1109/IALP61005.2023.10337300 Exploring bias evaluation techniques for quantifying large language model biases . In 2023 International Conference on Asian Language Processing (IALP), pages...

    2023 arXiv

  6. [14]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. https://openreview.net/forum?id=d7KBjmI3GmQ Measuring massive multitask language understanding . In International Conference on Learning Representations

    2021

  7. [15]

    Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. https://proceedings.mlr.press/v97/houlsby19a.html Parameter-efficient transfer learning for NLP . In Proceedings of the 36th In...

    2019

  8. [16]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. https://arxiv.org/abs/2106.09685 Lora: Low-rank adaptation of large language models . Preprint, arXiv:2106.09685

    2021 arXiv

  9. [17]

    Chengsong Huang, Qian Liu, Bill Yuchen Lin, Tianyu Pang, Chao Du, and Min Lin. 2024. https://openreview.net/forum?id=TrloAXEJ2B Lorahub: Efficient cross-task generalization via dynamic lo RA composition . In First Conference on Language Modeling

    2024

  10. [18]

    Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. 2023. https://openreview.net/forum?id=6t0Kwf8-jrj Editing models with task arithmetic . In The Eleventh International Conference on Learning Representations

    2023

  11. [19]

    Mojan Javaheripi, S \'e bastien Bubeck, Marah Abdin, Jyoti Aneja, Caio C \'e sar Teodoro Mendes, Weizhu Chen, Allie Del Giorno, Ronen Eldan, Sivakanth Gopi, et al. 2023. Phi-2: The surprising power of small language models. Microsoft Research Blog

    2023

  12. [20]

    Rossi, and Thien Huu Nguyen

    Viet Dac Lai, Chien Van Nguyen, Nghia Trung Ngo, Thuat Nguyen, Franck Dernoncourt, Ryan A. Rossi, and Thien Huu Nguyen. 2023. https://api.semanticscholar.org/CorpusID:260334562 Okapi: Instruction-tuned large language models in multiple languages with reinforcement learning fro...

    2023 arXiv

  13. [21]

    Yixiao Li, Yifan Yu, Chen Liang, Nikos Karampatziakis, Pengcheng He, Weizhu Chen, and Tuo Zhao. 2024. https://openreview.net/forum?id=LzPWWPAdY4 Loftq: Lo RA -fine-tuning-aware quantization for large language models . In The Twelfth International Conference on Learning Representations

    2024

  14. [22]

    Paul Pu Liang, Chiyu Wu, Louis-Philippe Morency, and Ruslan Salakhutdinov. 2021. https://api.semanticscholar.org/CorpusID:235623756 Towards understanding and mitigating social biases in language models . In International Conference on Machine Learning

    2021

  15. [23]

    Haokun Liu, Derek Tam, Muqeeth Mohammed, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin Raffel. 2022. https://openreview.net/forum?id=rBCvMG-JsPd Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning . In Advances in Neural Information Proc...

    2022

  16. [24]

    Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, and Adam Roberts. 2023. https://proceedings.mlr.press/v202/longpre23a.html The flan collection: Designing data and methods for effective instruction tuning . I...

    2023

  17. [25]

    Ilya Loshchilov and Frank Hutter. 2019. https://openreview.net/forum?id=Bkg6RiCqY7 Decoupled weight decay regularization . In International Conference on Learning Representations

    2019

  18. [26]

    Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. https://doi.org/10.18653/v1/D18-1260 Can a suit of armor conduct electricity? a new dataset for open book question answering . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language P...

    2018 doi

  19. [27]

    Mohammed Muqeeth, Haokun Liu, Yufan Liu, and Colin Raffel. 2024. https://api.semanticscholar.org/CorpusID:267547973 Learning to route among specialized experts for zero-shot generalization . ArXiv, abs/2402.05859

    2024 arXiv

  20. [28]

    Oleksiy Ostapenko, Zhan Su, Edoardo Ponti, Laurent Charlin, Nicolas Le Roux, Lucas Caccia, and Alessandro Sordoni. 2024. https://openreview.net/forum?id=0ZFWfeVsaD Towards modular LLM s by building and reusing a library of lo RA s . In Forty-first International Conference on M...

    2024

  21. [29]

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu ...

    2019

  22. [30]

    Jonas Pfeiffer, Sebastian Ruder, Ivan Vuli \'c , and Edoardo Ponti. 2023. https://openreview.net/forum?id=z9EkXfvxta Modular deep learning . Transactions on Machine Learning Research. Survey Certification

    2023

  23. [31]

    Edoardo Maria Ponti, Alessandro Sordoni, Yoshua Bengio, and Siva Reddy. 2023. https://doi.org/10.18653/v1/2023.eacl-main.49 Combining parameter-efficient modules for task-level generalisation . In Proceedings of the 17th Conference of the European Chapter of the Association fo...

    2023 doi

  24. [32]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. http://jmlr.org/papers/v21/20-074.html Exploring the limits of transfer learning with a unified text-to-text transformer . Journal of Machine Lea...

    2020

  25. [33]

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. https://doi.org/10.1145/3474381 Winogrande: an adversarial winograd schema challenge at scale . Commun. ACM, 64(9):99–106

    2021 doi

  26. [34]

    Mirac Suzgun, Nathan Scales, Nathanael Sch \"a rli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, and Jason Wei. 2023. https://doi.org/10.18653/v1/2023.findings-acl.824 Challenging BIG -bench tasks and whether chain-of-thought c...

    2023 doi

  27. [35]

    Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjana Arunkumar, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Lai, Ishan Purohit, Ishani Mondal, Jacob Anderson, Kirby Kuzni...

    2022

  28. [36]

    Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus

    Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022. https://openreview.net/forum?id=yzkSU5zd...

    2022

  29. [37]

    https://dumps.wikimedia.org Wikimedia downloads

    WikimediaFoundation. https://dumps.wikimedia.org Wikimedia downloads

  30. [38]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020 doi

  31. [39]

    Rowan Zellers, Yonatan Bisk, Roy Schwartz, and Yejin Choi. 2018. https://doi.org/10.18653/v1/D18-1009 SWAG : A large-scale adversarial dataset for grounded commonsense inference . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages ...

    2018 doi

  32. [40]

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. https://doi.org/10.18653/v1/P19-1472 H ella S wag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4791--4...

    2019 doi

  33. [41]

    Jinghan Zhang, Shiqi Chen, Junteng Liu, and Junxian He. 2023. https://openreview.net/forum?id=5r3e27I9Gy Composing parameter-efficient modules with arithmetic operation . In Thirty-seventh Conference on Neural Information Processing Systems

    2023

  34. [42]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  35. [43]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.